Microservice architecture root cause positioning method based on heterogeneous graph modeling and active learning
By employing heterogeneous graph modeling and active learning methods, the problems of heterogeneity and high manual annotation costs in large-scale microservice systems are solved, enabling efficient and accurate root cause localization and adapting to dynamic production environments.
Patent Information
- Application Number
- CN202511635570.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-10
- Publication Date
- 2026-02-24
AI Technical Summary
Existing technologies struggle to achieve accurate modeling and efficient root cause localization in large-scale microservice systems, particularly due to challenges related to heterogeneity and the cost of manual annotation.
We employ a heterogeneous graph modeling and active learning approach. By constructing a unified heterogeneous graph and combining it with multi-source observation data for feature extraction and topology analysis, we use active learning-driven semi-supervised training to optimize the root cause localization model, thereby achieving real-time fault detection and localization.
It addresses the challenges of managing heterogeneity and the high cost of manual annotation in large-scale microservice systems, improving the accuracy and efficiency of root cause analysis, reducing the need for manual annotation, and adapting to dynamic production environments.
Smart Images
Figure CN121560600A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent operation and maintenance and root cause localization technology, and relates to a root cause localization method for microservice architecture, especially a root cause localization method for microservice architecture based on heterogeneous graph modeling and active learning. Background Technology
[0002] With the rapid development of cloud computing and service-oriented technologies, microservice systems have become the mainstream architecture for modern internet services. Microservices break down complex monolithic services into a series of independent service units, giving the system high scalability, maintainability, and rapid iteration. However, the flexibility and loose coupling of this architecture also bring significant complexity to system operation and maintenance. When microservices reach thousands or even tens of thousands of instances, failures become inevitable, and a failure in any single service can propagate through complex dependencies, triggering cascading failures or even a global system outage.
[0003] Statistics show that large internet companies generate millions of abnormal alerts daily in their data centers, of which less than 1% are true system-level failures. However, locating and resolving these critical issues often requires significant human intervention and expert judgment. Traditional troubleshooting methods based on human experience can no longer meet the operational needs of ultra-large-scale systems. Intelligent and automated root cause localization has become an important research direction in the field of intelligent operations and maintenance.
[0004] In recent years, with the widespread application of microservice systems in distributed systems, their complexity and dynamism have placed higher demands on system operation and maintenance. For root cause analysis in microservice systems, researchers have proposed a series of innovative solutions by fusing multi-source observable data (such as logs, metrics, call chains, and deployment information) and applying machine learning and deep learning techniques. These solutions have improved the efficiency and accuracy of root cause analysis to some extent. However, due to the heterogeneous nature and large number of instances in microservice systems, achieving accurate modeling and efficient root cause analysis still faces the following two challenges: 1. Difficulty in Capturing System Heterogeneity: Microservice systems consist of hosts (physical entities) and services (abstract entities), which differ significantly in structural characteristics, dynamic behavior, and monitoring data. Hosts involve physical resources (such as CPU and disk I / O), and their failures typically manifest as resource bottlenecks or hardware malfunctions. Services, on the other hand, are logical functional units (such as API calls and business logic), and their failures are often related to call chain or business logic anomalies. Due to the heterogeneity of hosts and services in terms of topology and fault propagation patterns, root cause localization methods that rely solely on modeling a single type of node struggle to capture these differences, resulting in poor performance in locating host-level faults.
[0005] 2. High Cost of Manual Labeling: In microservice systems, the cost of manually labeling fault samples is high, and the quality of automatically generated labels is unstable. Supervised methods rely on manually labeled fault samples, but manual labeling is costly, especially in large-scale production environments where manually labeling all data is impractical. On the other hand, unsupervised methods avoid labeling costs, but their performance is highly dependent on specific assumptions (such as fault-free propagation or a single fault mode). When these assumptions are not true, the performance of unsupervised methods drops sharply, making it difficult to provide reliable root cause localization results. Therefore, traditional root cause localization methods struggle to achieve an effective balance between labeling costs and localization performance.
[0006] In summary, designing an accurate modeling method for heterogeneous microservice systems, achieving a balance between annotation cost and fault location performance, has become a key issue in the reliability engineering of microservice systems. To address this problem, this invention proposes a root cause localization method for microservice architectures based on heterogeneous graph modeling and active learning.
[0007] A search revealed no publicly available literature of the same or similar prior art as this invention. Summary of the Invention
[0008] This invention addresses the shortcomings of existing technologies by proposing a root cause localization method for microservice architectures based on heterogeneous graph modeling and active learning. This method can fully utilize the structural characteristics of microservice systems and has low annotation requirements, thus achieving a balance between task performance and annotation overhead. This provides a better solution for efficient root cause localization of microservice systems.
[0009] The above-mentioned objective of this invention is achieved through the following technical solution: A root cause localization method for microservice architecture based on heterogeneous graph modeling and active learning includes the following two main steps: Step 1: Extract features and topology from metrics, logs, call chains, and deployment information to form a unified heterogeneous graph model; Step 2: Perform clustering, label diffusion, and boundary sample selection on the heterogeneous graph model constructed in Step 1. Use active learning-driven semi-supervised training to continuously optimize the root cause localization model, and deploy the model online in the production environment to achieve real-time fault detection and localization.
[0010] Furthermore, the specific steps of step 1 include: (1) Obtain multi-source observation data from the microservice system; (2) Based on the call chain and configuration management data in the multi-source observation data, perform topology extraction and generate the skeleton of the heterogeneous graph; (3) Refine the edges of the generated heterogeneous graph skeleton, retaining only the high-value edges to form an optimized heterogeneous graph skeleton; (4) Extract events from indicators and log data in multi-source observation data; (5) Align and fuse the extracted events; (6) Combine the merged events and skeleton to form a heterogeneous graph with events; Moreover, the specific method of step (1) of step 1 is as follows: In the initial stage, observability engineering and persistence technologies are used to obtain multi-source observation data from the database of the microservice system, including metrics, logs, call chains and configuration management data.
[0011] Moreover, the specific steps of step 1, step (2) include: ① Configuration Management Database Retrieval: Using the configuration management database data obtained in the initial stage, retrieve the interaction relationships between hosts and the deployment relationships between services and hosts; ② Call chain extraction: Extracting inter-service call relationships from distributed call chain data; ③ Combining the above two parts of data, a directed acyclic graph is generated as the skeleton of the heterogeneous graph, including the directed edges between service nodes, host nodes and host nodes, and between service nodes and host nodes; the difference between the directed acyclic graph and the final required heterogeneous graph is that the directed acyclic graph does not have node data.
[0012] Furthermore, the specific steps of step 1, step (4) include: ① Indicator event extraction: For indicator data, firstly, remove sequences with zero variance (indicating that the sequence contains no information) and those with variance greater than five times the average variance (indicating that the sequence has too much noise or excessive short-term fluctuations). Then, remove periodic patterns in the indicators through STL decomposition, retaining only the residuals. Use the n-sigma rule to calculate the indicator deviation and obtain floating-point type indicator events (representing the degree of deviation of the indicator from the mean at a certain timestamp, in units of σ). This indicator event represents the degree of abnormality of the indicator.
[0013] ② Log event extraction: For log data, regular expressions are used to remove low-information content, retain semantic information, and filter out logs that only contain INFO level. After filtering, semantic vector embeddings of logs are generated through a pre-trained model, and DBSCAN is used to cluster them into K types of log events. K depends on the number of log templates and the number of indicator types (too large or too small a K may cause one type of data to overwhelm another).
[0014] Furthermore, the specific method of step (5) of step 1 is as follows: For service nodes, align and merge metric events and log events to form service node characteristics; for host nodes, since host nodes often do not have log data, only metric events are used to form host node characteristics.
[0015] Furthermore, the specific method for step (6) of step 1 is as follows: The fused events and skeleton are combined to form a heterogeneous graph with events; the skeleton serves as the edge data of the heterogeneous graph, and the events serve as the node data of the heterogeneous graph. The combination of the two forms a complete heterogeneous graph. Furthermore, the specific steps of step 2 include: (1) Clustering: After obtaining the embedding features of the heterogeneous graph, the DBSCAN clustering method is applied to cluster the embedding features to generate clusters and noise; (2) Cluster center annotation: Similar embedded features are grouped into the same cluster. The cluster center is calculated for each cluster, and the cluster center is manually annotated to obtain the cluster center annotation for each cluster.
[0016] (3) Label diffusion: After labeling the cluster center of each cluster, the cluster center label is selected as the label of all embedded features in the entire cluster. The labels formed in this way are called pseudo labels.
[0017] (4) Edge checking: Manually label the boundaries and noise of each cluster (the labeling scale depends on the affordable labeling cost), and incorporate the labeling results into the dataset containing pseudo-labels.
[0018] (5) Training: Through the above steps, a high-quality dataset containing pseudo-labels is obtained. Based on this dataset, a heterogeneous graph convolutional neural network is trained to generate a network for real-time root cause localization.
[0019] (6) Location: Deploy the network to the online system, perform root cause location when a fault occurs, output the root cause probability of each node, and complete the fault location task.
[0020] The advantages and positive effects of this invention are as follows: 1. This invention proposes a microservice root cause localization method based on heterogeneous graph modeling, specifically designed for large-scale cloud-native microservice environments, to address the challenge of traditional methods failing to characterize the heterogeneity of services and hosts. By explicitly constructing a heterogeneous graph, service nodes and host nodes are distinguished, and multiple types of edge relationships (such as call chain dependencies, metric associations, deployment relationships, etc.) are established, which can fully capture the differences in structural characteristics, dynamic behavior, and fault propagation patterns of nodes. Compared with traditional homogeneous graph modeling methods, this invention effectively avoids problems such as feature alignment difficulties, missing semantic levels, and data sparsity, achieving accurate modeling of the complex heterogeneous structure of microservice systems, thereby providing a more reliable input representation for subsequent root cause localization.
[0021] 2. To address the challenges of high annotation costs and limited available fault samples in microservice systems, this invention introduces an active learning strategy based on heterogeneous graphs. The method includes steps such as heterogeneous graph embedding clustering and uncertainty sampling based on label propagation, enabling the model to prioritize the nodes with the highest information gain for annotation under limited labeled data, thereby gradually optimizing the root cause localization capability of the graph neural network. This strategy significantly reduces the need for manual annotation while ensuring high-precision root cause localization even in dynamic production environments. It solves the performance limitations of existing supervised methods due to reliance on extensive manual annotation and avoids the performance drop-off of unsupervised methods when assumptions are invalid. Attached Figure Description
[0022] Figure 1 This is a general diagram of the root cause localization system of the present invention; a heterogeneous graph modeling module (left); and an active learning module (right).
[0023] Figure 2 This is a schematic diagram illustrating the difference between isomorphic graph modeling and heteromorphic graph modeling in this invention; Figure 3 This is a schematic diagram illustrating the conventional solution to the data sparsity problem in isomorphic graph modeling and the principle of how heteromorphic graph modeling avoids this problem. Figure 4 This is a flowchart of the processing of the present invention. Detailed Implementation
[0024] The structure of the present invention will be further described below with reference to the accompanying drawings and embodiments. It should be noted that these embodiments are descriptive and not limiting.
[0025] A root cause localization method for microservice architecture based on heterogeneous graph modeling and active learning, such as Figure 1 As shown, it includes the following steps: Step 1: Extract features and topology from metrics, logs, call chains, and deployment information to form a unified heterogeneous graph model; Objective: To construct a heterogeneous graph from multi-source observation data (metrics, logs, call chains, configuration management data, etc.).
[0026] The specific steps of step 1 include: (1) Obtain multi-source observation data from the microservice system; The specific method for step (1) of step 1 is as follows: In the initial stage, observability engineering and persistence technologies are used to obtain multi-source observation data from the database of the microservice system, including metrics, logs, call chains and configuration management data.
[0027] (2) Based on the multi-source observation data obtained from the microservice system, topology extraction is performed and the skeleton of the heterogeneous graph is generated; The specific steps of step 1, step (2) include: ① Configuration Management Database Retrieval: Using the configuration management database data obtained in the initial phase, retrieve the interaction relationships between hosts and the deployment relationships between services and hosts.
[0028] ② Call chain extraction: Extract the inter-service call relationships from the distributed call chain data.
[0029] ③ Combining the above two parts of data, generate the skeleton of the heterogeneous graph (directed acyclic graph), which includes service nodes, host nodes, and directed edges between service nodes and host nodes.
[0030] (3) Refine the edges of the generated heterogeneous graph skeleton: Topology extraction yields all three types of directed edges, but these edges have inconsistent information densities. Therefore, it is necessary to remove edges with low information density and retain only high-value edges to form a heterogeneous graph skeleton. This invention prunes low-relevance edges based on the Pearson correlation coefficient of event frequency between nodes and forces edges serving the host to be unidirectional, conforming to the fault propagation pattern, thus generating a refined heterogeneous graph skeleton.
[0031] (4) Extract events from the skeleton of the heterogeneous graph after edge refinement; The skeleton of a heterogeneous graph alone is insufficient to support model training. Therefore, it is necessary to extract events from metrics and logs to form node data of the heterogeneous graph for use in the active learning process.
[0032] The specific steps of step 1, step (4) include: ① Indicator event extraction: For indicator data, firstly remove sequences with low variance (no fluctuation) or high variance (fluctuation exceeding twice the global mean), then remove periodic patterns in the indicator through STL decomposition, retaining only the residuals; use the n-sigma rule to calculate the indicator deviation and obtain floating-point type indicator events, which represent the degree of abnormality of the indicator.
[0033] ② Log event extraction: For log data, regular expressions are used to remove low-information content (such as numeric IDs and IP addresses), retain semantic information, and filter out logs that only contain INFO level. After filtering, semantic vector embeddings of logs are generated through a pre-trained model (such as SentenceTransformer), and DBSCAN is used to cluster them into K types of log events.
[0034] (5) Merge the extracted events; The specific method for step (5) of step 1 is as follows: Event Fusion: After extracting metric events and log events, the formats of the two types of events are inconsistent, so event fusion is necessary. For service nodes, both metrics and logs are common and easily obtained through observability engineering, so metric events and log events are directly concatenated to form service features. For host nodes, logs are very rare, and in most cases only metrics (such as CPU usage and system I / O) can be obtained, so only metric events are used to form feature vectors.
[0035] (6) Combine the merged events and skeleton to form a heterogeneous graph with events; The specific method for step (6) of step 1 is as follows: Heterogeneous graph embedding: After the above steps are completed, the fused events and skeleton are combined to form a heterogeneous graph with events; the skeleton serves as the edge data (edge_data) of the heterogeneous graph, and the events serve as the node data (node_data) of the heterogeneous graph. The combination of the two forms a complete heterogeneous graph. These heterogeneous graphs are fed as data into a heterogeneous graph convolutional neural network for self-supervised training to form an embedding layer. The embedded features formed by the heterogeneous graph after passing through the embedding layer enter the next stage (active learning stage).
[0036] The working principle of step 1 is as follows: To adapt to the heterogeneous characteristics of microservice systems and address the challenge of "difficulty in grasping system heterogeneity", this invention adopts graph structure modeling, focusing on both service characteristics and host machine characteristics, and uses a heterogeneous graph to separate the two to ensure clear characteristic reference.
[0037] like Figure 2 As shown, this illustrates the difference between using isomorphic graph modeling (located on the top side of the graph) and heteromorphic graph modeling (located on the bottom side of the graph) in the process of modeling graph-structured data.
[0038] The core difference between homogeneous graph modeling and heterogeneous graph modeling lies in the handling of node types and the design of the feature space. Homogeneous graph modeling treats all nodes in the system as the same type, ignoring the essential differences between services and hosts, thus failing to reflect the characteristics and hierarchical relationships of different entities. In terms of feature space, homogeneous graph modeling typically unifies node features by inserting placeholders, which introduces significant redundancy, causing the model to consume more resources during graph neural network computation, reducing the quality of features in the graph structure data, and slowing down convergence. In contrast, heterogeneous graph modeling can distinguish between different types of nodes, reserving a dedicated feature space for each type of node, eliminating the need to manually fill in missing features, thereby improving feature quality, reducing computational burden, and accelerating model convergence.
[0039] Heterogeneous graph modeling is necessary for root cause analysis in microservice systems. Firstly, it alleviates the problems of feature sparsity and computational efficiency. Secondly, in microservice systems, service nodes generate rich, multi-source observation data, while host nodes primarily provide monitoring metrics, leading to uneven distribution of node features. Figure 3 As shown, this illustrates the conventional solution to the data sparsity problem in homogeneous graph modeling and the solution to this problem in heterogeneous graph modeling. As shown on the left, homogeneous graph modeling typically fills in missing features with placeholders. As shown on the right, heterogeneous graph modeling fundamentally avoids this problem.
[0040] Homogeneous graph modeling typically fills in missing features with placeholders. While this achieves uniformity, it introduces a large number of invalid features, increases the computational cost of message passing, and reduces model training efficiency. Heterogeneous graph modeling, on the other hand, allows different types of nodes to use their respective feature sets, avoiding redundancy, improving feature quality, and reducing computational burden. Therefore, root cause localization methods based on heterogeneous graph modeling are more efficient in large-scale systems.
[0041] Secondly, heterogeneous graph modeling preserves entity differentiation and hierarchical structure information, enhancing diagnostic accuracy. Homogeneous graph modeling ignores the different roles of services and hosts and lacks a hierarchical concept, making it difficult to model the propagation patterns of faults between the service and host layers. It also fails to utilize available domain knowledge, such as the directionality of fault propagation across different layers and data imbalance. Heterogeneous graph modeling, by explicitly distinguishing entity types and introducing hierarchical information, can combine domain knowledge to add loss weights to lower-labeled layers, accurately capturing inter-layer fault propagation patterns, thereby improving the accuracy and interpretability of diagnostic results.
[0042] like Figure 1 As shown on the left, the heterogeneous graph modeling module is responsible for integrating multi-source heterogeneous data into a unified graph structure model, providing a foundation for subsequent root cause localization.
[0043] Its main process is as follows: 1. Data Input and Feature Extraction Indicator data feature extraction: Anomaly detection and statistical feature extraction of the usage of key resources such as CPU, memory, network, and I / O.
[0044] Log data feature extraction: Semantic clustering and pattern extraction are performed on system logs to form semantic feature vectors.
[0045] Call relationship extraction: Extract the call relationships between services from the call chain, including call frequency and survey topology.
[0046] Deployment information extraction: Extract service-host node association information, topology structure and deployment dependencies.
[0047] 2. Redundant Information Filtering and Topology Construction: The system filters redundant information from multi-source data, removing duplicate or irrelevant information to reduce feature size and improve feature quality. It extracts topology features by calling relationships and deployment information, forming an initial heterogeneous graph model with multiple types of nodes and edges. The filtered features are then bound to the topology to form a complete heterogeneous graph model.
[0048] 3. Heterogeneous graph modeling and optimization The node types are clearly distinguished into service nodes and host nodes to ensure that the characteristics of different types of nodes are modeled independently; the edge types include call relationship edges, indicator dependency edges, and deployment association edges, which explicitly express multimodal dependency relationships; the edge weights and graph structure are optimized through correlation analysis, with edges that are highly related to faults being given higher weights and edges that are not related to faults being given lower weights, forming an optimized heterogeneous graph that can be used for graph neural network training.
[0049] In this system, the advantage of the heterogeneous graph modeling module is that it not only preserves the structural information and semantic features of the data, avoiding the semantic loss caused by the traditional homogeneous graph feature alignment, but also allows direct input into the graph neural network for training, improving the model's ability to perceive complex dependencies.
[0050] Step 2: Perform clustering, label diffusion, and boundary sample selection on the heterogeneous graph model constructed in Step 1. Continuously optimize the root cause localization model through semi-supervised training, and deploy the model online in the production environment to achieve real-time fault detection and localization.
[0051] Objective: To generate labels for the embedded features of the constructed heterogeneous graph, form a dataset containing pseudo-labels through label diffusion, improve the dataset through edge detection, and finally put it into training.
[0052] The specific steps of step 2 include: (1) Clustering: After obtaining the embedding features of the heterogeneous graph, the DBSCAN clustering method is applied to cluster the embedding features to generate clusters and noise.
[0053] (2) Cluster center annotation: Similar embedded features are grouped into the same cluster. At this time, it is only necessary to calculate the cluster center for each cluster and manually annotate the cluster center to obtain the cluster center annotation for each cluster, which greatly reduces the annotation requirements.
[0054] (3) Label diffusion: After labeling the cluster center of each cluster, the cluster center label is selected as the label of all embedded features in the entire cluster. The labels formed in this way are called pseudo labels.
[0055] (4) Edge checking: Although the clustering method greatly reduces the labeling requirements, the pseudo-labels obtained in this way will inevitably contain erroneous labels. In order to reduce the erroneous labels contained in the pseudo-labels, edge checking is required to check and label the embedded features of the cluster edges. Specifically, the boundaries and noise of each cluster are manually labeled (the labeling scale depends on the affordable labeling cost), and the labeling results are incorporated into the dataset containing pseudo-labels to improve the quality of pseudo-labels.
[0056] (5) Training: Through the above steps, a high-quality dataset containing pseudo-labels is obtained. Based on this dataset, a heterogeneous graph convolutional neural network is trained to generate a network for real-time root cause localization.
[0057] (6) Location: Deploy the network to the online system, perform root cause location when a fault occurs, output the root cause probability of each node, and complete the fault location task.
[0058] The working principle of step 2 is as follows: To address the challenge of high manual annotation costs in fault location tasks within microservice systems, this system employs semi-supervised active learning as the model training method, aiming to minimize the need for manual annotation during model training while meeting task requirements.
[0059] like Figure 1 As shown on the right, the active learning module of this system is based on heterogeneous graph embedding and uses a semi-supervised learning strategy to train and optimize the model, reducing the need for manual annotation. Its core process is as follows: 1. Graph Embedding Clustering Embedding learning is performed on the optimized heterogeneous graph to map the nodes to a high-dimensional vector space. Then, based on the embedding matrix, the DBSCAN method is used for clustering, thereby classifying all samples into several potential failure modes.
[0060] 2. Limited manual labeling and label dissemination A pseudo-labeling process is implemented for a small number of manually labeled nodes in the clustering results, spreading the manually labeled labels to unlabeled samples. This process significantly reduces the amount of labeling work while ensuring the accuracy of the pseudo-labels.
[0061] 3. Edge sample checking and iterative optimization Manual labeling and annotation are initiated for samples with high prediction uncertainty (marginal samples located at the boundary of the classification cluster), forming a feedback loop. This allows the newly labeled data to be used for model training, achieving iterative optimization and maximizing the utilization rate of manually labeled data.
[0062] 4. Online deployment The trained model can be deployed to a production environment and activated when a failure occurs for real-time root cause localization.
[0063] In this system, the advantages of the active learning module are: 1. Reduce manual labeling: Semi-supervised training is achieved through clustering and label propagation, reducing the operational burden.
[0064] 2. Improve positioning accuracy: Actively select boundary samples to enhance training effects and ensure high accuracy in complex fault scenarios.
[0065] 3. Adaptable to dynamic environments: It can handle microservice topology changes and data distribution drift, ensuring long-term stable operation of the system.
[0066] The working principle of this invention is: This invention achieves efficient root cause localization in large-scale microservice systems through heterogeneous graph modeling and active learning. The heterogeneous graph modeling module ensures the structured representation and semantic integrity of multi-source data; the active learning module optimizes the model with limited annotations using a semi-supervised strategy, reducing manual costs and adapting to dynamic production environments. The overall solution addresses the shortcomings of traditional methods in feature alignment, data sparsity, semantic hierarchy, and dynamic topology adaptability, achieving a balance between task performance and annotation overhead in root cause localization tasks within microservice systems.
[0067] This invention addresses two major challenges: difficulty in grasping heterogeneity and high cost of manual annotation. By modeling heterogeneous graphs and then performing active learning based on these models, it solves the problem of the difficulty in grasping heterogeneity and the high cost of manual annotation.
[0068] Heterogeneous graph modeling transforms multi-source data (metrics, logs, call chains, and configuration management data) into a heterogeneous graph model through topology extraction and event fusion. Nodes distinguish between services and hosts, while edges reflect the call and deployment relationships between services, between hosts, and between services and hosts. This modeling approach captures the heterogeneous structure of microservice systems, addressing the challenge of accurately capturing heterogeneity.
[0069] Active learning maximizes the use of manual annotation. Combining DBSCAN clustering and active learning strategies, this system efficiently generates pseudo-labels. Through cluster core annotation and label diffusion techniques, it significantly reduces the need for manual annotation, and further improves label quality by utilizing cluster boundary and noise point checks, thus achieving efficient training with limited annotations. Experiments show that this strategy can achieve over 95% accuracy in a fully supervised model with only 30% labeled data, solving the challenge of high costs associated with manual annotation.
[0070] The innovation of this invention lies in: This invention proposes a root cause localization method for microservice architectures based on heterogeneous graph modeling and active learning. This method addresses two major challenges in microservice system fault localization: difficulty in grasping system heterogeneity and high cost of manual annotation. It achieves significant results in accuracy, stability, and operational efficiency, fully demonstrating its practical value and innovative advantages in complex microservice architectures. Experimental results on two heterogeneous microservice datasets show that this invention outperforms several existing advanced methods in root cause localization, including representative algorithms such as CausalRCA based on causal reasoning, ART and Eadro based on graph learning, and DiagFusion based on multi-source fusion, demonstrating excellent overall performance.
[0071] First, in terms of root cause localization accuracy, this invention significantly outperforms similar methods. On the first dataset, this invention achieves a Top-1 accuracy of 82.5%, improving upon DiagFusion and ART (which have the next best performance) by approximately 30 percentage points. On the second dataset, the Top-1 accuracy reaches 68.75%, improving upon CausalRCA by approximately 50 percentage points. Further analysis of the Top-3 and Top-5 metrics shows that this invention consistently ranks first in accuracy at all levels, indicating superior discriminative ability in candidate ranking and multi-source feature fusion.
[0072] Secondly, in terms of algorithm efficiency, this invention achieves a good balance between training and inference. Experimental results show that the average offline training time of this invention on the two datasets is 71 seconds and 121 seconds, respectively, which is significantly shorter than complex models such as CausalRCA and DiagFusion; the average single root cause localization time in the online inference stage is 2.5 seconds and 1.2 seconds, respectively, which can meet the real-time and responsiveness requirements of production-grade systems.
[0073] Third, regarding the contribution and interpretability of the model modules, this invention verified through ablation experiments that each core module is effective in the root cause localization task. Experimental results show that removing any module leads to a decrease in model performance, with the Top-1 accuracy decreasing by an average of 8 percentage points. This indicates that the modules in this invention synergistically enhance the model's expressive power and stability, and that the new technologies used are effective and contributing.
[0074] In summary, the technical effects of this invention are mainly reflected in the following four aspects: 1. Improve model interpretability: By using heterogeneous graph modeling, the cross-level relationship between hosts and services can be accurately depicted, solving the problem of difficulty in heterogeneous modeling of systems. Different types of dependencies between nodes are clearly expressed, making both modeling data and model output more interpretable.
[0075] 2. Reduce labeling costs: The active learning strategy achieves effective training and high-precision localization with only a small amount of labeled data through a high-value sample selection mechanism, which greatly reduces the cost of manual fault labeling in practical applications.
[0076] 3. Balancing computational efficiency: The system boasts excellent performance and lightweight deployment, achieving a balance between offline training and online inference. It possesses rapid response capabilities, making it suitable for continuous monitoring and rapid decision-making in online service systems.
[0077] 4. Improved root cause localization accuracy: Compared with existing baseline methods, this invention achieves a 20% to 50% performance improvement across multiple evaluation metrics, and maintains high accuracy even in complex heterogeneous environments.
[0078] Although embodiments and drawings of the present invention have been disclosed for illustrative purposes, those skilled in the art will understand that various substitutions, variations and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the scope of the present invention is not limited to the contents disclosed in the embodiments and drawings.
Claims
1. A root cause localization method for microservice architecture based on heterogeneous graph modeling and active learning, characterized in that: Includes the following steps: Step 1: Extract features and topology from metrics, logs, call chains, and deployment information to form a unified heterogeneous graph model; Step 2: Perform clustering, label diffusion, and boundary sample selection on the heterogeneous graph model constructed in Step 1. Use active learning-driven semi-supervised training to continuously optimize the root cause localization model, and deploy the model online in the production environment to achieve real-time fault detection and localization.
2. The microservice architecture root cause localization method based on heterogeneous graph modeling and active learning according to claim 1, characterized in that: The specific steps of step 1 include: (1) Obtain multi-source observation data from the microservice system; (2) Based on the call chain and configuration management data in the multi-source observation data, perform topology extraction and generate the skeleton of the heterogeneous graph; (3) Refine the edges of the generated heterogeneous graph skeleton, retaining only the high-value edges to form an optimized heterogeneous graph skeleton; (4) Extract events from indicators and log data in multi-source observation data; (5) Align and fuse the extracted events; (6) Combine the merged events and skeleton to form a heterogeneous graph with events.
3. The microservice architecture root cause localization method based on heterogeneous graph modeling and active learning according to claim 2, characterized in that: The specific method for step (1) of step 1 is as follows: In the initial stage, observability engineering and persistence technologies are used to obtain multi-source observation data from the database of the microservice system, including metrics, logs, call chains and configuration management data.
4. The microservice architecture root cause localization method based on heterogeneous graph modeling and active learning according to claim 2, characterized in that: The specific steps of step 1, step (2) include: ① Configuration Management Database Retrieval: Using the configuration management database data obtained in the initial stage, retrieve the interaction relationships between hosts and the deployment relationships between services and hosts; ② Call chain extraction: Extracting inter-service call relationships from distributed call chain data; ③ Combining the above two parts of data, a directed acyclic graph is generated as the skeleton of the heterogeneous graph, including service nodes and service nodes, host nodes and host nodes, and directed edges between service nodes and host nodes; the difference between the directed acyclic graph and the final required heterogeneous graph is that the directed acyclic graph does not have node data.
5. The microservice architecture root cause localization method based on heterogeneous graph modeling and active learning according to claim 2, characterized in that: The specific steps of step 1, step (4) include: ① Indicator event extraction: For indicator data, firstly remove sequences with zero variance and variance greater than five times the average variance, then remove periodic patterns in the indicators through STL decomposition, retaining only the residuals; use the n-sigma rule to calculate the indicator deviation and obtain floating-point type indicator events, which represent the degree of abnormality of the indicator. ② Log event extraction: For log data, regular expressions are used to remove low-information content, retain semantic information, and filter out logs that only contain INFO level. After filtering, semantic vector embeddings of logs are generated through a pre-trained model, and DBSCAN is used to cluster them into K types of log events, where K depends on the number of log templates and the number of metric types.
6. The microservice architecture root cause localization method based on heterogeneous graph modeling and active learning according to claim 2, characterized in that: The specific method for step (5) of step 1 is as follows: For service nodes, align and merge metric events and log events to form service node characteristics; For host nodes, since host nodes often do not have log data, only indicator events are used to form host node characteristics.
7. The microservice architecture root cause localization method based on heterogeneous graph modeling and active learning according to claim 2, characterized in that: The specific method for step (6) of step 1 is as follows: The fused events and skeleton are combined to form a heterogeneous graph with events; the skeleton serves as the edge data of the heterogeneous graph, and the events serve as the node data of the heterogeneous graph. The combination of the two forms a complete heterogeneous graph.
8. The microservice architecture root cause localization method based on heterogeneous graph modeling and active learning according to claim 1, characterized in that: The specific steps of step 2 include: (1) Clustering: After obtaining the embedding features of the heterogeneous graph, the DBSCAN clustering method is applied to cluster the embedding features to generate clusters and noise; (2) Cluster center annotation: Similar embedded features are grouped into the same cluster. The cluster center is calculated for each cluster, and the cluster center is manually annotated to obtain the cluster center annotation for each cluster. (3) Label diffusion: After labeling the cluster center of each cluster, the cluster center label is selected as the label of all embedded features in the entire cluster. The labels formed in this way are called pseudo labels. (4) Edge detection: The boundaries and noise of each cluster are manually labeled, and the labeling results are merged into the dataset containing pseudo-labels; (5) Training: Through the above steps, a high-quality dataset containing pseudo-labels was obtained. Based on this dataset, a heterogeneous graph convolutional neural network was trained to generate a network for real-time root cause localization. (6) Location: Deploy the network to the online system, perform root cause location when a fault occurs, output the root cause probability of each node, and complete the fault location task.