Methods, systems, devices, processors, and media for microservice anomaly root cause localization based on multi-source data fusion and large language models.

Abstract

This invention relates to a method for microservice anomaly root cause localization based on multi-source data fusion and a large language model, comprising the following steps: parsing a set of anomaly cases input by the user, extracting indicator data, log data, and call chain data corresponding to the anomaly time period; performing multi-dimensional anomaly detection and data compression based on the anomaly time period to obtain indicator anomaly detection results, log anomaly detection results, and call chain anomaly detection results; constructing prompt words; performing reasoning based on preset scoring rules to output structured analysis results; validating the format of the structured analysis results, and generating a structured localization result file. The method, system, apparatus, processor, and computer-readable storage medium of this invention for microservice anomaly root cause localization based on multi-source data fusion and a large language model solve the problems of complex dependencies and heterogeneous data in microservice system fault localization, achieving high-precision, interpretable, and automated root cause analysis.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence operation and maintenance, and particularly to the field of root cause localization of microservice architecture faults. Specifically, it refers to a method, system, device, processor, and computer-readable storage medium for microservice anomaly root cause localization based on multi-source data fusion and large language model. Background Technology

[0002] The root cause of failures in a microservice architecture faces the following challenges: Complexity of fault propagation: Service calls form a directed graph, and anomalies spread through the call chain, creating a ripple effect. Traditional static graph analysis (such as MicroRCA) struggles to capture dynamic dependencies.

[0003] Data heterogeneity: Data from multiple sources such as metrics, logs, and call chains need to be integrated for analysis. Existing methods, such as patent CN114500209A, rely on a single data source and have insufficient recall.

[0004] Timeliness requirements: Industrial-grade systems need to locate the root cause within 3 minutes (such as Alibaba's microservice system), while traditional manual investigation takes more than 30 minutes.

[0005] Lack of interpretability: The output of graph embedding methods based on random walks (such as DyGNN) is not traceable, and the Top-1 accuracy is only 62%.

[0006] In existing technologies, Alibaba Cloud's patent (CN119496690A) accelerates localization through cluster center links, and AsiaInfo Technologies' patent (CN120034425A) uses fault impact graphs to trace root causes in reverse, but neither solves the problems of multi-source data fusion and inference interpretability. Furthermore, while the dynamic graph embedding method DGERCL introduces LSTM to capture temporal features, it does not combine with a large language model to improve semantic understanding capabilities. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system, device, processor and computer-readable storage medium for microservice anomaly root cause localization based on multi-source data fusion and large language model, which meets the requirements of high accuracy, interpretability and wide applicability.

[0008] To achieve the above objectives, the present invention provides a method, system, apparatus, processor, and computer-readable storage medium for microservice anomaly root cause localization based on multi-source data fusion and a large language model, as well as the following: The main feature of this microservice anomaly root cause localization method based on multi-source data fusion and large language model is that the method includes the following steps: (1) Parse the set of abnormal cases input by the user and extract the indicator data, log data and call chain data corresponding to the abnormal time period; (2) Based on the abnormal time period, multi-dimensional anomaly detection and data compression are performed on the indicator data, log data and call chain data in parallel to obtain the indicator anomaly detection results, log anomaly detection results and call chain anomaly detection results respectively; (3) Integrate system topology information, list of effective components, and results of abnormal indicator detection, abnormal log detection and abnormal call chain detection to construct prompt words; guide the large language model to perform reasoning based on preset scoring rules, and output structured analysis results containing root cause components, abnormal causes and reasoning chains; (4) Perform format verification on the structured analysis results and generate the final structured location result file.

[0009] Preferably, step (2) of the above-mentioned anomaly detection of indicator data specifically includes the following steps: (1-2.1) An integrated detection method is used, which combines the isolated forest algorithm, histogram outlier scoring algorithm and interquartile range algorithm. A weighted outlier score is generated for each time point, and global outliers are determined according to a preset threshold. (1-2.2) Identify local temporal anomaly patterns in the indicator data within the aforementioned abnormal time period; (1-2.3) If the proportion of abnormal points exceeds the preset threshold and the local time-series abnormality pattern exists, the indicator is determined to be abnormal.

[0010] Preferably, step (2) of performing anomaly detection on the log data specifically includes the following steps: (2-2.1) Abnormal log entries are identified using error keyword regular expression matching and the Drain3 log template parsing algorithm; (2-2.2) Determine whether the exception level is a Pod-level exception or a service-level exception based on the Pod range where the exception log entries are distributed; (2-2.3) Perform semantic similarity deduplication on the identified abnormal log entries.

[0011] Preferably, step (2) of performing anomaly detection on the call chain data specifically includes the following steps: (3-2.1) Based on error labels, high time consumption, and Pod distribution ratio, filter out abnormal call chains from the call chain data; (3-2.2) For the selected abnormal call chains, determine whether there is a sudden increase in errors based on Z-score; (3-2.3) Deduplicatize and merge the semantic information in the abnormal call chain.

[0012] Preferably, the step (3) of constructing prompt words specifically includes the following steps: (3.1) Describe the service topology and the call dependencies between services; (3.2) Perform abnormal input, including abnormal cases described in natural language, as well as the abnormal detection results of the aforementioned indicators, abnormal detection results of logs, and abnormal detection results of call chains; (3.3) Constrain the root cause components, which are matched with a predefined list of valid components and the reasoning chain is organized in chronological order.

[0013] Preferably, the preset scoring rules in step (3) specifically include the following steps: If a candidate component shows anomalies in multiple data sources, including metrics, logs, and call chains, then the first preset score is increased. If a candidate component is the callee in the call chain and its status code indicates an abnormality, then a second preset score is added. If the abnormal information associated with the candidate component contains preset high-risk keywords, then a third preset score will be added; If the candidate component is located at the very bottom of the service call topology, then a fourth preset score is added; If a candidate component experiences a restart event, the fifth preset score is increased.

[0014] Preferably, when multiple candidate components have the same total score, the root cause component is determined according to the following priority: components located further downstream in the dependency graph are given priority; components that occurred earlier are given priority; and components with more severe exception types are given priority.

[0015] This system for microservice anomaly root cause localization based on multi-source data fusion and a large language model is characterized by the following: The data input module is used to parse abnormal cases and extract abnormal time periods; An anomaly detection module, connected to the data input module, is used to perform multi-dimensional anomaly detection and data compression on indicator data, log data and call chain data in parallel based on the abnormal time period, and obtain indicator anomaly detection results, log anomaly detection results and call chain anomaly detection results respectively. The large model inference module, connected to the anomaly detection module, is used to integrate system topology information, a list of effective components, and the anomaly detection results of the indicators, logs, and call chains to construct prompt words; based on preset scoring rules, it guides the large language model to perform inference and outputs structured analysis results containing root cause components, anomaly causes, and inference chains. The results output module is connected to the large model inference module and is used to perform format verification on the structured analysis results and generate the final structured localization result file.

[0016] Preferably, the anomaly detection module includes: The indicator anomaly detection unit is connected to the data input module and the large model inference module, and is used to perform global anomaly scoring using an integrated algorithm and to identify local time-series anomaly patterns. The log anomaly detection unit, connected to the data input module and the large model inference module, is used to identify abnormal logs through regular expression matching and log template parsing, and to determine the anomaly level and perform semantic deduplication. The call chain anomaly detection unit is connected to the data input module and the large model inference module. It is used to filter abnormal call chains based on multi-dimensional features, detect sudden increases in errors, and compress semantic information.

[0017] Preferably, the large model inference module preloads system topology information and a list of valid components to constrain inference boundaries, verify the legality of output components, and provide a basis for scoring rules.

[0018] The device for microservice anomaly root cause localization based on multi-source data fusion and large language model is characterized in that the device includes: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the various steps of the above-described method for microservice anomaly root cause localization based on multi-source data fusion and a large language model.

[0019] The processor for microservice anomaly root cause localization based on multi-source data fusion and large language model is characterized in that the processor is configured to execute computer-executable instructions, and when the computer-executable instructions are executed by the processor, the various steps of the above-mentioned method for microservice anomaly root cause localization based on multi-source data fusion and large language model are implemented.

[0020] The computer-readable storage medium is characterized in that it stores a computer program that can be executed by a processor to implement the various steps of the above-described method for microservice anomaly root cause localization based on multi-source data fusion and a large language model.

[0021] The present invention employs a method, system, device, processor, and computer-readable storage medium for microservice anomaly root cause localization based on multi-source data fusion and a large language model, which solves the problems of complex dependencies and heterogeneous data in microservice system fault localization and achieves high-precision, interpretable, and automated root cause analysis. Attached Figure Description

[0022] Figure 1This is a flowchart of the microservice anomaly root cause localization method based on multi-source data fusion and large language model according to the present invention.

[0023] Figure 2 This is a schematic diagram of the system for microservice anomaly root cause localization based on multi-source data fusion and large language model according to the present invention. Detailed Implementation

[0024] To more clearly describe the technical content of the present invention, the following description is provided in conjunction with specific embodiments.

[0025] The present invention provides a method for microservice anomaly root cause localization based on multi-source data fusion and a large language model, comprising the following steps: (1) Parse the set of abnormal cases input by the user and extract the indicator data, log data and call chain data corresponding to the abnormal time period; (2) Based on the abnormal time period, multi-dimensional anomaly detection and data compression are performed on the indicator data, log data and call chain data in parallel to obtain the indicator anomaly detection results, log anomaly detection results and call chain anomaly detection results respectively; (3) Integrate system topology information, list of effective components, and results of abnormal indicator detection, abnormal log detection and abnormal call chain detection to construct prompt words; guide the large language model to perform reasoning based on preset scoring rules, and output structured analysis results containing root cause components, abnormal causes and reasoning chains; (4) Perform format verification on the structured analysis results and generate the final structured location result file.

[0026] In a preferred embodiment of the present invention, step (2) of detecting anomalies in the indicator data specifically includes the following steps: (1-2.1) An integrated detection method is used, which combines the isolated forest algorithm, histogram outlier scoring algorithm and interquartile range algorithm. A weighted outlier score is generated for each time point, and global outliers are determined according to a preset threshold. (1-2.2) Identify local temporal anomaly patterns in the indicator data within the aforementioned abnormal time period; (1-2.3) If the proportion of abnormal points exceeds the preset threshold and the local time-series abnormality pattern exists, the indicator is determined to be abnormal.

[0027] In a preferred embodiment of the present invention, step (2) of performing anomaly detection on log data specifically includes the following steps: (2-2.1) Abnormal log entries are identified using error keyword regular expression matching and the Drain3 log template parsing algorithm; (2-2.2) Determine whether the exception level is a Pod-level exception or a service-level exception based on the Pod range where the exception log entries are distributed; (2-2.3) Perform semantic similarity deduplication on the identified abnormal log entries.

[0028] In a preferred embodiment of the present invention, step (2) of performing anomaly detection on the call chain data specifically includes the following steps: (3-2.1) Based on error labels, high time consumption, and Pod distribution ratio, filter out abnormal call chains from the call chain data; (3-2.2) For the selected abnormal call chains, determine whether there is a sudden increase in errors based on Z-score; (3-2.3) Deduplicatize and merge the semantic information in the abnormal call chain.

[0029] In a preferred embodiment of the present invention, the step (3) of constructing prompt words specifically includes the following steps: (3.1) Describe the service topology and the call dependencies between services; (3.2) Perform abnormal input, including abnormal cases described in natural language, as well as the abnormal detection results of the aforementioned indicators, abnormal detection results of logs, and abnormal detection results of call chains; (3.3) Constrain the root cause components, which are matched with a predefined list of valid components and the reasoning chain is organized in chronological order.

[0030] As a preferred embodiment of the present invention, the preset scoring rules in step (3) specifically include the following steps: If a candidate component shows anomalies in multiple data sources, including metrics, logs, and call chains, then the first preset score is increased. If a candidate component is the callee in the call chain and its status code indicates an abnormality, then a second preset score is added. If the abnormal information associated with the candidate component contains preset high-risk keywords, then a third preset score will be added; If the candidate component is located at the very bottom of the service call topology, then a fourth preset score is added; If a candidate component experiences a restart event, the fifth preset score is increased.

[0031] In a preferred embodiment of the present invention, when multiple candidate components have the same total score, the root cause component is determined according to the following priority: the component located further downstream in the dependency graph is given priority; the component that occurred earlier is given priority; and the component with the more severe type of exception is given priority.

[0032] The present invention discloses a system for microservice anomaly root cause localization based on multi-source data fusion and a large language model, wherein the system comprises: The data input module is used to parse abnormal cases and extract abnormal time periods; An anomaly detection module, connected to the data input module, is used to perform multi-dimensional anomaly detection and data compression on indicator data, log data and call chain data in parallel based on the abnormal time period, and obtain indicator anomaly detection results, log anomaly detection results and call chain anomaly detection results respectively. The large model inference module, connected to the anomaly detection module, is used to integrate system topology information, a list of effective components, and the anomaly detection results of the indicators, logs, and call chains to construct prompt words; based on preset scoring rules, it guides the large language model to perform inference and outputs structured analysis results containing root cause components, anomaly causes, and inference chains. The results output module is connected to the large model inference module and is used to perform format verification on the structured analysis results and generate the final structured localization result file.

[0033] In a preferred embodiment of the present invention, the anomaly detection module includes: The indicator anomaly detection unit is connected to the data input module and the large model inference module, and is used to perform global anomaly scoring using an integrated algorithm and to identify local time-series anomaly patterns. The log anomaly detection unit, connected to the data input module and the large model inference module, is used to identify abnormal logs through regular expression matching and log template parsing, and to determine the anomaly level and perform semantic deduplication. The call chain anomaly detection unit is connected to the data input module and the large model inference module. It is used to filter abnormal call chains based on multi-dimensional features, detect sudden increases in errors, and compress semantic information.

[0034] In a preferred embodiment of the present invention, the large model inference module preloads system topology information and a list of valid components to constrain inference boundaries, verify the legality of output components, and provide a basis for scoring rules.

[0035] The device for microservice anomaly root cause localization based on multi-source data fusion and large language model of the present invention, wherein the device includes: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the various steps of the above-described method for microservice anomaly root cause localization based on multi-source data fusion and a large language model.

[0036] The processor of the present invention for microservice anomaly root cause localization based on multi-source data fusion and large language model is configured to execute computer-executable instructions, which, when executed by the processor, implement the various steps of the above-mentioned method for microservice anomaly root cause localization based on multi-source data fusion and large language model.

[0037] The computer-readable storage medium of the present invention stores a computer program that can be executed by a processor to implement the various steps of the above-described method for microservice anomaly root cause localization based on multi-source data fusion and large language model.

[0038] This invention relates to a microservice root cause localization method and system based on multi-source data fusion and large-model inference. The method includes: parsing abnormal events through a data input layer; fusing metrics, logs, and traces data using an anomaly detection layer, and combining Isolation Forest, HBOS, IQR algorithms, and log template parsing techniques for multi-dimensional anomaly detection; constructing prompts containing system topology, anomaly information, and inference rules through a large-model inference layer to guide the large language model to output root cause components, anomaly causes, and explainable inference chains; and finally generating structured localization results through a result output layer.

[0039] The microservice root cause localization method based on multi-source data fusion and large model inference of the present invention includes the following steps: (1) Parse the set of abnormal cases input by the user through the data input layer and extract the abnormal time period described in natural language; (2) In the anomaly detection layer: An integrated algorithm combining Isolation Forest, Histogram Outlier Score (HBOS), and Interquartile Range (IQR) was used to detect anomalies in the indicators. Filter call chain anomalies based on time consumption distribution, error labels, and Pod distribution ratio, and combine Z-score burst detection with Levenshtein distance semantic deduplication and compression data; Log anomalies are identified by matching error keywords using regular expressions and parsing Drain3 log templates.

[0040] (3) In the large model inference layer: The system integrates system architecture, service call relationships, and multi-source anomaly data to construct prompt words; Based on the scoring rules (multiple source evidence +2 points, callee in the call chain +2 points, high-risk keywords +1 point, downstream component +4 points, restart event +10 points), the large language model is guided to output the root cause component, abnormal cause and reasoning chain.

[0041] (4) Verify the format validity in the result output layer and generate a structured location result file.

[0042] The abnormality detection of indicators in step (2) specifically includes: Global detection module: Anomaly scoring is achieved by weighted integration of Isolation Forest, HBOS, and IQR algorithms, using the following formula: anomaly_score\=\0.4\astIF\+\0.3\astHBOS\+\0.3\astIQR Among them, astIF refers to, astHBOS refers to, and astIQR refers to.

[0043] Local detection module: identifies five types of time series patterns: sudden increase, sudden decrease, oscillation, etc., and determines an anomaly when the proportion of abnormal points exceeds the threshold.

[0044] The prompt word construction in step (3) includes: System description: Service topology, call dependencies (represented by "ComponentA calls ComponentB"); Abnormal input: Natural language description, anomaly detection results from metrics / logs / call chains; Output constraints: Root cause components must match a predefined list of valid components, and the inference chain is organized in chronological order.

[0045] The scoring rules for step (3) mentioned above should be selected first: Components that appear multiple times from multiple data sources; The callee (Callee) component in the call chain with a status code ≥ 500; Components containing high-risk keywords such as "timeout" and "connection refused"; The most downstream component in the service call topology; The component that experienced the restart event.

[0046] A microservice root cause localization system, comprising: Data input module: Analyzes abnormal cases and extracts time windows; Anomaly detection module: Execution metrics, logs, call chains, multi-dimensional anomaly detection and data compression; Large-scale model reasoning module: Constructs prompt words and calls the large language model to output root cause analysis results; Output module: Validates the result format and generates a structured file.

[0047] like Figure 1 As shown, this invention proposes a four-layer root cause localization scheme: 1. Data Input Layer: Based on alarm information, extract the corresponding metrics, logs, and call chain information for the abnormal time period; 2. Anomaly Detection Layer: Indicator detection: Integrates IF, HBOS, and IQR algorithms, combining global statistics with local temporal pattern recognition; Call chain detection: Filter anomalies based on error labels, high time consumption, and Pod distribution, and compress data through semantic deduplication; Log detection: Identifying abnormal logs using combined regular expression matching and Drain3 template parsing; 3. Large Model Inference Layer: The system topology (system_arch.json), the list of valid components (valid_components.json), and abnormal data are integrated. Design scoring rules to guide LLM reasoning and output root cause components, causes, and time sequence reasoning chains; 4. Output layer: Generates structured feedback results, consisting of three parts: root cause components, fault causes, and reasoning chains.

[0048] In a specific embodiment of the present invention, in a first aspect, the present invention provides a microservice anomaly root cause localization method based on multi-source data fusion and a large language model, which achieves accurate localization of system anomaly components and root causes through multi-stage collaborative processing. The specific implementation steps are as follows: (1) Data input layer. By accessing the alarm interface, the time period of the anomaly is obtained, and the corresponding indicators, logs and call chain information are matched.

[0049] (2) Anomaly Detection Layer. Based on the metrics, logs, and call chain information corresponding to the located abnormal time period, anomaly detection algorithms are applied to initially screen potentially abnormal record information. This layer includes three parts: metric anomaly detection, log anomaly detection, and call chain anomaly detection.

[0050] (2.1) The indicator anomaly detection module aims to obtain all indicator anomalies within the system during the detection period. This module performs parallel single-indicator anomaly detection on all indicators (apm, infrastructure, other, etc.) of objects at all levels of the system (pod, service, node, tidb, etc.), identifies potential fault symptoms, and provides structured anomaly information output. This module supports multi-granularity, multi-source, and multi-dimensional indicator monitoring, and combines global statistics with local time-series features to construct a highly robust indicator anomaly detection framework.

[0051] The indicator detection logic is mainly divided into two parts: a global detection module and a local detection module.

[0052] (2.1.1) Global Detection Module Global detection employs an ensemble detection algorithm for anomaly scoring and determination for each individual indicator sequence. Specifically, it integrates the following three mainstream unsupervised anomaly detection algorithms: a) Isolation Forest (IF) Isolation Forest is a fast outlier detection algorithm based on the concept of random forest. First proposed in 2008 by Professor Zhou Zhihua and others at Nanjing University, it boasts linear time complexity and high accuracy. Unlike other anomaly detection algorithms that characterize the degree of alienation between samples using quantitative metrics such as distance and density, Isolation Forest detects outliers by isolating sample points. Specifically, this algorithm utilizes a binary search tree structure called an isolation tree (iTree) to isolate samples. Due to the smaller number of outliers and their alienation from most samples, outliers are isolated earlier, meaning they are closer to the root node of the iTree, while normal values ​​are farther away. Furthermore, compared to traditional algorithms such as LOF and K-means, Isolation Forest exhibits better robustness to high-dimensional data. This algorithm is suitable for high-dimensional features, offers stable detection results, and does not require a standard distribution assumption.

[0053] b)Histogram-based Outlier Score (HBOS) HBOS is a fast anomaly detection algorithm based on histogram probability modeling [2]. It assumes that the features are independent of each other and constructs a histogram for each indicator, using the probability density of the indicator value to score. The univariate outlier score is defined as the reciprocal of the height of the box. The scoring model is as follows:

[0054] Rare values ​​correspond to low-probability intervals and are therefore marked as potential anomalies. HBOS is efficient in processing large-scale indicator data and performs well in global anomaly detection, but it cannot detect local outliers.

[0055] c) Interquartile Range (IQR) IQR is a classic anomaly detection method based on statistical distribution characteristics, commonly used to detect extreme values ​​or outliers in univariate data. Its core idea is that in a normal data distribution, most data points should fall within the quartile range; data points outside this range are likely outliers. IQR is defined as the difference between the upper quartile (Q3) and the lower quartile (Q1) of the data.

[0056] A data point is considered an outlier when it is either less than or greater than a given value. This method requires no model training, is computationally efficient, and is suitable for detecting stable monitoring metrics such as CPU utilization and latency. In this system, IQR is integrated with the Isolation Forest (IF) and Histogram-based Outlier Score (HBOS) algorithms as part of the global anomaly detection module. This integration helps identify faulty nodes and anomaly metrics from a statistical distribution perspective, improving the accuracy and robustness of anomaly detection.

[0057] The detection results from the three algorithms are integrated in a weighted manner to generate an anomaly score (anomaly_score) for each time point. The calculation formula is as follows:

[0058] The final exception detection logic is as follows:

[0059] Only points that simultaneously meet the global indicator judgment conditions will be retained as candidate anomalies.

[0060] (2.1.2) Local Detection Module To further enhance the temporal sensitivity of the detection, the local detection module introduces a mechanism for identifying time-series abrupt change patterns. This module focuses on whether the indicator exhibits a clear structural anomaly trend within abnormal time periods, such as: a) A sudden increase followed by a rapid decline b) A sudden increase and continued high level c) Rapid recovery after a sudden drop d) Sudden drop and sustained low level e) Violent fluctuations (sudden drops or increases) The system first calculates the proportion of outliers for the indicator within the detection period. If the proportion exceeds a preset threshold (e.g., 10%) and any of the above local anomaly patterns exist, the indicator can be determined to be abnormal.

[0061] Finally, the results are aggregated and summarized by object and indicator to form a result set consisting of a dictionary list of {abnormal object set, abnormal indicator set, and abnormal description}.

[0062] (2.2) The log anomaly detection module aims to identify abnormal log patterns of various services in the system within the detection period, and aggregate the results by pod dimension to form a dictionary list of {abnormal objects, abnormal log patterns, and the number of times the abnormal pattern occurs}, which serves as an important input for subsequent root cause analysis. The entire process mainly includes three stages: abnormal log identification, abnormal component determination, and abnormal log compression.

[0063] (2.2.1) Anomaly Log Identification This module combines two methods—error keyword regular expression matching and the Drain3 log parsing algorithm—to achieve accurate identification of abnormal logs during periods of failure. a) Regular expression matching method for incorrect keywords: By collecting typical error keywords (such as "error", "fail", "critical", "rejected", "thread pool starvation", etc.) from historical fault logs, a predefined regular expression pattern is constructed to match the log content during abnormal periods, so as to quickly identify significant error log entries.

[0064] b) Drain3 log parsing method: The Drain3 algorithm is used to convert unstructured logs into structured templates, and anomaly detection is performed based on template frequency and pattern changes. The specific process includes: i. Log cleaning: Remove timestamps, IP addresses, and other variable information, and replace numbers and hash values ​​with < Placeholder; ii. parse tree construction: the first level branch is based on the number of words in the log message, the middle level branch is based on words / tags at specific positions, and finally the log template is aggregated at the leaf nodes; iii. Anomaly detection: Within a specified time window, the frequency of each template is counted, and templates with sudden increases or decreases in frequency or newly appearing are detected to identify abnormal patterns.

[0065] (2.2.2) Identifying Abnormal Components Anomaly logs might only involve a subset of pods within a service, or they might affect the entire service. To differentiate between these two scenarios, the module first constructs a dictionary of services and pod lists based on normal logs, and then constructs a dictionary of services and anomalous pod lists based on the anomaly logs. The two dictionaries are then compared: a) If the list of abnormal pods does not cover all pods of the service, it is determined to be a pod-level abnormality; b) If it covers all pods, it is considered a service-level exception.

[0066] (2.2.3) Anomaly Log Compression To control the input size of the large language model and avoid token overflow due to excessive log volume, similarity deduplication is performed on the identified abnormal logs. A similarity matching algorithm is used to retain only log entries with similarity below a threshold, and to retain a representative record for content with high similarity (e.g., >95%), thereby maximizing the preservation of diverse abnormal information while compressing the data volume.

[0067] (2.3) The goal of the call chain anomaly detection module is to extract abnormal call paths from the call chain data to assist the large model in root cause localization. Considering that the original call chain data file is large in size and complex in structure, it is not suitable to directly input it into the large language model for analysis. Therefore, this module performs multi-stage preprocessing and anomaly screening before LLM calls to ensure that the input data is representative and has information density.

[0068] First, the call chain data preprocessing focuses on abnormal time periods, locating matching trace files and sorting the records by traceID and startTime. Then, basic information such as the node, container, and service is extracted from the process field of each call chain, laying the foundation for subsequent structured analysis.

[0069] Based on this, the system performs call chain anomaly localization from three dimensions: a) Error tag exception: Extract records from the call chain that contain tags.error=True or tags.http.status_code≥400 and mark them as exception call chains that may contain errors.

[0070] b) High time consumption exception: Calculate the time consumption distribution of all spans and select records whose time consumption exceeds the 95th percentile as performance bottleneck call chain exceptions.

[0071] c) Abnormal pod distribution ratio: Based on the distribution of each pod in each trace, significant statistical methods are used to identify abnormal deviations. If the frequency of a certain pod exceeds three standard deviations from the mean, it may indicate uneven service dependencies or abnormal call storms.

[0072] To further reduce the data size and enhance the diversity of exception information for the aforementioned set of anomalies, this module designs a call chain compression mechanism: a) Determine if there are sudden increases in error call chains. For traces with large error spans, use a recursive method based on Z-score to identify sudden error behaviors, thereby filtering subsequent rounds of errors based on sudden increases in errors and preventing certain call chains with fixed errors from entering the anomaly suspect set and interfering with the judgment of the subsequent large model.

[0073] (b) Introducing the Levenshtein distance algorithm to perform semantic deduplication on the tags.message field that exceeds the upper limit of the large model token. Messages with a similarity greater than 95% are merged to avoid wasting large model tokens due to semantic redundancy, thereby retaining more representative anomaly types.

[0074] (3) Large Model Inference Layer. This module is the core inference engine of the system. Relying on the understanding and reasoning capabilities of the Large Language Model (LLM), it automatically identifies the root cause components and anomalies of the microservice system with the support of multi-source abnormal data. The overall process follows "data construction → prompt generation → LLM inference → output cleaning and result extraction".

[0075] (3.1) Functional Objectives a) Receive and integrate anomaly detection results from the three sub-modules: metrics, logs, and traces; b) Generate a Prompt with contextual and reasoning guidance capabilities; c) Call LLM to complete causal inference based on anomaly information; d) Output structured JSON, specifying fields such as component, reason, reasoning_trace, and uuid.

[0076] (3.2) System architecture input dependencies To ensure clear inference boundaries and controllable results, the module loads two types of static structure information before running: system_arch.json contains system description, deployment information (number of main services, number of Pods per service, number of nodes, etc.), service call topology, and fault occurrence rules (each case has only one fault at a single node / service / pod level).

[0077] valid_components.json: As required by the question, the "component" field must be strictly consistent with the name in the annotation. To standardize the answers for large models, valid names for node / service / pod are provided.

[0078] These two types of constraints are used to limit the output space, verify the legality of component names, and provide a basis for rules such as "first-downstream component priority".

[0079] (3.3) Prompt construction logic a) System description information: system topology, deployment points, call dependencies (the path is shown by “componentA callscomponentB”), fault injection constraints and hierarchy examples.

[0080] b) Abnormal input information: Case natural language description and UUID, indicator anomaly detection results, log anomaly detection results, and call chain anomaly detection results.

[0081] c) Output format and rules: i. Output only JSON, including uuid / component / reason / reasoning_trace; ii.component must strictly match valid_components.json; iii. If indicator names / keywords appear in the reason field, they must be written exactly the same as those in the input (no abbreviations, no name changes); iv. Timing constraints: The reasoning trace must be organized in chronological order (timestamp) to avoid reversing causality; d) Scoring and selection strategies: Among multiple candidate components, a quantitative scoring system helps the LLM uniquely identify the most likely root cause component during output, avoiding multiple candidates or selecting the wrong target. The scoring mechanism prioritizes a comprehensive decision based on multi-source anomaly evidence, call chain position, anomaly type severity, and temporal causality.

[0082] ●+1 — Appears multiple times from multiple data sources Logic: If a component appears multiple times in different data sources (metrics, logs, traces), it is more likely to be the root cause.

[0083] Reason: A single data source may be an occasional anomaly, while mentioning multiple data sources simultaneously usually indicates a strong anomaly signal for that component.

[0084] Example: pod_cpu_usage error in metrics → Component A In the logs, "connection refused" indicates component A. →A adds 1 point ●+2—Target component of call chain exception Logic: In traces, if the component is the callee of the exception span and has a status code ≥500 or an exception such as a timeout, it is considered a more likely source of the exception.

[0085] Reason: Call chain exceptions usually originate from faults on the callee side, while upstream errors are mostly propagation effects.

[0086] Example: trace: checkoutservice→adservice (HTTP500) adservice is the callee → adservice +2 points ●+1—Abnormal keyword association Logic: If a component is accompanied by specific high-risk keywords in error_info or trace_info, extra points will be awarded.

[0087] Keywords: timeout, RPC error, I / O error, deadline exceeded, connection refused, etc.

[0088] Reason: These keywords usually directly reflect system availability issues.

[0089] Example: logs: paymentservice rpc timeout to currencyservice currencyservice appears in the context of an abnormal keyword → +1 point ●+4—The most downstream part of the service call topology diagram Logic: Among all candidate components, the component located at the lowest end of the service call topology graph receives bonus points.

[0090] Reason: The failure of downstream components can lead to the failure of the entire upstream link, which is a common failure propagation path in distributed systems.

[0091] Judgment method: Use the call dependency relationship in system_arch.json to build a directed graph, and find the downstream component in the call chain among the candidate components.

[0092] Example: frontend→checkoutservice→paymentservice→currencyservice If both currencyservice and paymentservice are candidates, currencyservice gets +4 points.

[0093] ●+10—Reboot error Logic: If error_info contains "restart", then add 10 points.

[0094] If multiple Pods of the same service restart, the overall service calculation will fail.

[0095] Reason: From the perspective of failure scenarios, service or Pod restart is a significant system anomaly event and is usually a direct manifestation of failure.

[0096] Example: logs: pod paymentservice-1 restarted→paymentservice+10 points At the same time, paymentservice-2 is also restarted → it will still only be added once (based on service count). ● Equal-split decision-making strategy If multiple components have the same total score, the decision will be made in the following order: Downstream priority: Components that are further downstream in the dependency graph are given priority.

[0097] Time priority: Components with earlier exception times take precedence.

[0098] Exception type priority: Restart > 5xx / timeout > Exception keywords > Frequency.

[0099] (3.4) Robustness and Design Advantages a) Multi-source integration and rule guidance: By unifying the Prompt injection system structure, service call topology, fault injection boundaries and scoring criteria, LLM is made "based on evidence".

[0100] b) Strong format output: Strict JSON constraints ensure that the results are parsable, comparable, and evaluable.

[0101] c) Component screening anti-illusion: Combining a list of legitimate components with a scoring mechanism such as "last-to-last priority" reduces the probability of selecting irrelevant / fictitious components.

[0102] d) Temporal causality: Organize the reasoning trace in chronological order, emphasizing "symptoms first, then spread", reducing the risk of treating secondary failures as the root cause.

[0103] e) Automatic retry and degradation: Set exponential backoff retries and timeout controls for LLM calls to improve stability under network fluctuations.

[0104] Results Output Layer. This module checks the validity of the LLM inference result format to ensure that the output results are fixed structured content (root cause components, fault causes, and inference chains) and do not change based on different input detection content, thereby ensuring the stability of data processing and utilization downstream of the algorithm.

[0105] Secondly, such as Figure 2As shown in the diagram, this specification provides a microservice root cause localization system based on multi-source data fusion and large model inference, comprising: a data input module for acquiring the time period in which the anomaly occurred and the corresponding metrics, logs, and call chain information; a metric anomaly detection module for acquiring all metric anomalies within the system during the detection time period; a log anomaly detection module for identifying abnormal log patterns of various services within the system during the detection time period; a call chain anomaly detection module for extracting abnormal call paths from the call chain data; an LLM root cause inference module for automatically identifying the root cause components and anomaly triggers of the microservice system based on multi-source anomaly data; and a result output module for outputting structured root cause localization results.

[0106] This invention also provides an integrated design for multi-source data anomaly detection. Indicator anomaly detection employs a weighted ensemble algorithm of "IF (0.4) + HBOS (0.3) + IQR (0.3)" combined with local temporal pattern recognition; log anomaly detection uses joint regular expression matching and Drain3 template parsing; and call chain anomaly detection combines multi-dimensional filtering, Z-score surge detection, and Levenshtein semantic deduplication. This solves the problems of insufficient detection accuracy of single algorithms and difficulty in uniformly processing heterogeneous multi-source data. Indicator detection considers both global statistics and local temporal features, while log / call chain detection balances accuracy and data compression efficiency. Existing technologies do not disclose such a multi-dimensional integrated detection scheme, significantly improving the recall and accuracy of anomaly detection.

[0107] This invention also provides a quantitative scoring guidance mechanism for large-scale model inference. By employing a reward mechanism, based on scoring rules of "multiple-source evidence +2 points, callee in the call chain +2 points, high-risk keywords +1 point, downstream component +4 points, and restart event +10 points," it guides LLM to focus on the core root cause. This overcomes the pain points of large-scale model inference being "unclearly guided and prone to illusions." By constraining the model's inference direction through quantitative rules and combining the characteristics of microservice fault propagation (downstream affects upstream, and multi-source evidence is more reliable), it significantly improves the accuracy of root cause localization.

[0108] This invention also provides a structured output design for interpretable reasoning chains.

[0109] The prompt requires the LLM to output the inference chain in chronological order, with the results in JSON format including "root cause component, fault cause, and inference chain" to ensure traceability. This solves the problem of "uninterpretable and difficult-to-verify results" in traditional root cause localization, meeting the traceability requirements of industrial-grade operation and maintenance scenarios. Existing technologies do not make "interpretable inference chain" a mandatory output requirement for large-scale model root cause localization.

[0110] This invention also provides an implementation method for extracting abnormal time period information from the data input layer. The data input layer extracts abnormal time period information through the following steps, which primarily rely on matching alarm interface data with a time window: (1) Alarm information access: receive alarm data through system alarm interfaces (such as PrometheusAlertmanager, enterprise-level operation and maintenance platform alarm API). The alarm data must include the core fields: alarm trigger time (trigger_time), alarm description (alert_description), related service / node (related_components), and alarm level (severity).

[0111] (2) Analysis of abnormal time periods: If the alarm data clearly indicates the start and end times of the anomaly (e.g., "2024-05-20 14:30:00-2024-05-20 14:45:00"), directly extract that time period as the abnormal time period.

[0112] If only the alarm trigger time is provided (e.g., "2024-05-20 14:35:00"), the time window will be adaptively expanded based on the alarm type: performance alarms (e.g., response timeout) will be expanded to "30 minutes before to 15 minutes after the trigger time", error alarms (e.g., service outage) will be expanded to "10 minutes before to 30 minutes after the trigger time", and the default window will be "1 hour before to 30 minutes after the trigger time".

[0113] (3) Standardized output of time period: Convert the finally determined abnormal time period into a standard format (such as "YYYY-MM-DDHH:MM:SS-YYYY-MM-DDHH:MM:SS") and synchronize it to the anomaly detection layer for filtering multi-source data within the corresponding time range.

[0114] This invention also provides an implementation method for constructing a Prompt based on anomaly detection results from metrics / logs / traces.

[0115] The Prompt build follows a three-tiered logic of "system constraints + abnormal data integration + inference guidance," and the specific implementation steps are as follows: (1) Basic information and system constraint injection System architecture description: Load service topology and call relationships from system_arch.json, specifying dependency paths in a dictionary-level structure; load the list of valid components from valid_components.json, constraining the scope of output components.

[0116] Fault rule declaration: Clearly define constraints such as "each case has only a single level of fault (node / service / pod)" and "inference chain is ordered chronologically".

[0117] (2) Structured integration of multi-source anomaly data Metrics anomaly data: Extract {anomaly object (pod / node / service), anomaly metric (such as CPU utilization), anomaly score, anomaly pattern (such as sudden increase), and occurrence time} from the anomaly detection results, and organize them into a list by "object grouping + time sorting".

[0118] Logs abnormal data: Extract {abnormal object, abnormal log template, error keyword, occurrence count, occurrence time}, merge log entries after deduplication based on similarity, and retain high-frequency and high-risk logs.

[0119] Traces exception data: Extract {exception call chain ID, involved components, exception type (error label / high time consumption / pod distribution exception), time consumption / status code, occurrence time}, and retain the core exception call chain after semantic deduplication.

[0120] (3) Injection of reasoning guidance rules and scoring mechanisms Clearly define the scoring rules: List the weighted logic for each item, such as "multiple source evidence +2 points, call chain callee +2 points, high-risk keywords +1 point, downstream component +4 points, and restart event +10 points".

[0121] Output format constraints: Specify JSON output fields (uuid, component, reason, reasoning_trace), requiring reasoning_trace to include "abnormal phenomenon → propagation path → root cause determination basis" in chronological order.

[0122] This invention also provides a specific algorithm for deduplication of abnormal log similarity.

[0123] The deduplication of abnormal logs uses an improved Levenshtein distance algorithm, optimized by combining log text characteristics, as detailed below: (1) Algorithm selection criteria: Levenshtein distance (edit distance) can quantify the difference between two strings (the minimum number of insertion, deletion and replacement operations), which is suitable for semantic similarity determination of unstructured log text, and has high computational efficiency, making it suitable for large-scale log processing.

[0124] (2) Similarity calculation and deduplication logic: Calculate the Levenshtein distance D between the two logs after preprocessing, with the maximum log length being L, and the similarity S = (LD) / L × 100%.

[0125] Set a similarity threshold (default 95%). When S≥95%, the two log entries are considered semantically redundant. The log entry that appears more often or earlier is retained, and the rest are deleted.

[0126] (3) Efficiency optimization: The “grouping and clustering + batch calculation” mode is adopted. First, the logs are grouped according to the component (pod / service) to which they belong, and then similarity calculation is performed within the group to avoid cross-component log comparison and improve processing speed.

[0127] For the specific implementation scheme of this embodiment, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0128] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0129] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0130] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0131] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0132] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The corresponding program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0133] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0134] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0135] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0136] The present invention employs a method, system, device, processor, and computer-readable storage medium for microservice anomaly root cause localization based on multi-source data fusion and a large language model, which solves the problems of complex dependencies and heterogeneous data in microservice system fault localization and achieves high-precision, interpretable, and automated root cause analysis.

[0137] In this specification, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and variations can be made without departing from the spirit and scope of the invention. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A method for microservice exception root cause positioning based on multi-source data fusion and large language model, characterized in that, The method includes the following steps: (1) Parse the set of abnormal cases input by the user and extract the indicator data, log data and call chain data corresponding to the abnormal time period; (2) Based on the abnormal time period, multi-dimensional anomaly detection and data compression are performed on the indicator data, log data and call chain data in parallel to obtain the indicator anomaly detection results, log anomaly detection results and call chain anomaly detection results respectively; (3) Integrate system topology information, list of effective components, and results of abnormal indicator detection, abnormal log detection and abnormal call chain detection to construct prompt words; guide the large language model to perform reasoning based on preset scoring rules, and output structured analysis results containing root cause components, abnormal causes and reasoning chains; (4) Perform format verification on the structured analysis results and generate the final structured location result file.

2. The method for microservice exception root cause location based on multi-source data fusion and large language model according to claim 1, characterized in that, Step (2) involves anomaly detection of the indicator data, specifically including the following steps: (1-2.1) An integrated detection method is used, which combines the isolated forest algorithm, histogram outlier scoring algorithm and interquartile range algorithm. A weighted outlier score is generated for each time point, and global outliers are determined according to a preset threshold. (1-2.2) Identify local temporal anomaly patterns in the indicator data within the aforementioned abnormal time period; (1-2.3) If the proportion of abnormal points exceeds the preset threshold and the local time-series abnormality pattern exists, the indicator is determined to be abnormal.

3. The method of claim 1, wherein the method is based on multi-source data fusion and large language model. Step (2) of the above-mentioned anomaly detection of log data specifically includes the following steps: (2-2.1) Abnormal log entries are identified using error keyword regular expression matching and the Drain3 log template parsing algorithm; (2-2.2) Determine whether the exception level is a Pod-level exception or a service-level exception based on the Pod range where the exception log entries are distributed; (2-2.3) Perform semantic similarity deduplication on the identified abnormal log entries.

4. The method for microservice exception root cause location based on multi-source data fusion and large language model according to claim 1, characterized in that, Step (2) involves anomaly detection of the call chain data, specifically including the following steps: (3-2.1) Based on error labels, high time consumption, and Pod distribution ratio, filter out abnormal call chains from the call chain data; (3-2.2) For the selected abnormal call chains, determine whether there is a sudden increase in errors based on Z-score; (3-2.3) Deduplicatize and merge the semantic information in the abnormal call chain.

5. The method for microservice exception root cause location based on multi-source data fusion and large language model according to claim 1, characterized in that, The construction of prompt words in step (3) specifically includes the following steps: (3.1) Describe the service topology and the call dependencies between services; (3.2) Perform abnormal input, including abnormal cases described in natural language, as well as the abnormal detection results of the aforementioned indicators, abnormal detection results of logs, and abnormal detection results of call chains; (3.3) Constrain the root cause components, which are matched with a predefined list of valid components and the reasoning chain is organized in chronological order.

6. The method for microservice exception root cause location based on multi-source data fusion and large language model according to claim 1, characterized in that, The preset scoring rules in step (3) specifically include the following steps: If a candidate component shows anomalies in multiple data sources, including metrics, logs, and call chains, then the first preset score is increased. If a candidate component is the callee in the call chain and its status code indicates an abnormality, then a second preset score is added. If the abnormal information associated with the candidate component contains preset high-risk keywords, then a third preset score will be added; If the candidate component is located at the very bottom of the service call topology, then a fourth preset score is added; If a candidate component experiences a restart event, the fifth preset score is increased.

7. The method for microservice exception root cause location based on multi-source data fusion and large language model according to claim 6, characterized in that, When multiple candidate components have the same total score, the root cause component is determined according to the following priority: the component located further downstream in the dependency graph takes priority; the component that occurred earlier takes priority; and the component with the more severe exception type takes priority.

8. A system for multi-source data fusion and large language model-based microservice exception root cause positioning, which implements the method of any one of claims 1 to 7, characterized in that, The system includes: The data input module is used to parse abnormal cases and extract abnormal time periods; An anomaly detection module, connected to the data input module, is used to perform multi-dimensional anomaly detection and data compression on indicator data, log data and call chain data in parallel based on the abnormal time period, and obtain indicator anomaly detection results, log anomaly detection results and call chain anomaly detection results respectively. The large model inference module, connected to the anomaly detection module, is used to integrate system topology information, a list of effective components, and the anomaly detection results of the indicators, logs, and call chains to construct prompt words; based on preset scoring rules, it guides the large language model to perform inference and outputs structured analysis results containing root cause components, anomaly causes, and inference chains. The results output module is connected to the large model inference module and is used to perform format verification on the structured analysis results and generate the final structured localization result file.

9. The system for microservice anomaly root cause localization based on multi-source data fusion and large language model according to claim 8, characterized in that, The anomaly detection module includes: The indicator anomaly detection unit is connected to the data input module and the large model inference module, and is used to perform global anomaly scoring using an integrated algorithm and to identify local time-series anomaly patterns. The log anomaly detection unit, connected to the data input module and the large model inference module, is used to identify abnormal logs through regular expression matching and log template parsing, and to determine the anomaly level and perform semantic deduplication. The call chain anomaly detection unit is connected to the data input module and the large model inference module. It is used to filter abnormal call chains based on multi-dimensional features, detect sudden increases in errors, and compress semantic information.

10. The system for microservice anomaly root cause localization based on multi-source data fusion and large language model according to claim 8, characterized in that, The large model inference module preloads system topology information and a list of valid components to constrain inference boundaries, verify the legality of output components, and provide a basis for scoring rules.

11. A device for microservice anomaly root cause localization based on multi-source data fusion and a large language model, characterized in that, The device includes: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method for microservice anomaly root cause localization based on multi-source data fusion and large language model as described in any one of claims 1 to 7.

12. A processor for microservice anomaly root cause localization based on multi-source data fusion and a large language model, characterized in that, The processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the method for microservice anomaly root cause localization based on multi-source data fusion and large language model as described in any one of claims 1 to 7.

13. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method for microservice anomaly root cause localization based on multi-source data fusion and large language model as described in any one of claims 1 to 7.

Images