Distributed system call chain interactive analysis method based on large language model
By constructing an interactive analysis method for distributed system call chains based on a large language model, the problem of the lack of interactivity and user-friendliness in call chain analysis methods in microservice systems is solved. This enables efficient fault root cause localization and system behavior understanding, and improves the accuracy and generalization ability of the analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FUDAN UNIVERSITY
- Filing Date
- 2026-01-27
- Publication Date
- 2026-05-12
AI Technical Summary
Existing call chain analysis methods in microservice systems lack interactive and user-friendly solutions, making it difficult to effectively assist operations and maintenance personnel in locating root causes of faults and understanding system behavior.
We construct an interactive analysis method for distributed system call chains based on a large language model. By using call chain to text representation and call chain to variable representation strategies, we convert call chain data into a processable representation. We design a unified analysis model and use graph neural networks and multilayer perceptrons for model training to generate analysis results for fault root cause localization and system behavior understanding.
It improves the accuracy and generalization ability of call chain analysis, enhances the analysis effect of existing large models, and shows good generalization ability, especially in unseen call chain analysis tasks and microservice systems, outperforming the existing powerful closed-source model GPT-4o.
Smart Images
Figure CN122018917A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of software engineering technology, specifically relating to an interactive analysis method for distributed system call chains based on a large language model. Background Technology
[0002] In recent years, distributed tracing technology has become a key component of the infrastructure of industrial microservice systems. The call chain generated during system operation records the complete execution flow of a business request across multiple service instances, and records rich attribute information involved in each service call, such as response status codes, execution latency, and HTTP methods. Therefore, call chains are widely used in critical tasks such as understanding the runtime behavior of microservice systems, anomaly detection, and root cause analysis.
[0003] Visual analytics is one of the mainstream methods for call chain analysis in current microservice systems. These methods typically first automatically execute one or more predefined call chain analysis tasks, such as end-to-end response time calculation, service call path identification, and call chain aggregation analysis. Then, they display the analysis results through dashboards or other visual formats, thereby assisting operations and maintenance personnel in carrying out downstream tasks such as anomaly detection and root cause localization based on practical experience. Summary of the Invention
[0004] The purpose of this invention is to provide a user-friendly, interactive call chain analysis method that supports various distributed system call chain analysis tasks, in order to assist operation and maintenance personnel in their work.
[0005] This invention provides an interactive analysis method for distributed system call chains based on a large language model. It constructs a unified analysis model for various call chain analysis tasks based on call chain data generated during the operation of a distributed system. This model assists operations and maintenance personnel in call chain analysis tasks such as fault root cause localization and system behavior understanding. A benchmark dataset in the form of an "instruction-response" model is constructed by collecting call chain data from a medium-sized open-source benchmark microservice system. This dataset contains 38 different call chain analysis tasks, involving various analysis granularities and objectives (see Table 1). Based on this dataset, supervised fine-tuning is performed on models based on call chain-to-text representation and models based on call chain-to-quantity representation to enhance the capabilities of the large model in call chain analysis tasks.
[0006] This invention mainly comprises three parts: call chain representation generation, call chain analysis model design, and call chain analysis model training. The specific steps are as follows:
[0007] (a) Generation of call chain representation
[0008] Two call chain representation strategies are designed: a call chain to text representation strategy and a call chain to vector representation strategy. These strategies are used to transform structured call chain data into a representation that large models can effectively understand and process. The call chain to text representation strategy aims to convert the call chain into a structured text representation based on a set of predefined grammatical rules, including node sequence representation, adjacency list-enhanced node sequence representation, edge list-enhanced node sequence representation, and class code representation. The call chain to vector representation strategy aims to encode the call chain into a dense vector representation using a graph neural network. Specifically, the strategy includes the following sub-steps:
[0009] (A) For the call chain to text representation strategy, the following four methods are used to convert the structured call chain into a text sequence representation:
[0010] (1) Node sequence representation: Each Span is represented as a dictionary consisting of "attribute name-attribute value", and a call chain is represented as a sequence of multiple Span dictionaries. Based on this representation, the large model needs to reconstruct the call relationship in the call chain according to the spanId and parentSpanId information of each Span.
[0011] (2) Adjacency list-enhanced node sequence representation: The adjacency list is used to explicitly enumerate the sub-spans of each span, representing the call relationship within the call chain in a compact and explicit manner. This method represents the call chain as a combination of node sequence and adjacency list.
[0012] (3) Edge table-enhanced node sequence representation: This method explicitly represents all call relationships in the call chain through an edge table. A call chain is represented as a combination of node sequence and edge table.
[0013] (4) Class code representation: This method converts the call chain into a highly structured XML representation.
[0014] (B) For the call chain to vector representation strategy, the call chain is encoded into a dense vector representation using a graph neural network, which mainly includes the following three steps:
[0015] (1) Call chain graph construction: Given a call chain t, this method first constructs the corresponding Span attribute graph. In this context, a node is represented by a text string formed by concatenating the attribute name and attribute value of a Span, and an edge represents the calling relationship between two Spans.
[0016] (2) Node representation initialization: For For each node in the algorithm, the method first uses the tokenizer of a pre-trained large model to encode the corresponding text string into a token sequence, and then maps these tokens into a continuous vector space to obtain a latent space representation. Since the text lengths of different spans are inconsistent, the method performs mean pooling on the latent space representations of all tokens within a node to generate a fixed-length vector representation as the initial representation of that node.
[0017] (3) Call the chain vector generation: after obtaining the initial representation of the node. and call chain graph After defining the caller-caller relationship A in the code, the vector representation of the call chain is calculated according to the following formula. :
[0018]
[0019] in, This represents the dimension of the latent space vector in a graph neural network. Used to generate vector representations of each node in the call chain graph; The vector representation of the call chain is obtained by pooling the vector representations of all nodes.
[0020] (ii) Design of call chain analysis model, including model design based on call chain to text representation and model design based on call chain to variable representation;
[0021] (A) For a model based on call chain to text representation, given a call chain (or a pair of call chains) and its corresponding instructions, the model first converts the structured call chain into a text representation based on predefined syntax rules; then, the generated call chain text is concatenated with the instructions and input into the large model; finally, the large model outputs the analysis results, including the step-by-step reasoning process and the final answer.
[0022] (B) For models based on call chain tovariant representations, given a call chain (or a pair of call chains) and its corresponding instructions, the model first generates a vector representation for the call chain, then further generates a vector representation for the corresponding text instructions, and finally concatenates the call chain vector representation with the instruction vector representation. The concatenated vector is then input into the larger model to generate a response, including the inference path and the final answer. Specifically, this includes the following sub-steps:
[0023] (1) Call chain processing: The model uses the call chain redirection representation method introduced in step (I) to generate the call chain vector. The vector is then mapped to the vector space of the larger model through a projection layer (i.e., a multilayer perceptron, or MLP). The resulting call chain vector representation... Calculated according to the formula:
[0024]
[0025] in, This represents the dimension of the latent space vector in a large model.
[0026] (2) Instruction processing: For each instruction The model uses a frozen large model word segmenter. The word segmentation process is as follows:
[0027]
[0028] Convert the instructions into discrete token sequences from the large model vocabulary. These tokens are then mapped to a continuous vector space:
[0029]
[0030] Where L represents the length of the token sequence; This represents the dimension of the latent space vector in a large model.
[0031] (3) Concatenation of call chain and instruction: Model concatenation of call chain vector representation With instruction vector representation The concatenated vector is then input into a larger model to generate a response, including the reasoning path and the final answer.
[0032] (III) Call Chain Analysis Model Training
[0033] In this invention, the model based on call chain to text representation and the model based on call chain to variable representation employ the same model training method. First, 70% of the instruction-response pairs are randomly selected from the benchmark dataset for each call chain analysis task to construct the training set, and the remaining 30% is used to construct the test set. Then, instruction fine-tuning training is performed on both the model based on call chain to text representation and the model based on call chain to variable representation based on the training set. Specifically, inference paths and final answers are used as supervision signals, and supervised fine-tuning is performed based on cross-entropy loss. For a given call chain or call chain pair and its corresponding instruction, the supervised fine-tuning loss measures the difference between the inference path and final answer predicted by the model and the true answer provided in the benchmark dataset. The supervised fine-tuning loss is defined as follows:
[0034]
[0035] in, Indicates the generated reasoning path; This represents the final answer generated; Indicates the input call chain or call chain pair; Indicates the input command; This represents the number of instruction-response pairs in the training dataset; Indicating targeting - The number of reasoning paths; These are the model parameters.
[0036] The main technical features and functional advantages of this invention are as follows:
[0037] (1) This invention is the first to construct a benchmark dataset for call chain analysis, which is used to evaluate and improve the call chain analysis capabilities of large models;
[0038] (2) This invention proposes an interactive analysis method for distributed system call chains based on a large language model. This invention constructs a unified analysis model for various call chain analysis tasks, which can handle various types of call chain analysis tasks. The analysis effect is better than the existing powerful closed-source model GPT-4o, and it shows a certain generalization ability in unseen call chain analysis tasks and unseen microservice systems. Attached Figure Description
[0039] Figure 1 This is a schematic diagram of the interactive analysis method for distributed system call chains based on a large language model, as described in this invention.
[0040] Figure 2 This is a schematic diagram illustrating the link-to-text representation in step one of this invention. (a) is a node sequence representation, (b) is a node sequence representation enhanced by an adjacency list, (c) is a node sequence representation enhanced by an edge list, and (d) is a class code representation. Detailed Implementation
[0041] This paper further introduces the fault root cause-assisted localization method based on multi-task interactive analysis of call chains, using OpenTelemetry as the specification for distributed call chains, PyTorch as the deep learning framework, and LLaMA-Factory as the large model fine-tuning framework.
[0042] (a) For fine-tuning the call chain analysis model, the specific process is as follows:
[0043] (1) Collect call chain data. Configure an OpenTelemetry agent for each program in the distributed system and set call chain collection rules. Collect call chains generated during normal system operation and during fault injection, and enhance the call chain dataset based on predefined rules.
[0044] (2) Collect call chain analysis tasks. Collect call chain analysis tasks from existing observability tools and existing research.
[0045] (3) Generate an instruction-response dataset. First, design an instruction template and a response template for each call chain analysis task. Then, for each call chain analysis task, randomly sample a set of call chain samples from the call chain dataset and generate several different instruction-response pairs for the task based on these samples.
[0046] (4) Training the call chain analysis model. The PyTorch and LLaMA-Factory frameworks were used to implement models based on call chain-to-text representation and call chain-to-quantum representation. Based on the dataset above, low-rank adaptation (LoRA) was used for fine-tuning, with the learning rate and number of training epochs set to 1e-5 and 3, respectively. The number of GAT layers, the dimension of the latent space vector, and the pooling strategy were set to 2, 1024, and summative pooling, respectively.
[0047] (II) The specific process for online call chain analysis is as follows:
[0048] (1) Trigger the call chain analysis process. When operations and maintenance personnel collect abnormal or critical call chain data, they can actively trigger the call chain analysis process.
[0049] (2) Call chain analysis process based on the call chain to text representation model. Given a call chain (or a pair of call chains) and its corresponding instructions, the model first converts the structured call chain into a text representation based on predefined syntax rules; then, the generated call chain text is concatenated with the instructions and input together into the large model; finally, the large model outputs the analysis results, including the step-by-step reasoning process and the final answer.
[0050] (3) Call chain analysis process based on the call chain to vector representation model. Given a call chain (or a pair of call chains) and its corresponding instructions, the model first generates a vector representation for the call chain; then it further generates a vector representation for the corresponding text instructions; finally, it concatenates the call chain vector representation and the instruction vector representation; and inputs the concatenated vector into the large model to generate a response, including the reasoning path and the final answer.
[0051] This invention conducts experimental evaluations based on an open-source benchmark microservice system of medium scale (TrainTicket) and small scale (OnlineBoutique); and constructs multiple benchmark methods for comparative analysis. Specifically, it selects the closed-source model GPT-4o and three open-source models Qwen-3-8B, Llama-3-8B and DeepSeek-R1-Distill-Llama-8B, and enhances their call chain analysis capabilities through prompt word engineering techniques (zero-sample prompt words and few-sample prompt words).
[0052] On one hand, the effectiveness of this invention in call chain analysis tasks was evaluated, specifically by assessing the accuracy metrics of the model based on call chain-to-text representation and the model based on call chain-to-quantity representation, as well as the aforementioned benchmark methods on the test set. Experimental results show that the two variants of this invention achieve average accuracy improvements of 51.34% and 18.20% respectively compared to existing open-source large models, and in the best case, still achieve a performance improvement of 21.66% compared to the closed-source large model GPT-4o.
[0053] On the other hand, the generalization ability of this invention on unseen call chain analysis tasks and unseen microservice systems was evaluated. Specifically, four call chain analysis tasks (identifying caller service instances, comparing Span wait times, identifying the most frequently accessed database, and checking the consistency of service instances in the call chain) were selected from the training set as unseen tasks and then removed from the training set, while keeping the test set unchanged. Finally, model training and testing were performed based on this training and test sets to evaluate the generalization ability of this invention on unseen call chain analysis tasks. A test set was constructed based on a small-scale open-source benchmark microservice system (OnlineBoutique) to evaluate the generalization ability of this invention on unseen microservice systems. Experimental results show that this invention exhibits good generalization ability on both unseen tasks and unseen systems, indicating that the method has the potential to be applied to other operation and maintenance scenarios.
[0054] Table 1 shows the 38 call chain analysis tasks and their classifications included in the benchmark dataset of this invention.
[0055] .
Claims
1. An interactive analysis method for distributed system call chains based on a large language model, characterized in that, Based on the call chain data generated during the operation of a distributed system, a unified analysis model is constructed for various call chain analysis tasks to assist operation and maintenance personnel in call chain analysis tasks such as fault root cause localization and system behavior understanding. A benchmark dataset in the form of "instruction-response" was constructed by collecting call chain data from a medium-sized open-source benchmark microservice system. This dataset contains 38 different call chain analysis tasks, involving four levels of analysis granularity and eight analysis objectives. Supervised fine-tuning was performed on models based on call chain-to-text representation and models based on call chain-to-quantity representation based on this dataset to enhance the capabilities of large models in call chain analysis tasks. The specific steps are as follows: (a) Generation of call chain representation; Two call chain representation strategies are designed: a call chain to text representation strategy and a call chain to vector representation strategy. These strategies are used to convert structured call chain data into a representation that large models can effectively understand and process. The call chain to text representation strategy aims to convert the call chain into a structured text representation based on a set of predefined grammar rules, including node sequence representation, adjacency list-enhanced node sequence representation, edge list-enhanced node sequence representation, and class code representation. The call chain to vector representation strategy aims to encode the call chain into a dense vector representation using a graph neural network. (ii) Design of call chain analysis model, including model design based on call chain to text representation and model design based on call chain to variable representation; For models based on call chain to text representation, given a call chain or a pair of call chains and their corresponding instructions, the model first converts the structured call chain into a text representation based on predefined syntax rules; then, the generated call chain text is concatenated with the instructions and input together into the larger model; finally, the larger model outputs the analysis results, including the step-by-step reasoning process and the final answer. For a model based on call chain torrent representation, given a call chain or a pair of call chains and their corresponding instructions, the model first generates a vector representation for the call chain, then generates a vector representation for the corresponding text instructions, and finally concatenates the call chain vector representation and the instruction vector representation. The concatenated vector is then input into the large model to generate a response, including the inference path and the final answer. (iii) Call chain analysis model training; Simultaneously train instruction fine-tuning on multiple call chain analysis tasks in the benchmark dataset; specifically, use inference paths and final answers as supervision signals, and perform supervised fine-tuning based on cross-entropy loss; for a given call chain or call chain pair and its corresponding instructions, the supervised fine-tuning loss measures the difference between the inference path and final answer predicted by the model and the true answer provided in the benchmark dataset.
2. The distributed system call chain multi-task interactive analysis method according to claim 1, characterized in that, In step (one): (A) For the call chain to text representation strategy, the structured call chain is converted into a text sequence representation using the following four methods: (1) Node sequence representation: Each Span is represented as a dictionary consisting of "attribute name-attribute value", and a call chain is represented as a sequence of multiple Span dictionaries; based on this representation, the large model reconstructs the call relationship in the call chain according to the spanId and parentSpanId information of each Span. (2) Adjacency list-enhanced node sequence representation: The adjacency list is used to explicitly enumerate the sub-spans of each span, representing the call relationship within the call chain in a compact and explicit manner, thereby representing the call chain as a combination of node sequence and adjacency list; (3) Edge table-enhanced node sequence representation: All call relationships in the call chain are explicitly represented by edge tables. A call chain is represented as a combination of node sequence and edge table. (4) Class code representation: Convert the call chain into a highly structured XML representation; (B) For the call chain to vector representation strategy, a graph neural network is used to encode the call chain into a dense vector representation, specifically including: (1) Call chain graph construction: Given a call chain t, first construct the corresponding Span attribute graph. ; where a node is represented by a text string formed by concatenating the attribute name and attribute value of a Span, and an edge represents the calling relationship between two Spans; (2) Node representation initialization: For For each node in the process, the corresponding text string is first encoded into a token sequence using the tokenizer of the pre-trained large model, and these tokens are mapped into a continuous vector space to obtain the latent space representation. Since the text lengths of different spans are inconsistent, the latent space representations of all tokens in the node are averaged and pooled to generate a fixed-length vector representation as the initial representation of the node. (3) Call the chain vector generation: after obtaining the initial representation of the node. and call chain graph After defining the caller-caller relationship A in the code, the vector representation of the call chain is calculated according to the following formula. : in, This represents the dimension of the latent space vector in a graph neural network. Used to generate vector representations of each node in the call chain graph; The vector representation of the call chain is obtained by pooling the vector representations of all nodes.
3. The distributed system call chain multi-task interactive analysis method according to claim 2, characterized in that, Step (II) involves constructing a model based on call chain redirection representation, which specifically includes the following sub-steps: (1) Call chain processing: The model uses the call chain redirection representation method described in step (I) to generate the call chain vector. The vector is then mapped to the vector space of the larger model through a projection layer, namely a multilayer perceptron (MLP); the call chain vector representation after projection... Calculated according to the formula: in, Represents the dimension of the latent space vectors of a large model; (2) Instruction processing: For each instruction The model uses a frozen large model word segmenter. The word segmentation process is as follows: Convert the instructions into discrete token sequences from the large model vocabulary. These tokens are then mapped to a continuous vector space: Where L represents the length of the token sequence; Represents the dimension of the latent space vectors of a large model; (3) Concatenation of call chain and instruction: Model concatenation of call chain vector representation With instruction vector representation The concatenated vector is then input into a larger model to generate a response, including the reasoning path and the final answer.
4. The distributed system call chain multi-task interactive analysis method according to claim 3, characterized in that, The definition of the supervised fine-tuning loss mentioned in step (iii) is as follows: in, Indicates the generated reasoning path; This represents the final answer generated; Indicates the input call chain or call chain pair; Indicates the input command; This represents the number of instruction-response pairs in the training dataset; Indicating targeting - The number of reasoning paths; These are the model parameters.