A multi-agent system fault attribution method and system

Abstract

The application discloses a kind of multi-agent system fault attribution method and system.The steps include: step 1: obtaining the running state data of multi-agent system execution trajectory;Step 2: extract semantic embedding vector and structure coding vector for each node, form double-view node representation;Step 3: the double-view node representation is input into the variational autoencoder encoder based on tree-shaped long short-term memory network, and the hidden variable representation of each node is obtained;Step 4: the hidden variable representation of each node is reconstructed by the decoder of variational autoencoder;Step 5: based on the degree of deviation, form candidate suspicious sub-trajectory;Step 6: the candidate suspicious sub-trajectory is submitted to downstream large language model attributor for fault root cause positioning.The application effectively alleviates the context dilution problem of large language model when processing long trajectory, significantly improves the accuracy and efficiency of fault attribution.

Description

Technical Field

[0001] This invention relates to the field of fault diagnosis and attribution in large language model multi-agent systems, and particularly to a fault attribution method and system for large language model multi-agent systems based on a dual-view variational autoencoder. Background Technology

[0002] With the rapid development of Large Language Model (LLM) technology, multi-agent systems (MAS) based on LLM have been widely applied to the automation of complex tasks, such as programming development, scientific discovery, and information retrieval. These systems demonstrate strong potential for solving complex multi-step reasoning tasks through collaborative reasoning, planning, and interaction with the environment among multiple agents. However, modern agent systems are increasingly evolving towards long-term tasks, and their execution processes typically exhibit complex, highly coupled hierarchical operation tree structures. This multi-step architecture introduces a critical vulnerability: subtle errors in the early stages may go undetected and subsequently propagate along execution branches, contaminating downstream decisions. This delayed manifestation of errors not only deviates from the final goal but also results in considerable computational overhead and wasted reasoning time.

[0003] Therefore, failure attribution (FA)—the process of tracing system-level failures back to their precise origins—becomes indispensable. By accurately pinpointing the root cause, failure attribution fundamentally enhances the debuggability, reliability, and cost-efficiency of LLM-based agent systems.

[0004] Existing fault attribution methods still have significant limitations. Topology- and spectrum-based methods (such as FAMAS and CDC-MAS) can utilize structural information (e.g., node dependencies) in trajectories, but often neglect fine-grained semantic cues, making it difficult to detect semantic faults related to hallucinations. In contrast, LLM-as-a-Judge methods offer a more promising direction because they can consider both structural and semantic information simultaneously. However, as trajectories become longer, LLM judges are increasingly affected by the "Lost in the Middle" problem, leading to decreased attribution accuracy and limiting their practicality in complex online systems. Directly feeding raw multi-agent system logs to the LLM judge incurs high token costs, increases latency, and leads to context dilution.

[0005] Therefore, there is an urgent need for a method and system that can perform anomaly detection, filter out normal information, and provide focused trajectory evidence for downstream LLM attribution before calling the LLM evaluator. Summary of the Invention

[0006] The purpose of this invention is to overcome the challenges of anomaly detection and fault attribution in the execution trajectory of large language model multi-agent systems, thereby solving the problems of context dilution, high token cost, and low attribution accuracy in existing methods when dealing with long trajectories.

[0007] In view of this, the present invention provides a method for attributing faults in a multi-agent system, comprising the following steps:

[0008] The system acquires the running status data of the execution trajectory of the faulty multi-agent system. The execution trajectory is a directed hierarchical graph structure. The execution trajectory includes multiple nodes and parent-child call dependencies between nodes. Each node stores the agent's identity text, the call operator text or tool, and unstructured text payload data.

[0009] For each node in the execution trajectory, a semantic embedding vector and a structural encoding vector are extracted to form a dual-view node representation. The semantic embedding vector is obtained by semantically encoding the agent's identity text and operator text, and the structural encoding vector is obtained by encoding the topological position of the node in the hierarchical graph structure.

[0010] The dual-view node representations are input into the encoder of the variational autoencoder based on a tree-structured long short-term memory network, and encoded along the hierarchical dependency structure of the execution trajectory to obtain the latent variable representations of each node.

[0011] The latent variable representation of each node is reconstructed by the decoder of the variational autoencoder. The dual-objective reconstruction includes semantic embedding vector reconstruction and token consumption prediction. The token consumption prediction is a probability prediction of the number of tokens consumed in the unstructured text payload data of the node. The anomaly score of each node is calculated based on the dual-objective reconstruction error.

[0012] Based on the statistical deviation of the accumulated abnormal scores propagating along the execution trajectory, the nodes are scored and ranked, and the top-ranked nodes are extracted to form candidate suspicious sub-trajectories.

[0013] The candidate suspicious sub-trajector is submitted to the downstream large language model attributor, which performs fault root cause localization. By screening abnormal nodes in the execution trajectory, the length of the execution trajectory processed by the large language model attributor is reduced.

[0014] As an improvement to the above method, the extraction of semantic embedding vectors from nodes specifically includes:

[0015] Using a pre-trained sentence encoder model, semantic encoding is performed on the agent identity text and operator text of the node to obtain their respective semantic embedding vectors;

[0016] The semantic embedding vector of the agent's identity text is concatenated with the semantic embedding vector of the operator text, and then the semantic embedding vector of the node is obtained after dimensionality reduction by a linear projection layer.

[0017] As an improvement to the above method, the tree-shaped long short-term memory network is a child node summation tree-shaped long short-term memory network. The update of its hidden state and cell state is achieved by combining the semantic embedding vector of the current node with the sum of the hidden states and cell states of all its child nodes. Furthermore, the semantic embedding vector of the current node does not include the actual token consumption information corresponding to the unstructured text payload of the current node, so as to force the encoder to encode only the topological context.

[0018] As an improvement to the above method, the step of performing bi-objective reconstruction of the latent variable representations of each node through the decoder of the variational autoencoder specifically includes:

[0019] The latent variable representation is input into the multilayer perceptron decoder to reconstruct the original semantic embedding vector, thus obtaining the semantic reconstruction result;

[0020] The latent variable representation is input into the multilayer perceptron decoder to predict the mean and variance parameters of the conditional Gaussian distribution that the token consumption of the current node follows, and the token consumption prediction result is obtained.

[0021] The calculation of anomaly scores for each node based on reconstruction errors specifically includes:

[0022] The mean squared error between the semantic embedding vector and the semantic embedding vector reconstruction result is taken as the semantic embedding vector reconstruction error;

[0023] The negative log-likelihood of the actual token consumption value under the predicted conditional Gaussian distribution is used as the token prediction error.

[0024] The anomaly score of the node is obtained by adding the semantic embedding vector reconstruction error to the token prediction error.

[0025] As an improvement to the above method, the step of dynamically standardizing and ranking nodes based on the statistical deviation degree propagating along the execution trajectory according to the cumulative abnormal score specifically includes:

[0026] The baseline mean and baseline standard deviation of the anomaly score are established using the successful execution trajectory. The anomaly score of each node is then standardized according to the baseline mean and baseline standard deviation to obtain the Z score of each node.

[0027] Sort all nodes in the execution trajectory from highest to lowest according to their Z-score;

[0028] The K nodes with the highest Z scores are extracted as candidate suspicious sub-trajectories, where K is much smaller than the total number of nodes N in the execution trajectory;

[0029] In the root cause localization step, the root cause is identified as the node corresponding to the earliest significant peak in the Z-score sequence, so as to mark the precise moment of logic failure.

[0030] On the other hand, the present invention provides a multi-agent system fault attribution system, comprising:

[0031] The data acquisition module is used to acquire the running status data of the execution trajectory of the faulty multi-agent system. The execution trajectory is a directed hierarchical graph structure, which includes multiple nodes and parent-child call dependencies between nodes.

[0032] The feature extraction module extracts semantic embedding vectors and structural encoding vectors for each node in the execution trajectory to form a dual-view node representation;

[0033] The trajectory encoding module is used to encode the hierarchical dependency structure of the execution trajectory by employing a tree-structured long short-term memory network as the encoder of the variational autoencoder, thereby obtaining the latent variable representation of each node.

[0034] The anomaly detection module performs bi-objective reconstruction of the latent variable representation of each node through the decoder of the variational autoencoder. The bi-objective reconstruction includes semantic embedding vector reconstruction and token consumption prediction. The token consumption prediction is a probability prediction of the number of tokens consumed in the unstructured text payload data of the node. The anomaly score of each node is calculated based on the bi-objective reconstruction error.

[0035] The dynamic scoring module is used to score and rank nodes based on the statistical deviation of the accumulated abnormal scores propagating along the execution trajectory, and extract the top-ranked nodes to form candidate suspicious sub-trajectories.

[0036] The fault attribution module is used to submit the candidate suspicious sub-trajector to the downstream large language model attributor, which then performs fault root cause localization.

[0037] As an improvement to the above system, the feature extraction module uses a pre-trained sentence encoder model to semantically encode the agent identity text and operator text of the node, and then concatenates the encoding results and reduces the dimensionality through a linear projection layer to obtain the semantic embedding vector of the node.

[0038] As an improvement to the above system, the tree-shaped long short-term memory network is a child node summation tree-shaped long short-term memory network. The update of its hidden state and cell state is achieved by combining the semantic embedding vector of the current node with the sum of the hidden states and cell states of all its child nodes. Furthermore, the semantic embedding vector of the current node excludes the actual token consumption information of the current node, so as to force the encoder to encode only the topological context.

[0039] As an improvement to the above system, the anomaly detection module performs a dual-objective reconstruction of the latent variable representation through a multilayer perceptron decoder: reconstructing the original semantic embedding vector to obtain the semantic embedding vector reconstruction result, and predicting the mean and variance parameters of the conditional Gaussian distribution that the current node's token consumption follows to obtain the token consumption prediction result; the anomaly score is the sum of the mean squared error of the semantic embedding vector reconstruction and the negative log-likelihood of the token consumption prediction.

[0040] As an improvement to the above system, the dynamic scoring module uses the successful execution trajectory to establish the baseline mean and baseline standard deviation of the abnormal score, standardizes the abnormal score of each node into a Z score, and extracts the top K nodes according to the Z score from high to low to form a candidate suspicious sub-trajectory, where K is much smaller than the total number of nodes N in the execution trajectory.

[0041] Compared with the prior art, the advantages of the present invention are:

[0042] (1) By using dual-view (semantic + structural) node representation and Tree-LSTM hierarchical encoding, the complex causal dependencies of multi-agent trajectories are effectively captured;

[0043] (2) By adopting the “detect first, then attribute” paradigm, the context length of the input LLM is greatly reduced, the “Lostin the Middle” problem is alleviated, and the attribution accuracy is significantly improved;

[0044] (3) Significantly reduced token consumption and inference latency, improving the scalability and usability of the system;

[0045] (4) As a plug-and-play module, it can be seamlessly integrated into various existing LLM-as-a-Judge attribution frameworks. Attached Figure Description

[0046] Figure 1 This is an overall architecture diagram of a multi-agent system fault attribution system provided in an embodiment of the present invention;

[0047] Figure 2 This is a flowchart illustrating a multi-agent system fault attribution method provided in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the dual-view node feature extraction and Tree-LSTM encoding process provided in this embodiment of the invention;

[0049] Figure 4 This is a schematic diagram of the process of dual-target reconstruction and anomaly detection of the variational autoencoder provided in the embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the dynamic scoring and candidate sub-trajectory extraction process provided in an embodiment of the present invention;

[0051] Figure 6A A comparison chart of overall agent attribution accuracy provided in embodiments of the present invention;

[0052] Figure 6B A comparison chart of overall step attribution accuracy provided for embodiments of the present invention;

[0053] Figure 7A This is a comparison chart of token consumption provided in an embodiment of the present invention;

[0054] Figure 7B A comparison chart of inference time provided for embodiments of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0056] This invention provides a fault attribution method for multi-agent systems, the method comprising the following steps:

[0057] S1: Obtain the runtime status data of the execution trajectory of the multi-agent system. The execution trajectory is represented as a directed hierarchical graph structure (tree structure), denoted as... ;

[0058] in, Represents a set of N nodes. This indicates the parent-child call dependency relationship between nodes.

[0059] Each node A tuple represents the behavior of an agent. ,in It is the identity text of the executing intelligent agent ( (the set of all available agents) It is the text of the operator or tool being called. (for a predefined set of operator texts) This is unstructured text payload data.

[0060] S2: Extract the semantic embedding vector and structural encoding vector for each node to form a dual-view node representation. For nodes... The pre-trained sentence encoder model is used to encode the agent identity text and operator text respectively, to obtain their respective semantic embedding vectors. and The two are concatenated and then subjected to dimensionality reduction via a linear projection layer to obtain the semantic embedding vector of the node. Simultaneously, the topological position of the node in the hierarchical graph structure (such as depth, number of child nodes, etc.) is encoded into a structural encoding vector. The semantic embedding vector and the structural encoding vector are then fused to obtain a dual-view node representation.

[0061] S3: A tree-structured long short-term memory network is used as the encoder of the variational autoencoder to encode the hierarchical dependency structure of the execution trajectory. A child-sum tree-structured long short-term memory network (Tree-LSTM) is used as the variational encoder to capture the hierarchical topology of the execution trajectory. Nodes Hidden state and cell state By combining the projected input of the current node with all its child nodes The state is updated. To prevent information leakage, the projected input explicitly excludes the actual token consumption information of the current node, forcing the encoder to rely solely on the topological context for encoding. The hidden state is then... Projection models macroscopic execution uncertainty using a parameterized Gaussian posterior distribution, and latent variables are obtained through reparameterization techniques. .

[0062] S4: A dual-objective reconstruction is performed on each node using a variational autoencoder decoder, and anomaly scores are calculated based on the reconstruction error. A unified multilayer perceptron (MLP) decoder is employed to analyze latent variables. Reconstruct the original dual-view input: ;

[0063] in, This represents the semantic embedding vector of the reconstructed agent identity. This represents the semantic embedding vector of the reconstructed operator; and These are the predicted mean and predicted log-variance of the conditional Gaussian distribution that the token consumption of this node follows, respectively.

[0064] The trained variational autoencoder can predict the semantic embedding vector and token consumption distribution of the agent based on the context of the execution trajectory. If it deviates from the normal pattern, it will produce a bias in the estimated value in the dual-objective reconstruction, thus being detected as an anomaly. The dual-objective reconstruction includes: (1) Semantic embedding vector reconstruction: reconstructing dense semantic embedding vectors ( , (2) Token consumption prediction: The token consumption is modeled as a conditional Gaussian distribution, and its mean is predicted. and variance The parameters are used to obtain a probabilistic prediction of the number of tokens consumed by the node's text payload. For each node, the mean squared error of the semantic embedding vector reconstruction and the negative log-likelihood of the token consumption prediction are calculated, and their sum is used as the anomaly score for that node. .

[0065] S5: Based on the statistical deviation of the cumulative anomaly scores propagating along the execution trajectory, dynamically standardize the scoring of nodes and rank them. Establish a baseline mean of the anomaly scores using the successful execution trajectory. and baseline standard deviation The anomaly scores of each node are standardized into Z-scores: ;

[0066] Represents a node The anomaly score is the sum of the mean squared error of semantic embedding vector reconstruction calculated in step S4 and the negative log-likelihood of token consumption prediction.

[0067] ;

[0068] in, For nodes The actual number of tokens consumed. Indicates For the mean, Let be the Gaussian probability density function of the variance.

[0069] Sort all N nodes in descending order of their Z scores, and extract the top K nodes (K is much smaller than N, preferably 20% of N) to form a candidate suspicious sub-trajectory. At the same time, the adjacency matrix is ​​used to preserve the topological dependencies between candidate nodes, forming a complete candidate suspicious sub-trajectory.

[0070] Sort all N nodes in descending order of their Z scores, and extract the top K nodes (K is much smaller than N) to form a candidate suspicious sub-trajectory. At the same time, the adjacency matrix is ​​used to preserve topological dependencies.

[0071] S6: Submit candidate suspicious subtrajectories to the downstream large language model attributor for final fault root cause localization. The results are passed to the downstream LLM-as-a-Judge attributor (such as CORRECT, ECHO, etc.), where the LLM performs the final root cause localization of the fault on the focused candidate sub-trajector.

[0072] Therefore, it can be seen that by first screening abnormal nodes in the execution trajectory, the present invention reduces the length of the execution trajectory processed by the large language model attributor, thereby significantly improving the attribution accuracy.

[0073] In step S2, the semantic embedding vector has a dimension of d=384, and the agent identity text embedding and operator text embedding are concatenated to form a dimension of 768, which is then reduced to 128 dimensions by a linear projection layer.

[0074] In step S3, the hidden state dimension of the tree-shaped long short-term memory network is 128-dimensional, and the hidden variable space dimension of the variational autoencoder is 64-dimensional.

[0075] In step S4, the optimization objective of the variational autoencoder is to maximize the lower bound of evidence (ELBO), which unifies the mean squared error loss of semantic embedding vector reconstruction, the negative log-likelihood loss of token consumption prediction, and the KL divergence loss of latent variable distribution, wherein the KL divergence weight β is set to 0.01.

[0076] The method also includes a data augmentation step: using large language model generation and prefix chain extraction to synthesize successful execution trajectory variants to expand the training dataset, enabling the variational autoencoder to establish a more robust normal execution distribution boundary.

[0077] In the root cause localization step, the root cause is identified as the node corresponding to the earliest significant peak in the Z-score sequence, so as to mark the precise moment of logic failure.

[0078] On the other hand, the present invention provides a multi-agent system fault attribution system, the system comprising:

[0079] The data acquisition module is used to acquire the running status data of the execution trajectory of the multi-agent system. The execution trajectory is represented as a directed hierarchical graph structure, including multiple nodes and parent-child call dependencies between nodes. Each node contains agent identity text, call operator text or tools, and unstructured text payload data.

[0080] The feature extraction module is used to extract semantic embedding vectors and structural encoding vectors for each node, forming a dual-view node representation. This module uses a pre-trained sentence encoder model to semantically encode the agent identification text and operator text of the node, respectively, and concatenates the encoding results and performs dimensionality reduction through a linear projection layer to obtain the semantic embedding vector of the node; at the same time, the topological position of the node in the hierarchical graph structure is encoded as a structural encoding vector.

[0081] The trajectory encoding module employs a tree-structured long short-term memory network as the encoder of the variational autoencoder to encode the hierarchical dependency structure of the execution trajectory, obtaining the latent variable representation of each node. The tree-structured long short-term memory network is a child-node summation tree-structured long short-term memory network, where the updates of the hidden states and cell states are achieved by combining the projected input of the current node with the states of all its child nodes. Furthermore, the projected input of the current node explicitly excludes the actual token consumption information of the current node.

[0082] The anomaly detection module performs a dual-objective reconstruction of the latent variable representations of each node using a variational autoencoder decoder. This dual-objective reconstruction includes semantic reconstruction and token consumption prediction, and an anomaly score is calculated for each node based on the reconstruction error. This module uses a multilayer perceptron decoder to perform the dual-objective reconstruction of the latent variable representations: reconstructing the original semantic embedding vector to obtain the semantic reconstruction result, and predicting the mean and variance parameters of the conditional Gaussian distribution that the current node's token consumption follows to obtain the token consumption prediction result. The anomaly score is the sum of the mean squared error of the semantic reconstruction and the negative log-likelihood of the token consumption prediction.

[0083] The dynamic scoring module is used to dynamically standardize and rank nodes based on the statistical deviation of cumulative abnormal scores propagating along the execution trajectory, and extract the top-ranked nodes to form candidate suspicious sub-trajectories. This module uses successful execution trajectories to establish the baseline mean and baseline standard deviation of abnormal scores, standardizes the abnormal scores of each node into Z-scores, and extracts the top K nodes according to their Z-scores from highest to lowest to form candidate suspicious sub-trajectories, where K is much smaller than the total number of nodes N in the execution trajectory.

[0084] The fault attribution module is used to submit the candidate suspicious sub-trajector to the downstream large language model attributor, which then performs fault root cause localization.

[0085] The fault attribution system for large language model multi-agent systems based on dual-view variational autoencoders is plug-and-play and can be seamlessly integrated into various existing LLM-as-a-Judge attribution frameworks.

[0086] This invention achieves anomaly detection of execution trajectories in multi-agent systems by deploying a variational autoencoder based on Tree-LSTM. Based on this, it submits only the abnormal sub-trajectories to the downstream LLM for fault attribution through the "detect first, then attribute" paradigm. Its advantages are: (1) It effectively captures the complex causal dependencies of multi-agent trajectories through dual-view (semantic + structural) node representation and Tree-LSTM hierarchical encoding; (2) Through the "detect first, then attribute" paradigm, it significantly reduces the context length of the input LLM, alleviates the "Lost in the Middle" problem, and significantly improves the attribution accuracy; (3) It significantly reduces token consumption and inference latency, and improves the scalability and practicality of the system; (4) As a plug-and-play module, it can be seamlessly integrated into various existing LLM-as-a-Judge attribution frameworks.

[0087] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0088] This embodiment provides a multi-agent system fault attribution system, including:

[0089] like Figure 1 The diagram shown is an overall architecture diagram of a multi-agent system fault attribution system provided in an embodiment of the present invention. Figure 1 As shown, the system mainly consists of six modules: a data acquisition module, a feature extraction module, a trajectory encoding module, an anomaly detection module, a dynamic scoring module, and a fault attribution module. The system adopts a pipeline paradigm of "detection first, then attribution": the original multi-agent execution trajectory first enters the system through the data acquisition module, where the feature extraction module extracts dual-view node representations, which are then encoded and reconstructed by a Tree-LSTM-based variational autoencoder. The anomaly detection module calculates the anomaly score for each node, the dynamic scoring module filters out candidate suspicious sub-trajectories, and finally, the system is submitted to the downstream LLM attributor for final fault root cause localization.

[0090] This embodiment provides a fault attribution method for a multi-agent system, including:

[0091] like Figure 2 The diagram shown is a flowchart illustrating a multi-agent system fault attribution method provided in an embodiment of the present invention. Figure 2 As shown, the main steps are as follows:

[0092] S201: Obtain the execution trajectory of the multi-agent system. Extract the fault execution trajectory from the multi-agent system's operation log and represent it as a directed hierarchical tree structure. Each node It includes agent identity, operator text, and text payload information.

[0093] The total number of nodes N in the execution trajectory may be very large (e.g., hundreds of steps), containing a large amount of redundant normal execution information and a small amount of critical abnormal information.

[0094] S202: Extract the dual-view node representation. For each node... Pre-trained sentence encoder models (such as all-MiniLM-L6-v2) are used to encode the agent identity text and operator text respectively, obtaining their respective semantic embedding vectors. and (Dimension d=384). The two are concatenated into a 768-dimensional vector, which is then reduced to 128 dimensions through a linear projection layer. Simultaneously, the structural features of the nodes (such as depth, number of child nodes, etc.) are extracted as structural encoding vectors. The semantic embedding vector and the structural encoding vector are fused to obtain a dual-view node representation.

[0095] S203: Tree-LSTM Encoding. A child-node summation Tree-LSTM is used as a variational encoder to encode along the hierarchical dependency structure of the execution trajectory. Node Hidden state and cell state By combining the projected input of the current node with all its child nodes The state is updated. To force the encoder to rely on the topological context rather than directly memorizing token consumption, the projected input explicitly excludes the actual token consumption information of the current node. Projecting parameterized Gaussian posterior distributions and obtaining latent variables through reparameterization techniques. (Dimension 64).

[0096] S204: Dual-target reconstruction and anomaly detection. A unified MLP decoder is used to perform dual-target reconstruction from the latent variable zᵢ: (1) Semantic embedding vector reconstruction - reconstructing the original semantic embedding ( , (2) Token consumption prediction - conditional Gaussian distribution parameters for predicting token consumption ( The mean squared error (MSE) of semantic embedding vector reconstruction and the negative log-likelihood (NLL) of token prediction are calculated, and their sum is used as the node anomaly score. .

[0097] The optimization objective of the variational autoencoder is to maximize the lower bound of evidence (ELBO).

[0098] ;

[0099] in, This represents the total loss function, used to jointly optimize three objectives: semantic reconstruction, token prediction, and latent variable regularization.

[0100] This represents the mathematical expectation operator, where the loss of the batch of samples is averaged.

[0101] Represents a node The token consumption is predicted to be negative log-likelihood loss.

[0102] KL divergence is used to measure the difference between two probability distributions.

[0103] Representing the variational posterior distribution, i.e., using the node hidden states As a condition, determined by encoder parameters A parameterized Gaussian distribution is used to approximate the true posterior.

[0104] Representing latent variables The prior distribution;

[0105] These are the weighting coefficients for the KL divergence, used to balance the relative contributions between the reconstruction loss and the regularization term. The value of 0.01 is chosen to avoid posterior collapse while maintaining the expressive power of the latent variable space.

[0106] S205: Dynamic scoring and candidate sub-trajectory extraction. Establishing anomaly score baselines using successful execution trajectories. The anomaly scores of each node are standardized into Z-scores: Sort all N nodes by their Z-score from highest to lowest, and extract the top K=20% of nodes to form a candidate suspicious sub-trajectory. Topological dependencies are preserved using an adjacency matrix.

[0107] S206: Downstream LLM attribution. The results are passed to downstream LLM-as-a-Judge attributors (such as CORRECT, ECHO, etc.), where the LLM performs final root cause localization on the focused candidate sub-trajectories. The root cause is identified as the node corresponding to the earliest significant peak in the Z-score sequence.

[0108] Figure 3 This is a schematic diagram illustrating the process of dual-view node feature extraction and Tree-LSTM encoding provided in an embodiment of the present invention. Figure 3As shown, each node in the original execution trajectory first passes through a sentence encoder (Sentence Transformer) to obtain the agent's identity embedding. operator text embedding The semantic embedding vector is obtained by concatenating the two and then reducing its dimensionality through a linear projection layer. Simultaneously, the structural features of the nodes are extracted. The semantic embedding vector and the structural encoding vector are fused and input into a Tree-LSTM encoder for hierarchical encoding. The Tree-LSTM recursively processes the data along the depth-first order of the execution trajectory. The state of each node is calculated by aggregating the states of its child nodes and combining them with its own projected input, ultimately outputting the latent variable representation of each node.

[0109] Figure 4 This is a schematic diagram of the variational autoencoder dual-target reconstruction and anomaly detection process provided in an embodiment of the present invention. Figure 4 As shown, the latent variables output by the Tree-LSTM encoder Obtained through reparameterized sampling The input is used for dual-objective reconstruction in the MLP decoder. The reconstruction objectives include: (1) semantic embedding vector reconstruction - decoder output ,and (2) Token consumption prediction - decoder output The NLL loss is calculated based on the actual token consumption. The sum of the two losses constitutes the anomaly score for that node. The entire VAE is optimized end-to-end using ELBO.

[0110] Figure 5 This is a schematic diagram of the dynamic scoring and candidate sub-trajectory extraction process provided in an embodiment of the present invention. Figure 5 As shown, after obtaining the anomaly scores for all nodes, the score statistics of the successful trajectories are used ( , The anomaly score is standardized to a Z-score. A higher Z-score indicates a more anomalous node. All nodes are sorted in descending order of Z-score, and the top K nodes (K is much smaller than N, preferably 20%-25% of N) are selected to form a candidate set. At the same time, the topological dependencies between candidate nodes are recovered using an adjacency matrix to form a complete candidate suspicious sub-trajectory.

[0111] To verify the effectiveness of the fault attribution method and system based on a dual-view variational autoencoder proposed in this invention, the inventors conducted experimental verification on the Who and When benchmark dataset. This dataset is specifically used for evaluating fault attribution tasks in multi-agent systems and contains 184 fault instances, divided into two subsets: Handcraft (58 instances) and Automated (126 instances), covering various task types such as information retrieval, mathematical logic reasoning, data analysis, multimodal processing, and programming debugging.

[0112] Six LLM-based baseline methods were selected for comparison, including DeepSeek-R1, Claude-Sonnet-4, GPT-5, Qwen3.5-plus, ECHO, and CORRECT. Two evaluation metrics were set: Agent Level accuracy (locating the agent that caused the failure) and Step Level accuracy (locating the specific step that caused the failure). All experiments were conducted in a unified computing environment.

[0113] like Figure 6A This is a comparison chart of Overall Agent Level attribution accuracy provided in an embodiment of the present invention. The horizontal axis represents six baseline methods (DeepSeek-R1, Claude-Sonnet-4, GPT-5, Qwen3.5-plus, ECHO, and CORRECT), and the vertical axis represents accuracy (%). Each group contains two bars: the left diagonal texture bar represents the accuracy of the original method, and the right mesh texture bar represents the accuracy after implementing the DuoTrace invention. The accuracy improvement difference Δ is marked above the bars, in percentage points.

[0114] Figure 6AAs shown, in terms of Overall Agent Level attribution accuracy, the accuracy of all baseline methods was improved after incorporating the DuoTrace invention. Specifically, the accuracy of Qwen3.5-plus increased from 49.5% to 63.0%, an improvement of 13.5 percentage points; GPT-5's accuracy increased from 46.2% to 59.1%, an improvement of 12.9 percentage points; ECHO's accuracy increased from 54.9% to 62.4%, an improvement of 7.5 percentage points; CORRECT's accuracy increased from 62.5% to 69.1%, an improvement of 6.6 percentage points; Claude-Sonnet-4's accuracy increased from 50.8% to 58.7%, an improvement of 7.9 percentage points; and DeepSeek-R1's accuracy increased from 61.4% to 63.6%, an improvement of 2.2 percentage points. Experimental results demonstrate that this invention, as a plug-and-play module, can stably improve the Agent Level attribution accuracy of various LLM-based attribution methods.

[0115] like Figure 6B As shown, this invention also demonstrates a significant improvement in overall step-level attribution accuracy. Specifically, the accuracy of GPT-5 increased from 16.3% to 29.0%, an improvement of 12.7 percentage points; DeepSeek-R1's accuracy increased from 22.4% to 29.9%, an improvement of 7.5 percentage points; CORRECT's accuracy increased from 41.9% to 49.0%, an improvement of 7.1 percentage points; Qwen3.5-plus's accuracy increased from 31.5% to 38.0%, an improvement of 6.5 percentage points; ECHO's accuracy increased from 35.9% to 40.7%, an improvement of 4.8 percentage points; and Claude-Sonnet-4's accuracy increased from 32.6% to 35.9%, an improvement of 3.3 percentage points. Step-level attribution requires finer-grained localization capabilities, and the improved accuracy at the step level further validates the effectiveness of this invention in accurate fault location.

[0116] The above experimental results fully demonstrate the effectiveness of the "detect first, then attribute" paradigm proposed in this invention. By introducing an anomaly detection filtering module based on a dual-view variational autoencoder before the LLM attributor, candidate suspicious sub-trajectories can be effectively screened out, thereby alleviating the context dilution problem of downstream LLM when processing long trajectories and significantly improving the accuracy of fault attribution.

[0117] like Figure 7AAs shown, in terms of token consumption, the token consumption of each method was significantly reduced after incorporating this invention. Qwen3.5-plus's token consumption decreased from 2.12 million to 500,000, a reduction of 1.62 million (76.4%); ECHO's token consumption decreased from 3.96 million to 2.34 million, a reduction of 1.62 million (40.9%); CORRECT's token consumption decreased from 2.04 million to 1.26 million, a reduction of 0.78 million (38.2%); and AgentTracer's token consumption decreased from 2.4 million to 1.44 million, a reduction of 0.96 million (40.0%).

[0118] like Figure 7B As shown, in terms of inference time, the inference time of each method is significantly reduced after incorporating this invention. The inference time of Qwen3.5-plus is reduced from 168.6 minutes to 60.6 minutes, a reduction of 108.0 minutes (64.1%); the inference time of ECHO is reduced from 168.0 minutes to 81.0 minutes, a reduction of 87.0 minutes (51.8%); the inference time of CORRECT is reduced from 81.4 minutes to 32.3 minutes, a reduction of 49.1 minutes (60.3%); and the inference time of AgentTracer is reduced from 144.0 minutes to 72.0 minutes, a reduction of 72.0 minutes (50.0%).

[0119] comprehensive Figure 7A and Figure 7B The experimental results show that this invention achieves significant reductions in both token consumption and inference time. Token consumption is reduced by up to 76.4%, and inference time is shortened by up to 64.1%, fully demonstrating that this invention significantly reduces computational resource consumption and inference latency while improving attribution accuracy, exhibiting good practicality and scalability.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A fault attribution method for a multi-agent system, characterized in that, Includes the following steps: The system acquires the running status data of the execution trajectory of the faulty multi-agent system. The execution trajectory is a directed hierarchical graph structure. The execution trajectory includes multiple nodes and parent-child call dependencies between nodes. Each node stores the agent's identity text, the call operator text or tool, and unstructured text payload data. For each node in the execution trajectory, a semantic embedding vector and a structural encoding vector are extracted to form a dual-view node representation. The semantic embedding vector is obtained by semantically encoding the agent's identity text and operator text, and the structural encoding vector is obtained by encoding the topological position of the node in the hierarchical graph structure. The dual-view node representations are input into the encoder of the variational autoencoder based on a tree-structured long short-term memory network, and encoded along the hierarchical dependency structure of the execution trajectory to obtain the latent variable representations of each node. The latent variable representation of each node is reconstructed by the decoder of the variational autoencoder. The dual-objective reconstruction includes semantic embedding vector reconstruction and token consumption prediction. The token consumption prediction is a probability prediction of the number of tokens consumed in the unstructured text payload data of the node. The anomaly score of each node is calculated based on the dual-objective reconstruction error. Based on the statistical deviation of the accumulated abnormal scores propagating along the execution trajectory, the nodes are scored and ranked, and the top-ranked nodes are extracted to form candidate suspicious sub-trajectories. The candidate suspicious sub-trajector is submitted to the downstream large language model attributor, which performs fault root cause localization. By screening abnormal nodes in the execution trajectory, the length of the execution trajectory processed by the large language model attributor is reduced.

2. The method as described in claim 1, characterized in that, The extraction of semantic embedding vectors from nodes specifically includes: Using a pre-trained sentence encoder model, semantic encoding is performed on the agent identity text and operator text of the node to obtain their respective semantic embedding vectors; The semantic embedding vector of the agent's identity text is concatenated with the semantic embedding vector of the operator text, and then the semantic embedding vector of the node is obtained after dimensionality reduction by a linear projection layer.

3. The method according to claim 1, characterized in that, The tree-shaped long short-term memory network is a child node summation tree-shaped long short-term memory network. The update of its hidden state and cell state is achieved by combining the semantic embedding vector of the current node with the sum of the hidden states and cell states of all its child nodes. The semantic embedding vector of the current node does not include the actual token consumption information corresponding to the unstructured text payload of the current node, so as to force the encoder to encode only the topological context.

4. The method according to claim 1, characterized in that, The step of reconstructing the latent variable representations of each node using a decoder derived from a variational autoencoder, specifically includes: The latent variable representation is input into the multilayer perceptron decoder to reconstruct the original semantic embedding vector, thus obtaining the semantic reconstruction result; The latent variable representation is input into the multilayer perceptron decoder to predict the mean and variance parameters of the conditional Gaussian distribution that the token consumption of the current node follows, and the token consumption prediction result is obtained. The calculation of anomaly scores for each node based on reconstruction errors specifically includes: The mean squared error between the semantic embedding vector and the semantic embedding vector reconstruction result is taken as the semantic embedding vector reconstruction error; The negative log-likelihood of the actual token consumption value under the predicted conditional Gaussian distribution is used as the token prediction error. The anomaly score of the node is obtained by adding the semantic embedding vector reconstruction error to the token prediction error.

5. The method according to claim 1, characterized in that, The method of dynamically standardizing and ranking nodes based on the statistical deviation degree propagated along the execution trajectory according to the cumulative abnormal score specifically includes: The baseline mean and baseline standard deviation of the anomaly score are established using the successful execution trajectory. The anomaly score of each node is then standardized according to the baseline mean and baseline standard deviation to obtain the Z score of each node. Sort all nodes in the execution trajectory from highest to lowest according to their Z-score; The K nodes with the highest Z scores are extracted as candidate suspicious sub-trajectories, where K is much smaller than the total number of nodes N in the execution trajectory; In the root cause localization step, the root cause is identified as the node corresponding to the earliest significant peak in the Z-score sequence, so as to mark the precise moment of logic failure.

6. A fault attribution system for a multi-agent system, characterized in that, include: The data acquisition module is used to acquire the running status data of the execution trajectory of the faulty multi-agent system. The execution trajectory is a directed hierarchical graph structure, which includes multiple nodes and parent-child call dependencies between nodes. The feature extraction module extracts semantic embedding vectors and structural encoding vectors for each node in the execution trajectory to form a dual-view node representation; The trajectory encoding module is used to encode the hierarchical dependency structure of the execution trajectory by employing a tree-structured long short-term memory network as the encoder of the variational autoencoder, thereby obtaining the latent variable representation of each node. The anomaly detection module performs bi-objective reconstruction of the latent variable representation of each node through the decoder of the variational autoencoder. The bi-objective reconstruction includes semantic embedding vector reconstruction and token consumption prediction. The token consumption prediction is a probability prediction of the number of tokens consumed in the unstructured text payload data of the node. The anomaly score of each node is calculated based on the bi-objective reconstruction error. The dynamic scoring module is used to score and rank nodes based on the statistical deviation of the accumulated abnormal scores propagating along the execution trajectory, and extract the top-ranked nodes to form candidate suspicious sub-trajectories. The fault attribution module is used to submit the candidate suspicious sub-trajector to the downstream large language model attributor, which then performs fault root cause localization.

7. The system according to claim 6, characterized in that, The feature extraction module uses a pre-trained sentence encoder model to semantically encode the agent identity text and operator text of the node, and then concatenates the encoding results and reduces the dimensionality through a linear projection layer to obtain the semantic embedding vector of the node.

8. The system according to claim 6, characterized in that, The tree-shaped long short-term memory network is a child node summation tree-shaped long short-term memory network. The update of its hidden state and cell state is achieved by combining the semantic embedding vector of the current node with the sum of the hidden states and cell states of all its child nodes. Furthermore, the semantic embedding vector of the current node excludes the actual token consumption information of the current node, so as to force the encoder to encode only the topological context.

9. The system according to claim 6, characterized in that, The anomaly detection module performs a dual-objective reconstruction of the latent variable representation through a multilayer perceptron decoder: reconstructing the original semantic embedding vector to obtain the semantic embedding vector reconstruction result, and predicting the mean and variance parameters of the conditional Gaussian distribution that the current node's token consumption follows to obtain the token consumption prediction result; the anomaly score is the sum of the mean squared error of the semantic embedding vector reconstruction and the negative log-likelihood of the token consumption prediction.

10. The system according to claim 6, characterized in that, The dynamic scoring module uses the successful execution trajectory to establish the baseline mean and baseline standard deviation of the abnormal score, standardizes the abnormal score of each node into a Z score, and extracts the top K nodes according to the Z score from high to low to form a candidate suspicious sub-trajectory, where K is much smaller than the total number of nodes N in the execution trajectory.

Images