A method for delay-aware spatio-temporal causal inference-based microservice fault root cause analysis

Abstract

The application discloses a micro-service fault root cause analysis method and system for delay-aware spatio-temporal causal inference, relates to the technical field of micro-service root cause analysis, and comprises the following steps: acquiring runtime index data and call chain data of each micro-service instance, and pre-processing the data to obtain a multi-modal input tensor; learning a dynamic multi-delay causal graph based on the multi-modal input tensor; taking the dynamic multi-delay causal graph as a structural prior, and using a delay-aware spatio-temporal attention mechanism to aggregate context signals under different delays from causal neighbors to obtain a causal spatio-temporal embedding of each micro-service instance; performing index prediction based on the causal spatio-temporal embedding, and generating a ranking list of root cause candidate instances and a fault propagation path by deducting an abnormal part explained by upstream propagation according to a prediction residual and the dynamic multi-delay causal graph. The application improves root cause positioning accuracy, reduces manual troubleshooting workload, and provides an interpretable propagation path.

Description

Technical Field

[0001] This invention relates to the field of microservice root cause analysis technology, and in particular to a method for microservice failure root cause analysis based on delay-aware spatiotemporal causal inference. Background Technology

[0002] With the rapid development of internet and cloud computing technologies, microservice architecture has become the mainstream deployment form for large-scale distributed systems. Through service decomposition, independent deployment, and elastic scaling, it effectively supports high-concurrency and high-availability business requirements. However, microservice systems involve a large number of services and complex call relationships, making it easy for faults to propagate along dependency chains across services. Minor anomalies in upstream services often take several seconds or even minutes to manifest in downstream services due to network buffering, retry mechanisms, and resource contention, ultimately evolving into large-scale service interruptions and severely impacting business continuity. Therefore, quickly and accurately locating the root cause of faults is a core requirement for ensuring the stable operation of microservice systems.

[0003] Current microservice root cause analysis (RCA) methods can be divided into two categories: single-dimensional methods and joint spatiotemporal methods. Single-dimensional methods analyze only time-series indicators or single call links, making it difficult to characterize the cross-service and multi-latency propagation characteristics of faults. They also fail to effectively correlate early upstream anomalies with downstream lagging symptoms, easily leading to biased root cause localization. While joint spatiotemporal methods attempt to integrate temporal dynamics and service topology, they generally adopt a synchronous modeling paradigm, treating fault propagation as an instantaneous interaction and ignoring the dynamic changes in fault propagation latency with load and scheduling in real systems. Furthermore, existing methods struggle to distinguish between true causal dependencies between services and synchronous fluctuations in indicators: sparse physical topology graphs only cover observed links, easily missing hidden dependencies; dense performance correlation graphs are susceptible to noise interference, often misjudging shared state changes as direct causal relationships, further reducing the accuracy of root cause analysis.

[0004] Microservice architecture has become the de facto infrastructure of the modern internet and cloud services. With the rapid growth in the number of services and the increasing complexity of dependencies between them, microservice systems are more prone to cascading failures: a localized failure can propagate along the dependency chain and eventually evolve into a large-scale service outage. Therefore, quickly and accurately identifying the root cause is crucial for business continuity and user experience. Actual failures rarely manifest as instantaneous, system-wide interruptions, but rather propagate gradually along the service dependency chain, accompanied by non-negligible and time-varying delays. As shown in Figure 1, upstream services... Slight vibrations may first affect the service being directly accessed. Indirect downstream services , Often in the extra delay Significant performance degradation only occurred later. Runtime mechanisms such as dynamic traffic, automatic scaling, and routing strategies constantly change the effective dependency graph and the time required for propagation.

[0005] Traditional RCA methods can be broadly categorized into two types. Single-dimensional methods focus on the temporal patterns of a single service or short-distance spatial dependencies, making it difficult to model the impact of upstream states on downstream states under different time delays. Joint spatiotemporal methods combine a temporal encoder with a microservice graph, but typically still separate temporal and spatial modeling and emphasize local interactions.

[0006] Existing methods typically rely on two types of graphs: physical topology graphs describing "who calls whom," and performance correlation graphs describing "which metrics fluctuate together over time." The former is interpretable but only covers observed communication links, easily overlooking hidden dependencies; the latter is dense but noisy, easily mistaking common causes or synchronous fluctuations for real fault propagation. Therefore, this paper aims to build a latency-aware spatiotemporal causal inference framework capable of recovering the true propagation chain of microservices. Challenge 1: Asynchronous propagation modeling under dynamic latency. The latency of upstream disturbances reaching different downstream services is jointly determined by load, scheduling, and resource contention, and varies over time. If interactions are treated as synchronous or fixed-latency, the model struggles to correlate the root cause at time t with time. The newly emerging symptoms are correctly correlated. Challenge 2: Bridging the gap between physical topology and service performance correlation. Physical topology is sparse but structurally accurate, while performance correlation graphs are dense but susceptible to common causes and noise. A path needs to be learned that respects physical topology while suppressing the propagation of non-causal performance-related edges.

[0007] like Figure 2The diagram illustrates metrics and call chains in a microservice failure. Accident diagnosis in microservice systems primarily relies on three types of observable data: logs, interaction traces, and runtime metrics. Runtime metrics record the performance and resource usage of instances over time; interaction traces record how a request flows through multiple service instances, along with corresponding time and status information; logs provide the textual context of individual events. Metrics are typically collected from each instance at fixed intervals, including request latency, throughput (QPS), error rate, and resource usage such as CPU and memory. Trace sequences consist of timestamped spans, recording which instance called which downstream instance. Aggregating spans yields the runtime call topology; observing instance metrics along this topology reveals when anomalies occur in each service and how failures are delayed and amplified along the path. Spatiotemporal graph modeling aims to jointly capture spatial dependencies and temporal dynamics in a network system. In microservice operations and maintenance, the task is often described as: modeling the evolution of service metrics on the service call dependency graph, enabling the algorithm to simultaneously utilize "who depends on whom" and "how the state of each node changes over time." Existing STGNNs typically employ a spatiotemporal decoupling paradigm: first, spatial modules aggregate neighbor information within each time slice, then RNNs, temporal convolutions, or Transformers are used along the time axis to capture sequence dynamics. This design implicitly assumes synchronous aggregation, meaning that a node's representation at time t is primarily updated by its neighbors' representations at the same time t. In microservice systems, fault propagation often exhibits asynchronous and multi-latency behavior across services.

[0008] Taking the cascading failure in Figure 1 as an example: Upstream service An abnormal delay and error rate occurred at time t. Due to retry mechanisms, message queue buffering, and timeout policies, the disturbance will not be immediately reflected downstream; it will directly affect the called service. It wasn't until several minutes later that the abnormality became obvious, and even further away... It will crash after a longer period of time. In other words, services do not fail simultaneously, but rather sequentially along the call chain. The synchronous aggregation model lacks an explicit mechanism to correlate upstream failures at time t with... The downstream offset is connected. When exist When an alarm is triggered, Early abnormalities in t only manifest as A fluctuation in its own history, without being explicitly linked to Current state. This is the source of Challenge 1. To verify whether multi-delay propagation is prevalent, all labeled incidents were statistically analyzed on the industrial dataset D1. The metric sampling interval was 1 minute, and the anomaly occurrence time was defined as the minute after the instance metric first became abnormal. Calculations were performed for each incident. This refers to the maximum difference between the time the root cause instance anomaly occurred and the time the anomaly occurred for any affected instance. Besides delayed propagation, similar fluctuations in service metrics can also mislead RCA. For example... At the same time to and Forwarding abnormal traffic, although and They don't physically interact with each other, but their metrics deteriorate almost simultaneously. If we only look at the operational metrics within the incident window... and They appear to be strongly correlated, but this does not mean there is a direct call or a real causal dependency between them.

[0009] Therefore, the limitations of existing technologies are mainly reflected in three aspects: First, they lack explicit modeling of multi-delay asynchronous propagation, making it impossible to accurately align anomalies at upstream time t with symptoms at downstream time t+τ, resulting in the weakening of the true upstream root cause and misjudgment of downstream affected services; Second, they fail to effectively bridge the gap between physical topology and performance correlation, cannot decouple stable call structures from dynamic interaction intensity, and struggle to suppress non-causal pseudo-dependency interference; Third, the online diagnosis phase does not correct for the impact of upstream propagation, making it prone to misjudging the root cause due to amplified downstream anomalies, and most methods lack interpretability in their output results, failing to provide a clear fault propagation path and hindering rapid handling by operations personnel. In summary, existing microservice root cause analysis technologies are ill-suited for large-scale, highly dynamic, and multi-delay real-world fault scenarios, exhibiting low location accuracy and high misjudgment rate, failing to meet the efficient and reliable fault diagnosis needs of enterprise-level microservice systems. Therefore, there is an urgent need for a root cause analysis solution that can explicitly model multi-delay fault propagation, accurately distinguish between causal dependencies and synchronous fluctuations, and provide interpretable propagation path outputs. Summary of the Invention

[0010] The main objective of this invention is to provide a delay-aware spatio-temporal causal inference method for microservice root cause analysis. It proposes a delay-aware spatio-temporal causal inference framework for microservice root cause analysis, STRCA (Lag-Aware Spatio-Temporal Causal Inference for Microservice Root Cause Analysis), which mainly includes two key modules: (1) multi-delay causal graph learning, which jointly utilizes physical topology and performance co-fluctuations to distinguish between relatively stable call structures and time-varying interaction strengths; and (2) delay-aware spatio-temporal representation learning, which uses the learned multi-delay causal graph as a structural prior and aggregates context signals under different delays from causal neighbors through spatio-temporal attention. The microservice RCA is formalized as a delay-aware spatio-temporal causal inference problem, and a STRCA framework for jointly learning multi-delay causal structures and spatio-temporal representations is proposed. A delay-aware propagation module is designed to couple the multi-delay causal graph with spatio-temporal attention, explicitly modeling the process of fault propagation along lag edges. A dual-branch causal graph learner is introduced to separate trace-based structural relationships and indicator-based interaction strengths, and then reassembles them under joint constraints. This can improve RCA accuracy and incident handling efficiency.

[0011] To achieve the above objectives, a first aspect of the present invention proposes a microservice failure root cause analysis method based on delay-aware spatiotemporal causal inference, comprising:

[0012] Obtain runtime metrics and call chain data for each microservice instance, and preprocess the data to obtain a multimodal input tensor; Based on the multimodal input tensor, a dynamic multi-delay causal graph is learned; Using the dynamic multi-latency causal graph as a structural prior, the causal spatiotemporal embedding of each microservice instance is obtained by aggregating context signals under different latencies from causal neighbors through a latency-aware spatiotemporal attention mechanism. Based on the causal spatiotemporal embedding, index prediction is performed, and based on the prediction residuals and the dynamic multi-delay causal graph, the abnormal parts explained by upstream propagation are deducted to generate a sorted list of root cause candidate instances and fault propagation paths.

[0013] In one embodiment of the present invention, obtaining runtime metric data and call chain data of each microservice instance includes: collecting infrastructure metrics and statistics obtained from the aggregation of distributed call chains, wherein the infrastructure metrics include CPU, memory and network I / O, and the statistics include request rate, error rate and latency quantile; The preprocessing of the data to obtain the multimodal input tensor includes: resampling the runtime metric data and call chain data to a uniform time interval, dividing them into windows of length L and step size s, and taking the mean of each channel within the window to obtain a time-aligned tensor; performing z-score normalization on the time-aligned tensor based on offline training set statistics, and using linear interpolation to fill in missing values ​​to obtain a normalized sequence for each microservice instance; mapping the normalized sequence to a latent vector in a compact health state space through lightweight learnable projection, and stacking the latent vectors of all microservice instances and all time steps to obtain the multimodal input tensor.

[0014] In one embodiment of the present invention, mapping the normalized sequence to a latent vector in a compact health state space via lightweight learnable projection includes: For each microservice instance i and time t, a multilayer perceptron is used to map the normalized sequence into a D-dimensional latent vector.

[0015] In one embodiment of the present invention, the dynamic multi-delay causal graph includes a discrete mask for indicating whether a fault propagation edge exists between an upstream instance and a downstream instance under different time delays, and dynamic weights for indicating the strength of the fault propagation edge; the learning of the dynamic multi-delay causal graph includes: By learning the structural skeleton, the system determines whether there are fault propagation edges between microservice instances based on the call chain topology and historical propagation patterns, and outputs a discrete mask. Through intensity adaptation, the influence strength of each candidate edge is estimated based on the co-fluctuation of the index and the current operating state, and dynamic weights are output. The discrete mask is multiplied element-wise with the dynamic weights to obtain a dynamic multi-delay causal graph.

[0016] In one embodiment of the present invention, the step of estimating the influence strength of each candidate edge based on index co-fluctuation and the current operating state through intensity adaptation, and outputting dynamic weights, includes: Dynamic low-rank parameterization driven by latent states is used to generate factor matrices shared across delays from two lightweight hypernetworks, and the complete matrix is ​​reconstructed from the product of these factor matrices to obtain dynamic weights.

[0017] In one embodiment of the present invention, the step of aggregating context signals under different delays from causal neighbors through a delay-aware spatiotemporal attention mechanism to obtain the causal spatiotemporal embedding of each microservice instance includes: The query vector, key vector, and value vector are calculated by employing a learnable relative delay embedding encoding time offset. Feature similarity and structural prior are both incorporated into the attention logits. Softmax is applied to all candidate upstream nodes and delays to obtain attention weights. The causal context vector is obtained by weighted summation of the value vectors. By fusing the causal context vector and the original latent state through a gated residual mechanism, a causal spatiotemporal embedding is output.

[0018] In one embodiment of the present invention, incorporating feature similarity and structural prior into attention logits includes: calculating attention logits according to the formula:

[0019] in, This represents the query vector of downstream instance i at time t. Let d represent the key vector of upstream instance j at time t-τ, where d is the vector dimension and γ is the structural prior weight. ε represents the causal influence strength from upstream instance j to downstream instance i at time t and delay τ, and ε is a numerically stable term.

[0020] In one embodiment of the present invention, the indicator prediction based on the causal spatiotemporal embedding includes: For each microservice instance and each metric, a fixed-length sliding window up to the current time t is taken and input into a shared Transformer time encoder to obtain the local pattern embedding; The instance-level causal state and the indicator-level local pattern are embedded and concatenated, and then input into the multilayer perceptron to predict the indicator value at the next time step.

[0021] In one embodiment of the present invention, the step of generating a ranked list of root cause candidate instances and a fault propagation path by subtracting the anomalous portion explained by upstream propagation based on the predicted residuals and the dynamic multi-delay causal graph includes: The dynamic multi-delay causal graph is aggregated into a single influence matrix according to exponential decay, so that short delays are given higher weights; Based on the anomaly scores and impact weights of upstream neighbors, calculate the expected propagation error for each microservice instance; calculate the final score for each microservice instance according to the formula:

[0022] in, Let i be the anomaly score of instance i at time t. Let β be the expected propagation error of instance i at time t, and β be the intensity of the propagation effect deduction. Starting from the high-ranking root cause candidates, a forward traversal is performed on the aggregated influence matrix, expanding along edges with high influence weights and exceptionally significant values ​​to construct a local propagation subgraph and obtain the fault propagation path.

[0023] In one embodiment of the present invention, obtaining runtime metric data and call chain data of each microservice instance includes: collecting infrastructure metrics and statistics obtained from the aggregation of distributed call chains, wherein the infrastructure metrics include CPU, memory and network I / O, and the statistics include request rate, error rate and latency quantile.

[0024] This invention presents a latency-aware spatiotemporal causal inference (STRCA) framework for microservice fault root cause analysis, proposing a framework for delay-aware STRCA. This framework models fault propagation using heterogeneous time delays, aligning upstream causes with delayed downstream symptoms. It also distinguishes between "whether one service causally affects another service" and "whether the metrics of two services fluctuate together," avoiding misjudging shared state changes as direct causal dependencies. STRCA also outputs interpretable propagation paths, helping operations personnel understand fault dynamics and take appropriate action.

[0025] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0026] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This diagram illustrates the propagation of delayed faults in a microservice system. Figure 2 A diagram illustrating the metrics and call chain during a microservice failure. Figure 3 A flowchart of a microservice fault root cause analysis method for delay-aware spatiotemporal causal inference provided in an embodiment of the present invention; Figure 4 The STRCA architecture diagram provided for embodiments of the present invention; Figure 5 This is a diagram of the STRCA deployment architecture provided in an embodiment of the present invention; Figure 6 This is a diagnostic result diagram of STRCA in a production accident provided by an embodiment of the present invention; Figure 7 This is a structural diagram of a microservice fault root cause analysis system for delay-aware spatiotemporal causal inference provided in an embodiment of the present invention. Detailed Implementation

[0027] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] The following describes, with reference to the accompanying drawings, a microservice fault root cause analysis method based on delay-aware spatiotemporal causal inference proposed according to an embodiment of the present invention.

[0030] Example 1 This embodiment provides a microservice fault root cause analysis method based on delay-aware spatiotemporal causal inference. For example... Figure 3 As shown, it includes: S1. Obtain runtime metrics data and call chain data for each microservice instance, and preprocess the data to obtain a multimodal input tensor.

[0031] Specifically, in the root cause analysis of a microservice system, it is first necessary to obtain runtime metric data continuously collected from each microservice instance during operation, as well as call chain data reflecting the inter-service call relationships. The runtime metric data includes, but is not limited to, performance metrics and resource usage metrics for each instance, such as request latency, throughput, error rate, CPU utilization, memory usage, and network input / output. The call chain data records the call edges, call directions, timestamps, and time consumption information formed when a request propagates among multiple service instances. Preprocessing of this multi-source heterogeneous data aims to eliminate differences in sampling frequency, time base, and units among different data sources, and integrate them into a unified input format.

[0032] Specifically, the preprocessing includes: resampling each data channel to a uniform time interval to align all indicators and call statistics on the time axis; dividing the continuous time series into fixed-length sliding windows and aggregating the observations of each channel within each window (e.g., taking the mean) to obtain a time-aligned multidimensional tensor; standardizing this tensor (e.g., z-score normalization based on statistics from the offline training set) to eliminate the influence of different indicator units; and interpolating to fill in any missing values ​​in the data (e.g., using linear interpolation) to ensure the integrity of the time series. After the above processing, a multimodal input tensor containing three dimensions—time step, instance dimension, and indicator channel—is finally formed. This tensor serves as the unified input basis for subsequent causal graph learning and spatiotemporal representation. As one implementation method, the original index sequence and call chain aggregate statistics of each instance can be resampled to a 1-minute interval, divided into windows of length L and step size s, and the mean value within the window is taken to obtain a time-aligned tensor. Then, the missing values ​​are filled in by z-score normalization and linear interpolation to obtain a standardized multimodal input tensor.

[0033] Through the above preprocessing steps, the original multi-source observable data can be transformed into time-aligned, dimensionally unified, and complete continuous multimodal input tensors, effectively eliminating time deviations and noise interference during data acquisition. This provides high-quality structured input for subsequent dynamic causal graph learning and delayed-aware spatiotemporal representation, thereby improving the robustness and accuracy of the entire root cause analysis framework.

[0034] S2, based on the multimodal input tensor, learn a dynamic multi-delay causal graph, the dynamic multi-delay causal graph including a discrete mask for indicating whether there is a fault propagation edge between upstream and downstream instances under different time delays, and dynamic weights for indicating the strength of the effect of the fault propagation edge.

[0035] Specifically, based on the multimodal input tensor, a dynamic multi-delay causal graph is learned. This dynamic multi-delay causal graph is used to characterize the fault propagation dependencies between microservice instances at different time delays. Specifically, this causal graph is defined collaboratively by two interrelated components: a discrete mask, used to indicate whether a fault propagation edge exists between upstream and downstream instances at a specific time delay, i.e., whether a causal influence is possible; and a dynamic weight, used to quantify the strength of the fault propagation edge at the current moment, i.e., the degree of influence of the upstream instance on the downstream instance. By modeling the existence of the structure (whether the edge exists) and the strength of the influence (how strong the edge is), this step can simultaneously capture the relatively stable call topology skeleton and the runtime interaction patterns that change over time in the microservice system.

[0036] In one implementation, a time-evolving factor matrix can be generated based on the instance latent states implied by the multimodal input tensor through a learnable low-rank parameterization method. A discretized mask matrix can be obtained using differentiable relaxation techniques (such as Gumbel-Sigmoid). At the same time, the learning process of the causal graph can be guided by combining topological priors (such as structural information obtained from the aggregation of call chain data) and regularization methods such as acyclic constraints, so that it both respects the physical topology and allows the discovery of hidden dependencies, thereby forming a set of dynamic causal graphs covering multiple time delays.

[0037] This step decouples discrete masks from dynamic weights, enabling joint modeling of structural stability and time-varying characteristics in microservice fault propagation. This allows for more accurate alignment of causal relationships between upstream causes and lagging downstream symptoms, avoiding misjudgments caused by synchronous fluctuations or fixed delay assumptions. Consequently, it provides reliable structural priors for subsequent delay-aware spatiotemporal representation learning and root cause inference, significantly improving the accuracy and interpretability of root cause localization.

[0038] S3, using the dynamic multi-delay causal graph as a structural prior, and through a delay-aware spatiotemporal attention mechanism, aggregate context signals under different delays from causal neighbors to obtain the causal spatiotemporal embedding of each microservice instance.

[0039] Specifically, after obtaining the dynamic multi-latency causal graph, it is used as a structural prior. Through a latency-aware spatiotemporal attention mechanism, contextual signals under different latency are aggregated from causal neighbors to obtain the causal spatiotemporal embedding of each microservice instance. Specifically, this step aims to address the temporal misalignment between upstream anomalies and downstream symptoms caused by time delays in microservice fault propagation. By explicitly modeling cross-service dependencies under different time delays, the model can align the state of upstream instances at earlier time points with the state of downstream instances at later time points.

[0040] To this end, we first obtain the latent state representation of each microservice instance at multiple time steps, which is obtained by instance embedding from the multimodal input tensor. Then, for the current time step of each downstream instance, we construct a query vector, and for multiple historical time steps of each candidate upstream instance, we construct key vectors and value vectors respectively. The key vector incorporates a learnable relative delay embedding to distinguish different propagation delays. Next, we combine feature similarity with the structural prior provided by the dynamic multi-delay causal graph to calculate an attention score, where the structural prior guides the model to prioritize upstream neighbors with causal propagation relationships and matching time delays. By normalizing all candidate upstream nodes and delays, we obtain attention weights, and then perform a weighted summation of the value vectors based on these weights to aggregate a causal context vector for each instance at each time step. Finally, we fuse this causal context vector with the original latent state through a gated residual mechanism, enabling each instance to adaptively determine the amount of information absorbed from causal neighbors and the degree to which it retains its own dynamic information, thereby outputting a causal spatiotemporal embedding.

[0041] As one implementation method, a relative delayed embedding can be learned, and feature similarity and structural prior can be put into attention logits. Attention weights are obtained through softmax, and the causal context vector is obtained by weighted summation of the value vector. Finally, the causal spatiotemporal embedding is output through gated residual fusion.

[0042] This step achieves explicit modeling of asynchronous, multi-delay fault propagation through a delay-aware spatiotemporal attention mechanism. This enables the model to accurately align upstream root causes with lagging downstream symptoms in time, thereby effectively improving the accuracy of root cause localization and avoiding the problem of downstream affected services being over-ordered due to time misalignment.

[0043] S4. Based on the causal spatiotemporal embedding, perform index prediction, and according to the prediction residual and the dynamic multi-delay causal graph, deduct the abnormal part explained by upstream propagation to generate a sorted list of root cause candidate instances and fault propagation paths.

[0044] Specifically, after obtaining the causal spatiotemporal embedding of each microservice instance, this step further utilizes the embedding to predict metrics, and performs source tracing analysis on anomalies based on the prediction results and the dynamic multi-latency causal graph.

[0045] Specifically, firstly, based on the aforementioned causal spatiotemporal embedding, a predictor is used to predict the indicator value for the next time step. This predictor is typically a learnable mapping function, taking the current causal spatiotemporal embedding as input and outputting an estimate of the future indicator value. During model training, the predictor is trained using historical data under normal conditions as a benchmark, ensuring that the residual between the predicted and actual observations is small when the system is running normally. However, when the system malfunctions, abnormal service instances will generate larger prediction residuals, reflecting the degree of anomaly of the instance that cannot be explained by historical patterns. Building upon this, and combined with the aforementioned dynamic multi-delay causal graph, upstream propagation explanations are further performed for the anomalies of each instance.

[0046] Specifically, using the fault propagation edges and their strengths indicated in the dynamic multi-delay causal graph, the propagation anomaly component that each instance may receive from its upstream neighbors is calculated; that is, the expected contribution of anomalies from upstream instances to the current instance through causal edges. Subsequently, this propagation anomaly component is subtracted from the instance's original prediction residual to obtain the upstream-adjusted anomaly score. This adjusted score reflects the degree of endogenous anomaly remaining in the instance after excluding the upstream propagation effect; a higher score indicates that the instance is more likely to be the root cause of the fault. Finally, all instances are sorted according to their adjusted anomaly scores to generate a ranked list of root cause candidate instances. Simultaneously, starting from the high-ranking root cause candidate, a forward traversal is performed along the causal edges with high strengths in the dynamic multi-delay causal graph to construct the fault propagation path from the root cause to the affected service.

[0047] In one implementation, the predictor can be implemented using a multilayer perceptron, the calculation of the upstream propagation anomaly component can be achieved by weighted aggregation of the anomaly score of the upstream instance and the influence strength of the causal edge, and the construction of the fault propagation path can limit the path length and the number of branches to maintain interpretability.

[0048] This step, by combining the indicator prediction residuals with a dynamic cause-effect graph and deducting the abnormal contributions from upstream propagation, can effectively distinguish between endogenous faults and propagational anomalies, preventing downstream affected services from being misjudged as root causes, thereby significantly improving the accuracy and reliability of root cause localization. At the same time, the generated fault propagation path provides operations and maintenance personnel with an intuitive view of fault spread, which helps to quickly understand the fault dynamics and take targeted measures.

[0049] Example 2 This invention proposes a latency-aware spatio-temporal causal inference framework (STRCA) for microservice root cause analysis. The overall architecture includes a data preprocessing stage and three core functional modules across two main processes: offline training and online diagnosis. (See attached...) Figure 3 The data processing flow shown in the figure first receives the indicator data and call chain data collected during the operation of the microservice instance as raw input. After data preprocessing, a multimodal input tensor is generated. Then, through the instance embedding module, the multimodal information is uniformly converted into the potential health status of each microservice instance, providing a unified basic representation for subsequent causal modeling, spatiotemporal feature learning and anomaly prediction.

[0050] Understandably, Appendix Figure 4 In this dataset, the indicator data refers to the performance and resource monitoring data continuously collected during the operation of each microservice instance, including request latency, throughput, error rate, CPU utilization, memory usage, network I / O, etc.; the call chain data records the call relationships, call directions, timestamps, and link time information formed by a single request flowing through the microservice cluster. After the two types of raw data are input into the data preprocessing module, they undergo standardization processes such as sampling frequency alignment, time window division, missing value completion, data normalization, and call edge aggregation, ultimately generating a unified format multimodal input tensor, realizing the transformation of raw observation data into model-usable data. The instance embedding module receives the preprocessed multimodal input tensor and compresses and maps the multidimensional indicator features and call statistics features corresponding to each microservice instance within each time window into a low-dimensional instance-level potential health state. This health state also integrates the instance's own indicator change features and cross-service call context information, serving as a unified input for subsequent causal graph learning, spatiotemporal representation learning, and anomaly prediction. The offline training phase mainly includes multi-latency causal graph construction, latency-aware spatiotemporal feature learning, and prediction model training.

[0051] The multi-delay causal graph learning module consists of a structural skeleton learning unit and an intensity adaptive learning unit. The structural skeleton learning unit outputs a discrete mask based on the call chain topology and historical fault propagation patterns to characterize the existence of fault propagation edges between instances. The intensity adaptive learning unit outputs dynamic weights based on the index co-fluctuation characteristics and real-time operating status to quantify the influence intensity of fault propagation edges. The discrete mask and dynamic weights are fused to form a multi-delay causal graph, used to characterize the fault propagation dependency of upstream instances on downstream instances under different time delays. The model introduces current time and historical time step identifiers to achieve temporal alignment between early upstream abnormal states and lagging downstream abnormal states, accurately characterizing the asynchronous propagation characteristics of multiple delays. The delay-aware cross-node attention module takes the potential health status of instances and the multi-delay causal graph as input. During the attention calculation process, it integrates the causal graph structure prior and prioritizes aggregating upstream neighbor features with real causal relationships and matching time delays. The index temporal encoder extracts local time pattern features such as trends, cycles, and mutations for each index sequence of a single instance. The two types of features are merged by a fusion layer and input into the predictor to complete the prediction of key performance indicators and the training of the anomaly scoring model, establishing a prediction benchmark under normal operating conditions. The online diagnostic phase utilizes a model trained offline to perform anomaly identification, causal propagation analysis, and root cause localization in real time. The system first calculates the residuals between the actual and predicted metrics for each instance, generating initial anomaly scores. Then, a graph aggregation module merges causal graphs with different latency dimensions into a unified fault propagation impact graph. Next, an upstream impact correction module removes anomaly components that can be explained by upstream fault propagation, eliminating interference from amplified anomalies in downstream affected services. Finally, it outputs a priority-ranked list of root cause candidate instances and an explainable fault propagation path from the suspected root cause to the affected service.

[0052] Specifically, in Figure 4, the metric data consists of performance and resource metrics continuously collected during the operation of each microservice instance, including latency, QPS, error rate, CPU, memory, and network I / O; the call chain data records the call edges, call directions, timestamps, and execution times formed during request propagation. Both types of data enter the preprocessing module, which performs sampling frequency unification, time window segmentation, missing value imputation, normalization, and Trace call edge aggregation, ultimately forming a multimodal input tensor X. The instance embedding module receives the multimodal input tensor X and compresses the multidimensional metrics and call statistics of each instance within each time window into a unified instance-level potential health state H. This state simultaneously includes metric change characteristics and call context, providing a unified input for subsequent causal graph learning, spatiotemporal representation learning, and anomaly prediction. During the offline training phase, a multi-delay causal graph learner is used, consisting of two parts: structural skeleton learning and intensity adaptation. Structural skeleton learning outputs a discrete mask M, used to determine the existence of fault propagation edges between instances. Intensity adaptation outputs dynamic weights W, used to characterize the strength of edge influence. M and W are fused to form a multi-delay causal graph A, used to represent the fault propagation dependencies between services under different time delays. The model incorporates the current time t and historical time t. 1 and delay time t τ is used to achieve temporal alignment between early upstream anomalies and late downstream anomalies. The delayed-aware spatiotemporal representation learning unit (LTV) receives instance states H and causal graphs A via a delayed-aware cross-node attention mechanism. When calculating attention, it integrates causal priors, prioritizes aggregating delayed-matching upstream causal neighbor signals, and outputs the causal context H_out. The index temporal encoder extracts single-index local time patterns and outputs index patterns E. H_out and E are fused and input into the predictor to predict the index value X_{t+1} at the next time step. KPI prediction and anomaly scoring training are completed using the loss function L. In the online diagnostic phase, root cause inference is performed: first, an instance anomaly score r is generated based on the prediction residual; then, multiple delayed causal graphs are aggregated into a propagation influence graph, and the upstream explainable anomaly component p is calculated; p is subtracted through upstream correction to obtain the corrected anomaly score s; finally, the root cause ranking list and explainable propagation chain are output. In summary, the complete data flow is as follows: Metric / Trace → Preprocessing → Tensor X → Instance embedding → Health state H → Backbone M and weights W → Causal graph A → Delayed perceptual attention → H_out, E → Fusion prediction → X_{t+1}, L → Online residual r, upstream component p, correction score s → Root cause ranking and propagation chain.

[0053] In summary, the complete data processing flow of this invention is as follows: indicator data and call chain data are preprocessed to generate multimodal input tensors, and the potential health status of instances is obtained through instance embedding; a multi-delay causal graph is constructed by a structural skeleton and dynamic weights; spatiotemporal feature learning is completed through a delay-aware attention mechanism and a temporal encoder; the fused features are output by a predictor to predict the indicator results and complete model training; in the online stage, based on the prediction residuals, causal graph aggregation, and upstream correction, accurate root cause ranking and interpretable propagation chains are generated to achieve efficient and reliable diagnosis of faults in large-scale microservice systems.

[0054] Example 3 This embodiment proposes a belief-state-based abductive reasoning method for computer system fault diagnosis, including Data Preprocessing and Unified Representation. The specific steps are as follows: TRCA uses microservice instances as the basic unit of analysis. For each instance i in the system, it comprehensively collects two types of key operational data: one is infrastructure metrics, including underlying resource parameters such as CPU utilization, memory usage, and network input / output traffic; the other is operational statistics aggregated from distributed call chain trace data, covering business performance metrics such as request rate, error rate, and latency quantiles. To ensure the consistency and comparability of time-series data, all data channels are uniformly resampled to the same time interval, and time windows are divided according to a preset window length L and sliding step s. Within each time window, the mean of each data channel is calculated, thereby obtaining the time-aligned tensor. Where T is the number of time steps, N is the number of instances, and M is the metric dimension. Subsequently, based on the statistical characteristics of the offline training dataset, z-score normalization is performed on the tensor X to eliminate differences in the dimensions of different metrics; simultaneously, linear interpolation is used to fill in missing values ​​in the time series, ultimately obtaining the standardized time series sequence corresponding to each instance. This ensures the integrity and standardization of the data.

[0055] Directly conducting causal relationship discovery in the high-dimensional original index space is not only extremely computationally inefficient, but also suffers from significant robustness defects: on the one hand, the causal search space increases with the product of the number of instances N and the index dimension M. The exponential growth of high-dimensional data significantly increases computational overhead. Furthermore, the noise, redundant indicators, and invalid features mixed in high-dimensional data can easily obscure the true fault propagation relationships between services, leading to distorted causal discovery results. To address this, STRCA introduces a lightweight learnable projection mechanism to compress high-dimensional observation data into a low-dimensional, compact health state space: for any instance i at time t, a multilayer perceptron with parameter θ is used... Standardized time series Mapped to a D-dimensional vector of potential health states Furthermore, all instances and the latent vectors corresponding to all time steps are stacked according to their dimensions to form a unified feature tensor. As Figure 2 The input is the same as the input for the preprocessing module and the offline training module.

[0056] This embodiment effectively eliminates interference from differences in data dimensions, temporal misalignment, and missing values ​​by performing standardized preprocessing, temporal alignment, and normalization on multi-source runtime data of microservice instances, ensuring the standardization, integrity, and consistency of input data. Simultaneously, it employs a lightweight, learnable projection mechanism to compress high-dimensional raw indicators into a low-dimensional unified health status representation, significantly reducing the search space and computational overhead for causal discovery. This avoids the problem of high-dimensional noise and redundant features obscuring the true fault propagation relationship, improving the computational efficiency, robustness, and accuracy of subsequent causal graph learning, spatiotemporal feature modeling, and root cause analysis. This provides a reliable data foundation and feature support for efficient fault diagnosis of large-scale microservice systems.

[0057] Example 4 This embodiment proposes a belief-state-based abductive reasoning method for computer system fault diagnosis, namely Dynamic Multi-Lag Causal Graph Learning, as detailed below: Microservice failures propagate across services with varying and time-varying latency, making a single static adjacency matrix insufficient to adequately describe dependencies. STRCA models potential system dependencies as a set of multi-latency causal graphs. Each of them Describe delay The following causal relationship This indicates that upstream node j at time t corresponds to downstream node i at time t. The direct impact.

[0058]

[0059] in Indicates whether a cross-instance edge exists. This indicates the strength of the edge's effect. It represents a relatively stable call structure. runtime statistics dependency Separation allows the model to both follow the physical topology and adapt to time-varying interaction patterns.

[0060] To reduce parameter size, STRCA uses dynamic low-rank parameterization driven by the latent state H. At each time t, two lightweight hypernetworks generate a factor matrix shared across their respective delays. and use Reconstruct the complete matrix, reducing the complexity from Down to During training, Gumbel-Sigmoid relaxation is used to obtain a differentiable binary skeleton.

[0061] Meanwhile, the model incorporates two structural regularization terms: one is a topological prior from the aggregated trace, which encourages edges to be consistent with the observation call path and allows the discovery of hidden dependencies when the trace is missing; the other is a NOTEARS-style acyclic constraint, which imposes a DAG constraint on the aggregated causal graph, making the propagation path more interpretable.

[0062] This invention addresses the cross-service propagation characteristics of microservice faults with multiple time intervals and dynamic delays. It constructs a multi-delay causal graph, overcoming the limitations of static adjacency matrices to accurately characterize causal dependencies between upstream and downstream services under different delays. By decoupling the structural skeleton from the strength weights, it effectively distinguishes between true causal dependencies and non-causal fluctuations, while simultaneously considering physical topology stability and runtime interaction dynamism. The introduction of low-rank parameterization and dynamic hypernetwork generation reduces model complexity, significantly decreasing the number of parameters and computational overhead, thus improving training and inference efficiency. Furthermore, by combining Gumbel-Sigmoid relaxation to achieve differentiability of the binary skeleton and applying topological prior regularization and acyclic DAG constraints, it ensures that the causal structure conforms to the actual call logic and can discover hidden dependencies, while also ensuring that the propagation path is acyclic and highly interpretable. This significantly improves the accuracy, robustness, and interpretability of causal modeling in complex and dynamic microservice scenarios.

[0063] Example 5 This embodiment proposes a belief-state-based abductive reasoning method for computer system fault diagnosis, namely Lag-Aware Spatio-Temporal Representation Learning, as detailed below: Specifically, after obtaining the multi-delay causal graphs, the model uses them as structural priors to guide the spatiotemporal information flow of delay awareness, avoiding the mixing of signals on misaligned delays. Let... Indicates the potential instance state. This represents the state of instance i at time t. To encode the time offset, the model employs a learnable relative delay embedding. And calculate query, key, and value:

[0064] Where h_{t,i} represents the potential health state of instance i at time t; , , These represent the query vector, key vector, and value vector, respectively. , , These are the corresponding learnable linear projection matrices; Let τ represent the learnable relative delay embedding used to distinguish different propagation delays; j represents the upstream candidate instance, and i represents the current downstream instance; t τ represents the state of the upstream instance at an earlier time step.

[0065] Then, feature similarity and structural prior are both incorporated into the attention logits:

[0066] Where Logits represents attention score; the superscript τ represents propagation delay; and the subscripts t,i,j represent the calculation of the attention relationship between downstream instance i and upstream instance j at time t. and These are the query vector and the key vector, respectively; d is the vector dimension, used to scale the dot product; γ is the structural prior weight, used to control the influence of the edge strength of the causal graph on the attention score; This represents the causal influence strength from upstream instance j to downstream instance i at time t and delay τ; ε is a numerically stable term to avoid invalid values ​​in logarithmic operations. Controlling structural bias strength This is a numerically stable term. The model performs a softmax operation on all candidate upstream nodes and delays to obtain attention weights, and then performs a weighted sum on the value vector to obtain the causal context vector for each instance at each time step. .

[0067] To mitigate the oversmoothing of deep graph models, STRCA uses gated residual connections to fuse causal context and the original latent state. The gating mechanism allows each instance to decide how much information to absorb from its causal neighbors and how much of its own dynamics to retain, outputting... As a causal spatiotemporal embedding for subsequent prediction and anomaly scoring.

[0068] For instance i and index m, the model takes a fixed-length sliding window with a cutoff time t, and inputs a shared Transformer temporal encoder to obtain the local pattern embedding. When predicting the next indicator value, the instance-level causal state is concatenated with the indicator-level embedding and input into the MLP. :

[0069] Predicting losses Defined as predicted value Compared with the true value The mean square error between them is calculated and averaged over the normal training window.

[0070] The overall objective function combines prediction loss, sparsity, and topological regularization terms. and DAG acyclic constraints :

[0071] This invention employs a multi-delay causal graph as a structural prior to guide precise alignment of spatiotemporal information flows, effectively avoiding interference from mixed signals of different delays. Through learnable relative delay embedding and a delay-aware attention mechanism, it simultaneously integrates feature similarity and causal structure information to accurately capture cross-service delay dependencies, significantly improving the accuracy and relevance of spatiotemporal feature modeling. The introduction of gated residual connections alleviates the oversmoothing problem of deep graph models, enabling adaptive fusion of causal context and original state, enhancing the model's ability to preserve local dynamics. Joint encoding of instance-level causal states and indicator-level temporal features is used, balancing global causal propagation and local indicator patterns to improve anomaly prediction accuracy. Through multi-constraint joint optimization, while ensuring prediction accuracy, it strengthens the sparsity, topological consistency, and acyclic interpretability of the causal graph, ultimately achieving delay-aware, structurally reliable, feature-effective, and prediction-accurate spatiotemporal causal modeling of microservice faults, significantly improving the reliability and interpretability of root cause analysis.

[0072] Example 6 This embodiment presents a belief-state-based abductive reasoning method for computer system fault diagnosis, proposing an upstream-adjusted root cause inference, as detailed below: Specifically, after training convergence, STRCA enters the online diagnostic phase. At each time window t, the model uses the prediction head to obtain the residuals for each indicator and aggregates them into an instance-level anomaly score vector. Meanwhile, the dynamic multi-delay causal graph provides the causal dependency structure between current instances.

[0073] If sorted directly by reconstruction error, downstream "victim" services might be ranked ahead of the true root cause due to amplified anomalies. STRCA therefore explicitly subtracts the bias that can be explained by upstream propagation. The model first aggregates multiple delay graphs into a single influence matrix based on exponential decay. This gives higher weight to short latency; then, based on the anomaly score and influence weight of upstream neighbors, the expected propagation error of each instance i is calculated. The final score is:

[0074] in The intensity of the deduction for the propagation effect is controlled. If the deviation of instance i can be largely explained by upstream anomalies, then... near Adjusted score If the residual is relatively small, then i is more likely to be the true root cause; if there is still a large residual, then i is more likely to be the true root cause.

[0075] In addition to root cause ranking, STRCA also starts from high-ranking root cause candidates, performs a forward traversal on the aggregated influence matrix, and expands along edges with high influence weights and exceptionally significant values ​​to construct a local propagation subgraph, for example... In actual deployments, path length and the number of branches are limited to keep the interpretation results readable and the online overhead manageable.

[0076] This embodiment enters the online diagnostic phase after model training convergence. It generates instance-level anomaly scores by aggregating predicted residuals to accurately quantify the anomaly severity of each microservice instance. Addressing the issue that direct sorting by residuals can easily lead to misjudgment of downstream affected services, an explicit upstream impact correction mechanism is proposed. This mechanism uses exponential decay aggregation of multi-latency causal graphs to form a unified impact matrix, effectively highlighting the dominant role of short-latency fault propagation. Based on this matrix, the interpretable propagation error of the upstream is calculated and dynamically subtracted, eliminating the sorting bias caused by downstream anomaly amplification and significantly improving the priority of identifying the true root cause. Simultaneously, it supports generating interpretable fault propagation paths starting from high-confidence root cause candidates. By limiting path length and the number of branches, it balances interpretability and online inference efficiency, enabling rapid, accurate, and interpretable fault location in large-scale microservice systems and significantly reducing operational and maintenance troubleshooting costs.

[0077] Example 7 This embodiment evaluates a belief-state-based abductive reasoning method for computer system fault diagnosis by addressing three questions: RQ1: Is STRCA superior to existing microservice root cause localization methods? RQ2: How much do each key component contribute to overall performance? RQ3: Is STRCA suitable for deployment in terms of computational cost and hyperparameter robustness?

[0078] Experiment Setup includes the following settings: Dataset D1 comes from a production-grade high-fidelity microservice system on the cloud infrastructure of a leading commercial bank. It contains 46 microservice instances and five common infrastructure failures: CPU anomalies, memory exhaustion, service interruption, storage capacity issues, and I / O contention. Dataset D2 is based on the open-source benchmark Online Boutique, deployed in a Kubernetes cluster. It contains 41 microservice instances and injects failures related to network, storage, resource pressure, Pod lifecycle, JVM runtime, and application logic. See Table 1 for details. Table 1

[0079] Evaluation indicators adopted and . This represents the probability that the true root cause appears in the top k candidates recommended by the method. Represents numbers from 1 to k average value.

[0080]

[0081]

[0082] Baseline methods include BARO, DyCause, MicroCause, RCD, CauseLens, and CausalRCA. Implementations were performed using Python 3.9.7, PyTorch 2.1.0, scikit-learn 1.0.2, and DGL 0.9.0. The experimental server was equipped with dual Intel Xeon Gold 5416S CPUs, 376GB of RAM, and eight NVIDIA RTX A6000 GPUs.

[0083] Among them, RQ1: The Overall Performance: Table 2 compares the root cause localization accuracy and time cost between D1 and D2. Train(s) is the total training time, and Test(s) is the average inference time per incident.

[0084] Table 2

[0085] Table 3 shows the performance of D1 categorized by fault type, as shown in Table 3: Table 3

[0086] Table 2 shows that STRCA consistently outperforms the baseline method on both datasets and all metrics, especially in... and The clear advantage indicates that it can more reliably prioritize the true root causes. The results in D1, categorized by fault type, also show that STRCA maintains a high score across all five fault categories. The advantage is not driven by a single type of failure.

[0087] Among them, the RQ2: Ablation Study ablation experiment is shown in Table 4: Table 4

[0088] Specifically, c1 removes causal structure learning and propagates only on the static call graph; c2 cancels structure-strength decoupling and learns a single dynamic adjacency matrix instead; c3 replaces L-STA with standard-time GCN; and c4 removes upstream adjustment and sorts only by the original reconstruction error. The results show that the four modules complement each other in delay cascading fault modeling, and removing any one module degrades performance.

[0089] Among them, RQ3: Hyperparameters Sensitivity. The delay window size K and the upstream adjustment coefficient were examined. Sparse regularization weights STRCA remains stable over a wide parameter range. If K is too small, it cannot cover the delay propagation; if it is too large, it will introduce historical noise. More suitable; When the value is moderate, it can suppress downstream false alarms; if it is too small, it degenerates into sorting by reconstruction error; if it is too large, it will overcorrect the true root cause score. exist arrive It is relatively stable within the range.

[0090] Example 8 This embodiment proposes a Deployment method based on belief states for computer system fault diagnosis, as described above. Figure 5 As shown: STRCA was deployed on the observable platform for three months, using the same GPU server configuration as the experiment. The deployment adopted a two-stage process: in the offline stage, the model was updated regularly using metrics and traces from the past three days to enable multi-latency causal graphs to track recent dependency changes; in the online stage, after a high-severity business alarm was triggered, the system connected real-time metrics and trace samples, ran STRCA, and output the root cause service ranking and candidate propagation chains.

[0091] Before deployment, complex cascading failures typically required 3–5 senior SREs across teams to review logs and dashboards, with an average root cause identification time exceeding 30 minutes. After deployment, the end-to-end latency from alarm to root cause ranking was less than 0.9 seconds; online inference consumed less than 10% of a CPU socket and 5% of a GPU. Over three months, the system automatically handled hundreds of incidents per month, based on experience gained from post-incident tagging and cross-validation. More than 80%.

[0092] Furthermore, Case Study analysis: like Figure 6 The image shows STRCA's diagnostic results in a production incident, a case from a real cascading failure in an online ticketing microservice system. The incident originated from a memory leak in the upstream Recommendation Service C, manifested as frequent Full GC, starting at 09:07. Due to middleware buffering and retry mechanisms, this disturbance did not immediately trigger an alert, but propagated to downstream Product Services B and D after 3–5 minutes, causing a surge in QPS, and ultimately resulting in a sharp drop in the success rate of Frontend Service A at 09:13.

[0093] STRCA accurately identifies Service C as significantly impacting Service B after a specific delay and further inducing anomalies in Service A through delay-aware causal inference and upstream impact calibration. For the drastic fluctuations in downstream B's metrics, the algorithm calculates and subtracts the expected error component from upstream C, thereby reducing B's endogenous anomaly score; while the anomaly of C cannot be explained by other nodes, thus its characteristics as an endogenous fault source are highlighted, ultimately ranking first in the root cause list.

[0094] Furthermore, Lessons Learned experience summary: Topology-guided learning: In online microservice systems, canary deployments and scheduling policy updates frequently alter the topology and propagation latency. Early prototypes treated call relationships and historical latency patterns as hard constraints, resulting in a significant performance degradation after topology changes. A more robust approach is to treat the topology and latency patterns as soft priors, used to define the search space and regularization, while allowing for moderate deviations to capture hidden dependencies.

[0095] Interpretability of Results: Returning only a single suspected root cause service and its confidence level is insufficient to support SRE decision-making. SREs are more concerned with where the fault originated, along which service paths it propagated, and how long it took to reach the current alarm point. Therefore, the system was changed to output a ranking of candidate propagation chains, along with a time-aligned view of key KPIs along the path.

[0096] Process Integration: Even with high offline accuracy, the RCA system must seamlessly integrate into existing incident management processes. STRCA outputs are integrated into raw alarms on the observable platform, along with suspected root causes, propagation chains, and brief summaries, facilitating SRE citation in work orders and debriefing reports.

[0097] Single-dimensional methods are still widely used in microservice root cause analysis. For example, BALANCE and BARO use Bayesian modeling and change point detection to locate root causes from multivariate metrics; PDiagnose combines KPIs, logs, and traces for unsupervised detection and rule-based diagnosis; Microscope, MS-Rank, and DyCause construct dependency or causal graphs from traces and metrics, and identify root causes through propagation analysis or reverse tracing. These methods are effective in practice, but they typically model temporal dynamics and structural propagation separately, making them susceptible to noise and hidden dependencies.

[0098] It is known that joint spatiotemporal methods integrate temporal dynamics with service topology or causal structure into a unified framework, such as ReconRCA, CauseLens, and CausalRCA. However, these methods still do not explicitly align temporal interactions with graph reasoning, limiting their ability to capture cross-service latency dependencies and distinguish between synchronization metric fluctuations and true causal propagation. This invention proposes STRCA, a latency-aware spatiotemporal causal inference framework for large-scale microservice incident diagnosis. STRCA uses multimodal observable data from metrics and traces to jointly learn a multi-latency causal graph and a latency-aware propagation encoder, decoupling the physical topology derived from traces from the interaction strength derived from metrics, improving instance-level localization accuracy, and providing a path-level explanation consistent with SRE's understanding of fault propagation in microservice call chains.

[0099] Example 9 To implement the methods of the above embodiments, the present invention also provides a microservice fault root cause analysis system 10 with delay-aware spatiotemporal causal inference, such as... Figure 7 As shown, it includes: The data preprocessing module 100 is used to obtain runtime metric data and call chain data of each microservice instance, and preprocess the data to obtain multimodal input tensors; The dynamic learning module 200 is used to learn a dynamic multi-delay causal graph based on the multimodal input tensor. The latency-aware module 300 is used to obtain the causal spatiotemporal embedding of each microservice instance by aggregating context signals under different latency from causal neighbors through the latency-aware spatiotemporal attention mechanism, using the dynamic multi-latency causal graph as a structural prior. The fault root cause confirmation module 400 is used to predict indicators based on the causal spatiotemporal embedding, and generate a sorted list of root cause candidate instances and fault propagation paths by deducting the abnormal parts explained by upstream propagation according to the prediction residuals and the dynamic multi-delay causal graph.

[0100] The microservice fault root cause analysis system with delayed perception spatiotemporal causal inference in this invention improves the accuracy of root cause localization, reduces the workload of manual investigation, and provides an interpretable propagation path.

[0101] It should be noted that the explanation of the aforementioned embodiment of the microservice fault root cause analysis method based on delay-aware spatiotemporal causal inference also applies to the microservice fault root cause analysis system based on delay-aware spatiotemporal causal inference in this embodiment, and will not be repeated here.

[0102] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0103] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0104] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A microservice fault root cause analysis method based on delay-aware spatiotemporal causal inference, characterized in that, include: Obtain runtime metrics and call chain data for each microservice instance, and preprocess the data to obtain a multimodal input tensor; Based on the multimodal input tensor, a dynamic multi-delay causal graph is learned; Using the dynamic multi-latency causal graph as a structural prior, the causal spatiotemporal embedding of each microservice instance is obtained by aggregating context signals under different latencies from causal neighbors through a latency-aware spatiotemporal attention mechanism. Based on the causal spatiotemporal embedding, index prediction is performed, and based on the prediction residuals and the dynamic multi-delay causal graph, the abnormal parts explained by upstream propagation are deducted to generate a sorted list of root cause candidate instances and fault propagation paths.

2. The method as described in claim 1, characterized in that, The process of obtaining runtime metrics data and call chain data for each microservice instance includes: collecting infrastructure metrics and statistics obtained from the aggregation of distributed call chains. The infrastructure metrics include CPU, memory, and network I / O, and the statistics include request rate, error rate, and latency quantile. The preprocessing of the data to obtain the multimodal input tensor includes: resampling the runtime metric data and call chain data to a uniform time interval, dividing them into windows of length L and step size s, and taking the mean of each channel within the window to obtain a time-aligned tensor; performing z-score normalization on the time-aligned tensor based on offline training set statistics, and using linear interpolation to fill in missing values ​​to obtain a normalized sequence for each microservice instance; mapping the normalized sequence to a latent vector in a compact health state space through lightweight learnable projection, and stacking the latent vectors of all microservice instances and all time steps to obtain the multimodal input tensor.

3. The method as described in claim 2, characterized in that, The step of mapping the normalized sequence to a latent vector in a compact health state space via lightweight learnable projection includes: For each microservice instance i and time t, a multilayer perceptron is used to map the normalized sequence into a D-dimensional latent vector.

4. The method as described in claim 1, characterized in that, The dynamic multi-delay cause-effect graph includes a discrete mask for indicating whether there is a fault propagation edge between upstream and downstream instances under different time delays, and a dynamic weight for indicating the strength of the effect of the fault propagation edge. The learning dynamic multi-delay causal graph includes: By learning the structural skeleton, the system determines whether there are fault propagation edges between microservice instances based on the call chain topology and historical propagation patterns, and outputs a discrete mask. Through intensity adaptation, the influence strength of each candidate edge is estimated based on the co-fluctuation of the index and the current operating state, and dynamic weights are output. The discrete mask is multiplied element-wise with the dynamic weights to obtain a dynamic multi-delay causal graph.

5. The method as described in claim 4, characterized in that, The process involves intensity adaptation, estimating the influence strength of each candidate edge based on index co-fluctuation and the current operating state, and outputting dynamic weights, including: Dynamic low-rank parameterization driven by latent states is used to generate factor matrices shared across delays from two lightweight hypernetworks, and the complete matrix is ​​reconstructed from the product of these factor matrices to obtain dynamic weights.

6. The method as described in claim 1, characterized in that, The process of aggregating context signals under different delays from causal neighbors through a latency-aware spatiotemporal attention mechanism to obtain the causal spatiotemporal embedding of each microservice instance includes: The query vector, key vector, and value vector are calculated by employing a learnable relative delay embedding encoding time offset. Feature similarity and structural prior are both incorporated into the attention logits. Softmax is applied to all candidate upstream nodes and delays to obtain attention weights. The causal context vector is obtained by weighted summation of the value vectors. The causal context vector and the original latent state are fused through a gated residual mechanism to output a causal spatiotemporal embedding.

7. The method as described in claim 6, characterized in that, The step of incorporating both feature similarity and structural prior into attention logits includes: calculating attention logits according to the formula: in, This represents the query vector of downstream instance i at time t. Let d represent the key vector of upstream instance j at time t-τ, where d is the vector dimension and γ is the structural prior weight. ε represents the causal influence strength from upstream instance j to downstream instance i at time t and delay τ, and ε is a numerically stable term.

8. The method as described in claim 1, characterized in that, The indicator prediction based on the causal spatiotemporal embedding includes: For each microservice instance and each metric, a fixed-length sliding window up to the current time t is taken and input into a shared Transformer time encoder to obtain the local pattern embedding; The instance-level causal state and the indicator-level local pattern are embedded and concatenated, and then input into the multilayer perceptron to predict the indicator value at the next time step.

9. The method as described in claim 1, characterized in that, The step of generating a ranked list of root cause candidate instances and fault propagation paths by subtracting the anomalous portion explained by upstream propagation based on the predicted residuals and the dynamic multi-delay causal graph includes: The dynamic multi-delay causal graph is aggregated into a single influence matrix according to exponential decay, so that short delays are given higher weights; Calculate the expected propagation error for each microservice instance based on the anomaly scores and impact weights of upstream neighbors; calculate the final score for each microservice instance using the formula: in, Let i be the anomaly score of instance i at time t. Let β be the expected propagation error of instance i at time t, and let β be the intensity of the propagation effect deduction. Starting from the high-ranking root cause candidates, a forward traversal is performed on the aggregated influence matrix, expanding along edges with high influence weights and exceptionally significant values ​​to construct a local propagation subgraph and obtain the fault propagation path.

10. A microservice fault root cause analysis system with delay-aware spatiotemporal causal inference, characterized in that, include: The data preprocessing module is used to obtain runtime metric data and call chain data of each microservice instance, and preprocess the data to obtain multimodal input tensors; The dynamic learning module is used to learn a dynamic multi-delay causal graph based on the multimodal input tensor. The latency-aware module is used to aggregate context signals under different latency from causal neighbors through the latency-aware spatiotemporal attention mechanism, using the dynamic multi-latency causal graph as a structural prior, to obtain the causal spatiotemporal embedding of each microservice instance. The fault root cause confirmation module is used to predict indicators based on the causal spatiotemporal embedding, and generate a sorted list of root cause candidate instances and fault propagation paths by deducting the abnormal parts explained by upstream propagation according to the prediction residuals and the dynamic multi-delay causal graph.

Images