System abnormity intelligent diagnosis and recovery method and system fusing time sequence logs

Abstract

The invention provides a system abnormity intelligent diagnosis and recovery method and system fusing time sequence logs, and relates to the field of distributed system operation and maintaining.The method comprises the steps that operation logs are collected from a plurality of service nodes of a distributed system, time sequence alignment is conducted, and calling relation and performance measurement data are extracted; time sequence change features are calculated in the sliding time window, dynamic feature vectors are generated through fusion coding, and a cross-service time sequence association graph is constructed; identifying an abnormal service node and matching the abnormal service node with historical fault data to obtain a historical propagation path and a feature vector; calculating a propagation convergence coefficient and a characteristic deviation degree based on a historical propagation path to obtain a causal intensity score; and identifying a fault influence degree according to the score, and adaptively adjusting a resource isolation and flow scheduling strategy. According to the invention, accurate diagnosis and efficient recovery of distributed system faults are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to distributed system operation and maintenance technology, and in particular to a system anomaly intelligent diagnosis and recovery method and system fusing time sequence logs. BACKGROUND

[0002] With the wide application of distributed systems, intelligent fault diagnosis and recovery has become a research hotspot. Existing fault diagnosis methods mainly perform anomaly detection based on performance indicators of a single service node, which is difficult to effectively identify fault propagation paths and root cause positioning in distributed systems.

[0003] Current distributed system fault diagnosis methods usually process the running logs and performance indicators of service nodes separately, lacking fusion analysis of multi-source heterogeneous data. This makes it difficult to fully grasp the running state of the system and accurately identify the propagation characteristics and impact range of faults.

[0004] Existing technologies also lack in-depth analysis of the calling relationship between service nodes, making it difficult to accurately grasp the propagation rules of faults in the system, and failing to fully utilize historical fault data for experience learning, resulting in poor diagnosis effect for similar fault scenarios; lacking self-adaptive ability in resource isolation and traffic scheduling, making it difficult to achieve precise fault control and system recovery.

[0005] With the continuous expansion of the scale of distributed systems and the continuous improvement of business complexity, traditional fault diagnosis and recovery methods have been unable to meet actual needs. An intelligent solution that can fuse time sequence log data, identify fault propagation paths, and adaptively adjust recovery strategies is urgently needed. SUMMARY

[0006] The embodiments of the present application provide a system anomaly intelligent diagnosis and recovery method and system fusing time sequence logs, which can solve the problems in the prior art.

[0007] In a first aspect of the embodiments of the present application, a system anomaly intelligent diagnosis and recovery method fusing time sequence logs is provided, comprising: Collecting running logs from multiple service nodes of a distributed system and performing time sequence alignment according to timestamps, extracting calling relationships and performance measurement data from the time sequence aligned running logs; Calculating time sequence variation characteristics of the performance measurement data within a sliding time window, fusing and encoding the time sequence variation characteristics and the calling relationships to generate dynamic feature vectors corresponding to the service nodes, and constructing a cross-service time sequence correlation graph based on the calling relationships and the dynamic feature vectors; Identifying abnormal service nodes from the cross-service time sequence correlation graph, matching the dynamic feature vectors corresponding to the abnormal service nodes with historical fault records in a historical fault database to obtain historical propagation paths and historical feature vectors in the historical fault records; calculate the feature deviation degree of the dynamic feature vector and the historical feature vector of each service node, and obtain the causal strength score of each service node by fusing the propagation convergence coefficient and the feature deviation degree; According to the causal strength score, the fault influence degree is identified, and the resource isolation strategy and the traffic scheduling strategy of the service node are adaptively adjusted based on the fault influence degree, so as to realize intelligent diagnosis and recovery of system fault.

[0008] Collect running logs from multiple service nodes of a distributed system and perform time sequence alignment according to timestamps, and extract calling relationship and performance measurement data from the time sequence aligned running logs, including: Collect original running logs from multiple service nodes, perform global clock synchronization calibration on the original running logs based on timestamps, sort the calibrated original running logs according to the chronological order of timestamps to construct time sequence aligned running logs; Extract request tracking identifiers from the time sequence aligned running logs, associate log records spanning multiple service nodes as complete calling links according to the request tracking identifiers, and construct weighted calling relationship based on the calling order and calling frequency of the service nodes in the calling links; Extract performance indicator data of each service node from the time sequence aligned running logs, perform time sequence slicing grouping on the performance indicator data according to timestamps, calculate performance indicator statistical feature values within each time sequence slice, and perform time sequence alignment matching between the performance indicator statistical feature values and the calling relationship to generate performance measurement data.

[0009] Calculate the time sequence variation characteristics of the performance measurement data within the sliding time window, fuse and encode the time sequence variation characteristics and the calling relationship to generate the dynamic feature vector corresponding to the service node, and construct the cross-service time sequence association graph based on the dynamic feature vector and the calling relationship, including: Set a sliding time window to segment the performance measurement data in time sequence, construct a time sequence phase space of the performance measurement data within each sliding time window, calculate the trajectory curvature and attractor feature of the performance measurement data in the time sequence phase space as the time sequence variation characteristics; Identify the hierarchical depth and branch density of the service node in the calling link from the calling relationship, and calculate the attenuation coefficient of the calling propagation path based on the hierarchical depth and branch density; Weight and aggregate the time sequence variation characteristics of the upstream service nodes in the calling relationship according to the calling attenuation coefficient, and fuse and encode the time sequence variation characteristics and the weighted and aggregated upstream time sequence characteristics across layers through a multi-head attention mechanism to generate the dynamic feature vector corresponding to the service node; Take the service node as the graph node and give the dynamic feature vector, calculate the cosine similarity between the dynamic feature vectors of adjacent service nodes in the call relationship, take the cosine similarity as the weight value of the graph edge, and construct the cross-service time sequence association graph.

[0010] Identify the abnormal service node from the cross-service time sequence association graph, match the dynamic feature vector corresponding to the abnormal service node with the historical fault record in the historical fault database, and obtain the historical propagation path and historical feature vector in the historical fault record. Calculate the dynamic feature vector difference matrix of adjacent time windows in the cross-service time sequence association graph, perform decomposition on the dynamic feature vector difference matrix to obtain the time sequence fluctuation component and the topological propagation component, and construct the state transition sequence based on the time sequence fluctuation component and the topological propagation component. Calculate the entropy values of the time sequence fluctuation component matrix and the topological propagation component matrix in the state transition sequence, generate an anomaly score by combining the time sequence fluctuation component entropy value and the topological propagation component entropy value, identify an abnormal service node based on the anomaly score, and extract an abnormal propagation link centered on the abnormal service node from the state transition sequence. Extract the time sequence fluctuation frequency spectrum component from the dynamic feature vector of the abnormal service node, extract the propagation delay spectrum component from the abnormal propagation link, combine the time sequence fluctuation frequency spectrum component and the propagation delay spectrum component to construct a dual-spectrum feature, and retrieve a matching historical fault record in the historical fault database according to the dual-spectrum feature. Obtain the historical propagation path and the historical feature vector based on the historical fault record.

[0011] Based on the historical propagation path, calculate the propagation convergence coefficient of each service node in the cross-service time sequence association graph, calculate the feature deviation degree of the dynamic feature vector and the historical feature vector of each service node, and fuse the propagation convergence coefficient and the feature deviation degree to obtain the causal strength score of each service node. Extract the directed propagation edge between the historical service nodes and the historical delay distribution corresponding to the directed propagation edge from the historical propagation path, identify a candidate propagation subgraph matching the topological structure of the historical propagation path in the cross-service time sequence association graph, and construct an upstream propagation tree in the in-degree direction of each target service node in the candidate propagation subgraph. Calculate the delay evolution deviation between the historical delay distribution of the directed propagation edge in the upstream propagation tree and the actual delay distribution of the corresponding directed propagation edge in the cross-service time sequence association graph. Based on the delay evolution deviation, perform propagation attenuation weighting and normalization aggregation on the directed propagation edges in the upstream propagation tree to obtain the propagation convergence coefficient of the target service node. Extract the time sequence component and the topological component from the dynamic feature vector and the historical feature vector of each target service node, respectively, calculate the time sequence deviation degree between the time sequence components, and the topological deviation degree between the topological components. The time sequence deviation degree and the topology deviation degree are combined by weighting to generate a feature deviation degree of each target service node. The topology centrality of the target service node in the cross-service time sequence association graph is calculated, the propagation convergence coefficient and the feature deviation degree are adaptively weighted and fused according to the topology centrality, and a causal strength score of each target service node is obtained.

[0012] According to the causal strength score, the fault influence degree is identified, the resource isolation strategy and the traffic scheduling strategy of the service node are adaptively adjusted based on the fault influence degree, and the intelligent diagnosis and recovery of the system fault are realized. The causal strength score is divided into a plurality of score sequences according to a fixed time window, the mean change amount and the variance change amount are extracted from the score sequence, the cumulative value of the mean change amount is calculated to obtain a fault diffusion degree, the cumulative value of the variance change amount is calculated to obtain a fault fluctuation degree, and the product of the fault diffusion degree and the fault fluctuation degree is taken as the fault influence degree. Based on the fault influence degree, the resource usage change of the service node is monitored, the growth rate and the decay rate of the resource usage are calculated, the time period in which the growth rate exceeds a preset growth threshold is marked as a resource isolation starting point, the time period in which the decay rate exceeds a preset decay threshold is marked as a resource isolation termination point, and the resource isolation strategy of the service node is generated according to the resource isolation starting point and the resource isolation termination point. The resource isolation strategy is converted into a resource limit vector, the computing resource limit value and the storage resource limit value are extracted from the resource limit vector, the concurrent processing threshold of the service node is determined by the computing resource limit value, the cache processing threshold of the service node is determined by the storage resource limit value, the traffic scheduling strategy of the service node is updated according to the concurrent processing threshold and the cache processing threshold, and the intelligent diagnosis and recovery of the system fault are realized.

[0013] In a second aspect of the embodiment of the application, a system abnormality intelligent diagnosis and recovery system fusing time sequence logs is provided, comprising: A log collection unit is configured to collect running logs from a plurality of service nodes of a distributed system and perform time sequence alignment according to timestamps, and extract call relationships and performance metric data from the time sequence aligned running logs. A feature extraction unit is configured to calculate time sequence change features of the performance metric data within a sliding time window, fuse and encode the time sequence change features and the call relationships, generate a dynamic feature vector corresponding to the service node, and construct the dynamic feature vector into a cross-service time sequence association graph based on the call relationships. A fault matching unit is configured to identify an abnormal service node from the cross-service time sequence association graph, match the dynamic feature vector corresponding to the abnormal service node with historical fault records in a historical fault database, and obtain a historical propagation path and a historical feature vector in the historical fault records. The causal analysis unit is used to calculate the propagation convergence coefficient of each service node in the cross-service time-series correlation graph based on the historical propagation path, calculate the feature deviation degree between the dynamic feature vector and the historical feature vector of each service node, and fuse the propagation convergence coefficient and the feature deviation degree to obtain the causal strength score of each service node. The fault recovery unit is used to identify the degree of fault impact based on the causal strength score, and adaptively adjust the resource isolation strategy and traffic scheduling strategy of the service node based on the degree of fault impact to realize intelligent diagnosis and recovery of system faults.

[0014] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0016] In this embodiment, by performing time-series alignment and fusion analysis of distributed system operation logs, effective integration of multi-source heterogeneous data is achieved, improving the accuracy of fault diagnosis. The method based on sliding time windows and dynamic feature vectors can capture dynamic changes in system state in a timely manner, improving the real-time performance of anomaly detection. Through experience learning and matching of historical fault data, combined with the calculation of propagation convergence coefficients and feature deviation, the propagation path and root cause location of the fault are accurately identified. The adaptive recovery strategy incorporating causal strength scoring achieves precise control of resource isolation and traffic scheduling, improving the efficiency of fault recovery. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the intelligent diagnosis and recovery method for system anomalies that integrates time-series logs according to an embodiment of the present invention. Figure 2 This is a flowchart of the causal strength calculation for multi-source feature fusion in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0020] Figure 1 This is a flowchart illustrating the intelligent diagnosis and recovery method for system anomalies that integrates time-series logs according to an embodiment of the present invention. Figure 1 As shown, the method includes: Run logs are collected from multiple service nodes in a distributed system and aligned to a time sequence by timestamps. Call relationships and performance metrics are then extracted from the time-series aligned run logs. Within a sliding time window, the time-series variation characteristics of performance metrics are calculated, and the time-series variation characteristics are fused and encoded with the call relationship to generate dynamic feature vectors corresponding to service nodes. Based on the call relationship, the dynamic feature vectors are used to construct a cross-service time-series association graph. Abnormal service nodes are identified from the cross-service time sequence correlation graph. The dynamic feature vectors corresponding to the abnormal service nodes are matched with historical fault records in the historical fault database to obtain the historical propagation paths and historical feature vectors in the historical fault records. Based on the historical propagation path, the propagation convergence coefficient of each service node in the cross-service temporal relationship graph is calculated, the feature deviation degree between the dynamic feature vector and the historical feature vector of each service node is calculated, and the causal strength score of each service node is obtained by fusing the propagation convergence coefficient and the feature deviation degree. The severity of a fault is identified by a causal strength score, and the resource isolation and traffic scheduling strategies of service nodes are adaptively adjusted based on the severity of the fault to achieve intelligent diagnosis and recovery of system faults.

[0021] Runtime logs are collected from multiple service nodes in a distributed system and aligned to a specific time sequence using timestamps. Call relationship and performance metric data are extracted from the time-aligned logs, including: Raw operation logs are collected from multiple service nodes. Global clock synchronization calibration is performed on the raw operation logs based on timestamps. The calibrated raw operation logs are then sorted according to the order of timestamps to construct time-aligned operation logs. Extract request tracing identifiers from time-aligned runtime logs, associate log records spanning multiple service nodes into a complete call chain based on the request tracing identifiers, and construct weighted call relationships based on the call order and call frequency of service nodes in the call chain; The performance metrics data of each service node are extracted from the time-aligned runtime logs. The performance metrics data are then grouped into time-series slices according to timestamps. The statistical feature values ​​of the performance metrics within each time-series slice are calculated. Finally, the statistical feature values ​​of the performance metrics are matched with the call relationships to generate performance measurement data.

[0022] The raw log collection phase involves deploying a log collection component to acquire logs from multiple service nodes. This component can be configured on each service node and supports real-time capture of log records generated by the service nodes. The collected raw logs include information such as timestamps, request identifiers, service names, interface names, execution status codes, and performance metrics. After collection, the log collection component transmits the raw logs to the central log processing module for further processing.

[0023] Global clock synchronization calibration is performed on the collected raw operation logs. Since the system clocks of different service nodes may deviate, resulting in inconsistent log timestamps, calibration is necessary. The calibration method is a clock synchronization algorithm based on the Network Time Protocol (NTP). A reference clock service is selected as the standard time source, and the time deviation between each service node and the standard time source is calculated. For a log record generated by a service node, its timestamp is added to the corresponding time deviation value to obtain the calibrated unified timestamp. For example, if the time deviation of service node A is +2 seconds, then the log record recorded on this node with the timestamp "2025-11-27 10:15:30" will have its timestamp updated to "2025-11-27 10:15:32" after calibration.

[0024] Time-order alignment involves sorting the calibrated raw runtime logs according to their timestamps. A quicksort algorithm is used to sort all log records, with the calibrated timestamps as the sorting key. After sorting, a globally consistent sequence of runtime logs arranged in chronological order is obtained; this sequence is called the time-order aligned runtime log.

[0025] The call chain construction process is based on request tracing identifier technology. Request tracing identifiers are extracted from time-aligned runtime logs. These identifiers are generated when a request enters the system and are passed throughout the request processing, identifying the processing of the same request on different service nodes. Request tracing identifiers are extracted from log records using regular expression pattern matching. Log records with the same request tracing identifier are grouped, forming a complete call chain. Each call chain contains the complete processing of the request from entry to exit, recording all service nodes the request passes through and their processing status.

[0026] Weighted call relationship construction is based on call chain analysis. The call order of service nodes in each call chain is analyzed to identify the call relationships between service nodes. For each identified call relationship, its call direction and call count are recorded. The call count is used as a weight value to represent the tightness of the call relationship. All call relationships obtained from call chain analysis and their weight values ​​are merged to construct a weighted call relationship graph for the entire system. In this graph, nodes represent service nodes, edges represent call relationships, and the edge weight represents the call frequency. For example, if service A calls service B 100 times, then the call relationship from A to B has a weight of 100.

[0027] Performance metric extraction involves extracting performance data for each service node from time-aligned runtime logs. Extracted performance metrics include response time, throughput, error rate, and resource utilization. Response time is calculated by dividing the start and end times of request processing; throughput is calculated by counting the number of requests processed per unit of time; error rate is calculated by the ratio of erroneous requests to total requests; and resource utilization is obtained by parsing resource monitoring logs.

[0028] Time-series slicing grouping involves grouping extracted performance metric data according to the time dimension. A fixed time window size, such as 5 minutes, is set, and the performance metric data is grouped according to the time window, resulting in a series of time-series slices. Each time-series slice contains the performance metric data of each service node within that time window.

[0029] The calculation of performance metric statistical characteristic values ​​involves statistical analysis of the performance metric data within each time slice. The calculation methods include calculating the average, median, maximum, minimum, standard deviation, and percentile values ​​of the performance metric data. These statistical characteristic values ​​comprehensively reflect the performance status of each service node within a specific time window.

[0030] Performance metric data generation is achieved by time-series alignment and matching of performance metric statistical features with call relationships. For each time slice, the performance metric statistical features within that time window are associated with the corresponding call relationships to generate performance metric data that includes time, service node, performance metric statistical features, and call relationships. This performance metric data comprehensively reflects the system's operating status and performance across various time windows.

[0031] This invention achieves comprehensive monitoring and anomaly diagnosis of the operating status of a distributed system by integrating time-series logs from multiple service nodes. It overcomes the limitations of traditional log analysis methods in multi-service environments, achieving time-series alignment of logs across service nodes and accurate construction of call relationships. Through time-series slicing and performance indicator statistical feature analysis, it can promptly detect performance anomalies and service call anomalies in the system, and, combined with call relationship graphs, perform root cause localization, improving the accuracy and efficiency of anomaly diagnosis. The time-series characteristics of performance measurement data help identify changes in system performance trends and potential risks, enabling preventative maintenance and improving system reliability and stability.

[0032] Within a sliding time window, the time-series variation characteristics of performance metrics are calculated. These characteristics are then fused and encoded with call relationships to generate dynamic feature vectors corresponding to service nodes. Based on the call relationships, these dynamic feature vectors are used to construct a cross-service time-series correlation graph, including: A sliding time window is set to segment the performance measurement data in time series. Within each sliding time window, a time series phase space of the performance measurement data is constructed. The trajectory curvature and attractor features of the performance measurement data are calculated in the time series phase space as time series variation features. Identify the level depth and branch density of service nodes in the call chain from the call relationship, and calculate the attenuation coefficient of the call propagation path based on the level depth and branch density; The temporal change features of upstream service nodes in the call relationship are weighted and aggregated according to the call attenuation coefficient. The temporal change features and the weighted aggregated upstream temporal features are then fused and encoded across levels through a multi-head attention mechanism to generate dynamic feature vectors corresponding to the service nodes. Using service nodes as graph nodes and assigning them dynamic feature vectors, the cosine similarity between the dynamic feature vectors of adjacent service nodes in the call relationship is calculated. The cosine similarity is then used as the weight value of the graph edge to construct a cross-service temporal association graph.

[0033] The temporal phase space of performance metric data is constructed using a delayed embedding method. For the performance metric data of each service node, an appropriate embedding dimension and delay parameter are selected to transform the one-dimensional time series into a trajectory in a multi-dimensional phase space. The embedding dimension is typically chosen to be 3 to 5 dimensions, and the delay parameter is determined based on the autocorrelation characteristics of the time series data, generally 1 to 3 times the sampling interval. For the response time metric, assuming a data sequence of [120ms, 135ms, 125ms, 180ms, 210ms] is collected within a sliding window, using the delayed embedding method with an embedding dimension of 3 and a delay parameter of 1, this sequence can be transformed into a set of points in the phase space: (120ms, 135ms, 125ms), (135ms, 125ms, 180ms), and (125ms, 180ms, 210ms). These points form a trajectory in the phase space representing the dynamic behavior of the system.

[0034] The curvature of the performance metric trajectory in the time-series phase space is calculated by analyzing the rate of change between adjacent points. In the constructed phase space, the change in the angle between adjacent points on the trajectory is calculated to obtain the local curvature of the trajectory. A larger curvature value indicates more drastic changes in performance metrics, potentially indicating anomalies in system behavior. The convergence region of the phase space trajectory is analyzed to identify attractor characteristics. Attractor characteristics include the type, size, and stability of the attractor, used to describe the long-term behavioral patterns of the system. Under normal operating conditions, system performance metrics typically form stable point attractors or periodic attractors; while under abnormal conditions, chaotic attractors or trajectory divergence may occur. Combining the trajectory curvature with attractor characteristics forms a time-series variation feature describing the dynamic changes in the performance of service nodes.

[0035] Further analysis is needed to identify the hierarchy depth and branch density of service nodes within the call chain. Hierarchical depth refers to the positional depth of the service node in the call chain, calculated from the entry service node, incrementing by one for each service call. Branch density represents the breadth of downstream service calls made by the service node, calculated as the number of different service nodes directly called by the node divided by the total number of service nodes. Both the hierarchy depth and branch density of a service node reflect its structural importance within the entire call network.

[0036] The attenuation coefficient of the call propagation path is calculated based on the hierarchy depth and branch density. The attenuation coefficient quantifies the influence strength between service nodes in the call relationship, taking into account information loss and influence attenuation during the service call process. The attenuation coefficient value is inversely proportional to the hierarchy depth and directly proportional to the branch density. Specifically, the hierarchy depth is first standardized to the [0, 1] interval, then the branch density is standardized to the [0, 1] interval. An exponential function is used to convert the standardized hierarchy depth into a basic attenuation value. The basic attenuation value is then adjusted according to the standardized branch density to obtain the final call attenuation coefficient.

[0037] Weighted aggregation of the temporal variation features of upstream service nodes is a feature fusion process utilizing call relationships. For each service node, all its upstream call nodes are identified, and the temporal variation features of these upstream nodes are obtained. These features are then weighted according to the call attenuation coefficient from each upstream node to the current node, resulting in weighted aggregated upstream temporal features. The weighting method uses a dot product operation between the call attenuation coefficient and the temporal variation feature. When a service node has multiple upstream call nodes, all weighted upstream temporal features are summed to obtain the final weighted aggregation result.

[0038] Multi-head attention (M&A) cross-level fusion encoding is a process of deep fusion of the temporal variation features of the service node itself with the weighted aggregated upstream temporal features. The M&A mechanism first projects the input features into three spaces: query, key, and value. Then, it computes attention in parallel in different representation subspaces. Finally, it concatenates the attention results from each subspace and performs a linear transformation to obtain the final output. Through M&A, the important correlations between the service node's own features and upstream features can be adaptively learned, achieving cross-level feature fusion. The fused features are processed through fully connected layers and non-linear activation functions to generate feature vectors representing the dynamic behavior of the service node, i.e., dynamic feature vectors.

[0039] Constructing a cross-service temporal relationship graph is the process of organizing service nodes and their call relationships into a graph structure. Service nodes are used as nodes in the graph, and each node is assigned a dynamic feature vector generated in the previous step as its node feature. For each pair of adjacent service nodes in the call relationship, the cosine similarity between their dynamic feature vectors is calculated and used as the weight value of the edge connecting the two nodes. Cosine similarity reflects the degree of similarity between the performance change patterns of two service nodes; the higher the similarity, the more coordinated the performance changes of the two nodes. In this way, a weighted undirected graph, namely the cross-service temporal relationship graph, is constructed. This graph not only contains service call structure information but also integrates the dynamic performance change characteristics of each service node and the performance coordination relationships between nodes.

[0040] This invention achieves precise characterization of the dynamic performance characteristics of a system through sliding time windows and phase space analysis techniques, overcoming the limitations of traditional static feature analysis. A cross-service feature fusion mechanism based on call relationships solves the problem of tracing anomaly propagation paths, improving the accuracy of root cause localization. A multi-head attention mechanism enables adaptive fusion of features across different service levels, enhancing the ability to identify anomaly patterns in complex call chains. A cross-service temporal correlation graph integrates service performance changes and topology information, providing a unified analytical framework for anomaly diagnosis, significantly improving the intelligence level of distributed system anomaly diagnosis, and providing a reliable basis for subsequent recovery strategy formulation.

[0041] Abnormal service nodes are identified from the cross-service time-series correlation graph. The dynamic feature vectors corresponding to the abnormal service nodes are matched with historical fault records in the historical fault database to obtain the historical propagation paths and historical feature vectors from the historical fault records, including: Calculate the dynamic feature vector difference matrix of adjacent time windows in the cross-service temporal correlation graph, decompose the dynamic feature vector difference matrix to obtain the temporal fluctuation component and the topology propagation component, and construct the state transition sequence based on the temporal fluctuation component and the topology propagation component. The entropy values ​​of the temporal fluctuation component matrix and the topology propagation component matrix in the state transition sequence are calculated. Anomaly scores are generated by combining the entropy values ​​of the temporal fluctuation component and the topology propagation component. Anomaly service nodes are identified based on the anomaly scores, and anomaly propagation links centered on the anomaly service nodes are extracted from the state transition sequence. The temporal fluctuation spectrum component is extracted from the dynamic feature vector of the abnormal service node, and the propagation delay spectrum component is extracted from the abnormal propagation link. The temporal fluctuation spectrum component and the propagation delay spectrum component are combined to construct a dual-spectrum feature. Based on the dual-spectrum feature, historical fault records are retrieved and matched in the historical fault database. Based on the historical fault records, historical propagation paths and historical feature vectors are obtained.

[0042] The dynamic feature vectors of the same service node within two adjacent time windows are subtracted element-wise to obtain a difference vector describing the performance change trend of that service node. This operation is performed on all service nodes, and all difference vectors are arranged in order of service node identification to form a dynamic feature vector difference matrix. For example, if the dynamic feature vector of a service node in time window T1 is [0.72, 0.35, 0.91, 0.48], and the dynamic feature vector in T2 is [0.75, 0.29, 1.25, 0.51], the calculated difference vector is [0.03, -0.06, 0.34, 0.03].

[0043] A decomposition operation is performed on the dynamic eigenvector difference matrix, decomposing it into a temporal fluctuation component and a topology propagation component. Non-negative matrix factorization is employed, with a latent dimension of 2, and the decomposition result is solved through an iterative optimization algorithm. The temporal fluctuation component describes the variation pattern of the service node's own performance indicators, while the topology propagation component reflects the propagation effect of performance changes caused by service call relationships. The decomposition process uses a multiplicative update rule to ensure the non-negativity of the result while minimizing the decomposition error. The maximum number of iterations is set to 100, and the convergence threshold is 0.001.

[0044] A state transition sequence is constructed based on the temporal fluctuation components and topology propagation components obtained from the decomposition. The decomposition results from multiple consecutive time windows are arranged chronologically to form a transition sequence representing the change of system state over time. This sequence contains a sequence of temporal fluctuation component matrices and a sequence of topology propagation component matrices, describing the spatiotemporal evolution of the system state. For data from 12 consecutive 30-second sliding windows within a 10-minute period, 11 sets of dynamic eigenvector difference matrices and their decomposition results are obtained, constructing a state transition sequence containing 11 time points.

[0045] Calculate the entropy values ​​of the temporal fluctuation component matrix and the topology propagation component matrix in the state transition sequence. For the temporal fluctuation component matrix, calculate the Shannon entropy of the row vector corresponding to each service node, measuring the uncertainty of the node's performance changes. For the topology propagation component matrix, calculate the Shannon entropy of the row vector corresponding to each service node, measuring the uncertainty of the node in the propagation of call relationships. When calculating the entropy value, the matrix row vectors are normalized so that the sum of the row vector elements is 1. The logarithmic weighted sum of the normalized element values ​​is then calculated to obtain the entropy value. The higher the entropy value, the greater the uncertainty and the higher the probability of anomalies.

[0046] An anomaly score is generated by combining the entropy values ​​of the temporal fluctuation component and the topology propagation component. A weighted summation method is used to combine the two entropy values ​​for each service node. The weighting coefficients are adjusted based on historical anomaly detection results; typically, the weight of the temporal fluctuation component entropy value is 0.6, and the weight of the topology propagation component entropy value is 0.4. The resulting anomaly score reflects the overall anomaly level of the service node, with scores ranging from 0 to 1; the closer the value is to 1, the greater the likelihood of an anomaly.

[0047] Anomaly scoring is used to identify abnormal service nodes. An anomaly scoring threshold of 0.8 is set, and service nodes with scores exceeding this threshold are identified as abnormal. If multiple nodes have anomaly scores exceeding the threshold, they are sorted from highest to lowest, and the node with the highest score is selected as the primary abnormal service node. If service node A has an anomaly score of 0.92, node B has 0.88, and node C has 0.75, then A and B are identified as abnormal service nodes, while C is not considered abnormal.

[0048] Extract the anomaly propagation links centered on the anomalous service nodes from the state transition sequence. Analyze the changing trends of the topology propagation components in the state transition sequence to identify the propagation paths of the anomalies. Examine the values ​​between the anomalous service nodes and other nodes in the topology propagation component matrix; larger values ​​indicate stronger propagation associations. Sort by propagation intensity and select service nodes whose propagation intensity exceeds a set threshold to form the anomaly propagation links. The threshold is typically set to 50% of the maximum value of the topology propagation components.

[0049] The temporal fluctuation spectral component is extracted from the dynamic feature vector of the abnormal service node. A Fast Fourier Transform (FFT) is performed on the dynamic feature vector sequence of the abnormal service node in the state transition sequence to convert the time-domain features into frequency-domain features. The amplitude portion of the transform result is taken as the temporal fluctuation spectral component, describing the frequency characteristics of the abnormal service node's performance changes. Typically, only the low-to-mid-frequency spectral information is retained, as the high-frequency portion often contains noise rather than effective signal.

[0050] The propagation delay spectral component is extracted from the anomaly propagation path. Cross-correlation analysis is performed on the anomaly score time series of each service node in the anomaly propagation path to calculate the anomaly propagation delay characteristics. The delay distribution is then subjected to Fourier transform to obtain the spectral representation of the propagation delay, i.e., the propagation delay spectral component, which reflects the temporal pattern characteristics of anomaly propagation between service nodes.

[0051] A dual-spectral feature is constructed by combining the temporal fluctuation spectral component and the propagation delay spectral component. A feature concatenation method is then used to combine the normalized two spectral components into a comprehensive feature vector. This feature integrates the performance variation characteristics and anomaly propagation characteristics of the abnormal service node, effectively characterizing the current anomaly pattern.

[0052] Based on bispectral features, historical fault records are retrieved from the historical fault database to match. Cosine similarity is used as the matching metric to calculate the similarity between the bispectral features of the current anomaly and the bispectral features of historical fault records. The historical fault record with the highest similarity exceeding a set threshold is selected as the matching result. If multiple records have similarity exceeding the threshold, the top few with the highest similarity are returned. The threshold is typically set to 0.85 to ensure sufficient similarity in the matching results.

[0053] Historical propagation paths and feature vectors are obtained based on matched historical fault records. Historical propagation path information, including the fault source node, propagation order, and scope, is extracted from the matched records. Simultaneously, historical feature vectors are extracted, including dynamic feature change patterns of each service node during the anomaly's development, for use in subsequent root cause analysis and recovery strategy formulation.

[0054] This invention achieves accurate identification and location of abnormal service nodes through dynamic analysis of cross-service temporal correlation graphs, solving the technical challenges of complex anomaly propagation and difficulty in tracing root causes in distributed environments. Based on a decomposition method of temporal fluctuations and topology propagation components, the temporal and spatial characteristics of anomalies are decoupled, improving the accuracy of anomaly detection. The innovative dual-spectrum feature characterization technology integrates performance fluctuation and propagation delay information, enhancing the ability to distinguish anomaly patterns. Through intelligent matching with historical faults, it enables the reuse of experience from similar faults, significantly shortening fault diagnosis time, lowering the professional threshold for maintenance personnel, providing a reliable decision-making basis for automated fault recovery, and effectively improving system reliability and service continuity.

[0055] like Figure 2 The diagram shows the flowchart for calculating the causal strength of multi-source feature fusion in this embodiment.

[0056] Based on historical propagation paths, the propagation convergence coefficient of each service node in the cross-service temporal correlation graph is calculated. The feature deviation between the dynamic feature vector and the historical feature vector of each service node is calculated. The causal strength score of each service node is obtained by fusing the propagation convergence coefficient and the feature deviation. The directed propagation edges between historical service nodes and the historical latency distribution corresponding to the directed propagation edges are extracted from the historical propagation path. Candidate propagation subgraphs that match the topology of the historical propagation path are identified in the cross-service temporal association graph. The upstream propagation tree is constructed by multi-hop backtracking along the in-degree direction for each target service node in the candidate propagation subgraph. Calculate the latency evolution deviation between the historical latency distribution of directed propagation edges in the upstream propagation tree and the actual latency distribution of the corresponding directed propagation edges in the cross-service time sequence association graph; Based on the time delay evolution deviation, the directed propagation edges in the upstream propagation tree are weighted by propagation attenuation and normalized to obtain the propagation convergence coefficient of the target service node. Temporal components and topological components are extracted from the dynamic feature vectors and historical feature vectors of each target service node, respectively. The temporal deviation between temporal components and the topological deviation between topological components are calculated. The temporal deviation and topology deviation are weighted and combined to generate the feature deviation of each target service node; Calculate the topological centrality of the target service node in the cross-service temporal correlation graph, and adaptively weight and fuse the propagation convergence coefficient and feature deviation degree based on the topological centrality to obtain the causal strength score of each target service node.

[0057] Historical propagation paths are typically recorded in a fault database, containing the propagation order and timing information of anomalies between service nodes when historical faults occurred. For each historical propagation path, service node pairs are extracted as directed propagation edges, and the time difference in anomaly propagation between these service node pairs is extracted as a historical latency value. If multiple historical propagation records exist between two service nodes, these latency values ​​are aggregated to obtain a latency distribution. A typical historical latency distribution might be the set of anomaly propagation latency between service node pairs A and B [2.1 seconds, 2.3 seconds, 1.9 seconds, 2.2 seconds], indicating that it typically takes about 2 seconds for an anomaly to propagate from node A to node B.

[0058] Candidate propagation subgraphs matching the historical propagation path topology are identified in the cross-service temporal association graph. A subgraph matching algorithm is employed, using the historical propagation path topology as a template to search for subgraphs with similar topologies in the current cross-service temporal association graph. The matching metric includes node similarity and edge similarity. Node similarity is calculated based on the functional type and interface characteristics of service nodes, while edge similarity is calculated based on the frequency and response time characteristics of service call relationships. Subgraphs with a matching score exceeding a set threshold are marked as candidate propagation subgraphs. The subgraph matching algorithm uses a heuristic search strategy, starting with highly similar nodes and gradually expanding to construct matching subgraphs. The algorithm complexity is controlled to be at the polynomial level of the number of service nodes.

[0059] For each target service node in the candidate propagation subgraph, a multi-hop backtracking process is performed along the in-degree direction to construct the upstream propagation tree. Target service nodes are typically abnormal nodes or critical business nodes in the candidate propagation subgraph. For each target node, a breadth-first traversal is performed starting from that node and proceeding in the reverse direction of service calls, constructing a directed tree structure with the target node as the root and upstream service nodes as branches. The traversal depth is typically limited to 3 to 5 hops to avoid constructing an excessively large propagation tree. During the construction process, the actual call relationships and latency between each node are recorded. If a target service node has multiple upstream service nodes, the upstream propagation tree will exhibit a branching structure.

[0060] The latency evolution deviation is calculated between the historical latency distribution of directed propagation edges in the upstream propagation tree and the actual latency distribution of the corresponding directed propagation edges in the cross-service temporal relationship graph. For each directed edge in the upstream propagation tree, its actual latency distribution in the current cross-service temporal relationship graph is extracted, typically based on service call logs within a recent period. The difference between the historical latency distribution and the actual latency distribution is calculated using chi-square distance to obtain the latency evolution deviation. In the chi-square distance calculation, the two latency distributions are first divided into equal latency intervals, the frequency within each interval is counted, and then the weighted sum of squares of the frequency differences is calculated. The latency evolution deviation reflects the degree of change in the current service call relationship relative to historical failures; a larger deviation indicates a greater difference between the current propagation pattern and the historical propagation pattern.

[0061] Based on the latency evolution deviation, the directed propagation edges in the upstream propagation tree are weighted by propagation attenuation and normalized to obtain the propagation convergence coefficient of the target service node. The smaller the latency evolution deviation, the closer the current propagation mode is to the historical failure mode, and the higher the propagation weight should be. An exponential decay function is used to convert the latency evolution deviation into propagation weights. The weight is maximum of 1 when the deviation is 0, and the weight decays exponentially as the deviation increases. For each path in the upstream propagation tree, the cumulative propagation weight is calculated step by step from the leaf node to the root node, taking into account the attenuation factor of the path length. Finally, the cumulative weights of all paths are normalized and aggregated to obtain the propagation convergence coefficient of the target service node, which reflects the probability that the target node is a potential source of failure.

[0062] Temporal and topological components are extracted from the dynamic and historical feature vectors of each target service node, respectively. The dynamic feature vectors contain information such as service node performance metrics and log event statistics, while the historical feature vectors are derived from feature data in historical fault records. Principal component analysis (PCA) is used to decompose the feature vectors into temporal and topological principal components. The temporal principal components primarily reflect the temporal characteristics of node performance changes, while the topological principal components primarily reflect the structural characteristics of the node within the service call network. Typically, the principal components with the largest explained variance are selected as representative components.

[0063] Calculate the temporal skewness between temporal components and the topological skewness between topological components. Use cosine distance to calculate the similarity between the current temporal component and historical temporal components of the target service node, and convert the similarity into temporal skewness. Calculate the topological skewness between the current topological component and historical topological components using the same method. The skewness value ranges from 0 to 1; the closer the value is to 0, the more similar the current feature is to the historical feature; the closer the value is to 1, the greater the difference.

[0064] The temporal deviation and topology deviation are weighted and combined to generate the feature deviation of each target service node. The weighting coefficients are adjusted according to the specific application scenario; typically, the weight of temporal deviation is 0.7 and the weight of topology deviation is 0.3. The two deviations are then weighted and summed to obtain the comprehensive feature deviation, which reflects the overall difference between the current state and the historical fault state of the target node.

[0065] The topological centrality of target service nodes in the cross-service temporal correlation graph is calculated. Based on the topological centrality, the propagation convergence coefficient and feature bias are adaptively weighted and fused to obtain a causal strength score for each target service node. Topological centrality uses a proximity centrality index, calculated as the reciprocal of the average shortest path length from a node to all other nodes in the graph. Nodes with higher centrality have greater influence during propagation, and their propagation convergence coefficient should have a correspondingly higher weight. The sigmoid transformation result of the topological centrality is used as the weight of the propagation convergence coefficient, with the remaining weights allocated to the feature bias. The inverse value of the propagation convergence coefficient is taken and weighted with the feature bias to obtain the causal strength score. The score ranges from 0 to 1; the closer the value is to 1, the greater the likelihood that the service node is the root cause of the current anomaly.

[0066] This invention achieves precise root cause localization of anomalies by organically combining the propagation convergence coefficient and the feature deviation degree. The propagation convergence coefficient provides causal reasoning from a topological perspective, while the feature deviation degree provides similarity analysis from a state feature perspective. The two complement each other, forming a multi-dimensional root cause assessment system. An adaptive weighting mechanism based on historical experience effectively balances the differences in importance of different service nodes in the anomaly propagation network, improving the accuracy of root cause analysis. It can effectively handle cascading failure scenarios in complex microservice architectures, shortening fault location time, reducing the workload of operations and maintenance personnel, improving service stability, and providing a reliable basis for subsequent intelligent recovery decisions, demonstrating significant value in ensuring business continuity.

[0067] The system identifies the degree of fault impact based on causality strength scores, and adaptively adjusts resource isolation and traffic scheduling strategies for service nodes based on this degree of impact to achieve intelligent fault diagnosis and recovery. The causal intensity score is divided into multiple score sequences according to a fixed time window. The mean change and variance change are extracted from the score sequences. The cumulative value of the mean change is calculated to obtain the fault diffusion degree, and the cumulative value of the variance change is calculated to obtain the fault volatility degree. The product of the fault diffusion degree and the fault volatility degree is used as the degree of fault impact. Based on the degree of impact of the fault, monitor the changes in resource usage of the service node, calculate the growth rate and decay rate of resource usage, mark the time period when the growth rate exceeds the preset growth threshold as the resource isolation start point, and mark the time period when the decay rate exceeds the preset decay threshold as the resource isolation end point, and generate the resource isolation strategy for the service node based on the resource isolation start point and resource isolation end point. The resource isolation strategy is converted into a resource constraint vector. Computational resource constraint values ​​and storage resource constraint values ​​are extracted from the resource constraint vector. The concurrent processing threshold of the service node is determined by the computational resource constraint value, and the cache processing threshold of the service node is determined by the storage resource constraint value. The traffic scheduling strategy of the service node is updated according to the concurrent processing threshold and the cache processing threshold, so as to realize intelligent diagnosis and recovery of system faults.

[0068] A sliding window approach was used, with a window size of 30 seconds and a sliding step of 10 seconds, to segment the causal strength score sequence for each service node. For each segmented score sequence, its statistical characteristics, including mean and variance, were calculated. The change in mean between adjacent windows reflects the overall trend of the causal strength score, while the change in variance reflects the degree of fluctuation in the score. In practical applications, the mean causal strength scores of a certain service node in three consecutive time windows were 0.35, 0.62, and 0.78, respectively. Therefore, the changes in mean between adjacent windows were 0.27 and 0.16; the variances were 0.05, 0.08, and 0.12, respectively, with changes in variance of 0.03 and 0.04.

[0069] The cumulative value of the mean change is used to determine the fault spread. The mean changes for each time window are accumulated sequentially. A time decay factor can be applied during the accumulation process to gradually reduce the influence of changes from earlier time windows on the cumulative result. The time decay factor uses an exponential decay form with a decay rate of 0.9. The fault spread reflects the persistence and cumulative effect of the causal strength score increase; a higher value indicates a more severe spread of the fault's impact.

[0070] The cumulative value of variance change is used to calculate fault volatility. Similar to the calculation method for fault diffusion, the variance change is accumulated sequentially over time, and a time decay factor is also applied. Fault volatility reflects the cumulative effect of fluctuations in the causal strength score; a larger value indicates higher uncertainty in the impact of the fault. For a certain service node, if the variance change over five consecutive time windows is 0.03, 0.04, 0.02, 0.05, and 0.06, the calculated fault volatility after considering the time decay factor is 0.17.

[0071] The product of fault propagation and fault volatility is used to determine the fault impact level. This multiplication considers both the expansion of the fault's impact range and the instability of its impact, resulting in a comprehensive assessment of the fault's severity. The fault impact level is typically normalized to a range of 0 to 1 for easier subsequent processing and decision-making. By setting a threshold for the fault impact level, service nodes can be categorized into three levels: high impact, medium impact, and low impact, corresponding to different resource isolation and traffic scheduling strategies.

[0072] The resource usage changes of service nodes are monitored based on the degree of fault impact. Resource usage mainly includes indicators such as CPU utilization, memory utilization, disk I / O rate, and network bandwidth utilization. A time-series sampling method is used, collecting values ​​for each resource indicator every second to form a time series of resource usage. The rate of change of resource usage within a short time window (e.g., 10 seconds) is calculated to obtain the growth rate and decay rate of resource usage.

[0073] For each resource indicator, a linear regression method within a sliding window is used to estimate its trend. The window size is set to 10 seconds, and the slope of the regression line represents the rate of change of that resource indicator. If the slope is positive, it is the growth rate; if the slope is negative, its absolute value is the decay rate. For CPU utilization, if the sampled values ​​within 10 seconds are 45%, 48%, 52%, 55%, 59%, 62%, 65%, 68%, 70%, and 73% respectively, then the growth rate calculated by linear regression is approximately 3.1% / second.

[0074] The time period during which the growth rate exceeds a preset growth threshold is marked as the starting point for resource isolation. The preset growth threshold is determined based on historical statistical data and is typically set to three times the resource usage growth rate under normal load. The growth threshold varies for different resource metrics; for example, the growth threshold for CPU utilization can be set to 2% / second, and the growth threshold for memory utilization can be set to 1% / second. When the growth rate of a resource metric exceeds its corresponding growth threshold for three consecutive sampling periods, the current time point is marked as the starting point for resource isolation.

[0075] The time period during which the decay rate exceeds the preset decay threshold is marked as the resource isolation termination point. The preset decay threshold is typically set to 50% to 70% of the growth threshold, ensuring that resource usage has significantly decreased before isolation is lifted. When the decay rate of a resource metric exceeds the decay threshold for five consecutive sampling periods, the current time point is marked as the isolation termination point for that resource.

[0076] Resource isolation policies for service nodes are generated based on the start and end points of resource isolation. These policies include three dimensions: isolation period, isolated resource type, and isolation level. The isolation level is determined by the severity of the fault and is typically categorized into three levels: complete isolation, partial isolation, and monitored but not isolated. Complete isolation means temporarily removing the service node from the service cluster; partial isolation means limiting the resource usage limit of the service node; and monitored but not isolated means only enhanced monitoring without taking actual isolation measures.

[0077] The resource isolation policy is converted into a resource constraint vector. A resource constraint vector is a multi-dimensional vector, with each dimension corresponding to a constraint value for a resource type, such as CPU core count limit or memory capacity limit. The constraint ratio for each dimension is set according to the isolation level. For partial isolation levels, resources are typically limited to 60% to 80% of the normal configuration; for full isolation levels, they are limited to the minimum runnable configuration, such as 20% to 40%. For a service node with a normal configuration of 4 CPU cores and 8GB of memory, and an assessment of moderate impact from a failure, a partial isolation policy could be adopted, and the resource constraint vector could be set to [2.4 cores, 5.6GB], i.e., limited to 70% of the normal configuration.

[0078] The resource constraint vector extracts computational and storage resource constraints. Computational resource constraints primarily refer to limitations on processor resources such as CPUs and GPUs, while storage resource constraints primarily refer to limitations on storage media such as memory and disks. Computational resource constraints directly affect the concurrent processing capability of service nodes, while storage resource constraints affect the data caching capability of service nodes.

[0079] The concurrent processing threshold for a service node is determined by the computing resource limit. The concurrent processing threshold refers to the maximum number of requests a service node can handle simultaneously. This is obtained by multiplying the number of requests that a unit of computing resource can handle by performance testing by the computing resource limit. For example, if it is measured that each CPU core can handle an average of 50 requests simultaneously, and the computing resource limit is 2.4 cores, then the concurrent processing threshold is set to 120 requests.

[0080] The cache processing threshold for service nodes is determined based on storage resource limits. The cache processing threshold refers to the maximum amount of data a service node can cache. Based on historical statistics, the amount of data that can be effectively cached per unit of storage resource is determined, and then multiplied by the storage resource limit to obtain the cache processing threshold. If approximately 100,000 records can be effectively cached per GB of memory, and the storage resource limit is 5.6 GB, then the cache processing threshold is set to 560,000 records.

[0081] The traffic scheduling strategy for service nodes is updated based on concurrency and cache thresholds to achieve intelligent diagnosis and recovery from system failures. The traffic scheduling strategy consists of two parts: inbound traffic control and outbound traffic redirection. Inbound traffic control is based on concurrency thresholds; when the current number of concurrent requests approaches 80% of the threshold, rate limiting begins; when the threshold is reached, new requests are rejected or forwarded to other healthy nodes. Outbound traffic redirection is based on service call chain analysis, identifying downstream nodes that frequently interact with the faulty node and redirecting calls to these nodes to other healthy instances, reducing the propagation of failures.

[0082] By analyzing the causal strength score over time, the severity of fault impact was accurately quantified, providing a scientific basis for subsequent resource isolation and traffic scheduling. An adaptive adjustment mechanism based on fault impact dynamically generates differentiated recovery strategies according to fault severity, avoiding the rigidity of traditional fixed threshold methods. The coordinated optimization of resource isolation and traffic scheduling ensures overall system availability while minimizing the impact on normal business operations. This provides strong support for the stable operation of core enterprise businesses and has broad application value in large-scale distributed systems.

[0083] A second aspect of this invention provides a system for intelligent diagnosis and recovery of system anomalies that integrates time-series logs, the system comprising: The log collection unit is used to collect runtime logs from multiple service nodes in the distributed system and align them according to timestamps. It then extracts call relationship and performance measurement data from the time-aligned runtime logs. The feature extraction unit is used to calculate the time-series variation features of performance metric data within a sliding time window, fuse and encode the time-series variation features with the call relationship, generate dynamic feature vectors corresponding to service nodes, and construct a cross-service time-series association graph based on the call relationship of the dynamic feature vectors. The fault matching unit is used to identify abnormal service nodes from the cross-service time sequence correlation graph, match the dynamic feature vectors corresponding to the abnormal service nodes with historical fault records in the historical fault database, and obtain the historical propagation paths and historical feature vectors in the historical fault records. The causal analysis unit is used to calculate the propagation convergence coefficient of each service node in the cross-service time-series correlation graph based on the historical propagation path, calculate the feature deviation degree between the dynamic feature vector and the historical feature vector of each service node, and fuse the propagation convergence coefficient and the feature deviation degree to obtain the causal strength score of each service node. The fault recovery unit is used to identify the degree of fault impact based on the causal strength score, and adaptively adjust the resource isolation strategy and traffic scheduling strategy of the service node based on the degree of fault impact to realize intelligent diagnosis and recovery of system faults.

[0084] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0085] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0086] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent diagnosis and recovery of system anomalies by integrating time-series logs, characterized in that, include: Run logs are collected from multiple service nodes in a distributed system and aligned to a time sequence by timestamps. Call relationships and performance metrics are then extracted from the time-series aligned run logs. Within a sliding time window, the time-series variation characteristics of performance metrics are calculated, and the time-series variation characteristics are fused and encoded with the call relationship to generate dynamic feature vectors corresponding to service nodes. Based on the call relationship, the dynamic feature vectors are used to construct a cross-service time-series association graph. Abnormal service nodes are identified from the cross-service time sequence correlation graph. The dynamic feature vectors corresponding to the abnormal service nodes are matched with historical fault records in the historical fault database to obtain the historical propagation paths and historical feature vectors in the historical fault records. Based on the historical propagation path, the propagation convergence coefficient of each service node in the cross-service temporal relationship graph is calculated, the feature deviation degree between the dynamic feature vector and the historical feature vector of each service node is calculated, and the causal strength score of each service node is obtained by fusing the propagation convergence coefficient and the feature deviation degree. The severity of a fault is identified by a causal strength score, and the resource isolation and traffic scheduling strategies of service nodes are adaptively adjusted based on the severity of the fault to achieve intelligent diagnosis and recovery of system faults.

2. The method according to claim 1, characterized in that, Runtime logs are collected from multiple service nodes in a distributed system and aligned to a specific time sequence using timestamps. Call relationship and performance metric data are extracted from the time-aligned logs, including: Raw operation logs are collected from multiple service nodes. Global clock synchronization calibration is performed on the raw operation logs based on timestamps. The calibrated raw operation logs are then sorted according to the order of timestamps to construct time-aligned operation logs. Extract request tracing identifiers from time-aligned runtime logs, associate log records spanning multiple service nodes into a complete call chain based on the request tracing identifiers, and construct weighted call relationships based on the call order and call frequency of service nodes in the call chain; The performance metrics data of each service node are extracted from the time-aligned runtime logs. The performance metrics data are then grouped into time-series slices according to timestamps. The statistical feature values ​​of the performance metrics within each time-series slice are calculated. Finally, the statistical feature values ​​of the performance metrics are matched with the call relationships to generate performance measurement data.

3. The method according to claim 1, characterized in that, Within a sliding time window, the time-series variation characteristics of performance metrics are calculated. These characteristics are then fused and encoded with call relationships to generate dynamic feature vectors corresponding to service nodes. Based on the call relationships, these dynamic feature vectors are used to construct a cross-service time-series correlation graph, including: A sliding time window is set to segment the performance measurement data in time series. Within each sliding time window, a time series phase space of the performance measurement data is constructed. The trajectory curvature and attractor features of the performance measurement data are calculated in the time series phase space as time series variation features. Identify the level depth and branch density of service nodes in the call chain from the call relationship, and calculate the attenuation coefficient of the call propagation path based on the level depth and branch density; The temporal change features of upstream service nodes in the call relationship are weighted and aggregated according to the call attenuation coefficient. The temporal change features and the weighted aggregated upstream temporal features are then fused and encoded across levels through a multi-head attention mechanism to generate dynamic feature vectors corresponding to the service nodes. Using service nodes as graph nodes and assigning them dynamic feature vectors, the cosine similarity between the dynamic feature vectors of adjacent service nodes in the call relationship is calculated. The cosine similarity is then used as the weight value of the graph edge to construct a cross-service temporal association graph.

4. The method according to claim 1, characterized in that, Abnormal service nodes are identified from the cross-service time-series correlation graph. The dynamic feature vectors corresponding to the abnormal service nodes are matched with historical fault records in the historical fault database to obtain the historical propagation paths and historical feature vectors from the historical fault records, including: Calculate the dynamic feature vector difference matrix of adjacent time windows in the cross-service temporal correlation graph, decompose the dynamic feature vector difference matrix to obtain the temporal fluctuation component and the topology propagation component, and construct the state transition sequence based on the temporal fluctuation component and the topology propagation component. The entropy values ​​of the temporal fluctuation component matrix and the topology propagation component matrix in the state transition sequence are calculated. Anomaly scores are generated by combining the entropy values ​​of the temporal fluctuation component and the topology propagation component. Anomaly service nodes are identified based on the anomaly scores, and anomaly propagation links centered on the anomaly service nodes are extracted from the state transition sequence. The temporal fluctuation spectrum component is extracted from the dynamic feature vector of the abnormal service node, and the propagation delay spectrum component is extracted from the abnormal propagation link. The temporal fluctuation spectrum component and the propagation delay spectrum component are combined to construct a dual-spectrum feature. Based on the dual-spectrum feature, historical fault records are retrieved and matched in the historical fault database. Based on the historical fault records, historical propagation paths and historical feature vectors are obtained.

5. The method according to claim 1, characterized in that, Based on historical propagation paths, the propagation convergence coefficient of each service node in the cross-service temporal correlation graph is calculated. The feature deviation between the dynamic feature vector and the historical feature vector of each service node is calculated. The causal strength score of each service node is obtained by fusing the propagation convergence coefficient and the feature deviation. The directed propagation edges between historical service nodes and the historical latency distribution corresponding to the directed propagation edges are extracted from the historical propagation path. Candidate propagation subgraphs that match the topology of the historical propagation path are identified in the cross-service temporal association graph. The upstream propagation tree is constructed by multi-hop backtracking along the in-degree direction for each target service node in the candidate propagation subgraph. Calculate the latency evolution deviation between the historical latency distribution of directed propagation edges in the upstream propagation tree and the actual latency distribution of the corresponding directed propagation edges in the cross-service time sequence association graph; Based on the time delay evolution deviation, the directed propagation edges in the upstream propagation tree are weighted by propagation attenuation and normalized to obtain the propagation convergence coefficient of the target service node. Temporal components and topological components are extracted from the dynamic feature vectors and historical feature vectors of each target service node, respectively. The temporal deviation between temporal components and the topological deviation between topological components are calculated. The temporal deviation and topology deviation are weighted and combined to generate the feature deviation of each target service node; Calculate the topological centrality of the target service node in the cross-service temporal correlation graph, and adaptively weight and fuse the propagation convergence coefficient and feature deviation degree based on the topological centrality to obtain the causal strength score of each target service node.

6. The method according to claim 1, characterized in that, The system identifies the degree of fault impact based on causality strength scores, and adaptively adjusts resource isolation and traffic scheduling strategies for service nodes based on this degree of impact to achieve intelligent fault diagnosis and recovery. The causal intensity score is divided into multiple score sequences according to a fixed time window. The mean change and variance change are extracted from the score sequences. The cumulative value of the mean change is calculated to obtain the fault diffusion degree, and the cumulative value of the variance change is calculated to obtain the fault volatility degree. The product of the fault diffusion degree and the fault volatility degree is used as the degree of fault impact. Based on the degree of impact of the fault, monitor the changes in resource usage of the service node, calculate the growth rate and decay rate of resource usage, mark the time period when the growth rate exceeds the preset growth threshold as the resource isolation start point, and mark the time period when the decay rate exceeds the preset decay threshold as the resource isolation end point, and generate the resource isolation strategy for the service node based on the resource isolation start point and resource isolation end point. The resource isolation strategy is converted into a resource constraint vector. Computational resource constraint values ​​and storage resource constraint values ​​are extracted from the resource constraint vector. The concurrent processing threshold of the service node is determined by the computational resource constraint value, and the cache processing threshold of the service node is determined by the storage resource constraint value. The traffic scheduling strategy of the service node is updated according to the concurrent processing threshold and the cache processing threshold, so as to realize intelligent diagnosis and recovery of system faults.

7. A system for intelligent diagnosis and recovery of system anomalies integrating time-series logs, used to implement the method of any one of claims 1-6, characterized in that, include: The log collection unit is used to collect runtime logs from multiple service nodes in the distributed system and align them according to timestamps. It then extracts call relationship and performance measurement data from the time-aligned runtime logs. The feature extraction unit is used to calculate the time-series variation features of performance metric data within a sliding time window, fuse and encode the time-series variation features with the call relationship, generate dynamic feature vectors corresponding to service nodes, and construct a cross-service time-series association graph based on the call relationship of the dynamic feature vectors. The fault matching unit is used to identify abnormal service nodes from the cross-service time sequence correlation graph, match the dynamic feature vectors corresponding to the abnormal service nodes with historical fault records in the historical fault database, and obtain the historical propagation paths and historical feature vectors in the historical fault records. The causal analysis unit is used to calculate the propagation convergence coefficient of each service node in the cross-service time-series correlation graph based on the historical propagation path, calculate the feature deviation degree between the dynamic feature vector and the historical feature vector of each service node, and fuse the propagation convergence coefficient and the feature deviation degree to obtain the causal strength score of each service node. The fault recovery unit is used to identify the degree of fault impact based on the causal strength score, and adaptively adjust the resource isolation strategy and traffic scheduling strategy of the service node based on the degree of fault impact to realize intelligent diagnosis and recovery of system faults.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images