An end-to-end metadata closed loop management method and system

Abstract

The application relates to the technical field of data management, and provides an end-to-end metadata closed-loop management method and system, which solves the problem that autonomous operation and maintenance is difficult in the prior art. The method comprises the following steps: acquiring calling chain topology data, time sequence data and temperature and humidity data; performing space-time alignment on the calling chain topology data and the time sequence data to obtain a space-time tensor, and performing physical feature extraction on the temperature and humidity data to obtain a physical vector; inputting the space-time tensor and the physical vector into a space-time graph neural network for dynamic fusion to generate a root cause analysis report; performing time sequence causal inference on the root cause analysis report based on a dynamic Bayesian network to obtain a causal structure diagram; performing decision planning on the causal structure diagram based on a Monte Carlo tree search to obtain a candidate action sequence, and calculating a stability risk coefficient, a resource consumption cost and a system recovery probability; and performing collaborative evaluation to obtain an optimal action sequence and issuing an execution engine. The application can realize end-to-end autonomous operation and maintenance and accurate intervention of microservice performance bottlenecks.

Description

Technical Field

[0001] This application relates to the technical field of data management, and in particular to an end-to-end closed-loop metadata management method and system. Background Technology

[0002] In scenarios where a unified data platform provides data interface services to the outside world, full-link monitoring and end-to-end autonomous operation and maintenance management of the performance of microservice systems have important application prospects for ensuring service level agreements and improving system stability.

[0003] Currently, the field of microservice governance mainly relies on traditional observability technologies such as distributed tracing, indicator monitoring, and log collection, and then uses preset thresholds or simple association rules for anomaly detection and alerts.

[0004] However, these existing methods have the following drawbacks: On the one hand, traditional monitoring methods are difficult to effectively integrate multimodal data such as call chain topology, dynamic load and underlying physical environment, and cannot accurately capture performance bottlenecks under complex dependencies; on the other hand, existing solutions mostly stop at root cause alarms and lack the ability to make autonomous decisions and execute closed-loop strategies for optimal intervention, which leads to low fault recovery efficiency and insufficient automation. Summary of the Invention

[0005] The purpose of this application is to provide an end-to-end closed-loop metadata management method and system to solve the problem of high difficulty in autonomous operation and maintenance in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides an end-to-end closed-loop metadata management method, comprising:

[0007] Obtain microservice call chain topology data, time-series data of each service instance, and data center rack-level temperature and humidity data;

[0008] The call chain topology data and the time series data are spatiotemporally aligned to obtain a spatiotemporal tensor. Physical features are extracted from the temperature and humidity data to obtain a physical vector.

[0009] The spatiotemporal tensor and the physical vector are input into a spatiotemporal graph neural network for dynamic fusion to generate a root cause analysis report.

[0010] Based on the root cause analysis report, a dynamic Bayesian network is used to perform temporal causal inference and generate a causal structure diagram.

[0011] Based on the causal structure graph, decision planning is performed using the Monte Carlo tree search algorithm to generate multiple candidate action sequences. For each candidate action sequence, the corresponding stability risk coefficient, resource consumption cost, and system recovery probability are calculated.

[0012] By combining the stability risk coefficient, the resource consumption cost, and the system recovery probability, a collaborative evaluation operation is performed on each candidate action sequence to obtain the optimal action sequence, and the optimal action sequence is sent to the execution engine to complete the metadata management operation.

[0013] Optionally, the step of using a dynamic Bayesian network to perform time-series causal inference based on the root cause analysis report and generating a causal structure graph includes:

[0014] The service nodes, physical nodes, and performance indicators in the root cause analysis report are respectively defined as the first variable node, the second variable node, and the third variable node in the dynamic Bayesian network.

[0015] Based on the contribution distribution vector in the root cause analysis report, the initial dependency relationship between the first variable node, the second variable node, and the third variable node is determined, and the network structure of the dynamic Bayesian network in the initial time slice is constructed based on the initial dependency relationship.

[0016] The time axis is divided into multiple consecutive time slices. Using the network structure as a template, a time-series slice network of the dynamic Bayesian network is constructed for each time slice. Multiple time-series slice networks are connected in chronological order to form the complete topology of the dynamic Bayesian network.

[0017] The belief propagation algorithm is used to perform iterative message passing in the complete topology to update the conditional probability distribution of each first variable node, second variable node and third variable node, and to calculate the causal strength coefficient between each variable node based on the updated conditional probability distribution.

[0018] Based on the causal strength coefficient, causal paths with a value greater than or equal to a preset threshold are selected from the complete topology, and all the causal paths are aggregated to form a causal structure graph.

[0019] Optionally, constructing the network structure of the dynamic Bayesian network in the initial time slice based on the initial dependencies includes:

[0020] Based on the contribution distribution vector, calculate the mutual information value between every two variable nodes, and use the mutual information value as the correlation metric between variable nodes;

[0021] Using the aforementioned correlation metric as a priori weight, a causal discovery algorithm based on an additive noise model is employed to infer the causal direction of each variable node, generating a set of edges with candidate directions.

[0022] The set of edges is used as the search space, and the tabu search algorithm is used to iteratively optimize the search space. Each iteration starts from the current structure and performs neighborhood operations such as adding, deleting, or reversing edges, and uses a tabu list to record the most recently performed neighborhood operations.

[0023] The iteration stops when the preset number of iterations is reached, and the structure of the last generation is used as the network structure of the dynamic Bayesian network in the initial time slice.

[0024] Optionally, the step of inputting the spatiotemporal tensor and the physical vector into a spatiotemporal graph neural network for dynamic fusion to generate a root cause analysis report includes:

[0025] The spatiotemporal tensor is input into the spatiotemporal graph neural network. The graph convolutional layer of the spatiotemporal graph neural network captures the spatial dependencies between service nodes and outputs spatial features. The temporal convolutional layer of the spatiotemporal graph neural network captures the temporal dependencies of each service node and outputs temporal features. The spatial features and the temporal features are concatenated to obtain the spatiotemporal feature vector of each service node.

[0026] Based on the preset mapping relationship between service instances and physical hosts, the physical vector is mapped to the corresponding service node to obtain the physical feature vector of each service node;

[0027] Using the spatiotemporal feature vector as the query and the physical feature vector as the key and value, the cross-modal attention weight of each service node is calculated through the multi-head attention layer of the spatiotemporal graph neural network, and the physical feature vector is weighted according to the cross-modal attention weight to obtain the weighted physical features of each service node.

[0028] The fusion layer of the spatiotemporal graph neural network dynamically calculates the fusion coefficient through a learnable gating weight vector, and based on the fusion coefficient, the spatiotemporal feature vector and the weighted physical features are weighted and summed to obtain the fusion feature vector of each service node.

[0029] The fully connected layer of the spatiotemporal graph neural network is used to perform nonlinear processing on the fused feature vector using the softmax function to generate contribution distribution vectors for multiple preset root cause types for each service node. The contribution distribution vectors of all preset root cause types are then aggregated to form a root cause analysis report.

[0030] Optionally, the step of performing decision planning based on the causal structure graph using the Monte Carlo tree search algorithm to generate multiple candidate action sequences, and calculating the corresponding stability risk coefficient, resource consumption cost, and system recovery probability for each candidate action sequence, includes:

[0031] Using the current system state of the service cluster as the root node and the causal path in the causal structure graph as the state transition constraint when the root node is expanded, a Monte Carlo search tree is constructed.

[0032] At each search node of the Monte Carlo search tree, based on the time delay parameters and confidence intervals corresponding to the causal structure graph, multiple initial actions that each search node can perform are determined.

[0033] The expansion operation is performed iteratively starting from the root node until a preset number of iterations is reached. Each expansion operation includes:

[0034] Using the upper confidence bound algorithm, based on the causal strength coefficient in the causal structure graph, the state transition after executing the initial action corresponding to the current node is simulated to generate a new child node;

[0035] When each expansion operation reaches a preset simulation depth, starting from the current child node, the random selection probability of each initial action is set according to the causal strength coefficient. According to the random selection probability, the initial action is randomly selected until the leaf node is reached to obtain the cumulative reward value of this simulation. The cumulative reward value is then backpropagated to update the average reward value of each node.

[0036] After reaching the preset number of iterations, select multiple candidate action sequences from each of the child nodes corresponding to the root node whose average return value meets the preset return condition;

[0037] For each candidate action sequence, the corresponding stability risk coefficient is calculated based on the confidence interval of each causal path in the causal structure graph. The corresponding resource consumption cost is calculated based on the action type and number contained in the candidate action sequence. The corresponding system recovery probability is calculated based on the cumulative reward value corresponding to the candidate action sequence.

[0038] Optionally, the step of combining the stability risk coefficient, the resource consumption cost, and the system recovery probability to perform a collaborative evaluation operation on each candidate action sequence to obtain the optimal action sequence includes:

[0039] The stability risk coefficient, the resource consumption cost, and the system recovery probability are used as three evaluation dimensions for the corresponding candidate action sequences to construct an evaluation vector for each candidate action sequence.

[0040] The evaluation vector is input into a pre-constructed multi-objective optimization model, which aims to minimize the stability risk coefficient, minimize the resource consumption cost, and maximize the system recovery probability.

[0041] The solution process of the multi-objective optimization model is modeled as a multi-objective Markov decision process. Based on the optimization objective, the Pareto front method is used to search for a set of candidate solutions that satisfy the Pareto optimality condition in the state space of the multi-objective Markov decision process through non-dominated sorting.

[0042] From the set of candidate solutions, each candidate solution is weighted and summed according to a preset decision preference weight to obtain the preference score of each candidate solution, and the candidate action sequence corresponding to the candidate solution with the highest preference score is selected as the optimal action sequence.

[0043] Optionally, the step of performing spatiotemporal alignment processing on the call chain topology data and the time series data to obtain a spatiotemporal tensor includes:

[0044] Based on the call chain topology data, the hierarchy depth and call relationship of each service instance in the service call chain are extracted, and a location encoding vector for each service instance is generated based on the hierarchy depth and the call relationship.

[0045] A third-order tensor is constructed using the time-series data of each service instance corresponding to each time point as the first dimension, each service instance as the second dimension, and the location encoding vector as the third dimension.

[0046] By adjusting the numerical distribution of different service instances in the third-order tensor at the same point in time using preset spatial constraints, the adjusted third-order tensor is obtained.

[0047] Tensor Tucker decomposition is performed on the adjusted third-order tensor to preserve the core tensor and the three factor matrices. With the goal of minimizing the reconstruction error, the core tensor and the three factor matrices are alternately optimized to generate a spatiotemporal tensor.

[0048] Secondly, this application provides an end-to-end metadata closed-loop management system, including:

[0049] The acquisition module is used to acquire microservice call chain topology data, time-series data of each service instance, and temperature and humidity data at the data center rack level.

[0050] The extraction module is used to perform spatiotemporal alignment processing on the call chain topology data and the time series data to obtain a spatiotemporal tensor, and to extract physical features from the temperature and humidity data to obtain a physical vector.

[0051] The fusion module is used to input the spatiotemporal tensor and the physical vector into the spatiotemporal graph neural network for dynamic fusion and generate a root cause analysis report.

[0052] The inference module is used to perform time-series causal inference based on the root cause analysis report using a dynamic Bayesian network to generate a causal structure diagram.

[0053] The calculation module is used to perform decision planning based on the causal structure graph using the Monte Carlo tree search algorithm, generate multiple candidate action sequences, and calculate the corresponding stability risk coefficient, resource consumption cost, and system recovery probability based on each candidate action sequence.

[0054] The evaluation module is used to perform a collaborative evaluation operation on each candidate action sequence by combining the stability risk coefficient, the resource consumption cost, and the system recovery probability to obtain the optimal action sequence, and then send the optimal action sequence to the execution engine to complete the metadata management operation.

[0055] Thirdly, this application provides an electronic device, comprising:

[0056] Memory, used to store computer programs;

[0057] A processor, used to execute the computer program, implements the steps of an end-to-end metadata closed-loop management method as described in the first aspect above.

[0058] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of an end-to-end metadata closed-loop management method as described in the first aspect above.

[0059] The end-to-end metadata closed-loop management method provided in this application has the following beneficial effects:

[0060] This application provides a multi-dimensional observational foundation for system analysis by acquiring microservice call chain topology data, time-series data, and rack temperature and humidity data. Next, the call chain topology data and time-series data are spatiotemporally aligned, and the physical features of the temperature and humidity data are extracted to eliminate heterogeneity differences among multi-source data, constructing a unified spatiotemporal tensor and physical vector. These are then dynamically fused into a spatiotemporal graph neural network, effectively capturing the spatial dependencies and temporal evolution patterns between service nodes and generating a root cause analysis report containing contribution distributions. Subsequently, based on this report, a dynamic Bayesian network is used for temporal causal inference, revealing the temporal dependencies and causal relationships between variables to form an interpretable causal structure graph. Then, Monte Carlo tree search is used for decision planning, generating candidate action sequences and quantifying their risks, costs, and recovery probabilities. Finally, multi-objective collaborative evaluation is performed on each sequence to select the optimal action sequence for execution, thus achieving a closed-loop control throughout the entire process from anomaly detection and root cause localization to autonomous decision-making.

[0061] Furthermore, this application defines service nodes, physical nodes, and performance indicators as variable nodes in a dynamic Bayesian network, and constructs an initial time-slice network based on the contribution distribution to determine the initial dependencies. Subsequently, this network is used as a template to expand along the time axis into multiple time-series slices and connect them into a complete topology. Then, the belief propagation algorithm is used to iteratively update the conditional probability distribution of each node and calculate the causal strength coefficient. Finally, high-intensity causal paths are selected to form a causal structure graph. This process can quantify the temporal causal strength between variables, thereby effectively discovering potential transmission paths of system anomalies and improving the interpretability and accuracy of root cause analysis. Attached Figure Description

[0062] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 A flowchart illustrating an end-to-end metadata closed-loop management method provided in an embodiment of this application;

[0064] Figure 2 A schematic diagram illustrating a specific implementation of an end-to-end metadata closed-loop management method provided in this application embodiment;

[0065] Figure 3 A schematic diagram of the structure of an end-to-end metadata closed-loop management system provided in an embodiment of this application;

[0066] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0067] In scenarios where a unified data foundation provides data interface services to the outside world, the performance and stability of a microservice system directly determine the level of achievement of service level agreements (SLAs). However, existing microservice governance technologies have the following obvious shortcomings: On the one hand, traditional monitoring methods are difficult to effectively integrate multi-dimensional data such as call chain topology, dynamic load, and underlying physical environment, and have limited ability to accurately locate performance bottlenecks in complex dependencies; on the other hand, existing solutions mostly stop at root cause alerts and lack the ability to make autonomous decisions and execute closed-loop strategies for optimal intervention, which leads to low fault recovery efficiency and insufficient automation.

[0068] To address this, this application proposes an end-to-end closed-loop metadata management method, the technical concept of which is as follows: First, multi-source observation data such as call chain topology, time-series data, and cabinet temperature and humidity are acquired and spatiotemporally aligned and physically extracted to eliminate heterogeneity differences; then, a root cause analysis report is generated through dynamic fusion using a spatiotemporal graph neural network, and a dynamic Bayesian network is used for temporal causal inference to reveal the transmission path between variables; finally, decision planning is performed based on the causal structure graph using Monte Carlo tree search, and the optimal action sequence is selected and executed after multi-objective collaborative evaluation. This scheme, by fusing multimodal data, revealing causal dependencies, and achieving autonomous decision-making, constructs a closed-loop control system covering the entire process from anomaly perception and root cause localization to strategy execution, which can solve the problems of inaccurate localization, delayed intervention, and lack of autonomy in existing technologies.

[0069] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] The core of this application is to provide an end-to-end closed-loop metadata management method, the flowchart of one specific implementation of which is shown below. Figure 1 As shown, the method includes:

[0071] S101. Obtain the call chain topology data of the microservice, the time sequence data of each service instance, and the temperature and humidity data at the data center rack level.

[0072] Among them, the call chain topology data refers to the data that records the call relationship and path structure between microservices, which can fully present all the service nodes that a business request passes through in a distributed system and their order.

[0073] Time-series data refers to performance metrics generated by each service instance at consecutive points in time. It typically includes key metrics such as response time, request volume, error rate, and CPU utilization, and is used to reflect changes in the service's operational status.

[0074] In step S101, the complete call path of business requests between service instances is collected by a distributed tracing system deployed in the microservice architecture, thereby generating call chain topology data; at the same time, the time-series data of each service instance is collected by the monitoring system at fixed time intervals; in addition, the temperature and humidity data at the rack level are acquired in real time by sensor devices deployed in the data center rack.

[0075] S102. Perform spatiotemporal alignment processing on the call chain topology data and the time series data to obtain a spatiotemporal tensor, and extract physical features from the temperature and humidity data to obtain a physical vector.

[0076] In one specific implementation, step S102 includes:

[0077] Step 1021: Based on the call chain topology data, extract the level depth and call relationship of each service instance in the service call chain, and generate the location encoding vector of each service instance based on the level depth and the call relationship.

[0078] In step 1021, firstly, based on the service call paths recorded in the call chain topology data, the layer depth of each service instance in the call chain is determined by a breadth-first traversal algorithm, i.e., the number of nodes traversed from the entry service to the current service. Simultaneously, based on the call relationships between service instances, an adjacency matrix is ​​constructed to describe the call relationships between services. Subsequently, the layer depth of each service instance is one-hot encoded to obtain depth features, and the row vector corresponding to the service in the adjacency matrix is ​​used as the relation features. Then, the depth features and relation features are concatenated and subjected to a nonlinear transformation, finally outputting a dense vector of fixed dimensions, which is the location encoding vector of the service instance.

[0079] Step 1022: Construct a third-order tensor using the time-series data of each service instance corresponding to each time point as the first dimension, each service instance as the second dimension, and the location encoding vector as the third dimension.

[0080] In step 1022, after obtaining the location encoding vectors of all service instances, a tensor basis is constructed with time points as the first dimension: the time series data of all service instances at each time point are arranged into a matrix, where the rows of the matrix correspond to service instances and the columns correspond to different performance indicators; then, the location encoding vector corresponding to each service instance is used as the third dimension, and an outer product operation is performed with the above matrix to construct a third-order tensor. The three dimensions of this third-order tensor represent: time point, service instance, and topological features encoded by the location encoding vector.

[0081] Step 1023: Adjust the numerical distribution of different service instances in the third-order tensor at the same point in time by using preset spatial constraints to obtain the adjusted third-order tensor.

[0082] In step 1023, the numerical distribution of different service instances in the constructed third-order tensor at the same time point may vary significantly, which may affect subsequent analysis. Therefore, a spatial constraint term based on the call similarity between service instances is introduced: First, the call frequency similarity between pairs of service instances is calculated based on the call chain topology data to generate a spatial similarity matrix; then, this matrix is ​​used as a regularization term to smooth the numerical distribution of different service instances at the same time point in the third-order tensor. Specifically, the adjusted third-order tensor is obtained by minimizing the difference between the original distribution and the distribution after spatial similarity weighting.

[0083] Step 1024: Perform tensor Tucker decomposition on the adjusted third-order tensor to retain the core tensor and the three factor matrices. With the goal of minimizing the reconstruction error, alternately optimize the core tensor and the three factor matrices to generate a spatiotemporal tensor.

[0084] Among them, the core tensor refers to the low-dimensional latent feature space extracted from the original three-dimensional data after tensor decomposition. It captures the high-order interaction relationships between the three dimensions and reflects the essential characteristics of the co-evolution of time patterns, service behaviors and topological structures during system operation.

[0085] In step 1024, tensor Tucker decomposition is performed on the adjusted third-order tensor to decompose it into a core tensor and three factor matrices. The decomposition process aims to minimize the reconstruction error, that is, to make the tensor reconstructed from the core tensor and the three factor matrices as close as possible to the original adjusted third-order tensor. Then, iterative optimization is performed using alternating least squares: two of the factor matrices are fixed, and the third is updated; this process is repeated until a preset convergence condition is met. The resulting core tensor is the desired spatiotemporal tensor, which integrates the temporal evolution pattern, service instance changes, and topological structure information. The convergence condition can be set to stop the iteration when any of the following conditions are met: the number of iterations reaches a preset maximum number of iterations, such as 100, or the relative change in reconstruction error between two adjacent iterations is less than a preset threshold, such as 10. -6 .

[0086] For example, the Tucker decomposition optimization process is as follows: ,in, This represents the adjusted third-order tensor. The core tensor is represented by A, B, and C, which are factor matrices for the time dimension, service instance dimension, and topology location dimension, respectively. This represents the product of a tensor and a matrix modulo n. This represents the Frobenius norm.

[0087] This application eliminates dimensional differences between data by uniformly representing heterogeneous multi-source data as a compact spatiotemporal tensor, thereby laying a structured data foundation for subsequent fusion analysis.

[0088] S103. Input the spatiotemporal tensor and the physical vector into the spatiotemporal graph neural network for dynamic fusion to generate a root cause analysis report.

[0089] In this embodiment, the spatiotemporal graph neural network can be designed with the following structure: the input layer receives a spatiotemporal tensor with a dimension of time step × number of service nodes × feature dimension; then three spatiotemporal convolutional blocks are stacked, each containing a graph convolutional layer and a temporal convolutional layer. The graph convolutional layer uses Chebyshev graph convolution with order K=3 and 64 output channels, while the temporal convolutional layer uses dilated causal convolution with a kernel size of 3, a dilation rate of [1, 2, 4], and 64 output channels; then a multi-head attention layer is connected, with 8 attention heads and a dimension of 16 for each head; finally, a fusion layer and a fully connected layer are connected, with an output dimension equal to the preset number of root cause types, such as 8 types; and the Adam optimizer is used for training with an initial learning rate of 0.001, 100 training epochs, an early stopping mechanism patience of 10, and a cross-entropy loss function. Root cause labeled data is used as a supervision signal for training.

[0090] In one specific implementation, step S103 includes:

[0091] Step 1031: Input the spatiotemporal tensor into the spatiotemporal graph neural network. The graph convolutional layer of the spatiotemporal graph neural network captures the spatial dependencies between service nodes and outputs spatial features. The temporal convolutional layer of the spatiotemporal graph neural network captures the temporal dependencies of each service node and outputs temporal features. Concatenate the spatial features and the temporal features to obtain the spatiotemporal feature vector of each service node.

[0092] In step 1031, the spatiotemporal tensor is first input into the spatiotemporal graph neural network. This network adopts a two-way parallel processing structure: one path performs convolution operations on the call relationship graph between service nodes through graph convolutional layers, that is, each service node aggregates the feature information of its neighboring nodes to capture the spatial dependencies between nodes and outputs spatial features containing topological structure information; the other path performs convolution operations on the time series data of each service node through temporal convolutional layers, that is, dilated causal convolution is used to expand the receptive field and capture the evolution of each node's own performance indicators over time, thereby outputting temporal features reflecting the changing trend; then, the spatial features and temporal features of the same service node are concatenated along the feature dimension to obtain the spatiotemporal feature vector of each service node. This vector simultaneously encodes the service's position information in the call topology and its own performance temporal evolution information.

[0093] Step 1032: Based on the preset mapping relationship between service instances and physical hosts, map the physical vector to the corresponding service node to obtain the physical feature vector of each service node.

[0094] In step 1032, based on the preset mapping relationship between service instances and physical hosts, the rack where the physical host deployed by each service node is located is queried. Then, the physical vector of the rack is copied and mapped to all service nodes deployed in the rack. For multiple service nodes deployed in the same rack, they share the physical vector of the rack. For service instances deployed across racks, the physical vector of their respective racks is obtained. Finally, through this mapping operation, a corresponding physical feature vector is generated for each service node.

[0095] Step 1033: Using the spatiotemporal feature vector as the query and the physical feature vector as the key and value, calculate the cross-modal attention weight of each service node through the multi-head attention layer of the spatiotemporal graph neural network, and weight the physical feature vector according to the cross-modal attention weight to obtain the weighted physical features of each service node.

[0096] In step 1033, the spatiotemporal feature vector of each service node is used as the query, and the physical feature vector of that node is used as the key and value. Both are input into the multi-head attention layer of the spatiotemporal graph neural network. This layer first calculates the similarity between the query and the key through dot product operation to obtain the original attention score. Then, the original attention score is normalized by the softmax function to obtain the cross-modal attention weight. Finally, the physical feature vector is weighted and summed according to the cross-modal attention weight to obtain the weighted physical feature of each service node.

[0097] Step 1034: Utilize the fusion layer of the spatiotemporal graph neural network to dynamically calculate the fusion coefficient through a learnable gating weight vector, and based on the fusion coefficient, perform a weighted summation of the spatiotemporal feature vector and the weighted physical features to obtain the fusion feature vector of each service node.

[0098] In step 1034, the spatiotemporal feature vector and weighted physical features of each service node are input into the fusion layer of the spatiotemporal graph neural network. This layer first initializes a learnable gating weight vector with the same dimension as the feature dimension. Then, the gating weight vector is mapped to the range of 0-1 using the sigmoid function to obtain the fusion coefficient. Finally, the spatiotemporal feature vector and weighted physical features are summed element-wise according to the coefficient to generate the fusion feature vector of each service node.

[0099] Step 1035: Through the fully connected layer of the spatiotemporal graph neural network, the softmax function is used to perform nonlinear processing on the fused feature vector to generate contribution distribution vectors for multiple preset root cause types for each service node. The contribution distribution vectors of all preset root cause types are collected to form a root cause analysis report.

[0100] In step 1035, the fused feature vector of each service node is input into the fully connected layer of the spatiotemporal graph neural network, and the features are mapped to a dimensional space with the same number of preset root cause types through a linear transformation. Then, the mapping result is nonlinearly processed by the softmax function, that is, the original score is converted into a probability distribution, thereby generating a contribution distribution vector for each service node corresponding to multiple preset root cause types. The value of each component in the vector represents the probability that the current service node will be abnormal due to that root cause type. Finally, the contribution distribution vectors of all service nodes are collected and organized according to the service node identifier and timestamp to form a complete root cause analysis report.

[0101] This application realizes deep interaction and feature extraction of multimodal information, and generates an analysis report containing the contribution of each service node to various root causes, which can provide accurate root cause localization basis for subsequent causal inference and decision planning.

[0102] S104. Based on the root cause analysis report, use a dynamic Bayesian network to perform temporal causal inference and generate a causal structure diagram.

[0103] In one specific implementation, such as Figure 2 As shown, step S104 includes:

[0104] Step 1041: Define the service nodes, physical nodes, and performance indicators in the root cause analysis report as the first variable node, the second variable node, and the third variable node in the dynamic Bayesian network, respectively.

[0105] In step 1041, all relevant service nodes, physical nodes, and performance metrics are first extracted from the root cause analysis report. Then, each service node is defined as the first variable node in the dynamic Bayesian network, each physical node is the second variable node, and each performance metric is defined as the third variable node. These three types of nodes together constitute the variable set for subsequent causal inference, laying the foundation for establishing the dependency relationship between nodes.

[0106] Step 1042: Based on the contribution distribution vector in the root cause analysis report, determine the initial dependency relationship between the first variable node, the second variable node, and the third variable node, and construct the network structure of the dynamic Bayesian network in the initial time slice based on the initial dependency relationship.

[0107] Step 1042 may specifically include the following steps:

[0108] Step a1: Calculate the mutual information value between every two variable nodes based on the contribution distribution vector, and use the mutual information value as the correlation metric between variable nodes.

[0109] Mutual information value is a statistic that measures the degree of interdependence between two variable nodes. The larger the value, the stronger the correlation between the two variables.

[0110] In step a1, for all variable nodes defined in step 1041, the mutual information value between every two variable nodes is calculated based on the contribution distribution vector in the root cause analysis report. The formula for calculating the mutual information value is as follows: Where X and Y represent two variable nodes, It is the joint probability distribution of the two. and These are their respective marginal probability distributions, and the calculated mutual information value is used as a measure of the association between the two variable nodes to characterize their statistical dependence strength.

[0111] Step a2: Using the correlation metric as the prior weight, a causal discovery algorithm based on an additive noise model is used to infer the causal direction of each variable node and generate a set of edges with candidate directions.

[0112] Among them, the causal discovery algorithm based on the additive noise model refers to an algorithm that infers the causal direction by testing whether there is a functional relationship between variables plus independent noise based on the assumption of the additive noise model. Its implementation process can be referred to relevant technologies, and will not be elaborated here.

[0113] In step a2, using the correlation metric as a prior weight, a causal discovery algorithm based on an additive noise model is used to infer the causal direction of each variable node. This algorithm determines the more likely causal direction by examining whether X can be expressed as a function of Y plus independent noise, and whether Y can be expressed as a function of X plus independent noise. Then, for each pair of variable nodes, the algorithm outputs an edge with a candidate direction, and then gathers all edges with directions to form a set of edges with candidate directions.

[0114] Step a3: Using the set of edges as the search space, perform iterative optimization in the search space using the tabu search algorithm. In each iteration, starting from the current structure, perform neighborhood operations such as adding, deleting, or reversing edges, and use the tabu list to record the most recently performed neighborhood operations.

[0115] In step a3, the set of edges with candidate directions is used as the search space. The tabu search algorithm is used to iteratively optimize the space. In each iteration, starting from the current network structure, one of three neighborhood operations is performed: adding an edge, deleting an edge, or reversing an edge, to generate a candidate new structure. Then, the algorithm evaluates the quality of the new structure according to a scoring function, such as BIC scoring, and uses a tabu list to record the most recently executed neighborhood operations to avoid repeating the same operation in a short period of time, which would lead to a loop. The iteration continues until the preset number of iterations is reached.

[0116] Step a4: Stop iterating when the preset number of iterations is reached, and use the structure of the last generation as the network structure of the dynamic Bayesian network in the initial time slice.

[0117] In step a4, when the number of iterations reaches the preset maximum number of iterations, such as 100, the iteration process of the tabu search algorithm is stopped, and the network structure of the last generation is used as the network structure of the dynamic Bayesian network in the initial time slice. This structure describes the dependency relationship of each variable node at the initial time.

[0118] Step 1043: Divide the time axis into multiple consecutive time slices. Using the network structure as a template, construct a time-series slice network of the dynamic Bayesian network for each time slice. Connect the multiple time-series slice networks in chronological order to form the complete topology of the dynamic Bayesian network.

[0119] In step 1043, the continuous time axis is first divided into multiple equally spaced time slices. Then, using the network structure of the initial time slice as a template, a corresponding time-series slice network is constructed for each time slice, and the inter-node dependencies within each time-series slice network are consistent with the initial network structure. Subsequently, the time-series slice networks of adjacent time slices are connected in chronological order. Specifically, a temporal dependency edge is added between each variable node and its own node in the next time slice, and the evolution of the variable over time is described. Finally, all time-series slice networks are connected to form the complete topology of the dynamic Bayesian network.

[0120] Step 1044: Use the belief propagation algorithm to perform iterative message passing in the complete topology to update the conditional probability distribution of each first variable node, second variable node and third variable node, and calculate the causal strength coefficient between each variable node based on the updated conditional probability distribution.

[0121] In step 1044, on the constructed complete topology, the belief propagation algorithm is used for iterative message passing. The algorithm updates the conditional probability distribution of each first variable node, second variable node, and third variable node by repeatedly sending and receiving messages between adjacent nodes. After the message passing converges, the causal strength coefficient between each variable node is calculated based on the updated conditional probability distribution. This coefficient can be quantified by comparing the change in the conditional probability of the child node when the parent node takes different values, so as to reflect the strength of the causal influence.

[0122] Step 1045: Based on the causal strength coefficient, select causal paths that are greater than or equal to a preset threshold from the complete topology, and gather all the causal paths to form a causal structure graph.

[0123] In step 1045, a preset threshold, such as 0.6, is set based on the causal strength coefficient between each variable node. Then, all causal edges in the complete topology are traversed to filter out edges with a causal strength coefficient greater than or equal to 0.6, and the paths formed by these edges are taken as valid causal paths. Finally, all causal paths that meet the conditions are aggregated to form the final causal structure graph, which clearly reveals the potential paths of fault propagation and the causal relationships between variables in the microservice system.

[0124] This application reveals the causal relationships and fault propagation paths between variable nodes in a microservice system by constructing a dynamic Bayesian network based on root cause analysis reports and performing time-series causal inference. It also generates an interpretable causal structure diagram, which can provide causal guidance for subsequent decision-making and planning.

[0125] S105. Based on the causal structure graph, a Monte Carlo tree search algorithm is used to perform decision planning, generating multiple candidate action sequences. Based on each candidate action sequence, the corresponding stability risk coefficient, resource consumption cost, and system recovery probability are calculated.

[0126] In one specific implementation, step S105 includes:

[0127] Step 1051: Using the current system state of the service cluster as the root node and the causal path in the causal structure graph as the state transition constraint when the root node is expanded, construct a Monte Carlo search tree.

[0128] Monte Carlo search tree refers to a tree-like data structure that is built by gradually expanding the current system state through random simulation, and is used to search for and evaluate the long-term effects of different decision sequences.

[0129] In step 1051, the current system state of the service cluster is first used as the root node of the Monte Carlo search tree. This node contains the performance indicators, physical environment state, and current anomalies of each service instance. When constructing the search tree, the causal structure graph is used as the constraint condition for state transition. That is, when expanding downward from the root node, only actions that conform to the causal path in the causal structure graph and the resulting state changes are allowed to be included in the search scope, so as to ensure that the search process is carried out under the constraints of real causal relationships.

[0130] Step 1052: At each search node of the Monte Carlo search tree, based on the time delay parameters and confidence intervals corresponding to the causal structure graph, determine multiple initial actions that each search node can perform.

[0131] Among them, the time delay parameter refers to the time delay required for the result variable to respond after a change in the causal variable is recorded in the causal structure diagram; the confidence interval refers to the confidence range of each causal path in the causal structure diagram, reflecting the degree of uncertainty of the causal relationship.

[0132] In step 1052, at each search node of the Monte Carlo search tree, all candidate actions are first obtained from the predefined action library based on the system state corresponding to the current node. Each candidate action corresponds to one or more variable nodes that can be intervened in the causal structure graph, such as restarting a service instance or adjusting resource quotas. Subsequently, for each candidate action, the causal path associated with the action is queried in the causal structure graph, and the time delay parameter and confidence interval of each path are obtained.

[0133] Then, a dual screening is performed based on the preset time delay tolerance threshold and confidence threshold: only when the time delay parameter does not exceed the maximum waiting time allowed by the current decision window, and the lower limit of the confidence interval is higher than the confidence threshold, such as 0.6, is the action considered executable in the current node state and included in the initial action of the node. This screening mechanism ensures that only intervention schemes that meet the timeliness requirements and have reliable causal relationships will be considered in subsequent searches.

[0134] Step 1053: Iteratively execute the expansion operation starting from the root node until a preset number of iterations is reached, wherein each expansion operation includes:

[0135] In step 1053, starting from the root node, the expansion operation is repeatedly executed, and each round of expansion attempts to explore new decision paths until the preset maximum number of iterations is reached, such as 1000 times. Each expansion operation follows the standard process of selection, expansion, simulation, and backtracking, gradually enriching the structure and information of the search tree.

[0136] Step 1054: Using the upper confidence bound algorithm, based on the causal strength coefficient in the causal structure graph, simulate the state transition after executing the initial action corresponding to the current node, and generate a new child node.

[0137] In step 1054, the upper confidence bound algorithm is used to guide the action selection of the current node. This algorithm combines the causal strength coefficient in the causal structure graph as prior knowledge and calculates the Q value of each candidate action plus the exploration term. The exploration term is inversely proportional to the number of times the action is tried and directly proportional to the logarithm of the total number of visits. Then, the action with the largest Q value plus the exploration term is selected for simulation execution, and the state change after the action is executed is predicted according to the causal structure graph to generate a new child node.

[0138] The formula for calculating the upper confidence boundary algorithm is as follows: ,in, This indicates the action chosen at time t. It is the average return estimate from action a up to time t. This is the exploration coefficient, which typically ranges from 1.0 to 2.0. It is the causal strength coefficient corresponding to action a. It is the number of times action a has been selected up to time t, where t is the current total number of simulations.

[0139] Step 1055: When each expansion operation reaches the preset simulation depth, starting from the current child node, set the random selection probability of each initial action according to the causal strength coefficient, and randomly select the initial action according to the random selection probability until the leaf node is reached, so as to obtain the cumulative reward value of this simulation, and backpropagate the cumulative reward value to update the average reward value of each node.

[0140] Among them, the preset simulation depth refers to the maximum number of steps that can be pushed forward in each simulation; the random selection probability refers to the random selection weight assigned to each action based on the causal strength coefficient during the simulation phase, and the higher the strength of the action, the greater the probability of it being randomly selected; the leaf node refers to the node when the simulation reaches the preset depth or reaches the termination state.

[0141] In step 1055, when each expansion operation reaches a preset simulation depth, such as 5 steps, the simulation phase begins from the current child node. During the simulation, a random selection probability is set for each initial action based on the causal strength coefficient in the causal structure graph, with higher-strength actions having a greater probability of being selected. Then, actions are randomly selected according to this probability and the simulation proceeds forward until a leaf node is reached. The cumulative reward value of the entire simulation path is recorded, and this value is then propagated upwards along the path to update the number of visits and average reward value of all nodes on the path, providing more accurate statistical information for subsequent selections.

[0142] Step 1056: After reaching the preset number of iterations, select multiple candidate action sequences from the child nodes corresponding to the root node whose average return value meets the preset return conditions.

[0143] Among them, the preset reward conditions refer to the thresholds or sorting rules used to filter candidate action sequences, such as selecting the K paths with the highest average reward value.

[0144] In step 1056, after reaching the preset number of iterations, starting from each child node corresponding to the root node, traversing downwards to the leaf nodes and extracting all complete decision paths; then, based on the average reward value of each node on each path, calculating the comprehensive score of the entire path, and then, according to preset reward conditions, such as selecting the top 10 paths with the highest scores, filtering out multiple candidate action sequences, which represent intervention schemes with high potential benefits that may be executed starting from the current state.

[0145] Step 1057: For each candidate action sequence, calculate the corresponding stability risk coefficient based on the confidence interval of each causal path in the causal structure graph, calculate the corresponding resource consumption cost based on the action type and number included in the candidate action sequence, and calculate the corresponding system recovery probability based on the cumulative reward value corresponding to the candidate action sequence.

[0146] Among them, the stability risk coefficient refers to the quantitative indicator of the risk that executing a certain candidate action sequence may lead to a decrease in system stability; the resource consumption cost refers to the comprehensive quantitative value of the cost of computing resources, network resources, etc. required to execute the candidate action sequence; and the system recovery probability refers to the estimated probability that the system can successfully recover to normal operation after executing the candidate action sequence.

[0147] In step 1057, for each selected candidate action sequence, three evaluation metrics are calculated: First, based on the confidence intervals of each causal path in the causal structure graph, the stability risk coefficient that may be caused by executing the sequence is calculated by weighted summation, and the path with a wider confidence interval contributes a greater risk value; Second, based on the action type and number contained in the candidate action sequence, and in conjunction with the preset action cost table, the total resource consumption cost is calculated; Finally, based on the cumulative reward value obtained by the candidate action sequence during the Monte Carlo tree search process, the corresponding system recovery probability is obtained through normalization. These three metrics together form the basis for subsequent multi-objective optimization evaluation.

[0148] This application uses Monte Carlo tree search based on causal structure graphs for decision planning and generates multiple candidate action sequences under causal constraints. Then, the risk, cost, and recovery probability of each sequence are quantified, which can provide a comprehensive evaluation basis for selecting the optimal intervention plan.

[0149] S106. Combining the stability risk coefficient, the resource consumption cost, and the system recovery probability, a collaborative evaluation operation is performed on each candidate action sequence to obtain the optimal action sequence, and the optimal action sequence is sent to the execution engine to complete the metadata management operation.

[0150] In one specific implementation, step S106 includes:

[0151] Step 1061: Use the stability risk coefficient, the resource consumption cost, and the system recovery probability as the three evaluation dimensions for the corresponding candidate action sequence to construct an evaluation vector for each candidate action sequence.

[0152] In step 1061, firstly, from each candidate action sequence generated in step S105, the corresponding stability risk coefficient, resource consumption cost, and system recovery probability are extracted. Then, these three values ​​are combined into a three-dimensional vector in a fixed order, such as risk first, then cost, and finally recovery probability, which is the evaluation vector of the candidate action sequence.

[0153] Step 1062: Input the evaluation vector into a pre-constructed multi-objective optimization model, wherein the multi-objective optimization model takes minimizing the stability risk coefficient, minimizing the resource consumption cost, and maximizing the system recovery probability as optimization objectives.

[0154] Among them, the pre-built multi-objective optimization model refers to a mathematical model with the optimization objectives of minimizing risk, minimizing cost, and maximizing recovery probability, which is used to find a balance solution among multiple conflicting objectives.

[0155] In step 1062, the evaluation vector of each candidate action sequence is input into a pre-built multi-objective optimization model. The optimization objective of the model is explicitly set as follows: in the set of candidate action sequences, find the sequence that minimizes the stability risk coefficient, minimizes the resource consumption cost, and maximizes the system recovery probability. Since these three objectives may conflict with each other, for example, reducing risk may require increasing cost, the model needs to make trade-offs under this constraint.

[0156] Step 1063: Model the solution process of the multi-objective optimization model as a multi-objective Markov decision process. Based on the optimization objective, use the Pareto front method to search for a set of candidate solutions that satisfy the Pareto optimality condition in the state space of the multi-objective Markov decision process through non-dominated sorting.

[0157] In this context, the state space of a multi-objective Markov decision process is composed of various evaluation vectors.

[0158] In step 1063, the solution process of the multi-objective optimization model is further modeled as a multi-objective Markov decision process, and each candidate action sequence is regarded as a decision path starting from the current state. Then, based on the three set optimization objectives, the Pareto front method is used to screen the candidate action sequences. Specifically, the candidate solution set is hierarchically divided by a non-dominated sorting algorithm: for any two candidate action sequences, if sequence C is not inferior to sequence D in all three objectives, and at least one objective is strictly superior to D, then C is said to dominate D. Subsequently, all candidate solutions are compared pairwise to find all sequences that are not dominated by any other solution, thus forming the first front Pareto optimal solution set. These solutions are the candidate solution set that achieves the best trade-off in the three dimensions of risk, cost, and recovery probability.

[0159] Step 1064: From the candidate solution set, according to the preset decision preference weight, perform a weighted summation on each candidate solution to obtain the preference score of each candidate solution, and select the candidate action sequence corresponding to the candidate solution with the highest preference score as the optimal action sequence.

[0160] Among them, the preset decision preference weights refer to the relative importance weights of three objectives, such as risk, cost, and recovery probability, which are pre-set according to business needs or operation and maintenance strategies. For example, the risk weight is 0.5, the cost weight is 0.3, and the recovery probability weight is 0.2.

[0161] In step 1064, preset decision preference weights are set from the candidate solution set according to business needs. For example, the importance of the stability risk coefficient is set to 0.5, the resource consumption cost is set to 0.3, and the system recovery probability is set to 0.2. Then, for each candidate solution in the set, its three index values ​​are normalized, multiplied by the corresponding weights, and summed to obtain the preference score of the solution. The candidate action sequence corresponding to the candidate solution with the highest preference score is selected as the final optimal action sequence. Subsequently, the sequence is sent to the execution engine, which executes the intervention actions in the sequence in sequence to complete the autonomous operation and maintenance and metadata management of the microservice system.

[0162] This application achieves a comprehensive balance between risk, cost, and recovery probability by conducting multi-objective collaborative evaluation of candidate action sequences, and selects the optimal intervention plan and issues it for execution. This enables closed-loop control of the entire process from anomaly detection and root cause localization to autonomous decision-making, thereby effectively improving the autonomous operation and maintenance capabilities of microservice systems.

[0163] Figure 3 This is a schematic diagram illustrating a specific implementation of an end-to-end metadata closed-loop management system provided in this application embodiment, with reference to... Figure 3 The system may include:

[0164] The acquisition module 31 is used to acquire microservice call chain topology data, time sequence data of each service instance, and temperature and humidity data at the data center rack level.

[0165] The extraction module 32 is used to perform spatiotemporal alignment processing on the call chain topology data and the time series data to obtain a spatiotemporal tensor, and to extract physical features from the temperature and humidity data to obtain a physical vector.

[0166] The fusion module 33 is used to input the spatiotemporal tensor and the physical vector into the spatiotemporal graph neural network for dynamic fusion and generate a root cause analysis report.

[0167] The inference module 34 is used to perform temporal causal inference based on the root cause analysis report using a dynamic Bayesian network to generate a causal structure diagram.

[0168] The calculation module 35 is used to perform decision planning based on the causal structure graph using the Monte Carlo tree search algorithm, generate multiple candidate action sequences, and calculate the corresponding stability risk coefficient, resource consumption cost, and system recovery probability based on each candidate action sequence.

[0169] The evaluation module 36 is used to perform a collaborative evaluation operation on each candidate action sequence by combining the stability risk coefficient, the resource consumption cost and the system recovery probability, to obtain the optimal action sequence, and to send the optimal action sequence to the execution engine to complete the management operation of metadata.

[0170] An end-to-end metadata closed-loop management system according to an embodiment of this application is used to implement the aforementioned end-to-end metadata closed-loop management method. Therefore, the specific implementation of the end-to-end metadata closed-loop management system can be found in the embodiment section of the end-to-end metadata closed-loop management method above. The specific implementation can be referred to the description of the corresponding embodiments, and will not be repeated here.

[0171] like Figure 4 As shown, this application also provides an electronic device, including: a memory 41 for storing a computer program; and a processor 42 for executing the computer program to implement the steps of any of the above-described end-to-end metadata closed-loop management methods.

[0172] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described end-to-end metadata closed-loop management methods.

[0173] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0174] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described end-to-end metadata closed-loop management method embodiments.

[0175] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0176] The foregoing has provided a detailed description of the end-to-end metadata closed-loop management method and system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. An end-to-end closed-loop metadata management method, characterized in that, include: Obtain microservice call chain topology data, time-series data of each service instance, and data center rack-level temperature and humidity data; The call chain topology data and the time series data are spatiotemporally aligned to obtain a spatiotemporal tensor. Physical features are extracted from the temperature and humidity data to obtain a physical vector. The spatiotemporal tensor and the physical vector are input into a spatiotemporal graph neural network for dynamic fusion to generate a root cause analysis report. Based on the root cause analysis report, a dynamic Bayesian network is used to perform temporal causal inference and generate a causal structure diagram. Based on the causal structure graph, decision planning is performed using the Monte Carlo tree search algorithm to generate multiple candidate action sequences. For each candidate action sequence, the corresponding stability risk coefficient, resource consumption cost, and system recovery probability are calculated. Combining the stability risk coefficient, the resource consumption cost, and the system recovery probability, a collaborative evaluation operation is performed on each candidate action sequence to obtain the optimal action sequence, and the optimal action sequence is sent to the execution engine to complete the management operation of metadata; The step of using a dynamic Bayesian network to perform time-series causal inference based on the root cause analysis report and generating a causal structure graph includes: The service nodes, physical nodes, and performance indicators in the root cause analysis report are respectively defined as the first variable node, the second variable node, and the third variable node in the dynamic Bayesian network. Based on the contribution distribution vector in the root cause analysis report, the initial dependency relationship between the first variable node, the second variable node, and the third variable node is determined, and the network structure of the dynamic Bayesian network in the initial time slice is constructed based on the initial dependency relationship. The time axis is divided into multiple consecutive time slices. Using the network structure as a template, a time-series slice network of the dynamic Bayesian network is constructed for each time slice. Multiple time-series slice networks are connected in chronological order to form the complete topology of the dynamic Bayesian network. The belief propagation algorithm is used to perform iterative message passing in the complete topology to update the conditional probability distribution of each first variable node, second variable node and third variable node, and to calculate the causal strength coefficient between each variable node based on the updated conditional probability distribution. Based on the causal strength coefficient, causal paths with a value greater than or equal to a preset threshold are selected from the complete topology, and all the causal paths are aggregated to form a causal structure graph.

2. The end-to-end metadata closed-loop management method according to claim 1, characterized in that, The step of constructing the network structure of the dynamic Bayesian network in the initial time slice based on the initial dependency relationship includes: Based on the contribution distribution vector, calculate the mutual information value between every two variable nodes, and use the mutual information value as the correlation metric between variable nodes; Using the aforementioned correlation metric as a priori weight, a causal discovery algorithm based on an additive noise model is employed to infer the causal direction of each variable node, generating a set of edges with candidate directions. The set of edges is used as the search space, and the tabu search algorithm is used to iteratively optimize the search space. Each iteration starts from the current structure and performs neighborhood operations such as adding, deleting, or reversing edges, and uses a tabu list to record the most recently performed neighborhood operations. The iteration stops when the preset number of iterations is reached, and the structure of the last generation is used as the network structure of the dynamic Bayesian network in the initial time slice.

3. The end-to-end metadata closed-loop management method according to claim 1, characterized in that, The step of dynamically fusing the spatiotemporal tensor and the physical vector into a spatiotemporal graph neural network to generate a root cause analysis report includes: The spatiotemporal tensor is input into the spatiotemporal graph neural network. The graph convolutional layer of the spatiotemporal graph neural network captures the spatial dependencies between service nodes and outputs spatial features. The temporal convolutional layer of the spatiotemporal graph neural network captures the temporal dependencies of each service node and outputs temporal features. The spatial features and the temporal features are concatenated to obtain the spatiotemporal feature vector of each service node. Based on the preset mapping relationship between service instances and physical hosts, the physical vector is mapped to the corresponding service node to obtain the physical feature vector of each service node; Using the spatiotemporal feature vector as the query and the physical feature vector as the key and value, the cross-modal attention weight of each service node is calculated through the multi-head attention layer of the spatiotemporal graph neural network, and the physical feature vector is weighted according to the cross-modal attention weight to obtain the weighted physical features of each service node. The fusion layer of the spatiotemporal graph neural network dynamically calculates the fusion coefficient through a learnable gating weight vector, and based on the fusion coefficient, the spatiotemporal feature vector and the weighted physical features are weighted and summed to obtain the fusion feature vector of each service node. The fully connected layer of the spatiotemporal graph neural network is used to perform nonlinear processing on the fused feature vector using the softmax function to generate contribution distribution vectors for multiple preset root cause types for each service node. The contribution distribution vectors of all preset root cause types are then aggregated to form a root cause analysis report.

4. The end-to-end metadata closed-loop management method according to claim 1, characterized in that, Based on the causal structure graph, a Monte Carlo tree search algorithm is used for decision planning to generate multiple candidate action sequences, including: Using the current system state of the service cluster as the root node and the causal path in the causal structure graph as the state transition constraint when the root node is expanded, a Monte Carlo search tree is constructed. At each search node of the Monte Carlo search tree, based on the time delay parameters and confidence intervals corresponding to the causal structure graph, multiple initial actions that each search node can perform are determined. The expansion operation is performed iteratively starting from the root node until a preset number of iterations is reached. Each expansion operation includes: Using the upper confidence bound algorithm, based on the causal strength coefficient in the causal structure graph, the state transition after executing the initial action corresponding to the current node is simulated to generate a new child node; When each expansion operation reaches a preset simulation depth, starting from the current child node, the random selection probability of each initial action is set according to the causal strength coefficient. According to the random selection probability, the initial action is randomly selected until the leaf node is reached to obtain the cumulative reward value of this simulation. The cumulative reward value is then backpropagated to update the average reward value of each node. After reaching the preset number of iterations, select multiple candidate action sequences from the child nodes corresponding to the root node whose average return value meets the preset return conditions.

5. The end-to-end metadata closed-loop management method according to claim 1, characterized in that, The optimal action sequence is obtained by combining the stability risk coefficient, the resource consumption cost, and the system recovery probability to perform a collaborative evaluation operation on each candidate action sequence, including: The stability risk coefficient, the resource consumption cost, and the system recovery probability are used as three evaluation dimensions for the corresponding candidate action sequences to construct an evaluation vector for each candidate action sequence. The evaluation vector is input into a pre-constructed multi-objective optimization model, which aims to minimize the stability risk coefficient, minimize the resource consumption cost, and maximize the system recovery probability. The solution process of the multi-objective optimization model is modeled as a multi-objective Markov decision process. Based on the optimization objective, the Pareto front method is used to search for a set of candidate solutions that satisfy the Pareto optimality condition in the state space of the multi-objective Markov decision process through non-dominated sorting. From the set of candidate solutions, each candidate solution is weighted and summed according to a preset decision preference weight to obtain the preference score of each candidate solution, and the candidate action sequence corresponding to the candidate solution with the highest preference score is selected as the optimal action sequence.

6. The end-to-end metadata closed-loop management method according to claim 1, characterized in that, The process of performing spatiotemporal alignment on the call chain topology data and the time series data to obtain a spatiotemporal tensor includes: Based on the call chain topology data, the hierarchy depth and call relationship of each service instance in the service call chain are extracted, and a location encoding vector for each service instance is generated based on the hierarchy depth and the call relationship. A third-order tensor is constructed using the time-series data of each service instance corresponding to each time point as the first dimension, each service instance as the second dimension, and the location encoding vector as the third dimension. By adjusting the numerical distribution of different service instances in the third-order tensor at the same point in time using preset spatial constraints, the adjusted third-order tensor is obtained. Tensor Tucker decomposition is performed on the adjusted third-order tensor to preserve the core tensor and the three factor matrices. With the goal of minimizing the reconstruction error, the core tensor and the three factor matrices are alternately optimized to generate a spatiotemporal tensor.

7. An end-to-end metadata closed-loop management system, characterized in that, A method for implementing an end-to-end closed-loop metadata management system as described in any one of claims 1 to 6 includes: The acquisition module is used to acquire microservice call chain topology data, time-series data of each service instance, and temperature and humidity data at the data center rack level. The extraction module is used to perform spatiotemporal alignment processing on the call chain topology data and the time series data to obtain a spatiotemporal tensor, and to extract physical features from the temperature and humidity data to obtain a physical vector. The fusion module is used to input the spatiotemporal tensor and the physical vector into the spatiotemporal graph neural network for dynamic fusion and generate a root cause analysis report. The inference module is used to perform time-series causal inference based on the root cause analysis report using a dynamic Bayesian network to generate a causal structure diagram. The calculation module is used to perform decision planning based on the causal structure graph using the Monte Carlo tree search algorithm, generate multiple candidate action sequences, and calculate the corresponding stability risk coefficient, resource consumption cost, and system recovery probability based on each candidate action sequence. The evaluation module is used to perform a collaborative evaluation operation on each candidate action sequence by combining the stability risk coefficient, the resource consumption cost, and the system recovery probability to obtain the optimal action sequence, and then send the optimal action sequence to the execution engine to complete the metadata management operation.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of an end-to-end metadata closed-loop management method as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables an end-to-end metadata closed-loop management method as described in any one of claims 1 to 6.

Images