Server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph
By using multi-source heterogeneous data fusion and dynamic knowledge graphs, we have solved the data barriers and root cause identification difficulties in server cluster operation and maintenance, achieved efficient fault diagnosis and adaptive operation and maintenance, and improved operation and maintenance efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 金品计算机科技(天津)有限公司
- Filing Date
- 2026-02-24
- Publication Date
- 2026-05-26
AI Technical Summary
Existing server cluster operations and maintenance suffer from problems such as data multimodal barriers, alarm storms and difficulties in root cause localization, lack of adaptability in static models, and lack of closed-loop feedback in handling, resulting in low operational efficiency.
We adopt a method based on multi-source heterogeneous data fusion and dynamic knowledge graph. By collecting performance indicators, log text and topology data, we use an unsupervised deep learning model to analyze the health baseline, a graph neural network to locate the root cause node, and an incremental learning mechanism to optimize the treatment strategy, so as to realize a closed-loop operation and maintenance of perception-decision-execution-learning.
It enables unified analysis of multi-source data, improves the accuracy of fault detection and the efficiency of root cause localization, reduces alarm storms, and enhances the system's adaptability and closed-loop feedback mechanism for operation and maintenance.
Smart Images
Figure CN121705077B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer technology, and in particular relates to a server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph. Background Technology
[0002] With the widespread adoption of microservice architectures, containerization technologies (such as Kubernetes), and cloud-native technologies, the scale and complexity of server clusters are growing exponentially. Despite the existence of various monitoring tools, existing server cluster maintenance practices still suffer from numerous pain points:
[0003] Multimodal Data Silos: Numerical monitoring metrics, textual logs, and relational topology data are usually collected independently by different tools, lacking a unified time and space alignment mechanism, making joint inference difficult.
[0004] Alarm storms and difficulties in root cause localization: Traditional monitoring based on static thresholds is prone to generating "alarm storms." In the complex call chains of microservices, upper-layer application errors caused by underlying hardware (such as disk failures) often mask the true root cause, making it difficult for operations and maintenance personnel to quickly construct causal links from massive amounts of alarms.
[0005] Static models lack adaptability: Most existing AIOps models are trained offline and cannot adapt to dynamic changes in cluster topology (such as the elastic scaling of containers).
[0006] Lack of closed-loop feedback in handling: Fault handling is mostly one-way instructions, lacking a mechanism to feed back the handling results (such as whether it was successful or not, and the performance recovery status) to the AI model, which makes the system unable to learn from historical experience and lacks "growth potential".
[0007] Therefore, there is an urgent need for an intelligent operation and maintenance system that can integrate multi-source heterogeneous data, dynamically construct fault propagation maps, and realize a closed loop of "perception-decision-execution-learning". Summary of the Invention
[0008] In view of the above, the present invention aims to address at least one of the aforementioned technical problems.
[0009] To achieve the above objectives, the technical solution of the present invention is implemented as follows:
[0010] The first aspect of this invention proposes a server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph, comprising the following steps:
[0011] Collect performance metrics data, log text data, and topology data of the server cluster, and concatenate the performance metrics data and log text data based on a unified time window to obtain a multimodal feature sequence;
[0012] An unsupervised deep learning model is used to analyze multimodal feature sequences to obtain a dynamic health baseline, and a health profile is generated based on the deviation between real-time data and the dynamic health baseline.
[0013] When the health profile detects an anomaly in the server cluster, the anomaly event is mapped to a dynamic topology graph built based on the topology data.
[0014] The graph neural network algorithm is used to analyze the probability of fault propagation between nodes in a dynamic topology graph. The root cause node is located based on the probability of fault propagation. Based on the location of the root cause node, a handling strategy is generated and the corresponding handling operation is executed.
[0015] The cluster state recovery data after the collection and processing operations are used as feedback signals to update the unsupervised deep learning model based on the incremental learning mechanism.
[0016] Furthermore, the unified time window is specifically a timestamp-based sliding window mechanism. The sliding window mechanism resamples performance index data with different sampling frequencies based on time sequence to align the time granularity of performance index data with log vector data.
[0017] Furthermore, the process of concatenating performance metric data with log text data is as follows:
[0018] Set a uniform time window length;
[0019] Calculate the statistical feature values of numerical performance index data within a uniform time window, and use the statistical feature values to generate index feature vectors;
[0020] Log templates are extracted from log text data using log parsing algorithms. The frequency of occurrence of each log template within a uniform time window is obtained. Log vector data is then extracted from the log text data based on the frequency of occurrence using a natural language model.
[0021] The indicator feature vector and log vector data are spatially indexed and aligned using entity identifiers and then concatenated to obtain a multimodal feature matrix.
[0022] Furthermore, the log parsing algorithm is specifically a tree structure parsing method based on the longest common subsequence, which separates the unstructured log text into a constant template part and a variable parameter part.
[0023] Log vector data is obtained by weighted combination of semantic embedding of constant template and statistical distribution characteristics of variable parameters.
[0024] Furthermore, the unsupervised deep learning model is specifically a long short-term memory variational autoencoder;
[0025] The loss function for training an unsupervised deep learning model consists of a weighted sum of reconstruction loss and KL divergence loss. When the reconstruction loss value of real-time data exceeds the dynamic threshold calculated based on the extreme value theory, it is judged as an anomaly.
[0026] The calculation process of the loss function is as follows: ;
[0027] in, This is the original multimodal feature sequence. To reconstruct the sequence, Let KL divergence be the KL divergence. To balance the hyperparameters.
[0028] Furthermore, the process of using a graph neural network algorithm to analyze the probability of fault propagation between nodes in a dynamic topology graph includes:
[0029] After mapping abnormal events to a dynamic topology graph constructed based on topology data, the dynamic weight attributes of edges in the dynamic topology graph are calculated by combining static topology dependencies and dynamic correlations. The dynamic weight attributes of each edge in the dynamic topology graph are used as the probability of fault propagation.
[0030] The calculation process for dynamic weight attributes is as follows: ;
[0031] in, This indicates whether there is a physical or logical call relationship between node i and node j. α represents the Pearson correlation coefficient of the key performance indicator sequences of node i and node j within the current time window, where α is the weighting adjustment factor.
[0032] Furthermore, the method of locating the root cause node based on the fault propagation probability is implemented using either a random walk algorithm or a graph attention network, wherein:
[0033] The implementation process of the random walk algorithm includes:
[0034] The node where the anomaly is detected is taken as the starting node. The node is traversed in reverse along the edge where the failure propagation probability is greater than the preset threshold. The failure contribution score of each neighboring node is calculated, and the node with the highest score is determined as the root cause node.
[0035] The implementation process of the graph attention network includes:
[0036] By using graph attention networks to analyze the propagation weights of faults in a dynamic topology graph, and by aggregating information from neighboring nodes, the source node that triggers the cascading effect can be identified.
[0037] Furthermore, the dynamic topology graph is a heterogeneous graph;
[0038] The node types of a dynamic topology graph must include at least physical machines, virtual machines, container pods, microservices, API interfaces, and configuration change events; the edge attributes of a dynamic topology graph must include at least network bandwidth usage, service call latency, and logical dependencies.
[0039] And configure the change event node as a temporal causal constraint node to participate in root cause localization, including:
[0040] Within a unified time window, collect change events associated with server entities and service component entities;
[0041] Change events are mapped to event nodes in a dynamic topology graph, and the causal correlation between event nodes and abnormal nodes is calculated based on the time interval between the occurrence time of the change event and the occurrence time of the abnormality.
[0042] During the root cause localization process, the causal correlation is used as an adjustment factor for the dynamic weight attribute of each edge of the dynamic topology graph.
[0043] When the failure contribution of a change event node exceeds a preset threshold, the server entity or service component entity corresponding to the change event is determined to be the root cause node.
[0044] Furthermore, the incremental learning mechanism is constructed based on the reward function mechanism of reinforcement learning, and the execution process of the incremental learning mechanism includes:
[0045] If the cluster health recovers to the dynamic health baseline range within a preset time after the handling strategy is executed, a positive reward will be given, and the correlation weight between the handling strategy and the current fault characteristics will be increased.
[0046] If the problem is not resolved or a new anomaly is triggered, a negative penalty will be imposed.
[0047] The second aspect of this invention proposes a server cluster operation and maintenance system based on multi-source heterogeneous data fusion and dynamic knowledge graph, applying the method described in the first aspect, including:
[0048] Heterogeneous data spatiotemporal alignment and vectorization module: used to collect physical layer, virtualization layer and application layer data of server cluster, and perform spatiotemporal alignment and entity ID mapping of time series indicators, unstructured logs and configuration change events collected at different frequencies through a unified spatiotemporal coordinate system to generate multimodal feature tensors;
[0049] Dynamic baseline and health profile generation module: used to build cluster health baseline and learn the multimodal feature distribution under normal conditions using an unsupervised anomaly detection algorithm, generate health profile and detect anomalies;
[0050] The root cause localization module based on spatiotemporal knowledge graphs maintains a dynamic fault propagation graph, calculates the propagation probability weights between nodes, and uses graph neural networks or random walk algorithms to locate the root cause nodes.
[0051] Intelligent decision-making and incremental learning closed-loop module: used to recommend treatment plans and update model parameters and strategy library online based on the effect evaluation results after treatment, so as to realize the self-evolution of the system.
[0052] Compared with existing technologies, the server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph described in this invention has the following beneficial effects:
[0053] By integrating log vectorization with metrics, the model can simultaneously utilize "numerical fluctuations" and "error semantics" for joint diagnosis, thereby improving the accuracy of fault detection.
[0054] By utilizing dynamic knowledge graph technology, it is possible to clearly reconstruct the fault propagation path across the physical layer, virtual layer, and application layer, effectively suppress alarm storms, and directly locate the root cause.
[0055] The unique feedback loop mechanism makes the system more adaptable as the business operation time increases. Attached Figure Description
[0056] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:
[0057] Figure 1 This is a schematic diagram illustrating the workflow of the server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph as described in an embodiment of the present invention. Detailed Implementation
[0058] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.
[0059] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.
[0060] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0061] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0062] like Figure 1 As shown, the server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph includes the following steps:
[0063] S1. Collect performance index data, log text data, and topology data of the server cluster, and concatenate the performance index data and log text data based on a unified time window to obtain a multimodal feature sequence;
[0064] S2. Use an unsupervised deep learning model to analyze multimodal feature sequences to obtain a dynamic health baseline, and generate a health profile based on the deviation between real-time data and the dynamic health baseline.
[0065] S3. When the health profile detects an anomaly in the server cluster, the anomaly event is mapped to a dynamic topology graph constructed based on the topology data.
[0066] S4. Use graph neural network algorithm to analyze the failure propagation probability between nodes in the dynamic topology graph, locate the root cause node based on the failure propagation probability, generate a handling strategy based on the location result of the root cause node, and execute the corresponding handling operation.
[0067] S5. Collect cluster state recovery data after processing as a feedback signal, and update the unsupervised deep learning model based on the incremental learning mechanism.
[0068] The unified time window is specifically a timestamp-based sliding window mechanism. The sliding window mechanism resamples performance index data with different sampling frequencies based on time sequence to align the time granularity of performance index data with log vector data.
[0069] The process of concatenating performance metric data and log text data is as follows:
[0070] Set a uniform time window length;
[0071] Calculate the statistical feature values of numerical performance index data within a uniform time window, and use the statistical feature values to generate index feature vectors;
[0072] Log templates are extracted from log text data using log parsing algorithms. The frequency of occurrence of each log template within a uniform time window is obtained. Log vector data is then extracted from the log text data based on the frequency of occurrence using a natural language model.
[0073] The indicator feature vector and log vector data are spatially indexed and aligned using entity identifiers and then concatenated to obtain a multimodal feature matrix.
[0074] The log parsing algorithm is specifically a tree structure parsing method based on the longest common subsequence, which separates the unstructured log text into a constant template part and a variable parameter part.
[0075] Log vector data is obtained by weighted combination of semantic embedding of constant template and statistical distribution characteristics of variable parameters.
[0076] In some embodiments, to achieve the fusion of heterogeneous data (performance metric data and log vector data), a spatiotemporal alignment and vectorization engine was designed, and its processing flow is as follows:
[0077] Step 1: Temporal Alignment
[0078] Define a global sliding window (e.g., Window_Size=60s, Step=10s).
[0079] Monitoring metrics are typically performed at a fixed frequency (e.g., one point every 5 seconds). Within a 60-second window, there may be 12 data points. The system calculates the statistical characteristics (Avg, Max, P99, Slope) of these 12 points and compresses them into a set of statistical feature vectors. ;
[0080] Logs occur discretely (there may be 100 or 0 logs within 60 seconds). The system counts the number of various "log templates" in the window (LogCountVector) and extracts the TF-IDF values of key error words (such as "Timeout" and "Error") to form a log feature vector. .
[0081] Step 2, Spatial Alignment:
[0082] Establish a unified entity mapping table.
[0083] Metrics are typically bound to IP addresses (such as 192.168.1.10), while logs are typically bound to hostnames or pod IDs.
[0084] The system uses CMDB metadata to uniformly map IP, Hostname, and PodID to a unique Node_UUID.
[0085] Step 3, Tensor Concatenation:
[0086] For each Node_UUID in each time window T, perform a vector concatenation operation. =Concat( , );
[0087] The final output is a tensor in the format [Batch_Size, Sequence_Length, Feature_Dimension], which can be directly input into the LSTM / Transformer model.
[0088] The unsupervised deep learning model is specifically a long short-term memory variational autoencoder.
[0089] The loss function for training an unsupervised deep learning model consists of a weighted sum of reconstruction loss and KL divergence loss. When the reconstruction loss value of real-time data exceeds the dynamic threshold calculated based on the extreme value theory, it is judged as an anomaly.
[0090] The calculation process of the loss function is as follows: ;
[0091] in, This is the original multimodal feature sequence. To reconstruct the sequence, Let KL divergence be the KL divergence. To balance the hyperparameters.
[0092] The process of using a graph neural network algorithm to analyze the probability of fault propagation between nodes in a dynamic topology graph includes:
[0093] After mapping abnormal events to a dynamic topology graph constructed based on topology data, the dynamic weight attributes of edges in the dynamic topology graph are calculated by combining static topology dependencies and dynamic correlations. The dynamic weight attributes of each edge in the dynamic topology graph are used as the probability of fault propagation.
[0094] The calculation process for dynamic weight attributes is as follows: ;
[0095] in, This indicates whether there is a physical or logical call relationship between node i and node j. α represents the Pearson correlation coefficient of the key performance indicator sequences of node i and node j within the current time window, where α is the weighting adjustment factor.
[0096] The method of locating the root cause node based on the failure propagation probability is implemented by either a random walk algorithm or a graph attention network, wherein:
[0097] The implementation process of the random walk algorithm includes:
[0098] The node where the anomaly is detected is taken as the starting node. The node is traversed in reverse along the edge where the failure propagation probability is greater than the preset threshold. The failure contribution score of each neighboring node is calculated, and the node with the highest score is determined as the root cause node.
[0099] The implementation process of the graph attention network includes:
[0100] By using graph attention networks to analyze the propagation weights of faults in a dynamic topology graph, and by aggregating information from neighboring nodes, the source node that triggers the cascading effect can be identified.
[0101] The dynamic topology graph is a heterogeneous graph;
[0102] The node types of a dynamic topology graph must include at least physical machines, virtual machines, container pods, microservices, API interfaces, and configuration change events; the edge attributes of a dynamic topology graph must include at least network bandwidth usage, service call latency, and logical dependencies.
[0103] And configure the change event node as a temporal causal constraint node to participate in root cause localization, including:
[0104] Within a unified time window, collect change events associated with server entities and service component entities;
[0105] Change events are mapped to event nodes in a dynamic topology graph, and the causal correlation between event nodes and abnormal nodes is calculated based on the time interval between the occurrence time of the change event and the occurrence time of the abnormality.
[0106] During the root cause localization process, the causal correlation is used as an adjustment factor for the dynamic weight attribute of each edge of the dynamic topology graph.
[0107] When the failure contribution of a change event node exceeds a preset threshold, the server entity or service component entity corresponding to the change event is determined to be the root cause node.
[0108] The incremental learning mechanism is constructed based on the reward function mechanism of reinforcement learning, and the execution process of the incremental learning mechanism includes:
[0109] If the cluster health recovers to the dynamic health baseline range within a preset time after the handling strategy is executed, a positive reward will be given, and the correlation weight between the handling strategy and the current fault characteristics will be increased.
[0110] If the problem is not resolved or a new anomaly is triggered, a negative penalty will be imposed.
[0111] In some embodiments, the core logical chain of processing data using the server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph is as follows:
[0112] 1. Data Access and Preprocessing Layer:
[0113] Deploy a lightweight eBPF probe and Log Agent to collect data; set a basic time window (e.g., 1 minute) within this window, calculate statistical values (mean, P99 value) of metrics such as CPU / memory; simultaneously, aggregate application logs within this window and extract log template IDs and error keyword vectors.
[0114] 2. Anomaly Detection Layer:
[0115] Using the LSTM-VAE (Long Short-Term Memory-Variational Autoencoder) model, the input is the above-aligned multimodal vector (the sum of the error keyword vector within the basic time window, the log template ID, and the statistical values of each indicator); the model attempts to reconstruct the input data, and if the reconstruction error exceeds the dynamic threshold calculated based on the extreme value theory (EVT), it is marked as an anomalous segment.
[0116] 3. Graph Construction and Reasoning Layer:
[0117] Real-time synchronization of topology data from CMDB and Kubernetes API is used to build an infrastructure graph. When an anomaly occurs, the anomaly event is mapped to the graph as an "activation node". The GAT (Graph Attention Network) algorithm is applied to analyze the feature correlation of the activation node and its neighboring nodes, calculate the weight of each edge (representing the probability of fault propagation), and the path with the highest weight is the fault propagation chain.
[0118] In other embodiments, the above-mentioned server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph is applied to an e-commerce business server cluster.
[0119] If a microservice suddenly becomes slow to respond, perform the following steps:
[0120] Step A: The system detected that the P99 latency indicator of "Order Service" deviated from the dynamic baseline. At the same time, a large number of "Connection Timeout" errors appeared in the log within the same time window.
[0121] Step B Analysis: The system activates the "Order Service" node in the knowledge graph; by tracing the call chain backward through GNN, it is found that although the "Redis Cache Node-03" on which "Order Service" depends has normal indicators, the "Disk I / O Wait Rate" of its host physical machine "Server-A" has an abnormal surge; at the same time, the graph is associated with "Server-A" having performed a scheduling change for a "Big Data Offline Analysis Task" 10 minutes ago.
[0122] Step C: Location Analysis: The AI model determined the root cause to be: "Offline tasks running on Server-A compete for disk I / O resources, causing Redis on the same node to slow down, which in turn slows down the upper-layer order service."
[0123] Step D: The system automatically recommends the following solutions: "Isolate the offline task on Server-A" or "Migrate Redis to an idle node". After confirmation by operations and maintenance, the API is called to execute the Pod migration.
[0124] Step E Feedback: After the migration is completed, business metrics return to normal within 2 minutes. The system records this successful handling and strengthens the weight of the "I / O contention causing upper-layer timeout" pattern in the model.
[0125] A server cluster operation and maintenance system based on multi-source heterogeneous data fusion and dynamic knowledge graphs is implemented, including the following methods:
[0126] Heterogeneous data spatiotemporal alignment and vectorization module: used to collect physical layer, virtualization layer and application layer data of server cluster, and perform spatiotemporal alignment and entity ID mapping of time series indicators, unstructured logs and configuration change events collected at different frequencies through a unified spatiotemporal coordinate system to generate multimodal feature tensors;
[0127] Dynamic baseline and health profile generation module: used to build cluster health baseline and learn the multimodal feature distribution under normal conditions using an unsupervised anomaly detection algorithm, generate health profile and detect anomalies;
[0128] The root cause localization module based on spatiotemporal knowledge graphs maintains a dynamic fault propagation graph, calculates the propagation probability weights between nodes, and uses graph neural networks or random walk algorithms to locate the root cause nodes.
[0129] Intelligent decision-making and incremental learning closed-loop module: used to recommend treatment plans and update model parameters and strategy library online based on the effect evaluation results after treatment, so as to realize the self-evolution of the system.
[0130] In some embodiments, the server cluster operation and maintenance system based on multi-source heterogeneous data fusion and dynamic knowledge graph is implemented as an AI-based intelligent operation and maintenance monitoring and fault handling system for the entire lifecycle of a server cluster, including:
[0131] The heterogeneous data spatiotemporal alignment and vectorization module is used to collect data from the physical layer, virtualization layer, and application layer of the server cluster.
[0132] By introducing a unified spatiotemporal coordinate system, time-series metrics collected at different frequencies, unstructured text logs, and configuration change events are associated through time window alignment and entity ID mapping. Natural language processing models (such as BERT) are used to convert the log text into high-dimensional vectors, which are then concatenated and fused with numerical metrics to construct a multimodal feature tensor.
[0133] The dynamic baseline and health profile generation module is used to build cluster health baselines;
[0134] An unsupervised anomaly detection algorithm (such as VAE variational autoencoder) is used to learn the multimodal feature distribution under normal conditions. When the reconstruction error of real-time data exceeds the dynamic threshold, it is judged as an anomaly, and a health profile containing multiple dimensions such as "resource saturation", "service response latency" and "hardware health" is generated.
[0135] The root cause localization module, based on a spatiotemporal knowledge graph, is used to construct and update a dynamic fault propagation graph in real time. The dynamic fault propagation graph includes:
[0136] A node consists of at least physical entities (servers, switches), logical entities (Pods, Services, API interfaces), and event entities (alarms, changes).
[0137] An edge must contain at least physical connections, a logical call graph, and a temporal causal relationship;
[0138] By using graph neural networks (GNNs) or random walk algorithms to perform inference on the graph, the propagation probability between anomalous nodes is calculated, thereby identifying the root cause node located at the top of the propagation chain.
[0139] The intelligent decision-making and incremental learning closed-loop module is used to generate disposal strategies and optimize the model;
[0140] Based on the root cause analysis results, the contingency plan database is retrieved to recommend appropriate actions (such as expansion, restart, or downgrade).
[0141] A deployment effect evaluation unit is used to monitor the recovery of the cluster status after the treatment. The "fault characteristics-treatment actions-effect scores" are used as new training samples. The prediction model and recommendation strategy are updated online through incremental learning technology to achieve the self-evolution of the system.
[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.
[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph, characterized in that, Includes the following steps: Collect performance metrics data, log text data, and topology data of the server cluster, and concatenate the performance metrics data and log text data based on a unified time window to obtain a multimodal feature sequence; An unsupervised deep learning model is used to analyze multimodal feature sequences to obtain a dynamic health baseline, and a health profile is generated based on the deviation between real-time data and the dynamic health baseline. When the health profile detects an anomaly in the server cluster, the anomaly event is mapped to a dynamic topology graph built based on the topology data. The graph neural network algorithm is used to analyze the probability of fault propagation between nodes in a dynamic topology graph. The root cause node is located based on the probability of fault propagation. Based on the location of the root cause node, a handling strategy is generated and the corresponding handling operation is executed. The cluster state recovery data after the collection and processing operations are used as feedback signals to update the unsupervised deep learning model based on the incremental learning mechanism.
2. The server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph as described in claim 1, characterized in that: The unified time window is specifically a timestamp-based sliding window mechanism. The sliding window mechanism resamples performance index data with different sampling frequencies based on time sequence to align the time granularity of performance index data with log text data.
3. The server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph as described in claim 1, characterized in that, The process of concatenating performance metric data and log text data is as follows: Set a uniform time window length; Calculate the statistical feature values of numerical performance index data within a uniform time window, and use the statistical feature values to generate index feature vectors; Log templates are extracted from log text data using log parsing algorithms. The frequency of occurrence of each log template within a uniform time window is obtained. Log vector data is then extracted from the log text data based on the frequency of occurrence using a natural language model. The indicator feature vector and log vector data are spatially indexed and aligned using entity identifiers and then concatenated to obtain a multimodal feature matrix.
4. The server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph as described in claim 3, characterized in that: The log parsing algorithm is specifically a tree structure parsing method based on the longest common subsequence, which separates the unstructured log text into a constant template part and a variable parameter part. Log vector data is obtained by weighted combination of semantic embedding of constant template and statistical distribution characteristics of variable parameters.
5. The server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph as described in claim 1, characterized in that: The unsupervised deep learning model is specifically a long short-term memory variational autoencoder. The loss function for training an unsupervised deep learning model consists of a weighted sum of reconstruction loss and KL divergence loss. When the reconstruction loss value of real-time data exceeds the dynamic threshold calculated based on the extreme value theory, it is judged as an anomaly. The calculation process of the loss function is as follows: ; in, This is the original multimodal feature sequence. To reconstruct the sequence, Let KL divergence be the KL divergence. To balance the hyperparameters.
6. The server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph as described in claim 1, characterized in that, The process of using a graph neural network algorithm to analyze the probability of fault propagation between nodes in a dynamic topology graph includes: After mapping abnormal events to a dynamic topology graph constructed based on topology data, the dynamic weight attributes of edges in the dynamic topology graph are calculated by combining static topology dependencies and dynamic correlations. The dynamic weight attributes of each edge in the dynamic topology graph are used as the probability of fault propagation. The calculation process for dynamic weight attributes is as follows: ; in, This indicates whether there is a physical or logical call relationship between node i and node j. Let α represent the Pearson correlation coefficient between the key performance indicator sequences of node i and node j within the current time window, where α is the weighting adjustment factor.
7. The server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph as described in claim 6, characterized in that, The method of locating the root cause node based on the fault propagation probability is implemented using either a random walk algorithm or a graph attention network, wherein: The implementation process of the random walk algorithm includes: The node where the anomaly is detected is taken as the starting node. The node is traversed in reverse along the edge where the failure propagation probability is greater than the preset threshold. The failure contribution score of each neighboring node is calculated, and the node with the highest score is determined as the root cause node. The implementation process of the graph attention network includes: By using graph attention networks to analyze the propagation weights of faults in a dynamic topology graph, and by aggregating information from neighboring nodes, the source node that triggers the cascading effect can be identified.
8. The server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph as described in claim 7, characterized in that: The dynamic topology graph is a heterogeneous graph; The node types of a dynamic topology graph must include at least physical machines, virtual machines, container pods, microservices, API interfaces, and configuration change events; the edge attributes of a dynamic topology graph must include at least network bandwidth usage, service call latency, and logical dependencies. And configure the change event node as a temporal causal constraint node to participate in root cause localization, including: Within a unified time window, collect change events associated with server entities and service component entities; Change events are mapped to event nodes in a dynamic topology graph, and the causal correlation between event nodes and abnormal nodes is calculated based on the time interval between the occurrence time of the change event and the occurrence time of the abnormality. During the root cause localization process, the causal correlation is used as an adjustment factor for the dynamic weight attribute of each edge of the dynamic topology graph. When the failure contribution of a change event node exceeds a preset threshold, the server entity or service component entity corresponding to the change event is determined to be the root cause node.
9. The server cluster operation and maintenance method based on multi-source heterogeneous data fusion and dynamic knowledge graph as described in claim 1, characterized in that, The incremental learning mechanism is constructed based on the reward function mechanism of reinforcement learning, and the execution process of the incremental learning mechanism includes: If the cluster health recovers to the dynamic health baseline range within a preset time after the handling strategy is executed, a positive reward will be given, and the correlation weight between the handling strategy and the current fault characteristics will be increased. If the problem is not resolved or a new anomaly is triggered, a negative penalty will be imposed.
10. A server cluster operation and maintenance system based on multi-source heterogeneous data fusion and dynamic knowledge graph, using the method described in any one of claims 1-9, characterized in that, include: Heterogeneous data spatiotemporal alignment and vectorization module: used to collect physical layer, virtualization layer and application layer data of server cluster, and perform spatiotemporal alignment and entity ID mapping of time series indicators, unstructured logs and configuration change events collected at different frequencies through a unified spatiotemporal coordinate system to generate multimodal feature tensors; Dynamic baseline and health profile generation module: used to build cluster health baseline and learn the multimodal feature distribution under normal conditions using an unsupervised anomaly detection algorithm, generate health profile and detect anomalies; The root cause localization module based on spatiotemporal knowledge graphs maintains a dynamic fault propagation graph, calculates the propagation probability weights between nodes, and uses graph neural networks or random walk algorithms to locate the root cause nodes. Intelligent decision-making and incremental learning closed-loop module: used to recommend treatment plans and update model parameters and strategy library online based on the effect evaluation results after treatment, so as to realize the self-evolution of the system.