Cloud environment-oriented data center intelligent operation and maintenance management system and method

Abstract

The invention provides a cloud environment-oriented data center intelligent operation and maintenance management system and method, and relates to the technical field of computer system monitoring, and the method comprises the steps: carrying out the hierarchical compression of a multi-source heterogeneous operation and maintenance data flow, and obtaining a multi-modal data flow; constructing a data operation graph based on the multi-modal data flow, extracting dual-path time sequence characteristics from the data operation graph, and carrying out coupling modeling on the dual-path time sequence characteristics to obtain a capacity prediction result; in the data operation map, reverse traceability and influence surface deduction are carried out on the capacity prediction result to obtain a multi-modal root cause; generating a self-adaptive operation and maintenance instruction according to the capacity prediction result and the multi-modal root cause diagnosis report through a multi-target tradeoff model; according to the method, the execution feedback data of the self-adaptive operation and maintenance instruction is collected, iterative optimization of the operation and maintenance case is carried out through the execution feedback data and the historical event data, automatic management from multi-source heterogeneous operation and maintenance data to accurate closed-loop operation and maintenance decision can be achieved, and the accuracy of data anomaly diagnosis is improved.

Description

Technical Field

[0001] This application relates to the field of computer system monitoring technology, and more specifically, to a data center intelligent operation and maintenance management system and method for cloud environments. Background Technology

[0002] With the rapid development of information technology, cloud computing has become the core paradigm supporting modern digital infrastructure. As the physical carrier of cloud services, data centers have experienced a dramatic increase in scale and complexity, posing unprecedented challenges to operation and maintenance management. Computer system monitoring mainly relies on threshold alarms for infrastructure performance indicators, combined with post-event analysis using system logs and distributed tracing. This responsive and single-point monitoring approach forms the basis of automated operation and maintenance.

[0003] Intelligent operation and maintenance management of data centers in cloud environments aims to improve the initiative, accuracy, and efficiency of operation and maintenance by leveraging artificial intelligence and big data technologies. Its core lies in the comprehensive analysis of massive, multi-source, and heterogeneous operation and maintenance data (indicators, logs, traces, etc.) to achieve advanced goals such as fault prediction, root cause localization, and capacity planning. However, existing technical solutions still have significant limitations: First, at the data processing level, facing the continuously generated massive amounts of operation and maintenance data, most methods employ general compression or direct storage, lacking a semantically aware intelligent layered compression mechanism for subsequent intelligent analysis scenarios, resulting in high storage costs and low data analysis efficiency. Second, at the anomaly detection and diagnosis level, existing methods often analyze indicators, logs, or trace data in isolation, failing to deeply explore the inherent relationships between multimodal data, and lacking means to couple and model the topological dependencies of the system runtime with the dynamic characteristics of time series. Third, at the prediction and decision-making level, existing capacity prediction models are mostly based on historical time-series data, failing to effectively integrate real-time system topology and abnormal state information, resulting in poor robustness of prediction results. Therefore, how to achieve automated management from multi-source heterogeneous operation and maintenance data to precise closed-loop operation and maintenance decisions, so as to improve the accuracy of data anomaly diagnosis, is a challenge faced by the industry. Summary of the Invention

[0004] This application provides a data center intelligent operation and maintenance management system and method for cloud environments, which can realize automated management from multi-source heterogeneous operation and maintenance data to accurate closed-loop operation and maintenance decisions, so as to improve the accuracy of data anomaly diagnosis.

[0005] Firstly, this application provides an intelligent operation and maintenance management method for data centers in a cloud environment, the operation and maintenance management method comprising the following steps: Obtain heterogeneous operation and maintenance data streams from the data center, and perform layered compression on the multi-source heterogeneous operation and maintenance data streams to obtain multimodal data streams; A data operation map is constructed based on the multimodal data stream, and dual-path time series features are extracted from the data operation map. The dual-path time series features are coupled and modeled to obtain the capacity prediction results. In the data operation graph, the capacity prediction results are traced back to their source and the impact surface is extrapolated to obtain the multimodal root causes; Adaptive operation and maintenance instructions are generated based on the capacity prediction results and the multimodal root cause diagnosis report using a multi-objective trade-off model. The execution feedback data of the adaptive operation and maintenance instructions is collected, and the operation and maintenance cases are iteratively optimized using the execution feedback data and historical event data.

[0006] In this embodiment, heterogeneous operation and maintenance data streams of the data center are obtained through data probes.

[0007] In this embodiment, intelligent hierarchical compression is performed on the multi-source heterogeneous operation and maintenance data stream to obtain a multimodal data stream, specifically including: Semantic parsing, template matching, and deduplication are performed on the log data in the multi-source heterogeneous operation and maintenance data stream to obtain a lightweight log event stream. The lightweight log event stream is then subjected to hybrid compression to obtain a multimodal data stream.

[0008] In this embodiment, constructing a data operation map based on the multimodal data stream specifically includes: The distributed tracing context in the multimodal data stream is parsed and aggregated to identify and extract the set of operation and maintenance entities and the explicit call dependency pairs between entities; Based on the set of operation and maintenance entities and the explicit call dependency pairs between the entities, the vector representations of the entities are learned using a graph embedding model, and then a dependency graph is constructed. The dependency graph is updated based on the reachability of the intermediary nodes to obtain the data operation graph.

[0009] In this embodiment, extracting dual-path temporal features from the data execution graph specifically includes: The data running graph is output through multi-layer dilated convolution of a temporal convolutional network to produce a sequence of inter-point feature vectors. The inter-point feature vector sequence is divided into an inter-segment feature vector sequence by a sliding window. The inter-point feature vector sequence and the inter-segment feature vector sequence are fused using a gating network to obtain dual-path temporal features.

[0010] In this embodiment, the coupled modeling of the dual-path temporal features to obtain the capacity prediction result specifically includes: The dual-path temporal features and the data execution graph are fused to obtain the enhanced temporal features of the topological dependency context; A capacity prediction sequence is generated based on the enhanced time-series features using a parallel architecture that includes a long-term trend prediction branch and a short-term fluctuation prediction branch. The capacity prediction value sequence is dynamically calibrated to obtain the capacity prediction result.

[0011] In this embodiment, the multimodal root causes are obtained by reverse tracing and impact surface extrapolation of the capacity prediction results in the data operation map, specifically including: A backpropagation simulation is performed on the data operation graph to obtain the set of anomaly source nodes; Multimodal anomaly evidence is determined by the set of anomaly source nodes, thereby obtaining the root cause nodes; Starting from the root cause node, a positive impact propagation analysis is performed on the data operation graph based on the multimodal anomaly evidence to obtain the multimodal root cause.

[0012] In this embodiment, generating adaptive operation and maintenance instructions based on the capacity prediction results and the multimodal root cause diagnosis report using a multi-objective trade-off model specifically includes: The multimodal root causes and the capacity prediction results are analyzed to generate a set of decision constraints; The simulation evaluation matrix is ​​determined based on the decision constraint set and the preset operation and maintenance strategy library; The simulation evaluation matrix is ​​input into a multi-objective trade-off model for optimization and solution, thereby generating adaptive operation and maintenance instructions.

[0013] In this embodiment, the execution feedback data of the adaptive operation and maintenance instructions is collected through the log interface.

[0014] Secondly, this application provides a data center intelligent operation and maintenance management system for cloud environments, used to execute a data center intelligent operation and maintenance management method for cloud environments, the operation and maintenance management system comprising: The hierarchical compression module is used to acquire heterogeneous operation and maintenance data streams from the data center, and to perform hierarchical compression on the multi-source heterogeneous operation and maintenance data streams to obtain multimodal data streams. The coupled modeling and prediction module is used to construct a data operation map based on the multimodal data stream, extract dual-path time series features from the data operation map, perform coupled modeling on the dual-path time series features, and then obtain the capacity prediction result. The source tracing and inference module is used to perform reverse source tracing and impact surface inference on the capacity prediction results in the data operation map to obtain multimodal root causes; The multi-objective trade-off module is used to generate adaptive operation and maintenance instructions based on the capacity prediction results and the multimodal root cause diagnosis report through the multi-objective trade-off model; The feedback optimization module is used to collect execution feedback data of the adaptive operation and maintenance instructions, and to iteratively optimize the operation and maintenance cases based on the execution feedback data and historical event data.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The process involves acquiring heterogeneous operation and maintenance data streams from the data center, performing layered compression on these multi-source heterogeneous data streams to obtain multimodal data streams, constructing a data operation graph based on these multimodal data streams, extracting dual-path temporal features from the data operation graph, and performing coupled modeling on these dual-path temporal features to obtain capacity prediction results. Within the data operation graph, the capacity prediction results are traced back to their source and their impact surface is extrapolated to obtain multimodal root causes. An adaptive operation and maintenance instruction is generated based on the capacity prediction results and the multimodal root cause diagnostic report using a multi-objective trade-off model. Execution feedback data of the adaptive operation and maintenance instruction is collected, and operation and maintenance cases are iteratively optimized using the execution feedback data and historical event data.

[0016] Therefore, this application can achieve automated management from multi-source heterogeneous operation and maintenance data to precise closed-loop operation and maintenance decisions, thereby improving the accuracy of data anomaly diagnosis. Firstly, by intelligently layering and compressing the heterogeneous operation and maintenance data streams of the data center, data storage and transmission overhead can be significantly reduced while retaining key semantic information, and a unified and regular multimodal data stream can be produced, providing a lightweight and high-quality data foundation for subsequent analysis. Furthermore, based on the multimodal data stream, a data operation graph is constructed, extracting dual-path temporal features containing local details and macro-cycles for coupled modeling. This facilitates the deep integration of the system's static topology structure and dynamic temporal behavior, ensuring that capacity prediction results not only reflect historical trends but also contain the potential impact of service dependencies, improving the context awareness and accuracy of predictions. Secondly, in the data operation graph, combined with capacity... By reverse-engineering and extrapolating the impact of capacity prediction results, it is possible to correlate anomalies with resource trends, accurately pinpoint the root cause of fault propagation paths, and quantify their potential impact on upstream and downstream services. This facilitates the transition from perceiving anomalies to diagnosing root causes and assessing risks. Then, through a multi-objective trade-off model, the root cause diagnosis report and forward-looking capacity prediction results are integrated to generate adaptive operation and maintenance instructions that balance service availability, resource efficiency, and cost constraints. This transforms diagnostic insights into actionable optimization actions. Finally, by collecting instruction execution feedback data and combining iterative optimization with historical cases, each operation and maintenance loop is transformed into experience to strengthen the prediction model, diagnostic algorithm, and decision-making strategy. This creates a virtuous cycle that becomes increasingly intelligent with use, continuously improving the overall accuracy and automation level of anomaly diagnosis and operation and maintenance management.

[0017] In summary, the technical solution adopted in this application can realize automated management from multi-source heterogeneous operation and maintenance data to precise closed-loop operation and maintenance decisions, thereby improving the accuracy of data anomaly diagnosis. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a data center intelligent operation and maintenance management method for cloud environments provided in this application; Figure 2 This is an exemplary flowchart of constructing a data operation graph according to the present application; Figure 3 This is an exemplary flowchart for determining multimodal root causes provided in this application; Figure 4 This is a module structure diagram of a cloud-oriented intelligent operation and maintenance management system for data centers, provided in this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] This application provides a data center intelligent operation and maintenance management system and method for cloud environments. Its core is to acquire heterogeneous operation and maintenance data streams from the data center, perform layered compression on these multi-source heterogeneous operation and maintenance data streams to obtain multimodal data streams; construct a data operation graph based on the multimodal data streams, extract dual-path temporal features from the data operation graphs, and perform coupled modeling on these dual-path temporal features to obtain capacity prediction results; perform reverse tracing and impact surface deduction on the capacity prediction results within the data operation graphs to obtain multimodal root causes; generate adaptive operation and maintenance instructions based on the capacity prediction results and the multimodal root cause diagnostic report using a multi-objective trade-off model; collect execution feedback data of the adaptive operation and maintenance instructions, and iteratively optimize operation and maintenance cases using the execution feedback data and historical event data.

[0022] Example 1: To better understand the above technical solution, the following will provide a detailed description of the technical solution in conjunction with the accompanying drawings and specific implementation methods. (Refer to...) Figure 1 As shown in the figure, this is a flowchart of a data center intelligent operation and maintenance management method for a cloud environment according to this embodiment of the present application. The operation and maintenance management method includes the following steps: In step S1, the heterogeneous operation and maintenance data stream of the data center is obtained, and the multi-source heterogeneous operation and maintenance data stream is layered and compressed to obtain a multimodal data stream.

[0023] In specific implementation, heterogeneous operation and maintenance data streams of the data center are obtained through data probes. These data probes are software agents that can be deployed on monitoring entities to proactively capture multi-dimensional and multi-format operational status data from the infrastructure, platform, and application layers. It should be noted that the heterogeneous operation and maintenance data streams in this application are data sets representing the operational status of the data center, including: indicator data, log data, and tracing data. Indicator data are quantified values ​​of system performance and resource usage recorded in time-series format, such as CPU utilization, memory usage, network input / output throughput, and disk read / write latency. Log data are semi-structured or unstructured text records generated by the operating system, middleware, and applications during operation, used to describe system events, error messages, user operations, etc., such as system logs, application error logs, and security audit logs. Tracing data, under a distributed service architecture, is structured data used to record the call paths and time consumption of a single user request across multiple services, including fields such as tracing identifier, span information, service name, timestamp, and parent span identifier, used to analyze inter-service dependencies and performance bottlenecks.

[0024] In this embodiment, intelligent hierarchical compression is performed on the multi-source heterogeneous operation and maintenance data stream to obtain a multimodal data stream, which can be achieved in the following manner: Semantic parsing, template matching, and deduplication are performed on the log data in the multi-source heterogeneous operation and maintenance data stream to obtain a lightweight log event stream. The lightweight log event stream is then subjected to hybrid compression to obtain a multimodal data stream.

[0025] In practice, firstly, log data can be read using a log parsing engine (such as the open-source Drain parser). The variable parts (such as timestamps, IP addresses, and numeric IDs) in the log data are separated from the constant text parts to obtain a log template sequence. Then, a hash algorithm is used to perform content-based deduplication on the log template sequence to obtain a lightweight log event stream. Next, a dynamic dictionary is created to store frequently occurring log template identifier sequence patterns. The lightweight log event stream is then scanned with a sliding window, and the dynamic dictionary is used to perform dictionary encoding and variable-length integer encoding on the lightweight log event stream to obtain a compressed data stream. Lossless compression (such as Zlib compression) is then used to align and encapsulate the indicator data stream and tracking data stream in the heterogeneous operation and maintenance data stream. The structured data packet composed of the compressed data stream and the aligned and encapsulated data stream is then used as a multimodal data stream.

[0026] It should be noted that lightweight log event streams refer to structured data sequences that have had their original text redundancy removed, retaining only log event types (templates) and time-series information. These sequences are used to characterize the order and frequency of various events occurring during system operation. Hybrid compression refers to data reduction techniques that combine multiple encoding strategies (such as dictionary encoding, variable-length encoding, and general lossless compression) based on the characteristics of the data types themselves (such as repetition patterns and numerical ranges). Multimodal data streams refer to a collection of indicator data, log data, and tracking data that have undergone layered compression processing. These streams can provide high-quality input for subsequent construction of data operation maps and extraction of time-series features.

[0027] In step S2, a data operation map is constructed based on the multimodal data stream, and dual-path time-series features are extracted from the data operation map. The dual-path time-series features are coupled and modeled to obtain the capacity prediction result.

[0028] Preferably, in this embodiment, reference Figure 2 As shown, this diagram is an exemplary flowchart for constructing a data operation graph according to the present application. In this embodiment, the construction of the data operation graph based on the multimodal data stream can be implemented using the following steps: First, in step S21, the distributed tracing context in the multimodal data stream is parsed and aggregated to identify and extract the set of operation and maintenance entities and the explicit call dependency pairs between entities; Then, in step S22, based on the set of operation and maintenance entities and the explicit call dependency pairs between the entities, the vector representation of the entities is learned using a graph embedding model, and then a dependency graph is constructed. Finally, in step S23, the dependency graph is updated based on the reachability of the intermediary nodes to obtain the data operation graph.

[0029] In practice, the first step is to call the tracing data processor to read the tracing data (i.e., the distributed tracing context) stored in a structured format. Key fields for each span record are then extracted, including: a unique span identifier, a parent span identifier, the name of the service to which it belongs, the operation name, the start timestamp, and the duration. Spans belonging to the same service and having a parent-child relationship in the call sequence are aggregated. The number of successful calls, average latency, and number of errors for the service within a specified time window are then calculated. The service name is used as the operation entity, and the above calculation results are used as explicit call dependency pairs. After traversing all the tracing data, the set of operation entities and the explicit call dependency pairs between entities can be obtained. Then, each service name in the set of operational entities is treated as a node, and each explicit call dependency pair is considered as a directed edge connecting the two nodes. The weight of the edge can be set to the number of calls. An initial dependency graph is constructed using these nodes and edges. The Node2Vec graph embedding algorithm is then used to learn the initial dependency graph. The Node2Vec graph embedding algorithm generates a node sequence by performing random walks on the graph, and then uses word vector training technology to map each node into a low-dimensional real vector, obtaining the vector representation of all nodes. The vector representation replaces the service name of the corresponding node, and the replaced graph is used as the dependency graph. Finally, in the graph theory algorithm library, for any two non-directly connected nodes in the dependency graph, it is determined whether there is a shortest path such that the path passes through a candidate intermediary node, that is, the two nodes establish an indirect reachability relationship through the intermediary node. A new virtual edge can be created for this indirect reachability relationship, and the virtual edge is updated in the dependency graph. The updated dependency graph is used as the data operation graph.

[0030] It should be noted that in this application, the set of operation and maintenance entities forms the basis of the graph structure, using service names or hostnames as unique identifiers; explicit invocation of dependencies traces the parent-child span relationship in the data, used to characterize the observable and direct interactive behavior of the system; vector representation refers to mapping each node in the set of operation and maintenance entities to a point in a low-dimensional, continuous, and dense real-number vector space through graph embedding technology; the dependency graph is an intermediate form from a pure structure graph to the input of a graph neural network with computable features, which can attach a structural semantic entity vector representation to each node; candidate intermediate nodes refer to nodes selected from the operation and maintenance entities based on network centrality indicators. Key nodes selected from the entity set that lie on multiple shortest paths in the graph; indirect reachability, as opposed to explicit call dependency, refers to the logical connectivity between two entities that do not have direct call records but are established through the transmission of candidate intermediary nodes. It can be used to fill in potential dependency links that are missing due to monitoring sampling, asynchronous calls, or indirect interactions through public services; the data runtime graph is a graph model that describes the runtime dependency structure and risk propagation path of a complex distributed system, including direct and observable call dependencies. Indirect fault propagation channels can be modeled by introducing candidate intermediary nodes and indirect reachability.

[0031] In this embodiment, the extraction of dual-path time-series features from the data runtime graph can be specifically carried out in the following manner: The data running graph is output through multi-layer dilated convolution of a temporal convolutional network to produce a sequence of inter-point feature vectors. The inter-point feature vector sequence is divided into an inter-segment feature vector sequence by a sliding window. The inter-point feature vector sequence and the inter-segment feature vector sequence are fused using a gating network to obtain dual-path temporal features.

[0032] In practical implementation, firstly, the indicator data associated with each node in the data flow graph (such as the CPU usage sequence of the past hour) can be used as input to a temporal convolutional network. This temporal convolutional network model consists of multiple stacked convolutional layers, some of which employ dilated convolution techniques to increase the receptive field without significantly increasing the parameters. After passing through each layer of this temporal convolutional network model, and undergoing nonlinear activation function processing and feature extraction, the final layer of the network outputs a sequence of inter-point feature vectors. Then, a sliding window of fixed time length, such as 10 minutes, can be used to slide across the inter-point feature vector sequence with a fixed stride. After aggregation operations through the window, feature vectors representing the overall characteristics of the time period can be generated. The sequence formed by arranging all feature vectors according to the sliding window order is used as the inter-segment feature vector sequence. Finally, the inter-point feature vector sequences and inter-segment feature vector sequences of inconsistent lengths are aligned in the time dimension. For example, upsampling is used to make the two sequences have the same number of time steps. Then, a gated fusion unit is constructed. This unit independently calculates a fusion weight vector for each aligned time step. The actual calculation method is as follows: the inter-point feature vector and inter-segment feature vector of the current time step are concatenated and input into a small fully connected neural network. After passing through the Sigmoid activation function, a weight vector with a value between 0 and 1 is output. This weight vector is multiplied element-wise with the inter-point feature vector. At the same time, the weight vector is subtracted from 1 and then multiplied element-wise with the inter-segment feature vector. Finally, the two product results are added together. The feature sequence composed of the fusion feature vectors generated in all time steps is then used as the dual-path temporal feature.

[0033] It should be noted that the inter-point feature vector sequence is a sequence obtained after extracting features point by point from a high-dimensional index time series. Each feature vector encodes local pattern information of the corresponding original time point and its neighboring time points (determined by the receptive field of the convolutional kernel), such as short-term fluctuations and abrupt change features, focusing on capturing fine-grained time dependencies. The inter-segment feature vector sequence is a sequence obtained by aggregating the inter-point feature vector sequence through a sliding window. Each vector represents a statistical summary or overall trend of the original data within the corresponding time window, such as mean level, direction of change, and periodic contour, focusing on capturing coarse-grained macro-evolution patterns spanning multiple time points. The gating network is an information weighting mechanism for adaptively fusing two-path features. It can dynamically generate a set of weights based on the specific content of the two input features at each time step to determine the proportion of local detail information from the inter-point path and macro-trend information from the inter-segment path in the final output. The dual-path time series features contain key information features of the time series at both the micro-local and macro-segment scales, providing a more comprehensive input representation for capacity prediction and anomaly detection.

[0034] In this embodiment, the capacity prediction result is obtained by coupling and modeling the dual-path time-series features in the following manner: The dual-path temporal features and the data execution graph are fused to obtain the enhanced temporal features of the topological dependency context; A capacity prediction sequence is generated based on the enhanced time-series features using a parallel architecture that includes a long-term trend prediction branch and a short-term fluctuation prediction branch. The capacity prediction value sequence is dynamically calibrated to obtain the capacity prediction result.

[0035] In practice, firstly, for each node in the data execution graph, a dual-path temporal feature vector is used as its initial feature. The data execution graph is then processed using a graph attention network layer. By stacking multiple such graph attention network layers, the final output feature vector of each node after processing by multiple graph attention networks is used as the enhanced temporal feature of the topological dependency context. Then, a parallel prediction model with two independent sub-networks is constructed. The long-term trend prediction branch typically uses a recurrent neural network (such as LSTM) or its variants, with the input being an enhanced temporal series feature sequence from a longer past time window (e.g., the past 24 hours). Its goal is to learn and predict the overall baseline of resource capacity changes over a longer future time span (e.g., the next 6 hours), and output a relatively smooth capacity baseline value sequence. The short-term fluctuation prediction branch typically uses a temporal convolutional network or a shallower fully connected network, with the input being an enhanced temporal series feature sequence from a shorter past time window (e.g., the past 30 minutes). Its goal is to capture and predict the rapid fluctuations or corrections in capacity around the baseline value in the next short time span (e.g., the next 30 minutes), and output a capacity correction sequence. The capacity baseline value sequence output by the long-term trend prediction branch and the capacity correction sequence output by the short-term fluctuation prediction branch are then added together at corresponding time steps, and the sum is used as the capacity prediction value sequence. Finally, real-time anomaly detection results are obtained, and anomaly nodes and their propagation paths in the data flow graph are located. For capacity prediction values, if the corresponding node is located on the anomaly propagation path, an attenuation coefficient is set based on the number of hops (propagation depth) from the path source, and the prediction value is adjusted accordingly. Then, the adjusted capacity prediction values ​​of all nodes are used as the capacity prediction results. It should be noted that, in this application, the enhanced temporal features of topological dependency context refer to the dual-path temporal features of nodes processed by a graph attention network, which can provide an information basis for predicting capacity changes propagated due to dependencies; the long-term trend prediction branch is the branch in the parallel prediction architecture responsible for capturing and predicting slow, continuous capacity changes, and can extract macroscopic patterns such as cycles and trends from longer historical data; the short-term fluctuation prediction branch is the branch in the parallel prediction architecture responsible for capturing and predicting rapid, temporary capacity changes, and can identify microscopic disturbances such as random fluctuations and sudden loads from recent data; dynamic calibration refers to the calibration based on the capacity prediction value and the results of real-time anomaly detection. The risk-aware correction process involves the following: when the system detects an anomaly and forms a propagation path in the graph, it is believed that the capacity performance of the nodes on the path may deviate from the prediction based on pure historical patterns in the future. Therefore, it is necessary to conservatively lower the predicted value or increase the uncertainty based on the intensity of the anomaly's impact and the propagation distance, so that the final capacity prediction result is more in line with the real-time health status of the system and has risk warning value. The capacity prediction result refers to the estimated set of predicted usage of key resources (such as CPU and memory) of various operation and maintenance entities (such as servers and services) in the data center, which can provide data basis for resource elastic scaling, fault preventive scheduling, and cost optimization.

[0036] In step S3, the capacity prediction results are reverse-engineered and influence surface extrapolated in the data operation graph to obtain multimodal root causes.

[0037] Preferably, in this embodiment, reference Figure 3 As shown, this diagram is an exemplary flowchart for determining multimodal root causes according to the present application. In this embodiment, the multimodal root causes are obtained by reverse tracing and influence surface extrapolation of the capacity prediction results in the data operation map, which can be achieved by the following steps: First, in step S31, a backpropagation simulation is performed in the data operation graph to obtain a set of anomaly source nodes; Then, in step S32, multimodal anomaly evidence is determined through the set of anomaly source nodes, thereby obtaining the root cause node; Finally, in step S33, starting from the root cause node, a positive impact propagation analysis is performed on the data operation graph based on the multimodal anomaly evidence to obtain the multimodal root cause.

[0038] In specific implementation, firstly, a restart-based random walk algorithm is used to perform reverse simulation on the data operation graph. That is, a random walk is performed along the opposite direction of the edges in the data operation graph (i.e., from the callee to the caller, or from the affected party to the source of influence). Each time the algorithm reaches a node, it has a certain probability of jumping back to the initial abnormal node, thereby ensuring that the simulation is carried out around the abnormal area, and thus the set of abnormal source nodes can be identified. Then, for each node in the set of anomaly source nodes, the multimodal data stream associated with that node is synchronously backtracked and queried. Three aspects of multimodal anomaly evidence are extracted from the multimodal data stream associated with that node: anomaly index segments that significantly deviate from the historical baseline are extracted from the index data; sequences and frequencies of anomaly log templates such as errors and warnings are extracted from the log data; and a tracking subgraph with the node as the vertex and abnormal response time or call failure is extracted from the tracking data. A multimodal evidence aggregation network (e.g., a shallow fully connected neural network) is then constructed. The feature vectors of the above three types of heterogeneous evidence are used as input, so that the multimodal evidence aggregation network can learn the contribution weight of different evidence to the judgment of the root cause after training. It can output a comprehensive confidence score, the confidence score of each node in the set of anomaly source nodes, and sort them from high to low. The node ranked first is taken as the root cause node. Finally, starting from the root cause node, a directed breadth-first search is performed along the normal direction of the edges in the data operation graph (i.e., from the caller to the callee, or from the source of influence to the affected party). An impact assessment is performed on each downstream node in the search path. The impact assessment involves analyzing the historical normal capacity baseline, current capacity prediction results, and abnormal patterns from the root cause node (such as increased latency or increased error rate) of the downstream node. The degree of performance degradation that the downstream node may suffer is calculated and an impact level (such as high, medium, or low) is marked. The search stops when the graph boundary is reached or the impact level is lower than a specific threshold. This generates a visual graph centered on the root cause node, clearly marking each affected node and its impact level. The visual graph, combined with the multimodal anomaly evidence from the root cause node, is used as the multimodal root cause.

[0039] It should be noted that, in this application, the set of abnormal source nodes refers to the set of potential nodes that are the source of the failure; multimodal anomaly evidence refers to the abnormal data fragments extracted from three different types of data—indicators, logs, and traces—that support or refute the root cause of a specific suspected node; the root cause node refers to the initial failure point or core failure component that is determined to be most likely to cause a series of currently observed anomalies based on the fusion analysis of multimodal anomaly evidence; positive impact propagation analysis refers to the process of reasoning and assessing the possible scope and severity of the failure from the identified root cause node, along the normal dependencies or calls of the system; and multimodal root causes are failure analysis reports used to guide repair and prevention.

[0040] In step S4, an adaptive operation and maintenance instruction is generated based on the capacity prediction results and the multimodal root cause diagnosis report using a multi-objective trade-off model.

[0041] In this embodiment, the adaptive operation and maintenance instructions generated by the multi-objective trade-off model based on the capacity prediction results and the multi-modal root cause diagnosis report can be specifically implemented in the following manner: The multimodal root causes and the capacity prediction results are analyzed to generate a set of decision constraints; The simulation evaluation matrix is ​​determined based on the decision constraint set and the preset operation and maintenance strategy library; The simulation evaluation matrix is ​​input into a multi-objective trade-off model for optimization and solution, thereby generating adaptive operation and maintenance instructions.

[0042] In practical implementation, firstly, the diagnostic report and prediction results are parsed to generate a set of decision constraints. A rule parsing engine is used to extract root cause nodes, fault types, affected nodes, and capacity prediction values ​​to form the decision constraint set. Then, based on the decision constraint set and a pre-defined operation and maintenance strategy library, a simulation evaluation matrix is ​​determined. Basic operation and maintenance units are selected and combined from the strategy library to form multiple candidate strategy sequences. Using the current system state as the initial environment, a discrete event simulator is used to simulate the execution process of each strategy sequence, evaluating its impact on three objectives: service level agreement, resource utilization, and operation and maintenance cost. Evaluation vectors are obtained and combined into a simulation evaluation matrix. The pre-defined operation and maintenance strategy library is a knowledge base for standardized operation and maintenance operations, which can be established based on actual operation and maintenance experience to provide composable strategy units. Finally, the simulation evaluation matrix is ​​input into a multi-objective trade-off model for optimization. A multi-objective optimization algorithm is used to find the Pareto optimal solution set that achieves the best trade-off among multiple objectives. The most suitable solution is selected according to a preset priority, and its corresponding candidate strategy sequence is converted into a specific executable command sequence as adaptive operation and maintenance instructions.

[0043] It should be noted that, in this application, the decision constraint set refers to the set of boundary conditions of the optimization problem transformed from diagnostic and prediction conclusions, which can formalize complex operation and maintenance problems into a constrained optimization problem; the simulation evaluation matrix is ​​a pre-evaluation matrix of the performance of candidate strategies under multiple objectives, used to characterize the potential gains and losses of different strategies on multiple objectives, and is the data basis for scientific trade-offs; the multi-objective trade-off model is an algorithmic model used to find balanced solutions among conflicting objectives; and the adaptive operation and maintenance instructions are capable of generating the most suitable response solutions for different operation and maintenance scenarios.

[0044] In step S5, the execution feedback data of the adaptive operation and maintenance instructions is collected, and the operation and maintenance cases are iteratively optimized using the execution feedback data and historical event data.

[0045] In practice, the execution feedback data of the adaptive operation and maintenance instructions is collected through the log interface. The log interface refers to a standardized application programming interface that follows a structured data exchange format (such as JSON format). This interface can be provided by the underlying resource management platform (such as the Nova service of the Kubernetes cluster). When the adaptive operation and maintenance instructions are distributed to the execution engine and completed, the execution engine will call this log interface to backfill and report the detailed records of the instruction execution process in the form of predefined key-value pairs. It should be noted that execution feedback data refers to structured data records that document the status and effects of adaptive operation and maintenance instructions from issuance to completion. This data provides factual evidence for model optimization and case study learning, including: instruction execution status pipeline, target resource status changes, and system-level related impact indicators. The instruction execution status pipeline includes the instruction's unique identifier, start timestamp, end timestamp, total execution time, final execution result (success, failure, partial success), and error codes or exceptions that may be thrown during execution. This is used to determine whether the operation and maintenance actions were completed correctly and in a timely manner. Target resource status changes refer to snapshots comparing key indicators of the operation and maintenance entity (such as the target virtual machine, container, or service) before and after execution, such as CPU utilization, memory usage, service response time, and network connection count. This is used to quantitatively evaluate the direct effects of the operation and maintenance actions on the target object. System-level related impact indicators refer to changes in key performance indicators of related services or the entire system affected by the instruction within the instruction's execution time window, such as overall service availability, request success rate, average latency, and system load. This is used to assess the broader side effects or indirect impacts that the operation and maintenance actions may produce.

[0046] It should be noted that, in this embodiment, the iterative optimization of operation and maintenance cases using the execution feedback data and historical event data can be carried out in the following way: First, a structured operation and maintenance case unit is constructed, that is, the multimodal root causes of operation and maintenance events, capacity prediction results, adaptive operation and maintenance instructions, execution feedback data, and calculated performance indicators (such as fault recovery time) are integrated into a structured record in a unified format to form a complete case. Then, new cases are stored in the operation and maintenance case knowledge base and correlation analysis is performed. The knowledge base uses a dedicated database to store cases, retrieves historical similar cases based on case features, and compares the operation and maintenance instructions and performance indicators of new and old cases. Finally, incremental updates and optimizations are performed based on the analysis results. If the new instructions are more effective, their recommendation weight is strengthened; if the effect is not good, the reasons are analyzed, and the relevant model parameters are fine-tuned according to the conclusions. For example, the feedback data is added to the training set of the prediction model for incremental training, or the weight coefficients of the multi-objective trade-off model are adjusted. The system achieves self-evolution through continuous learning.

[0047] It should be noted that a structured operation and maintenance case unit refers to a complete record of operation and maintenance event data packaged in a unified format; an operation and maintenance case knowledge base refers to a system for storing, indexing, and managing cases to support querying and analysis; a correlation analysis process refers to an intelligent process that automatically matches new and old cases and compares and analyzes them to learn from experience; incremental updates and optimizations refer to the gradual calibration and fine-tuning of model parameters using new data; and iterative optimization of operation and maintenance cases refers to the process of achieving system self-learning and continuous performance improvement through feedback loops.

[0048] In summary, the technical solution adopted in this application can realize automated management from multi-source heterogeneous operation and maintenance data to precise closed-loop operation and maintenance decisions, thereby improving the accuracy of data anomaly diagnosis.

[0049] Example 2: This application provides a data center intelligent operation and maintenance management system for cloud environments, referencing... Figure 4 As shown, this diagram is a modular structure diagram of a data center intelligent operation and maintenance management system for cloud environments provided in this application. The operation and maintenance management system includes: The hierarchical compression module 100 is used to acquire heterogeneous operation and maintenance data streams from the data center, and to perform hierarchical compression on the multi-source heterogeneous operation and maintenance data streams to obtain multimodal data streams. The coupled modeling and prediction module 200 is used to construct a data operation map based on the multimodal data stream, extract dual-path time series features from the data operation map, perform coupled modeling on the dual-path time series features, and then obtain capacity prediction results. The source tracing and inference module 300 is used to perform reverse source tracing and influence surface inference on the capacity prediction results in the data operation map to obtain multimodal root causes; The multi-objective trade-off module 400 is used to generate adaptive operation and maintenance instructions based on the capacity prediction results and the multimodal root cause diagnosis report through the multi-objective trade-off model; The feedback optimization module 500 is used to collect the execution feedback data of the adaptive operation and maintenance instructions, and to iteratively optimize the operation and maintenance cases through the execution feedback data and historical event data.

[0050] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0051] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0052] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. A method for intelligent operation and maintenance management of data centers in a cloud environment, characterized in that, The operation and maintenance management method includes the following steps: Obtain heterogeneous operation and maintenance data streams from the data center, and perform layered compression on the multi-source heterogeneous operation and maintenance data streams to obtain multimodal data streams; A data operation map is constructed based on the multimodal data stream, and dual-path time series features are extracted from the data operation map. The dual-path time series features are coupled and modeled to obtain the capacity prediction results. In the data operation graph, the capacity prediction results are traced back to their source and the impact surface is extrapolated to obtain the multimodal root causes; Adaptive operation and maintenance instructions are generated based on the capacity prediction results and the multimodal root cause diagnosis report using a multi-objective trade-off model. The execution feedback data of the adaptive operation and maintenance instructions is collected, and the operation and maintenance cases are iteratively optimized using the execution feedback data and historical event data.

2. The intelligent operation and maintenance management method for data centers in a cloud environment as described in claim 1, characterized in that, Obtain heterogeneous operation and maintenance data streams from the data center using data probes.

3. The intelligent operation and maintenance management method for data centers in a cloud environment as described in claim 1, characterized in that, The intelligent layered compression of the multi-source heterogeneous operation and maintenance data streams to obtain multimodal data streams specifically includes: Semantic parsing, template matching, and deduplication are performed on the log data in the multi-source heterogeneous operation and maintenance data stream to obtain a lightweight log event stream. The lightweight log event stream is then subjected to hybrid compression to obtain a multimodal data stream.

4. The intelligent operation and maintenance management method for data centers in a cloud environment as described in claim 1, characterized in that, Constructing a data operation map based on the multimodal data stream specifically includes: The distributed tracing context in the multimodal data stream is parsed and aggregated to identify and extract the set of operation and maintenance entities and the explicit call dependency pairs between entities; Based on the set of operation and maintenance entities and the explicit call dependency pairs between the entities, the vector representations of the entities are learned using a graph embedding model, and then a dependency graph is constructed. The dependency graph is updated based on the reachability of the intermediary nodes to obtain the data operation graph.

5. The intelligent operation and maintenance management method for data centers in a cloud environment as described in claim 1, characterized in that, Extracting dual-path temporal features from the data runtime graph specifically includes: The data running graph is output through multi-layer dilated convolution of a temporal convolutional network to produce a sequence of inter-point feature vectors. The inter-point feature vector sequence is divided into an inter-segment feature vector sequence by a sliding window. The inter-point feature vector sequence and the inter-segment feature vector sequence are fused using a gating network to obtain dual-path temporal features.

6. The intelligent operation and maintenance management method for data centers in a cloud environment as described in claim 1, characterized in that, The coupling modeling of the dual-path time-series features to obtain capacity prediction results specifically includes: The dual-path temporal features and the data execution graph are fused to obtain the enhanced temporal features of the topological dependency context; A capacity prediction sequence is generated based on the enhanced time-series features using a parallel architecture that includes a long-term trend prediction branch and a short-term fluctuation prediction branch. The capacity prediction value sequence is dynamically calibrated to obtain the capacity prediction result.

7. The intelligent operation and maintenance management method for data centers in a cloud environment as described in claim 1, characterized in that, In the data operation graph, the capacity prediction results are traced back to their source and the impact surface is extrapolated to obtain the multimodal root causes, which specifically include: A backpropagation simulation is performed on the data operation graph to obtain the set of anomaly source nodes; Multimodal anomaly evidence is determined by the set of anomaly source nodes, thereby obtaining the root cause nodes; Starting from the root cause node, a positive impact propagation analysis is performed on the data operation graph based on the multimodal anomaly evidence to obtain the multimodal root cause.

8. The intelligent operation and maintenance management method for data centers in a cloud environment as described in claim 1, characterized in that, The adaptive operation and maintenance instructions generated by the multi-objective trade-off model based on the capacity prediction results and the multi-modal root cause diagnosis report specifically include: The multimodal root causes and the capacity prediction results are analyzed to generate a set of decision constraints; The simulation evaluation matrix is ​​determined based on the decision constraint set and the preset operation and maintenance strategy library; The simulation evaluation matrix is ​​input into a multi-objective trade-off model for optimization and solution, thereby generating adaptive operation and maintenance instructions.

9. The intelligent operation and maintenance management method for data centers in a cloud environment as described in claim 1, characterized in that, The execution feedback data of the adaptive operation and maintenance instructions is collected through the log interface.

10. A data center intelligent operation and maintenance management system for cloud environments, used to execute a data center intelligent operation and maintenance management method for cloud environments as described in any one of claims 1 to 9, characterized in that, The operation and maintenance management system includes: The hierarchical compression module is used to acquire heterogeneous operation and maintenance data streams from the data center, and to perform hierarchical compression on the multi-source heterogeneous operation and maintenance data streams to obtain multimodal data streams. The coupled modeling and prediction module is used to construct a data operation map based on the multimodal data stream, extract dual-path time series features from the data operation map, perform coupled modeling on the dual-path time series features, and then obtain the capacity prediction result. The source tracing and inference module is used to perform reverse source tracing and impact surface inference on the capacity prediction results in the data operation map to obtain multimodal root causes; The multi-objective trade-off module is used to generate adaptive operation and maintenance instructions based on the capacity prediction results and the multimodal root cause diagnosis report through the multi-objective trade-off model; The feedback optimization module is used to collect execution feedback data of the adaptive operation and maintenance instructions, and to iteratively optimize the operation and maintenance cases based on the execution feedback data and historical event data.

Images