Intelligent operation and maintenance method and system based on big data

Abstract

The application discloses an intelligent operation and maintenance method and system based on big data, relates to the technical field of intelligent operation and maintenance, and is based on time series analysis and a deep generation learning model, dynamically generates a self-adaptive multi-dimensional baseline for each key performance indicator in a unified data view, and quantitatively calculates an abnormality degree score of a real-time state of the indicator relative to the baseline; by means of cross-modal feature fusion and graph structure reasoning technology, the abnormality degree score and deep semantic features of multi-source heterogeneous data are jointly modeled and aggregated, and a globally consistent abnormal event is output. The application realizes unified management and efficient utilization of operation and maintenance data, solves the problem of scattered traditional operation and maintenance data and difficulty in collaborative analysis. The generation of a dynamically self-adaptive baseline improves the accuracy and adaptability of abnormality detection and effectively reduces the false alarm and missed alarm rates. Root cause positioning technology based on causal inference and counterfactual reasoning realizes accurate positioning of a root cause entity and a propagation path and improves fault troubleshooting efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology, and in particular to an intelligent operation and maintenance method and system based on big data. Background Technology

[0002] As IT operations and maintenance (O&M) systems become larger and more complex, O&M data exhibits multi-source and heterogeneous characteristics, encompassing various types such as time-series performance metrics, log texts, and call chain topologies. Traditional O&M methods often rely on static thresholds for anomaly detection, which struggles to adapt to dynamic changes in business load and is prone to false positives and false negatives.

[0003] Meanwhile, existing operation and maintenance technologies mostly rely on correlation analysis to locate the root cause of faults, which cannot distinguish between causal relationships and correlation relationships, resulting in low accuracy and efficiency in root cause location, making it difficult to achieve early prediction and automatic self-healing of faults.

[0004] Furthermore, a large number of fault cases and repair experiences generated during the operation and maintenance process have not been effectively accumulated, making it impossible to form a closed-loop optimization mechanism. As a result, the adaptability and accuracy of the operation and maintenance model are difficult to continuously improve, making it difficult to meet the needs of large-scale, high-reliability operation and maintenance scenarios. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides an intelligent operation and maintenance method and system based on big data. The following technical solution is adopted: A big data-based intelligent operation and maintenance method includes the following steps: Step 1: Collect, standardize, associate, and merge multi-source heterogeneous data obtained from the operation and maintenance system to generate a unified data view with consistency in the time dimension and completeness in the semantic dimension; Step 2: Based on time series analysis and deep generative learning models, dynamically generate adaptive multidimensional baselines for each key performance indicator in the unified data view, and quantify the anomaly score of the real-time status of the indicator relative to the baseline. Step 3: Using cross-modal feature fusion and graph structure reasoning techniques, perform joint modeling and aggregation analysis on anomaly scores and deep semantic features of multi-source heterogeneous data, and output globally consistent anomaly events; Step 4: Construct a dynamic topology structure that represents the causal relationships between entities within the operation and maintenance system. Through causal inference and counterfactual reasoning logic, locate the root cause entity and the anomaly propagation path from globally consistent anomaly events. Step 5: Based on the spatiotemporal sequence graph neural network, predict and learn the evolution state of the operation and maintenance system, output the probability of failure and time window, and match and execute automated operation and maintenance decisions according to the root cause entity and the anomaly propagation path.

[0006] Optionally, the method for dynamically generating the adaptive multidimensional baseline in step 2 is: Seasonal trend decomposition was performed on the time series data of key performance indicators to separate the periodic component, trend component and residual component. A variational autoencoder network is constructed to extract and reconstruct latent features from time series data within a preset time window, and a reconstruction error index is generated. The baseline upper and lower limit thresholds are adaptively adjusted based on the real-time distribution statistical characteristics of the residual components. The variational autoencoder network parameters are continuously updated using an online gradient descent strategy to adapt to the conceptual drift of the operation and maintenance system and achieve continuous dynamic iteration of the baseline. The abnormality score is calculated using the following formula: ; ; in It is the anomaly score at time t, and its value ranges from 1 to 2. A larger value indicates a higher degree of abnormality. It is a preset weight balancing coefficient with a value range of [0,1], used to balance the contribution of numerical anomalies and reconstruction error anomalies; It is an anomaly score calculated based on the degree of numerical deviation; The measured values ​​of key performance indicators at time t. This is the upper limit of the dynamic baseline threshold. This is the lower limit of the dynamic baseline threshold; It is anomaly score calculated based on the reconstruction error of variational autoencoder.

[0007] Optionally, the method for cross-modal feature fusion and graph structure reasoning in step 3 is: Multi-source heterogeneous data is decomposed into time-series index data, unstructured text data, and topological data according to data type. Features are extracted from the three types of data using a time-series attention network, a pre-trained language model, and a graph embedding model, respectively. The extracted multi-modal features are fused through a cross-modal attention alignment mechanism to generate a fused feature vector. Based on the fused feature vector, the anomaly confidence of globally consistent abnormal events is calculated.

[0008] Optionally, the method for constructing the dynamic topology representing the causal relationships between entities within the operation and maintenance system in step 4 is: During normal operation, an initial causal topology graph is constructed based on a constrained causal structure learning algorithm, and the initial causal topology graph is constrained and corrected by incorporating domain expert rules. During the period when an anomaly occurs, the node weights of the causal topology graph are dynamically adjusted according to the anomaly severity score to generate a dynamic causal topology structure.

[0009] Optionally, in step 4, the method for locating the root cause entity and the abnormal propagation path through causal inference and counterfactual reasoning logic is as follows: Perform a random walk causal propagation algorithm on a dynamic causal topology to calculate the causal influence score for each node. A candidate root cause entity set is selected. For each entity in the candidate root cause entity set, a counterfactual reasoning operation is performed, that is, the entity's state value is replaced with the normal baseline value to simulate the system's operating state. After the counterfactual reasoning operation, the total decrease in the anomalousness score of all anomalous entities is calculated. The entity with the largest total decrease is selected as the final root cause entity, and the corresponding anomalous propagation path is traced using the following formula: ; It is the causal influence score of node i. The higher the score, the greater the likelihood that the node is the root cause. It is a set of abnormal nodes, containing all operation and maintenance entity nodes whose abnormality scores exceed a preset threshold; It is the causal path influence value from the abnormal node j to the candidate node i, which is determined by the path length, the weight of each edge on the path, and the causal strength.

[0010] Optionally, the method for predicting the evolutionary state of the operation and maintenance system based on the spatiotemporal sequence graph neural network in step 5 is as follows: The unified data view is constructed as a dynamic graph structure, where nodes represent operation and maintenance entities and edges represent relationships between entities. The dynamic graph structure is input into the spatiotemporal sequence graph neural network for training to learn the spatiotemporal evolution law of the operation and maintenance system. Based on the trained model, the probability of failure and the specific time of occurrence within a preset time window are predicted.

[0011] Optionally, the method for matching and executing automated operation and maintenance decisions in step 5 is: A strategy library containing various automated operation and maintenance strategies is constructed, and the strategy library is associated and bound with the anomaly propagation path and root cause entity; based on the located root cause entity and anomaly propagation path, the corresponding automated operation and maintenance strategy is matched from the strategy library; and the matched automated operation and maintenance strategy is executed within the time window according to priority.

[0012] Optionally, step 6 is also included, which involves constructing an operation and maintenance knowledge graph from the fault occurrence probability, root cause entity, abnormal propagation path, and automated operation and maintenance decision results. The entity includes fault phenomena, root cause entities, and operation and maintenance strategies, and the relationship includes causal relationship and trigger relationship. The operation and maintenance knowledge graph is then subjected to embedding representation learning to generate vector representations of entities and relationships. Based on the vector representations, the model and reasoning logic described in S2 to S5 are periodically fine-tuned and optimized.

[0013] Optionally, the multi-source heterogeneous data in step 1 includes time-series performance metrics, log text, call chain topology data, and network event data; the standardized association and fusion processing includes timestamp alignment, entity unique identifier association, and semantic terminology normalization of the data.

[0014] A big data-based intelligent operation and maintenance system is used to implement a big data-based intelligent operation and maintenance method. The system includes a data acquisition and fusion module, a dynamic baseline generation and anomaly calculation module, a multimodal anomaly fusion and reasoning module, a causal root cause localization module, and a fault prediction and self-healing decision-making module. The data acquisition and fusion module is used to collect multi-source heterogeneous data within the operation and maintenance system, and to perform timestamp alignment, entity association and semantic normalization processing on the multi-source heterogeneous data to generate a unified data view. The dynamic baseline generation and anomaly calculation module generates an adaptive multidimensional baseline based on temporal decomposition and variational autoencoder, and calculates the anomaly score of each key performance indicator. The multimodal anomaly fusion and reasoning module is used to output globally consistent anomaly events through cross-modal feature fusion and graph structure reasoning; The causal root cause localization module is used to construct a dynamic causal topology and, together with causal inference and counterfactual reasoning, locate root cause entities and abnormal propagation paths. The fault prediction and self-healing decision module is used to predict the probability of fault occurrence and time window based on spatiotemporal sequence graph neural networks, and to match and execute automated operation and maintenance decisions.

[0015] In summary, the present invention has at least one of the following beneficial technical effects: by standardizing and integrating multi-source heterogeneous data, unified management and efficient utilization of operation and maintenance data are achieved, solving the problem of traditional operation and maintenance data being scattered and difficult to analyze collaboratively.

[0016] The generation of dynamic adaptive baselines improves the accuracy and adaptability of anomaly detection, effectively reducing false positives and false negatives. Root cause localization technology based on causal inference and counterfactual reasoning enables precise location of root cause entities and propagation paths, improving troubleshooting efficiency.

[0017] The application of spatiotemporal sequence graph neural networks enables early prediction of faults, and combined with automated operation and maintenance decisions, achieves fault self-healing and reduces manual operation and maintenance costs.

[0018] The construction of the operation and maintenance knowledge graph and the optimization of the closed-loop model enable the accumulation and reuse of operation and maintenance experience, and continuously improve the adaptive capability and intelligence level of the operation and maintenance system. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating an intelligent operation and maintenance method based on big data according to the present invention. Figure 2 This is a schematic diagram of the architecture of an intelligent operation and maintenance system based on big data according to the present invention; Detailed Implementation

[0020] The present invention will be further described in detail below with reference to the accompanying drawings.

[0021] This invention discloses an intelligent operation and maintenance method and system based on big data.

[0022] Reference Figure 1 and Figure 2 Example 1: A smart operation and maintenance method based on big data includes the following steps: Step 1: Collect, standardize, associate, and merge multi-source heterogeneous data obtained from the operation and maintenance system to generate a unified data view with consistency in the time dimension and completeness in the semantic dimension; Step 2: Based on time series analysis and deep generative learning models, dynamically generate adaptive multidimensional baselines for each key performance indicator in the unified data view, and quantify the anomaly score of the real-time status of the indicator relative to the baseline. Step 3: Using cross-modal feature fusion and graph structure reasoning techniques, perform joint modeling and aggregation analysis on anomaly scores and deep semantic features of multi-source heterogeneous data, and output globally consistent anomaly events; Step 4: Construct a dynamic topology structure that represents the causal relationships between entities within the operation and maintenance system. Through causal inference and counterfactual reasoning logic, locate the root cause entity and the anomaly propagation path from globally consistent anomaly events. Step 5: Based on the spatiotemporal sequence graph neural network, predict and learn the evolution state of the operation and maintenance system, output the probability of failure and time window, and match and execute automated operation and maintenance decisions according to the root cause entity and the anomaly propagation path.

[0023] By adopting the above technical solutions, a unified data foundation is first established through data acquisition and fusion, solving the problem of difficult collaborative analysis of multi-source heterogeneous data and providing standardized data support for all subsequent operation and maintenance operations. Based on this, an adaptive baseline is dynamically generated and anomaly scores are calculated, enabling real-time monitoring of the system's operating status and accurate identification of anomaly signs. Through cross-modal fusion and graph structure reasoning, the anomaly features of multiple data types are integrated to form globally consistent anomaly events, avoiding misjudgments caused by noise in single-source data. Using causal topology and reasoning logic, the root cause and propagation path of the fault are accurately located, providing a clear direction for fault handling. Finally, a spatiotemporal sequence graph neural network is used to predict the fault development trend and match it with automated strategies to execute self-healing operations, forming a complete operation and maintenance chain from monitoring, identification, and location to prediction and repair, comprehensively improving the intelligence and efficiency of operation and maintenance.

[0024] Example 2: The method for dynamically generating adaptive multidimensional baselines in step 2 is as follows: Seasonal trend decomposition was performed on the time series data of key performance indicators to separate the periodic component, trend component and residual component. A variational autoencoder network is constructed to extract and reconstruct latent features from time series data within a preset time window, and a reconstruction error index is generated. The baseline upper and lower limit thresholds are adaptively adjusted based on the real-time distribution statistical characteristics of the residual components. The variational autoencoder network parameters are continuously updated using an online gradient descent strategy to adapt to the conceptual drift of the operation and maintenance system and achieve continuous dynamic iteration of the baseline. The abnormality score is calculated using the following formula: ; ; in It is the anomaly score at time t, and its value ranges from 1 to 2. A larger value indicates a higher degree of abnormality. It is a preset weight balancing coefficient with a value range of [0,1], used to balance the contribution of numerical anomalies and reconstruction error anomalies; It is an anomaly score calculated based on the degree of numerical deviation; The measured values ​​of key performance indicators at time t. This is the upper limit of the dynamic baseline threshold. This is the lower limit of the dynamic baseline threshold; It is anomaly score calculated based on the reconstruction error of variational autoencoder.

[0025] By adopting the above technical solution, the time series data of key performance indicators are first decomposed seasonally to separate periodic, trend, and residual components. This accurately captures the inherent variation patterns of the data, eliminates interference from periodic fluctuations and long-term trends in anomaly detection, and focuses on true anomalies. A variational autoencoder network is constructed to extract and reconstruct latent features from time series data within a preset time window. The reconstruction error reflects the normality of the data, overcoming the limitations of single numerical analysis. The baseline upper and lower thresholds are dynamically adjusted based on the real-time distribution statistical characteristics of the residual components, enabling the baseline to adapt to dynamic changes in business load and avoiding false alarms or missed alarms caused by fixed thresholds. The variational autoencoder network parameters are continuously updated using an online gradient descent strategy, which can adapt to conceptual drift in the operation and maintenance system in a timely manner, ensuring continuous dynamic iteration of the baseline and maintaining consistency with the system's operating status. The anomaly score is calculated by balancing numerical anomalies and reconstruction error anomalies with weights, combining two dimensions of anomaly judgment to further improve the accuracy of anomaly detection, ensuring that the anomaly score truly reflects the degree of anomaly in the system.

[0026] Example 3, the method of cross-modal feature fusion and graph structure reasoning in step 3 is as follows: Multi-source heterogeneous data is decomposed into time-series index data, unstructured text data, and topological data according to data type. Features are extracted from the three types of data using a time-series attention network, a pre-trained language model, and a graph embedding model, respectively. The extracted multi-modal features are fused through a cross-modal attention alignment mechanism to generate a fused feature vector. Based on the fused feature vector, the anomaly confidence of globally consistent abnormal events is calculated.

[0027] By adopting the above technical solution, multi-source heterogeneous data is first decomposed into time-series index data, unstructured text data, and topological structure data. Corresponding feature extraction models are used to address the feature differences of different data types, ensuring that the deep semantic features of each type of data can be fully mined. The time-series attention network can focus on key time node features in the time-series index data, the pre-trained language model can accurately extract semantic information from log text, and the graph embedding model can transform topological structure data such as call chains into computable feature vectors. These three models are adapted to the processing needs of different data types. Through a cross-modal attention alignment mechanism, the extracted multimodal features are fused, achieving complementarity between different types of features, highlighting key features related to anomalies, and weakening irrelevant noise. Based on the fused feature vectors, the anomaly confidence of globally consistent anomaly events is calculated, integrating anomaly information from multiple sources to generate unified and accurate anomaly events, avoiding biased judgments caused by single-source data anomalies, and improving the comprehensiveness and reliability of anomaly event identification.

[0028] Example 4: The method for constructing the dynamic topology representing the causal relationships between entities within the operation and maintenance system in step 4 is as follows: During normal operation, an initial causal topology graph is constructed based on a constrained causal structure learning algorithm, and the initial causal topology graph is constrained and corrected by incorporating domain expert rules. During the period when an anomaly occurs, the node weights of the causal topology graph are dynamically adjusted according to the anomaly severity score to generate a dynamic causal topology structure.

[0029] By adopting the above technical solution, during normal system operation, an initial causal topology graph is constructed based on a constrained causal structure learning algorithm. By analyzing the correlations between various indicators and entities under normal operation, potential causal logic is uncovered, ensuring that the initial topology graph reflects the inherent causal relationships of the system. Integrating domain expert rules to constrain and correct the initial causal topology graph can compensate for the limitations of the algorithm's learning. Combining professional experience in the operation and maintenance field corrects unreasonable causal relationships, improving the accuracy of the topology graph. During periods of anomaly, the node weights of the causal topology graph are dynamically adjusted according to the anomaly severity score, giving higher weights to nodes with higher anomaly severity. This highlights anomaly-related entities and causal relationships, allowing the topology structure to adapt to the system's abnormal state. This provides more targeted causal support for subsequent root cause localization, ensuring that root cause localization focuses on key anomaly nodes.

[0030] Example 5, step 4, the method for locating the root cause entity and abnormal propagation path through causal inference and counterfactual reasoning logic is as follows: Perform a random walk causal propagation algorithm on a dynamic causal topology to calculate the causal influence score for each node. A candidate root cause entity set is selected. For each entity in the candidate root cause entity set, a counterfactual reasoning operation is performed, that is, the entity's state value is replaced with the normal baseline value to simulate the system's operating state. After the counterfactual reasoning operation, the total decrease in the anomalousness score of all anomalous entities is calculated. The entity with the largest total decrease is selected as the final root cause entity, and the corresponding anomalous propagation path is traced using the following formula: ; It is the causal influence score of node i. The higher the score, the greater the likelihood that the node is the root cause. It is a set of abnormal nodes, containing all operation and maintenance entity nodes whose abnormality scores exceed a preset threshold; It is the causal path influence value from the abnormal node j to the candidate node i, which is determined by the path length, the weight of each edge on the path, and the causal strength.

[0031] By employing the above technical solution, a random walk causal propagation algorithm is first executed on a dynamic causal topology to calculate the causal impact score of each node. By simulating the propagation process of anomalies in the causal network, the influence of each node on the abnormal event is quantified, and a set of candidate root cause entities with high causal impact scores is selected, narrowing the scope of root cause investigation. Counterfactual reasoning is then performed on each entity in the candidate root cause entity set, replacing the candidate entity's state value with the normal baseline value. This simulates the system's operating state after the entity returns to normal, allowing for a direct assessment of whether the entity is the root cause of the anomaly. The total decrease in the anomaly score of all abnormal entities after counterfactual reasoning is statistically analyzed, and the candidate entity with the largest total decrease is identified as the final root cause entity. This logic accurately verifies the causal relationship between candidate entities and abnormal events, avoiding misjudging related nodes as root causes. Tracing the anomaly propagation path through the causal impact score formula clearly demonstrates how the root cause entity influences other nodes through causal relationships, forming an anomaly propagation chain, providing clear path guidance for fault repair.

[0032] Example 6: The method for predicting the evolutionary state of the operation and maintenance system based on a spatiotemporal sequence graph neural network in step 5 is as follows: The unified data view is constructed as a dynamic graph structure, where nodes represent operation and maintenance entities and edges represent relationships between entities. The dynamic graph structure is input into the spatiotemporal sequence graph neural network for training to learn the spatiotemporal evolution law of the operation and maintenance system. Based on the trained model, the probability of failure and the specific time of occurrence within a preset time window are predicted.

[0033] By adopting the above technical solution, the unified data view is first constructed as a dynamic graph structure, with operation and maintenance entities as nodes and the relationships between entities as edges. This structure accurately reflects the structural characteristics and operational status of the operation and maintenance system, transforming fragmented data into structured graph data, facilitating the capture of spatiotemporal relationships between entities. The dynamic graph structure is then input into a spatiotemporal sequence graph neural network for training. This network can simultaneously capture the spatial relationships within the graph structure and the temporal evolution patterns in the time-series data, fully learning the spatiotemporal evolution characteristics of the operation and maintenance system and grasping the potential patterns of fault occurrence. Based on the trained model, the probability and specific timing of fault occurrence within a preset time window can be predicted, enabling early identification of potential faults. This breaks the traditional passive response mode of operation and maintenance, providing time support for subsequent automated operation and maintenance decisions, and achieving early intervention and prevention of faults.

[0034] Example 7, the method for matching and executing automated operation and maintenance decisions in step 5 is as follows: A strategy library containing various automated operation and maintenance strategies is constructed, and the strategy library is associated and bound with the anomaly propagation path and root cause entity; based on the located root cause entity and anomaly propagation path, the corresponding automated operation and maintenance strategy is matched from the strategy library; and the matched automated operation and maintenance strategy is executed within the time window according to priority.

[0035] The core of this technical solution is to achieve automatic fault self-healing, reduce manual operation and maintenance costs, and improve fault handling efficiency. First, a strategy library containing various automated operation and maintenance strategies is constructed. These strategies are associated and bound to anomaly propagation paths and root cause entities, enabling the matching of corresponding handling strategies based on different fault scenarios, ensuring the strategies' relevance and effectiveness. Based on the located root cause entity and anomaly propagation path, the corresponding automated operation and maintenance strategy is matched from the strategy library, accurately matching the fault's root cause and propagation characteristics, avoiding repair failures due to strategy mismatch. The matched automated operation and maintenance strategies are executed within the predicted time window according to priority, prioritizing faults with a wide impact and high urgency, ensuring efficient fault repair. Simultaneously, the strategy is deployed in advance using the predicted time window, enabling timely fault self-healing and reducing the impact of faults on the operation and maintenance system.

[0036] Example 8 further includes step 6, constructing an operation and maintenance knowledge graph from the fault occurrence probability, root cause entity, anomaly propagation path, and automated operation and maintenance decision results, wherein entities include fault phenomena, root cause entities, and operation and maintenance strategies, and relationships include causal relationships and triggering relationships; performing embedding representation learning on the operation and maintenance knowledge graph to generate vector representations of entities and relationships; and periodically fine-tuning and optimizing the model and reasoning logic described in S2 to S5 based on the vector representations.

[0037] By adopting the above technical solutions, a closed-loop optimization mechanism for operation and maintenance (O&M) is constructed to achieve the accumulation of O&M experience and continuous improvement of model performance. An O&M knowledge graph is built by constructing fault occurrence probabilities, root cause entities, anomaly propagation paths, and automated O&M decision results. This clarifies the causal and triggering relationships between entities such as fault phenomena, root cause entities, and O&M strategies, enabling the systematic accumulation of various experience data during the O&M process and forming reusable knowledge resources. Embedded representation learning is applied to the O&M knowledge graph to generate vector representations of entities and relationships, transforming structured knowledge into computable vectors for easy model reading and utilization. Based on these vector representations, the model and inference logic in steps 2 to 5 are periodically fine-tuned and optimized. This integrates the accumulated O&M experience into model training, enabling the model to adapt to long-term changes in the O&M system and continuously improve the accuracy and adaptability of anomaly detection, root cause localization, and fault prediction, forming a closed-loop link of O&M-accumulation-optimization-improvement.

[0038] Example 9: In step 1, the multi-source heterogeneous data includes time-series performance indicators, log text, call chain topology data, and network event data; the standardized association and fusion processing includes timestamp alignment, entity unique identifier association, and semantic terminology normalization of the data.

[0039] By adopting the above technical solutions, the multi-source heterogeneous data is clearly defined to include time-series performance indicators, log text, call chain topology data, and network event data, covering the core data types of the operation and maintenance system. This ensures the comprehensiveness of data collection and avoids operational judgment biases caused by missing data. Data is timestamped to ensure consistency in the time dimension for data from different sources and of different types, facilitating subsequent time-series and correlation analysis. Entity unique identifier association is performed to uniformly identify the same operation and maintenance entity involved in different data, achieving interconnectedness between data and facilitating the discovery of relationships and causal relationships between entities. Semantic terminology normalization is performed to unify the semantic expressions in different data, avoiding misunderstandings and analytical errors caused by inconsistent terminology, ensuring the semantic integrity and consistency of the data, and providing reliable data support for subsequent model processing and inference analysis.

[0040] Example 10: A big data-based intelligent operation and maintenance system for implementing a big data-based intelligent operation and maintenance method. The system includes a data acquisition and fusion module, a dynamic baseline generation and anomaly calculation module, a multimodal anomaly fusion and reasoning module, a causal root cause localization module, and a fault prediction and self-healing decision-making module. The data acquisition and fusion module is used to collect multi-source heterogeneous data within the operation and maintenance system, and to perform timestamp alignment, entity association and semantic normalization processing on the multi-source heterogeneous data to generate a unified data view. The dynamic baseline generation and anomaly calculation module generates an adaptive multidimensional baseline based on temporal decomposition and variational autoencoder, and calculates the anomaly score of each key performance indicator. The multimodal anomaly fusion and reasoning module is used to output globally consistent anomaly events through cross-modal feature fusion and graph structure reasoning; The causal root cause localization module is used to construct a dynamic causal topology and, together with causal inference and counterfactual reasoning, locate root cause entities and abnormal propagation paths. The fault prediction and self-healing decision module is used to predict the probability of fault occurrence and time window based on spatiotemporal sequence graph neural networks, and to match and execute automated operation and maintenance decisions.

[0041] By adopting the above technical solutions, the data acquisition and fusion module is responsible for the acquisition and standardization of multi-source heterogeneous data, generating a unified data view to provide basic data support for the entire system, which is a prerequisite for the work of all subsequent modules. The dynamic baseline generation and anomaly calculation module, based on temporal decomposition and variational autoencoders, generates adaptive multidimensional baselines and calculates anomaly scores, enabling real-time monitoring of the system's operating status and providing data support for anomaly event identification. The multimodal anomaly fusion and inference module, through cross-modal feature fusion and graph structure inference, outputs globally consistent anomaly events, accurately identifies system anomalies, and provides direction for root cause localization. The causal root cause localization module constructs a dynamic causal topology structure, combining causal inference and counterfactual reasoning to locate root cause entities and anomaly propagation paths, providing a clear basis for fault repair. The fault prediction and self-healing decision module predicts faults based on spatiotemporal sequence graph neural networks, matches and executes automated operation and maintenance strategies, achieving early fault prediction and automatic self-healing. Each module collaborates sequentially and performs its respective function, forming a complete system architecture, ensuring the efficient implementation of intelligent operation and maintenance methods, and improving the intelligence level and operational reliability of the operation and maintenance system.

[0042] The following specific embodiments illustrate the implementation principle of the present invention: The core e-commerce platform operation and maintenance system of a large Internet company carries hundreds of millions of daily visits and involves various operation and maintenance entities such as server clusters, database clusters, microservice application systems, network switching equipment and edge computing nodes. The load difference between peak business periods and daily periods is significant, and there is a complex architecture with multiple regional nodes operating in coordination. The requirements for the real-time performance, accuracy and self-healing capability of the operation and maintenance system are extremely high.

[0043] Specific implementation process Step 1: Unified collection, standardized association, and fusion of multi-source heterogeneous data; The data acquisition and fusion module is activated, synchronously collecting multi-source heterogeneous data from the core business links of the e-commerce platform. Specifically, it covers time-series performance indicators such as server CPU utilization, memory usage, and disk I / O throughput; unstructured text data such as application running logs and interface call error logs; service call chain topology data; and network event data such as network packet loss rate and latency fluctuations.

[0044] During the data collection process, data from different sources and in different formats undergoes standardized association and fusion processing: First, timestamp alignment is performed to ensure that all data remains consistent in the time dimension. For example, performance metrics collected from the server every 10 seconds are matched with the corresponding interface call logs. Next, unique entity identifiers are associated to unify the identifiers of the same operation and maintenance entity across different data sources. For example, the hardware serial number of a server node is used as a unified identifier to associate the corresponding performance metrics, log information, and call chain data. Finally, semantic terminology normalization is completed to unify the semantic expressions in different data. For example, expressions such as memory usage rate and memory occupancy ratio are standardized to memory utilization rate, eliminating semantic ambiguity.

[0045] Through the above processing, a unified data view with consistency in the time dimension and completeness in the semantic dimension is generated, providing standardized data support for subsequent operation and maintenance.

[0046] Step 2: Dynamic adaptive multidimensional baseline generation and anomaly score calculation; The dynamic baseline generation and anomaly calculation module is based on time series analysis and deep generative learning models. It performs dynamic adaptive multidimensional baseline generation and anomaly score calculation for each key performance indicator in the unified data view.

[0047] Taking the order processing interface response time of a core e-commerce platform as a key performance indicator, we first perform seasonal trend decomposition on its time series data, using the STL decomposition algorithm to separate the periodic component, trend component, and residual component. The e-commerce platform's business exhibits a clear diurnal cycle: high traffic and fluctuating response time during the day, and low traffic and relatively stable response time at night. The periodic component accurately captures this periodic pattern, the trend component reflects the long-term trend of response time as the business develops, and the residual component eliminates the interference of periodicity and trend, focusing on real abnormal fluctuations.

[0048] Subsequently, a variational autoencoder network is constructed. Time-series data of order processing interface response times within a preset time window over the past 24 hours are selected and input into the variational autoencoder network for latent feature extraction and reconstruction, generating a reconstruction error index. Combining the real-time distribution statistical characteristics of the residual components, the upper and lower baseline thresholds are dynamically adjusted. Specifically, based on the mean and standard deviation of the residual components, the upper limit Ut and lower limit Lt of the dynamic baseline threshold are determined, enabling the baseline to adapt to dynamic changes in business load.

[0049] Meanwhile, the variational autoencoder network parameters are updated every 15 minutes using an online gradient descent strategy to adapt to conceptual shifts in the operation and maintenance system in a timely manner. For example, when a new promotional activity causes a sudden change in load, the model parameters are updated quickly to ensure that the baseline is always consistent with the system's operating status.

[0050] The abnormality score is calculated using a formula. ; ;in, The value is set to 0.6 to balance the contributions of numerical anomalies and reconstruction error anomalies; xt is the measured value of the order processing interface response time at the current moment. For outlier scores calculated based on the degree of numerical deviation, when Exceeding or below hour, Take the corresponding value; otherwise, return 0. The anomaly score is calculated based on the reconstruction error of the variational autoencoder and is positively correlated with the reconstruction error of the input time window.

[0051] At a certain moment, the measured response time of the order processing interface was 800 milliseconds, while the current dynamic baseline threshold upper limit was... 600 milliseconds, lower limit When it is 200 milliseconds, =max(0,(800-600) / (600-200))=0.5; If the variational autoencoder reconstruction error corresponds to If it is 0.4, then at this time =0.6×0.5+0.4×0.4=0.46, this score indicates that there is an anomaly in the interface response time, and the degree of anomaly is 0.46.

[0052] Step 3: Cross-modal feature fusion and graph structure reasoning generate globally consistent anomalous events; The multimodal anomaly fusion inference module decomposes multi-source heterogeneous data into time-series index data, unstructured text data, and topological structure data according to data type, and uses corresponding feature extraction models to perform deep feature mining for each.

[0053] For time-series metrics data, a time-series attention network is used to focus on key time-node features in metrics such as order processing interface response time and server CPU utilization, for example, capturing the sudden changes in metrics during peak business periods; for unstructured text data, a pre-trained language model is used to extract semantic information from application runtime logs and error logs, for example, identifying key semantics such as database connection timeouts and service call failures in the logs; for topological data, a graph embedding model is used to transform the service call chain topology data into computable feature vectors, clearly presenting the relationships between services.

[0054] A cross-modal attention alignment mechanism is used to fuse the multimodal features extracted by the three models to generate a fused feature vector. This process highlights key features related to anomalies while weakening irrelevant noise. Based on the fused feature vector, the anomaly confidence of globally consistent anomalies is calculated. When the anomaly confidence exceeds a preset threshold of 0.7, a globally consistent anomaly is determined to exist.

[0055] In this implementation case, the order processing interface response time was abnormal, the server CPU utilization exceeded the normal range, and a large number of database connection timeout semantic information appeared in the logs. The anomaly confidence score calculated after cross-modal feature fusion was 0.82, which exceeded the threshold. Therefore, it was determined that there was a globally consistent anomaly event in the core business link of the e-commerce platform, and the anomaly type was a chain response timeout caused by database connection anomaly.

[0056] Step 4: Construct a dynamic causal topology and locate the root cause entity and the abnormal propagation path; The causal root cause localization module first constructs an initial causal topology graph based on the constrained causal structure learning algorithm during the normal operation of the system. It analyzes the relationships between entities such as servers, databases, and microservice applications under normal conditions and explores potential causal logic. For example, under normal conditions, an increase in server CPU utilization will lead to a decrease in the processing capacity of microservice applications, which in turn will affect the response time of the order processing interface.

[0057] The initial causal topology graph is constrained and corrected using rules from domain experts. For example, based on professional experience in operations and maintenance, it is clearly stated that database connection anomalies will directly lead to service call failures, thus correcting unreasonable causal relationships in the initial topology graph. During the period when an anomaly occurs, the node weights in the causal topology graph are dynamically adjusted according to the anomaly severity scores of each operations and maintenance entity. Nodes with high anomaly severity, such as order processing interfaces and database servers, receive higher weights, generating a dynamic causal topology structure.

[0058] Perform a random walk causal propagation algorithm on a dynamic causal topology to calculate the causal influence score for each node. The candidate root cause entity set is selected through a formula. ; Calculate the score, where A is the set of abnormal nodes, including server nodes, database nodes, interface nodes, etc. whose abnormality scores exceed a preset threshold of 0.5; The causal path influence value from the abnormal node j to the candidate node i is determined by the path length, edge weight, and causal strength.

[0059] The calculation results show the causal impact score of the database server node. The value was 0.92, significantly higher than other nodes, and it was included in the candidate root cause entity set. Counterfactual reasoning was performed on the database server node in the candidate root cause entity set, replacing the node's state value with the normal baseline value to simulate the system's operating state after recovery.

[0060] After counterfactual reasoning, the anomaly score of the order processing interface response time decreased from 0.46 to 0.12, and the anomaly score of the server CPU utilization decreased from 0.58 to 0.15. The total decrease in the anomaly score of all abnormal entities reached 0.77, which is the maximum among all candidate entities. Therefore, the database server node was determined to be the final root cause entity.

[0061] By tracing the anomaly propagation path using the causal impact scoring formula, the anomaly propagation path is clearly defined as follows: database server connection anomaly - microservice application database call timeout - order processing interface response time extended - server CPU utilization increases due to continuous waiting for requests, clearly showing the propagation process of the fault from the root cause to other nodes.

[0062] Step 5: Fault prediction and automated operation and maintenance decision execution; The fault prediction and self-healing decision-making module constructs a unified data view as a dynamic graph structure, with servers, databases, microservice applications, interfaces, etc. as nodes and the calling and association relationships between entities as edges, to truly reflect the structural characteristics and operating status of the e-commerce platform's operation and maintenance system.

[0063] The dynamic graph structure is input into a spatiotemporal sequence graph neural network for training. This network simultaneously captures the spatial relationships within the graph structure and the temporal evolution patterns in the time-series data, learning the spatiotemporal evolution characteristics of the operation and maintenance system. After training, the probability of a failure occurring and its specific time point within the next hour are predicted. According to the model prediction, if the database server connection anomaly is not handled, the probability of the failure worsening within the next 30 minutes is 92%, and it may also trigger more service node anomalies.

[0064] A strategy library containing various automated operation and maintenance strategies is constructed. The strategy library is associated and bound with the anomaly propagation path and root cause entity. For the root cause of "database connection anomaly" and its corresponding propagation path, the preset automated operation and maintenance strategies include: database connection pool expansion strategy, database connection timeout parameter optimization strategy, and database service restart strategy.

[0065] Based on the identified root cause entity and the abnormal propagation path, the highest priority database connection pool expansion strategy is matched from the strategy library. The strategy is executed within the predicted time window according to the priority order, immediately expanding the database server connection pool resources by 20%, and at the same time triggering connection timeout parameter optimization, adjusting the connection timeout time from 5 seconds to 3 seconds.

[0066] Step 6: Construction of the operation and maintenance knowledge graph and optimization of the model closed loop; The operation and maintenance knowledge graph and closed-loop optimization module constructs the operation and maintenance knowledge graph based on the probability of occurrence of this fault, the root cause entity, the anomaly propagation path, and the results of automated operation and maintenance decisions. The entities include database connection anomaly fault phenomena, database server root cause entities, connection pool expansion strategies, connection timeout parameter optimization strategies, etc. The relationships include the causal relationship between the fault phenomena and the root cause entity, and the triggering relationship between the root cause entity and the operation and maintenance strategy.

[0067] Embedded representation learning is performed on the operations and maintenance knowledge graph to generate vector representations of entities and relationships, transforming structured operations and maintenance experience into computable vectors. Based on these vector representations, the models and inference logic in steps two through five are fine-tuned and optimized periodically (every 7 days). The current operations and maintenance experience is integrated into the model training, enabling models such as variational autoencoder networks and spatiotemporal sequence graph neural networks to better adapt to the characteristics of business scenarios such as database server connection anomalies, continuously improving the accuracy and adaptability of anomaly detection, root cause localization, and fault prediction.

[0068] Through the implementation of this case study, the operational efficiency of the core e-commerce platform of this large internet enterprise has been significantly improved: the false alarm rate of anomaly detection has decreased from 28% before implementation to 5%, and the false negative rate has decreased from 12% to 2%; the average time for root cause location of faults has been shortened from 2 hours to 15 minutes; the accuracy of fault prediction has reached 90%, enabling the prediction of fault risks 30 minutes in advance; the success rate of automated operation and maintenance strategy execution has reached 98%, successfully avoiding 3 potential business interruption events, ensuring the stable operation of hundreds of millions of daily visits, while reducing the cost of manual operation and maintenance intervention by 70%, forming a complete intelligent operation and maintenance closed loop.

[0069] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A big data-based intelligent operation and maintenance method, characterized in that: Includes the following steps: Step 1: Collect, standardize, associate, and merge multi-source heterogeneous data obtained from the operation and maintenance system to generate a unified data view with consistency in the time dimension and completeness in the semantic dimension; Step 2: Based on time series analysis and deep generative learning models, dynamically generate adaptive multidimensional baselines for each key performance indicator in the unified data view, and quantify the anomaly score of the real-time status of the indicator relative to the baseline. Step 3: Using cross-modal feature fusion and graph structure reasoning techniques, perform joint modeling and aggregation analysis on anomaly scores and deep semantic features of multi-source heterogeneous data, and output globally consistent anomaly events; Step 4: Construct a dynamic topology structure that represents the causal relationships between entities within the operation and maintenance system. Through causal inference and counterfactual reasoning logic, locate the root cause entity and the anomaly propagation path from globally consistent anomaly events. Step 5: Based on the spatiotemporal sequence graph neural network, predict and learn the evolution state of the operation and maintenance system, output the probability of failure and time window, and match and execute automated operation and maintenance decisions according to the root cause entity and the anomaly propagation path. 2.The big data-based intelligent operation and maintenance method according to claim 1, characterized in that: The method for dynamically generating adaptive multidimensional baselines in step 2 is as follows: Seasonal trend decomposition was performed on the time series data of key performance indicators to separate the periodic component, trend component and residual component. A variational autoencoder network is constructed to extract and reconstruct latent features from time series data within a preset time window, and a reconstruction error index is generated. The baseline upper and lower limit thresholds are adaptively adjusted based on the real-time distribution statistical characteristics of the residual components. The variational autoencoder network parameters are continuously updated using an online gradient descent strategy to adapt to the conceptual drift of the operation and maintenance system and achieve continuous dynamic iteration of the baseline. The abnormality score is calculated using the following formula: ； ； in It is the anomaly score at time t, and its value ranges from 1 to 2. A larger value indicates a higher degree of abnormality. It is a preset weight balancing coefficient with a value range of [0,1], used to balance the contribution of numerical anomalies and reconstruction error anomalies; It is an anomaly score calculated based on the degree of numerical deviation; The measured values ​​of key performance indicators at time t. This is the upper limit of the dynamic baseline threshold. This is the lower limit of the dynamic baseline threshold; It is anomaly score calculated based on the reconstruction error of variational autoencoder. 3.The big data-based intelligent operation and maintenance method according to claim 2, characterized in that: The method for cross-modal feature fusion and graph structure reasoning in step 3 is as follows: Multi-source heterogeneous data is decomposed into time-series index data, unstructured text data, and topological data according to data type. Features are extracted from the three types of data using a time-series attention network, a pre-trained language model, and a graph embedding model, respectively. The extracted multi-modal features are fused through a cross-modal attention alignment mechanism to generate a fused feature vector. Based on the fused feature vector, the anomaly confidence of globally consistent abnormal events is calculated. 4.The intelligent operation and maintenance method based on big data according to claim 3, characterized in that: The method for constructing the dynamic topology representing the causal relationships between entities within the operation and maintenance system in step 4 is as follows: During normal operation, an initial causal topology graph is constructed based on a constrained causal structure learning algorithm, and the initial causal topology graph is constrained and corrected by incorporating domain expert rules. During the period when an anomaly occurs, the node weights of the causal topology graph are dynamically adjusted according to the anomaly severity score to generate a dynamic causal topology structure. 5.The big data-based intelligent operation and maintenance method according to claim 4, characterized in that: In step 4, the method for locating the root cause entity and the abnormal propagation path through causal inference and counterfactual reasoning is as follows: Perform a random walk causal propagation algorithm on a dynamic causal topology to calculate the causal influence score for each node. The candidate root cause entity set is selected; for each entity in the candidate root cause entity set, the counterfactual reasoning operation is performed, that is, the state value of the entity is replaced with the normal baseline value to simulate the system operation state. After statistical counterfactual reasoning, the total decrease in the anomalousness score of all anomalous entities is calculated. The entity that maximizes this total decrease is selected as the root cause entity, and the corresponding anomalous propagation path is traced using the following formula: ； is the causal impact score of node i, the higher the score, the more likely the node is to be the root cause; is an abnormal node set, containing all operation and maintenance entity nodes whose abnormality degree scores exceed a preset threshold; is the causal path influence value from abnormal node j to candidate node i, which is determined by path length, weights of edges on the path, and causal strength.

6. The intelligent operation and maintenance method based on big data according to claim 5, characterized in that: The method for predicting the evolutionary state of the operation and maintenance system based on the spatiotemporal sequence graph neural network in step 5 is as follows: The unified data view is constructed as a dynamic graph structure, where nodes represent operation and maintenance entities and edges represent relationships between entities. The dynamic graph structure is input into the spatiotemporal sequence graph neural network for training to learn the spatiotemporal evolution law of the operation and maintenance system. Based on the trained model, the probability of failure and the specific time of occurrence within a preset time window are predicted. 7.The intelligent operation and maintenance method based on big data according to claim 6, characterized in that: The method for matching and executing automated operation and maintenance decisions in step 5 is as follows: A strategy library containing various automated operation and maintenance strategies is constructed, and the strategy library is associated and bound with the anomaly propagation path and root cause entity; based on the located root cause entity and anomaly propagation path, the corresponding automated operation and maintenance strategy is matched from the strategy library. Execute the matching automated operation and maintenance strategy within the time window according to priority. 8.The intelligent operation and maintenance method based on big data according to claim 7, characterized in that: The method also includes step 6, which constructs an operation and maintenance knowledge graph by combining the failure occurrence probability, root cause entity, abnormal propagation path and automated operation and maintenance decision results. The entity includes failure phenomenon, root cause entity and operation and maintenance strategy, and the relationship includes causal relationship and trigger relationship. The operation and maintenance knowledge graph is subjected to embedding representation learning to generate vector representations of entities and relationships. Based on the vector representation, the model and reasoning logic described in S2 to S5 are periodically fine-tuned and optimized.

9. The intelligent operation and maintenance method based on big data according to claim 8, characterized in that: The multi-source heterogeneous data in step 1 includes time-series performance metrics, log text, call chain topology data, and network event data; Standardized association and fusion processing includes timestamp alignment, entity unique identifier association, and semantic terminology normalization of data.

10. A big data-based intelligent operation and maintenance system, characterized in that: To implement the intelligent operation and maintenance method based on big data as described in claim 9, the system includes a data acquisition and fusion module, a dynamic baseline generation and anomaly calculation module, a multimodal anomaly fusion and reasoning module, a causal root cause localization module, and a fault prediction and self-healing decision-making module. The data acquisition and fusion module is used to collect multi-source heterogeneous data within the operation and maintenance system, and to perform timestamp alignment, entity association and semantic normalization processing on the multi-source heterogeneous data to generate a unified data view. The dynamic baseline generation and anomaly calculation module generates an adaptive multidimensional baseline based on temporal decomposition and variational autoencoder, and calculates the anomaly score of each key performance indicator. The multimodal anomaly fusion and reasoning module is used to output globally consistent anomaly events through cross-modal feature fusion and graph structure reasoning; The causal root cause localization module is used to construct a dynamic causal topology and, together with causal inference and counterfactual reasoning, locate root cause entities and abnormal propagation paths. The fault prediction and self-healing decision module is used to predict the probability of fault occurrence and time window based on spatiotemporal sequence graph neural networks, and to match and execute automated operation and maintenance decisions.

Images