Multi-source data fusion IT operation and maintenance fault root cause analysis method and system

Abstract

The invention discloses a multi-source data fusion IT operation and maintenance fault root cause analysis method, which comprises the following steps: data acquisition: acquiring data in real time from a plurality of data sources in an IT system, the data sources specifically being system logs, performance index data and network topology data, and the data sources being data of the IT system; the data acquisition is carried out according to a preset time interval or an event triggering mechanism; data preprocessing: preprocessing the multi-source data collected in the step S10, specifically comprising: a data cleaning step: removing noise, repeated data and incomplete data in the data; in the data conversion step, data in different formats are converted into data in a unified structured format, and in the data normalization step, a minimum-maximum normalization method is adopted. The multi-source data fusion IT operation and maintenance fault root cause analysis method and system aim to realize rapid and accurate identification of IT operation and maintenance fault root causes by fusing multiple data sources and utilizing an advanced data analysis technology.

Description

Technical Field

[0001] The present invention relates to the technical field of IT operation and maintenance, and specifically provides a method and system for analyzing the root cause of IT operation and maintenance failures through multi-source data fusion. Background Art

[0002] With the rapid progress of information technology, the IT systems of modern enterprises have become increasingly large and complex, encompassing a wide variety of hardware devices, intricate software systems, and widely connected network components. These systems are intertwined with each other to jointly support the daily operations and business development of enterprises. However, in such a complex environment, IT operation and maintenance face unprecedented challenges. The occurrence of failures is like an undercurrent, difficult to predict and avoid. Each failure may bring serious consequences such as system downtime and business interruption, thereby causing huge economic losses and reputational damage.

[0003] Currently, traditional methods for analyzing the root cause of failures mainly rely on the experience of operation and maintenance personnel and simple rule matching. This approach is inefficient and inaccurate when dealing with complex and ever-changing IT environments. At the same time, some existing analysis methods do not fully utilize the multi-source heterogeneous data in IT systems, such as system logs, performance metrics, network topologies, etc., resulting in the inability to comprehensively and deeply explore the root causes of failures. Summary of the Invention

[0004] The purpose of the present invention is to propose a method and system for analyzing the root cause of IT operation and maintenance failures through multi-source data fusion, which can quickly and accurately identify the root cause of IT operation and maintenance failures by integrating multiple data sources and using advanced data analysis techniques.

[0005] The technical solution for the present invention to solve the above technical problems is as follows: A method for analyzing the root cause of IT operation and maintenance failures through multi-source data fusion, comprising the following steps: S10. Data collection, real-time collecting data from multiple data sources in the IT system, where the data sources are specifically system logs, performance metric data, and network topology data, and the data collection is carried out according to a predetermined time interval or event trigger mechanism; S20. Data preprocessing, preprocessing the multi-source data collected in step S10, specifically including: a data cleaning step to remove noise, duplicate data, and incomplete data in the data; a data conversion step to convert data in different formats into a unified structured format; and a data normalization step to map the feature values of the data into the interval of [0, 1] using the min-max normalization method; S30. Multi-source data fusion, performing data fusion by combining feature fusion and model fusion; S40, Fault Pattern Recognition: Use the fused data to train a CNN fault pattern recognition model. Divide the historical fault data and normal data into a training set, a validation set, and a test set according to a certain ratio. Use the backpropagation algorithm to adjust the parameters of the CNN model so that the model achieves optimal performance on the validation set. In real-time monitoring, the currently collected and fused data is input into the trained CNN model to determine whether a fault exists and the type of fault. S50, Root Cause Analysis: After a fault is identified, root cause analysis is performed using a Bayesian network. Based on historical data and expert experience, a Bayesian network structure is constructed to determine the causal relationship between various factors. Multi-source data from the time of the current fault is input into the Bayesian network as evidence. By calculating the posterior probability of each node, the most likely root cause of the fault is identified. S60: Result feedback: The result of the fault root cause analysis and the corresponding solution suggestions are sent to the operation and maintenance personnel via email. At the same time, a message prompt will pop up on the operation and maintenance management platform to inform the detailed information of the fault and the solution suggestions.

[0006] On the basis of the above technical solution, the present invention can also be improved as follows.

[0007] Furthermore, in the S10, data collection, data collection tools are used to collect data. The data collection tools specifically include the rsyslog tool, Prometheus tool and network topology discovery tool of the Linux system, which collect data from the hardware facilities of the server and network equipment, as well as the operating system and application.

[0008] Furthermore, in the data cleaning in the data preprocessing in S20, for system logs, duplicate records and records with incorrect formats are removed; for performance indicator data, abnormal outliers are eliminated, and the data can be converted into JSON format.

[0009] Furthermore, in the S30, multi-source data fusion, the feature fusion is specifically to splice the key features in the system log, the features of the performance indicator data and the features of the network topology data to form a feature vector containing multi-source information, and the model fusion is specifically to use the decision tree model to model the system log data, use the support vector machine model to model the performance indicator data, use the neural network model to model the network topology data, and then use the voting method to fuse the prediction results of these three models.

[0010] Furthermore, in the S40, fault pattern recognition, the fault pattern recognition model can also be trained using a recurrent neural network deep learning model. In the S50, root cause analysis, before constructing the Bayesian network, it also includes a step of learning the Bayesian network, specifically including determining the nodes in the network, the status of the nodes, and the causal relationship between the nodes based on historical data and expert experience, and calculating the conditional probability table between each node.

[0011] Furthermore, in the said S50, root cause analysis, when using the Bayesian network to perform root cause analysis, it also includes the step of reasoning the Bayesian network, specifically including inputting the multi-source data when the current fault occurs as evidence into the Bayesian network, and using the Bayesian network's reasoning algorithm, including Bayes' theorem or Markov chain Monte Carlo method, to calculate the posterior probability of each node, so as to find the root cause that is most likely to cause the fault.

[0012] Furthermore, the S60, result feedback, also includes a step of providing feedback confirmation to the operation and maintenance personnel, specifically including the operation and maintenance personnel confirming the received fault root cause analysis results and solution suggestions, and making adjustments or supplements based on actual conditions.

[0013] A multi-source data fusion IT operation and maintenance fault root cause analysis system, including: Data acquisition module: collects data from multiple data sources in the IT system and adopts a distributed data acquisition framework to ensure efficient and stable data acquisition; Data preprocessing module: cleans, converts and normalizes the collected multi-source data, including data cleaning algorithms, data conversion functions and data normalization models to ensure data quality and consistency; Multi-source data fusion module: realizes feature fusion and model fusion, including feature fusion unit and model fusion unit to improve the accuracy of fault identification; Fault pattern recognition module: This module uses the fused data to train a convolutional neural network fault pattern recognition model and determines the presence and type of faults during real-time monitoring. This module uses a deep learning framework for model training and deployment. Root Cause Analysis Module: Once a fault is identified, it uses Bayesian networks to perform root cause analysis and determine the underlying cause of the fault. This module models and analyzes causal relationships based on Bayesian networks and can automatically optimize the Bayesian network structure using machine learning methods. Result feedback module: Feedback the results of fault root cause analysis and solution suggestions to operation and maintenance personnel, ensuring that operation and maintenance personnel obtain information in a timely manner through various means.

[0014] Compared with the prior art, the technical solution of this application has the following beneficial technical effects: The present invention can collect data in real time from multiple data sources such as system logs, performance metric data, and network topology data according to a predetermined time interval or event trigger mechanism, avoiding the cumbersome process of manual item-by-item inspection, greatly improving the efficiency and comprehensiveness of data collection. Through steps such as data cleaning, transformation, and normalization, noise and redundant information in the data are removed, and data in different formats are converted into a unified structured format, providing a high-quality data foundation for subsequent data fusion and model training. At the same time, advanced machine learning and probabilistic inference algorithms are adopted. In the fault mode recognition stage, the fused data is used to train a CNN fault mode recognition model, and the model parameters are adjusted through the backpropagation algorithm to make the model achieve the best performance on the validation set. This deep learning-based fault mode recognition method can automatically learn complex features and patterns in the data, improving the accuracy of fault recognition. In the root cause analysis stage, a Bayesian network is used to construct a network structure based on historical data and expert experience, determine the causal relationships between various factors, and input the multi-source data at the time of the current fault occurrence as evidence into the network. The root cause most likely to cause the fault is found by calculating the posterior probability. This root cause analysis method based on probabilistic inference can comprehensively consider various factors and their mutual relationships, improving the accuracy and reliability of root cause analysis. Furthermore, various types of data such as the text information of system logs, the time series data of performance metrics, and the structural information of network topologies are organically fused to generate a more comprehensive and richer fault feature representation. This multi-source data fusion method can fully exploit the associations and complementary information between different data sources, providing more comprehensive data support for subsequent fault mode recognition and root cause analysis. At the same time, by jointly using the CNN model and the Bayesian network, the processing ability for complex fault scenarios is further improved. Finally, the present invention sends the results of the fault root cause analysis and corresponding solution suggestions to the operation and maintenance personnel by email and pops up a message prompt on the operation and maintenance management platform. This timely and effective feedback mechanism can help the operation and maintenance personnel quickly understand the details of the fault and the handling suggestions, shorten the fault response time, and reduce the impact of the fault on the business. At the same time, by continuously accumulating fault cases and analysis results, it can also provide strong support for subsequent fault prevention and optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of a method for root cause analysis of IT operation and maintenance faults with multi-source data fusion according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0017] A method for root cause analysis of IT operation and maintenance failures with multi-source data fusion according to the present invention includes the following steps: S10. Data collection, collecting data in real time from multiple data sources in the IT system. The data sources are specifically system logs, performance metric data, and network topology data. The data collection is carried out according to a predetermined time interval or event trigger mechanism. S20. Data preprocessing, preprocessing the multi-source data collected in step S10, specifically including: a data cleaning step to remove noise, duplicate data, and incomplete data in the data; a data conversion step to convert data in different formats into a unified structured format; and a data normalization step to map the feature values of the data into the interval of [0, 1] using the min-max normalization method. S30. Multi-source data fusion, performing data fusion by combining feature fusion and model fusion. S40. Fault mode recognition, training a CNN fault mode recognition model using the fused data, dividing historical fault data and normal data into a training set, a validation set, and a test set according to a certain ratio, adjusting the parameters of the CNN model through the backpropagation algorithm to make the model achieve the best performance on the validation set. In real-time monitoring, input the currently collected and fused data into the trained CNN model to determine whether there is a fault and the type of the fault. S50. Root cause analysis, when a fault is identified, using a Bayesian network for root cause analysis, constructing a Bayesian network structure based on historical data and expert experience to determine the causal relationship between various factors, inputting the multi-source data at the time of the current fault occurrence as evidence into the Bayesian network, and finding the most likely root cause of the fault by calculating the posterior probability of each node. S60. Result feedback, sending the result of the root cause analysis of the fault and the corresponding solution suggestions to the operation and maintenance personnel by email, and at the same time popping up a message prompt on the operation and maintenance management platform to inform the operation and maintenance personnel of the detailed information of the fault and the processing suggestions.

[0018] In a preferred embodiment of the present invention, it can be further configured as follows: S10. In data collection, data collection tools are used to collect data. The data collection tools specifically include the rsyslog tool, Prometheus tool, and network topology discovery tool of the Linux system, which collect data from the hardware facilities of servers and network devices, as well as operating systems and application programs respectively. By explicitly specifying the rsyslog tool, Prometheus tool, and network topology discovery tool of the Linux system as data collection tools, efficient and automated collection of multi-source heterogeneous data is achieved. The rsyslog tool can capture the log information of servers and application programs in real time, including error logs, access logs, etc., providing rich text semantic information for fault analysis. The Prometheus tool focuses on the collection of performance metric data, such as CPU usage, memory occupancy, disk I / O, etc. Its high-frequency sampling ability can capture subtle changes in the system state. The network topology discovery tool constructs a network topology map by scanning network devices, parsing configuration files, etc., clarifying the connection relationships and data flows between devices. The collaborative use of these tools not only covers multiple levels such as hardware facilities, operating systems, and application programs in the IT system, but also integrates various data types such as logs, metrics, and topologies, significantly improving the comprehensiveness and efficiency of data collection, and providing a solid data foundation for subsequent root cause analysis of faults.

[0019] Through the configuration of flexible log rules, the rsyslog tool can filter and forward different levels of log events (such as DEBUG, INFO, WARN, ERROR), ensuring that key log information is not overwhelmed. At the same time, combined with the log parsing module, the collected logs are preprocessed to extract key fields such as timestamps, event types, and correlation IDs, facilitating subsequent feature extraction and correlation analysis. The Prometheus tool collects performance metric data in real time through the Exporter components deployed on target servers and network devices, and uses its powerful query language PromQL to aggregate, calculate, and visualize the metrics, helping operation and maintenance personnel quickly locate performance bottlenecks. For the network topology discovery tool, it adopts a combination of active detection and passive listening. On the one hand, it sends query requests to network devices through the SNMP protocol to obtain device information, interface status, etc.; on the other hand, it listens to network traffic, parses protocol packets such as ARP and ICMP, and dynamically updates the network topology map. In addition, this tool also supports version management of the topology map, which can record the change history of the topology structure, providing temporal context information for root cause analysis of faults. Through the integrated application of these tools, the present invention realizes all-round and multi-level data collection of the IT system, provides rich and accurate data support for root cause analysis of faults, and effectively overcomes the limitations of data collection in traditional methods.

[0020] In a preferred embodiment of the present invention, it can be further configured as follows: In S20, during data cleaning in data preprocessing, for system logs, duplicate records and records with format errors are removed; for performance metric data, abnormal outliers are excluded. At the same time, the data can be converted into JSON format. By defining specific rules for data cleaning and formulating cleaning strategies for system logs and performance metric data respectively, the data quality is effectively improved. For system logs, removing duplicate records and records with format errors can reduce data redundancy and avoid interference of invalid information on the analysis model; for performance metric data, excluding abnormal outliers can eliminate the negative impact of noise data on statistical analysis and model training. At the same time, converting the data into JSON format not only unifies the data representation method but also facilitates subsequent data storage, transmission, and parsing, significantly improving the flexibility and scalability of data processing.

[0021] For data cleaning of system logs, first, the hash algorithm is used to uniquely identify log records to quickly identify and remove duplicate records, avoiding redundant information occupying storage resources and analysis computing power. For records with format errors, regular expressions are used to match fixed fields of the logs (such as timestamp, log level, message content, etc.), and records that do not conform to the predefined format are marked or discarded. At the same time, a log parser (such as Logstash or Fluentd) is introduced to convert unstructured logs into structured data, extract key fields, and add metadata tags (such as source device, collection time, etc.) for subsequent feature engineering and correlation analysis. For performance metric data, the exclusion of abnormal outliers adopts a statistical method (such as the 3σ principle or box plot method), combined with the business scenario to set a reasonable threshold range to automatically identify and correct or delete abnormal values outside the range. In terms of data format conversion, data conversion tools (such as Pandas or Spark DataFrame) are used to convert the cleaned metric data into JSON format, defining a unified key-value pair structure to ensure the compatibility and interoperability of data between different systems. In addition, to further improve data quality, a data verification mechanism can be introduced to check the structural integrity and value range legality of the converted JSON data to ensure the accuracy and reliability of data cleaning and conversion.

[0022] In a preferred embodiment of the present invention, it can be further configured as follows: S30. In multi-source data fusion, the key features in system logs specifically include event type and timestamp, the features of performance metric data specifically include CPU usage rate and memory usage rate, and the features of network topology data specifically include node connection relationships. By explicitly extracting key features such as event type and timestamp in system logs, CPU usage rate and memory usage rate in performance metric data, and node connection relationships in network topology data, the effective integration of multi-source heterogeneous data is achieved. The event type and timestamp can reflect the temporal logic and abnormal patterns of system behavior, the CPU usage rate and memory usage rate directly reflect the load status of system resources, and the node connection relationship reveals the structural characteristics and data flow of the network topology. The fusion of these features can construct a more comprehensive fault feature vector, capture potential associations between different data sources, thereby significantly improving the accuracy and efficiency of root cause analysis, and effectively overcoming the analysis limitations caused by data silos and single features in traditional methods.

[0023] For the feature extraction of system logs, first, the log messages are segmented and part-of-speech tagged through natural language processing (NLP) technology to identify key event types (such as "service startup failure", "database connection timeout", etc.), and a temporal event chain is constructed in combination with the timestamp to analyze the causal relationship and temporal dependence between events. For example, through association rule mining algorithms (such as Apriori or FP-Growth), frequently co-occurring event patterns are discovered to identify potential fault propagation paths. For performance metric data, in addition to CPU usage rate and memory usage rate, key metrics such as disk I / O and network bandwidth can also be extended and extracted, and time series analysis techniques (such as sliding window statistics, exponential smoothing) are used to perform trend prediction and anomaly detection on the metrics to identify resource bottlenecks and performance degradation points. For example, an LSTM neural network is used to model the CPU usage rate to predict future load trends and discover overload risks in advance. For network topology data, the extraction of node connection relationships is not limited to physical connections, and a logical topology map can also be constructed in combination with traffic monitoring data to identify key paths and bottleneck nodes. For example, the importance score of nodes is calculated through the PageRank algorithm to locate the core devices that have the greatest impact on network stability. In addition, to enhance the relevance between features, a graph neural network (GNN) can be introduced to combine node connection relationships with performance metric data to construct a dynamic topology-aware feature representation. For example, the CPU usage rate is used as a node attribute to be embedded in the topology graph, and the state information of neighbor nodes is aggregated through the message passing mechanism of the GNN to generate a feature vector that combines topology structure and performance metrics. Through these refinement measures, the present invention realizes the deep mining and effective fusion of multi-source data features, provides richer and more relevant feature inputs for fault root cause analysis, and significantly improves the accuracy and reliability of the analysis.

[0024] In a preferred embodiment of the present invention, it can be further configured as follows: S30. In multi-source data fusion, feature fusion specifically refers to splicing the key features in system logs, the features of performance metric data, and the features of network topology data to form a feature vector containing multi-source information. Model fusion specifically refers to using a decision tree model to model system log data, using a support vector machine model to model performance metric data, and using a neural network model to model network topology data. Then, the prediction results of these three models are fused using the voting method. By feature fusion, the key features of system logs, performance metrics, and network topology data are spliced into a unified feature vector, breaking data silos and achieving the organic integration of multi-source information. At the same time, in model fusion, a decision tree, a support vector machine, and a neural network are used to model log, metric, and topology data respectively, and the prediction results are fused based on the voting method, giving full play to the advantages of different models in processing different types of data, significantly improving the accuracy and robustness of root cause analysis. This multi-model collaborative approach can effectively handle the complex and changeable fault modes of IT systems, overcome the analysis bias caused by a single model in traditional methods, and provide a more comprehensive and reliable solution for fault root cause location.

[0025] In the feature fusion stage, the event types and timestamps in the system logs are first subjected to one-hot encoding or embedding representation to convert the text features into numerical vectors; the performance metric data (such as CPU usage rate and memory usage rate) is normalized to eliminate the dimensional differences; the node connection relationships in the network topology data are represented using an adjacency matrix or graph embedding techniques (such as Node2Vec) to retain the topological structure information. Subsequently, these three types of feature vectors are concatenated by dimension to form a comprehensive feature vector containing multi-source information. For example, if the log feature vector is 128-dimensional, the metric feature vector is 32-dimensional, and the topology feature vector is 64-dimensional, then the dimension of the concatenated feature vector is 224-dimensional. In the model fusion stage, the decision tree model constructs classification rules for the system log data, selects the optimal splitting attribute through information gain or Gini coefficient, and captures the logical associations between log events; the support vector machine model performs a high-dimensional space mapping on the performance metric data, uses a kernel function (such as the RBF kernel) to handle non-linear relationships, and identifies abnormal metric patterns; the neural network model (such as a multi-layer perceptron MLP or a graph convolutional network GCN) performs deep feature extraction on the network topology data and learns the complex dependencies between nodes. After the three models are independently trained, the soft voting method is used to fuse the prediction results: each model outputs the probability distribution of the fault category, and the probability distributions of all models are summed according to weights (which can be dynamically adjusted based on model performance), and the category with the highest probability is taken as the final prediction result. For example, if the decision tree predicts the probability of a certain fault to be 0.6, the support vector machine to be 0.7, and the neural network to be 0.8, and the weights are all 1 / 3, then the probability of this fault after fusion is (0.6 + 0.7 + 0.8) / 3 = 0.7, which is determined as the final fault category. In addition, to further improve the fusion effect, a meta-learning mechanism can be introduced, and a "meta-model" (such as XGBoost or LightGBM) is added before the fusion layer. Using the prediction results of the three basic models and the original feature vectors as inputs, it learns how to dynamically adjust the fusion weights. For example, when the performance metric data fluctuates violently, the weight of the support vector machine model is automatically increased; when the network topology structure changes significantly, the contribution of the neural network model is enhanced. Through this multi-level fusion strategy, the present invention can adaptively integrate the multi-modal information of multi-source data, significantly improving the accuracy and generalization ability of root cause analysis of faults.

[0026] In a preferred embodiment, the present invention can be further configured as follows: In S40, during fault mode recognition, the fault mode recognition model can also be trained using a recurrent neural network deep learning model. In S50, during root cause analysis, before constructing the Bayesian network, there is also a step of learning the Bayesian network, specifically including determining the nodes, node states, and causal relationships between nodes in the network according to historical data and expert experience, and calculating the conditional probability table between each node. By introducing a recurrent neural network (such as LSTM or GRU) for fault mode recognition, it can effectively handle the temporal dependence of data such as log sequences and metric time series, improving the ability to capture complex fault modes. In addition, adding a learning step before constructing the Bayesian network, automatically determining the network nodes, states, and causal relationships based on historical data and expert experience, and calculating the conditional probability table (CPT), not only reduces manual intervention, but also enables the network structure to dynamically adapt to system changes, significantly improving the accuracy and efficiency of root cause analysis, and overcoming the defects of model staticization and analysis subjectivity in traditional methods.

[0027] The fault mode recognition model adopts a bidirectional LSTM (BiLSTM) structure to simultaneously capture the temporal characteristics of fault precursors and subsequent impacts. The input data is a sequence of fused multi-source feature vectors (such as log event embeddings, metric time series values, topology node states). Through the gating mechanism (input gate, forget gate, output gate) of the LSTM unit, key information is dynamically screened to avoid the problem of gradient disappearance. For example, for a system log sequence, BiLSTM can identify the "database connection timeout" event following a "service startup failure" and associate it with the memory leak trend in performance metrics to form a complete fault mode feature representation. In the training phase, the cross-entropy loss function is used to optimize the model parameters, and overfitting is prevented by the early stopping method. In the Bayesian network learning step of root cause analysis, first, an initial network topology is constructed based on historical fault cases and expert knowledge. The nodes are defined as fault components (such as servers, network devices, application modules), and the node states are "normal" or "faulty". Through causal discovery algorithms (such as the PC algorithm or FCI algorithm), the conditional independence relationships between nodes are learned from the data to determine the direction of the directed edges. For example, if historical data shows that the probability of "application service unavailable" when the "database server fails" is significantly higher than other cases, a causal edge of "database server → application service" is established. The calculation of the conditional probability table (CPT) uses maximum likelihood estimation (MLE) or Bayesian estimation to smooth the probability distribution by combining prior knowledge and observed data. For example, for the node "network device", the conditional probability of its state "faulty" may depend on the state combinations of parent nodes such as "CPU overload" and "port error packet rate". To enhance the dynamic adaptability of the network, the network structure and CPT can be re-learned regularly (such as weekly or monthly) based on newly collected data, or the probability distribution can be adjusted in real-time through an online learning mechanism (such as incremental Bayesian update). In addition, to further improve the accuracy of root cause analysis, the evidence theory can be introduced to fuse the output of the Bayesian network with other diagnostic results (such as expert scores, monitoring alarms), quantify the uncertainty, and generate a comprehensive credibility report. For example, when the Bayesian network infers that the failure probability of a certain component is 70%, while the expert score indicates that the failure probability of this component is 80%, the final credibility is obtained as 85% through the evidence synthesis rule, providing a more reliable basis for operation and maintenance decisions. Through these refined measures, the present invention realizes the temporal intelligence of fault mode recognition and the dynamic adaptability of root cause analysis, significantly improving the accuracy and practicality of IT operation and maintenance root cause analysis.

[0028] In a preferred embodiment of the present invention, it can be further configured as follows: S50. In root cause analysis, when using a Bayesian network for root cause analysis, it further includes a step of reasoning about the Bayesian network, specifically including using the multi-source data at the time of the current fault occurrence as evidence and inputting it into the Bayesian network, and using the reasoning algorithms of the Bayesian network, including Bayes' theorem or Markov Chain Monte Carlo method, to calculate the posterior probability of each node, so as to find out the root cause most likely to cause the fault. By introducing the Bayesian network reasoning step, using the multi-source data at the time of the current fault occurrence as evidence and inputting it into the network, and using Bayes' theorem or Markov Chain Monte Carlo (MCMC) method to calculate the node posterior probability, the quantitative analysis and uncertainty modeling of root cause inference are realized. This method can comprehensively consider the combined effects of multiple fault factors, dynamically update the probability distribution of root cause hypotheses, and significantly improve the accuracy and reliability of root cause analysis. At the same time, the probability reasoning characteristics of the Bayesian network make the analysis results interpretable, and the operation and maintenance personnel can clearly understand the contribution degree of each factor to the fault, overcoming the defects of fuzzy root cause inference and lack of quantitative basis in traditional methods.

[0029] The inference steps of the Bayesian network first need to map the multi-source data of the current fault (such as abnormal events in system logs, records of performance metrics exceeding thresholds, and link interruption information in network topologies) to the evidence states of network nodes. For example, if the CPU usage rate of a certain server node exceeds the 90% threshold, the evidence probability of the "overloaded" state of this node is set to 1, and the probabilities of other states are set to 0. For uncertain evidence (such as the vague description of "performance degradation" in the log), fuzzy sets or probability distributions can be used to represent its evidence strength. The choice of inference algorithm depends on the network scale and complexity: for small networks, exact inference algorithms (such as variable elimination method or clique tree propagation algorithm) can be directly applied to calculate the posterior probability based on Bayes' theorem. For example, for the fault probability P(A|E) of node A, it is calculated through the formula P(A|E) = P(E|A)P(A) / P(E), where P(E|A) is the likelihood of evidence E when A fails, P(A) is the prior probability, and P(E) is the normalization constant. For large and complex networks, the Markov chain Monte Carlo (MCMC) method is used for approximate inference, generating samples through Gibbs sampling or Metropolis-Hastings algorithm to estimate the posterior probability distribution. For example, in Gibbs sampling, the states of other nodes are fixed in turn, and conditional sampling is performed on the target node. After multiple iterations, it converges to a stationary distribution, and the sample mean is the estimated value of the posterior probability. To enhance the robustness of inference, an evidence discount factor can be introduced to handle conflicting or unreliable evidence. For example, if a certain log evidence may be distorted due to acquisition delay, its evidence probability is multiplied by a discount coefficient (such as 0.8) to reduce its impact on the inference result. In addition, the visualization of inference results is crucial: the posterior probabilities of each node are displayed through a heat map, and the thickness of the directed edges represents the causal strength to generate a root cause analysis report. Further, to cope with the dynamically changing IT environment, an incremental Bayesian inference mechanism can be designed: when new evidence arrives, only the affected local network area is updated, rather than re-inferring globally. For example, through local conditional probability table (CPT) update and belief propagation algorithm, the root cause probability distribution is quickly adjusted to achieve real-time root cause tracking. Through these refined measures, the present invention realizes the efficiency, accuracy, and interpretability of Bayesian network inference, significantly improving the technical level of root cause analysis for IT operation and maintenance faults.

[0030] In a preferred embodiment, the present invention can be further configured as follows: S60. In the result feedback, it further includes a step of feedback confirmation for the operation and maintenance personnel, specifically including that the operation and maintenance personnel confirm according to the received root cause analysis result of the fault and the recommended solution, and adjust or supplement according to the actual situation. By introducing the feedback confirmation step of the operation and maintenance personnel, the operation and maintenance personnel can verify and adjust the root cause analysis result and the recommended solution based on on-site experience, effectively bridging the gap between the algorithm output and the operation and maintenance practice. The feedback confirmation step of the operation and maintenance personnel is implemented through a visual interaction interface. The interface displays the key conclusions of the fault root cause analysis (such as the most likely root cause node, probability distribution), the associated evidence chain (such as log fragments, metric trend charts, topological paths), and the recommended solution (such as restarting the service, replacing hardware, adjusting configuration parameters). The operation and maintenance personnel can confirm and adjust based on the following dimensions: Verification of root cause accuracy: The operation and maintenance personnel compare the on-site inspection records (such as the status of device indicators, manual command outputs) with the evidence chain in the analysis result. If evidence is found to be missing or contradictory (such as the analysis shows that a certain switch port is faulty, but the on-site inspection shows that the physical connection of the port is normal), it can be marked as "doubtful" and on-site data can be supplemented (such as uploading a port capture file). Feasibility assessment of the solution: The operation and maintenance personnel evaluate the execution risk of the recommended solution according to the current system status (such as maintenance window, business priority). For example, if it is recommended to restart the core database service, but it is currently the business peak period, the operation and maintenance personnel can select "postpone execution" and add a note "need to coordinate with the business team to confirm the maintenance time". Supplement operation and maintenance knowledge: For special situations not covered in the analysis (such as a specific fault mode of a certain old device), the operation and maintenance personnel can add custom rules or cases to the knowledge base for subsequent analysis models to learn. For example, input the operation and maintenance experience of "device model X is prone to memory leakage under a specific firmware version, and the firmware needs to be upgraded first". The feedback confirmation process is designed as a closed-loop iterative mechanism: After the operation and maintenance personnel submit the adjustment, the system automatically updates the analysis record and triggers a secondary verification process. If the adjustment involves a change in the root cause, the posterior probability of the relevant nodes is recalculated; if the adjustment only involves the solution, a versioned record is generated (such as "V1.0 original recommendation → V1.1 Operation and maintenance adjustment suggestions), in addition, the system parses the remarks of operation and maintenance personnel through natural language processing (NLP), extracts key terms (such as "firmware upgrade", "traffic bypass") and associates them with the knowledge graph to realize the explicitization of implicit operation and maintenance experience. To further improve the feedback efficiency, intelligent auxiliary tools can be introduced: for example, when the operation and maintenance personnel have doubts about a certain root cause node, the system automatically recommends similar cases (such as "The similarity between the historical failure case #20230915 and the current situation is 85%") for reference; when the solution involves complex operations, generate a step-by-step operation guide (such as "Step 1: Back up the configuration file → Step 2: Execute the firmware upgrade command → Step 3: Verify the service status") and embed risk warnings (such as "Check the license compatibility after the upgrade"). Through these refined measures, the present invention realizes the deep integration of the analysis results and operation and maintenance practices, significantly improving the practical value of root cause analysis of faults and the scientific nature of operation and maintenance decisions.

[0031] The present invention also provides an IT operation and maintenance fault root cause analysis system for multi-source data fusion, including: Data acquisition module: Collect data from multiple data sources of the IT system, and adopt a distributed data acquisition framework to ensure efficient and stable data acquisition; Data preprocessing module: Clean, transform and normalize the collected multi-source data, including data cleaning algorithms, data transformation functions and data normalization models to ensure the quality and consistency of the data; Multi-source data fusion module: Realize feature fusion and model fusion, including a feature fusion unit and a model fusion unit to improve the accuracy of fault identification; Fault mode recognition module: Use the fused data to train a convolutional neural network fault mode recognition model, and judge whether there is a fault and the type of the fault during real-time monitoring. This module uses a deep learning framework for model training and deployment; Root cause analysis module: When a fault is identified, use the Bayesian network for root cause analysis to determine the root cause of the fault. This module realizes the modeling and analysis of causal relationships based on the Bayesian network, and can also automatically optimize the Bayesian network structure using machine learning methods; Result feedback module: Feedback the results of the fault root cause analysis and solution suggestions to the operation and maintenance personnel to ensure that the operation and maintenance personnel can obtain information in a timely manner through various methods.

[0032] This system breaks data islands through a distributed data acquisition framework to achieve efficient integration of multi-source data; it uses data cleaning, transformation, and normalization processes to improve data quality, laying a reliable foundation for subsequent analysis. The multi-source data fusion module significantly enhances the accuracy of fault mode recognition through feature fusion and model fusion. The introduction of a convolutional neural network effectively captures spatial or temporal patterns in the data, improving the sensitivity of fault detection; a Bayesian network combines machine learning to optimize the structure, enabling dynamic causal modeling and precise root cause location. The result feedback module ensures the effective transmission of analysis results to operation and maintenance personnel through multi-channel notifications and visual reports, forming a closed loop of "analysis - feedback - optimization", significantly overcoming the defects of insufficient data utilization, single analysis model, and fragmented result application in traditional methods, and greatly improving the intelligence level and practicality of root cause analysis for IT operation and maintenance faults.

[0033] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0034] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A root cause analysis method for IT operation and maintenance failures with multi-source data fusion, characterized in that, The following steps are involved: S10, data collection, real-time data collection from multiple data sources in the IT system, specifically system logs, performance indicator data, and network topology data, and data collection is performed according to predetermined time intervals or event triggering mechanisms; S20, data preprocessing, preprocessing the multi-source data collected in step S10, specifically including: a data cleaning step, removing noise, duplicate data, and incomplete data in the data; a data conversion step, converting data in different formats into a unified structured format; a data normalization step, using a minimum-maximum normalization method to map the eigenvalues of the data to the interval [0,1]; S30, multi-source data fusion, using a combination of feature fusion and model fusion to perform data fusion; S40, Fault Pattern Recognition: Use the fused data to train a CNN fault pattern recognition model. Divide the historical fault data and normal data into a training set, a validation set, and a test set according to a certain ratio. Use the backpropagation algorithm to adjust the parameters of the CNN model so that the model achieves optimal performance on the validation set. In real-time monitoring, the currently collected and fused data is input into the trained CNN model to determine whether a fault exists and the type of fault. S50, Root Cause Analysis: After a fault is identified, root cause analysis is performed using a Bayesian network. Based on historical data and expert experience, a Bayesian network structure is constructed to determine the causal relationship between various factors. Multi-source data from the time of the current fault is input into the Bayesian network as evidence. By calculating the posterior probability of each node, the most likely root cause of the fault is identified. S60: Result feedback: The result of the fault root cause analysis and the corresponding solution suggestions are sent to the operation and maintenance personnel via email. At the same time, a message prompt will pop up on the operation and maintenance management platform to inform the detailed information of the fault and the solution suggestions.

2. The IT operation and maintenance fault root cause analysis method for multi-source data fusion according to claim 1, characterized in that, In the S10, data collection, data is collected using data collection tools, which specifically include the rsyslog tool, Prometheus tool and network topology discovery tool of the Linux system, which collect data from the hardware facilities of the server and network equipment, as well as the operating system and application programs.

3. A method for analyzing the root cause of IT operation and maintenance failures with multi-source data fusion according to claim 1, characterized in that, In the data cleaning in the data preprocessing S20, for system logs, duplicate records and records with incorrect formats are removed; for performance indicator data, abnormal outliers are removed, and the data can be converted into JSON format.

4. A root cause analysis method for IT operation and maintenance faults with multi-source data fusion according to claim 1, characterized in that, In the S30, multi-source data fusion, the key features in the system log specifically include event type and timestamp, the features of the performance indicator data specifically include CPU usage and memory usage, and the features of the network topology data specifically include node connection relationships.

5. A method for analyzing the root cause of IT operation and maintenance failures through multi-source data fusion according to claim 1, characterized in that, In S30, in multi-source data fusion, feature fusion specifically involves concatenating the key features in system logs, the features of performance metric data, and the features of network topology data to form a feature vector containing multi-source information. Model fusion specifically involves using a decision tree model to model system log data, using a support vector machine model to model performance metric data, using a neural network model to model network topology data, and then using the voting method to fuse the prediction results of these three models.

6. A root cause analysis method for IT operation and maintenance faults with multi-source data fusion according to claim 1, characterized in that, In S40, in fault mode recognition, the fault mode recognition model can also be trained using a recurrent neural network deep learning model. In S50, in root cause analysis, before constructing the Bayesian network, there is also a step of learning the Bayesian network, which specifically includes determining the nodes in the network, the states of the nodes, and the causal relationships between the nodes based on historical data and expert experience, and calculating the conditional probability tables between each pair of nodes.

7. A root cause analysis method for IT operation and maintenance faults with multi-source data fusion according to claim 1, characterized in that, In S50, in root cause analysis, when using the Bayesian network for root cause analysis, there is also a step of reasoning the Bayesian network, which specifically includes using the multi-source data at the time of the current fault occurrence as evidence and inputting it into the Bayesian network, and using the reasoning algorithms of the Bayesian network, including Bayes' theorem or Markov chain Monte Carlo method, to calculate the posterior probabilities of each node, so as to find out the root cause most likely to cause the fault.

8. A root cause analysis method for IT operation and maintenance faults with multi-source data fusion according to claim 1, characterized in that, In S60, in result feedback, there is also a step of feedback confirmation to the operation and maintenance personnel, which specifically includes the operation and maintenance personnel confirming according to the received root cause analysis results of the fault and the proposed solutions, and making adjustments or supplements according to the actual situation.

9. An IT operation and maintenance fault root cause analysis system for multi-source data fusion, characterized in that, Using the IT operation and maintenance fault root cause analysis method of multi-source data fusion according to any one of claims 1 to 8, including: Data acquisition module: Collect data from multiple data sources of the IT system, and use a distributed data acquisition framework to ensure efficient and stable data acquisition; Data preprocessing module: Clean, transform, and normalize the collected multi-source data, including data cleaning algorithms, data transformation functions, and data normalization models, to ensure the quality and consistency of the data; Multi-source data fusion module: Implement feature fusion and model fusion, including a feature fusion unit and a model fusion unit, to improve the accuracy of fault recognition; Fault mode recognition module: Use the fused data to train a convolutional neural network fault mode recognition model, and determine whether there is a fault and the type of the fault during real-time monitoring. This module uses a deep learning framework for model training and deployment; Root cause analysis module: When a fault is recognized, use the Bayesian network for root cause analysis to determine the root cause of the fault. This module realizes the modeling and analysis of causal relationships based on the Bayesian network, and can also automatically optimize the Bayesian network structure using machine learning methods; Result feedback module: Feedback the root cause analysis results of the fault and the proposed solutions to the operation and maintenance personnel, and ensure that the operation and maintenance personnel can obtain information in a timely manner through various means.

Images