Operation and maintenance fault self-healing system and method based on AI algorithm

Abstract

The invention discloses an operation and maintenance fault self-healing system and method based on an AI algorithm. The system comprises a data acquisition module, a data preprocessing module, an anomaly detection module, a root cause analysis module, a self-healing strategy generation module and an automatic execution module. The method comprises the following steps: S1, collecting user system data and preprocessing the collected data; s2, performing anomaly detection on the preprocessed data information, and performing root cause analysis on the detected abnormal data; and S3, generating an optimal self-healing strategy based on the fault root cause and the user system data, and then executing the optimal self-healing strategy. According to the method, the AI algorithm is used for automatically analyzing the monitoring data and automatically restarting the fault application, so that the operation and maintenance efficiency is greatly improved, the fault positioning efficiency is greatly improved, the fault recovery time is greatly shortened, and the system availability is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of software operation and maintenance technology, and specifically to an operation and maintenance fault self-healing system and method based on AI algorithm. Background Art

[0002] The increasing scale and complexity of various information systems presents significant challenges for operations and maintenance. Traditional operations and maintenance monitoring systems only collect and display data. When a system failure occurs, operations and maintenance personnel must rely on their experience to analyze the monitoring system data and restart related application services. Summary of the Invention

[0003] In order to solve the above problems, the present invention provides an operation and maintenance fault self-healing system based on AI algorithm, including a data acquisition module, a data preprocessing module, an anomaly detection module, a root cause analysis module, a self-healing strategy generation module, and an automated execution module; The data acquisition module is used to collect user system data; The data preprocessing module is used to preprocess the collected data; The anomaly detection module is used to detect anomalies in the processed data and issue fault warnings for anomalies; The root cause analysis module is used to analyze the detected abnormal data and locate the root cause of the fault; it uses GNN to build the system's service dependency topology map, takes the abnormal data as input, analyzes the association relationship between nodes through the GNN model, and locates possible fault nodes.

[0004] The self-healing strategy generation module is used to generate an optimal self-healing strategy based on the root cause of the fault and the user system status; The automatic execution module is used to automatically execute corresponding repair operations based on the generated optimal self-healing measurement.

[0005] Furthermore, the collected user system data specifically includes: system performance indicator data, business system log data, and call chain data; wherein, the system performance indicator data includes: CPU usage, memory usage, and network bandwidth; the business system log data includes: system logs and application logs.

[0006] Furthermore, the anomaly detection module includes an anomaly detection submodule and an anomaly threshold determination submodule; The anomaly detection submodule uses IForest to perform anomaly detection on the pre-processed log data, uses Holt-Winters to monitor the system operation status in real time, and detects whether there are abnormal calls in the call link through the call link topology structure and performance indicators; The abnormal threshold determination submodule assumes that the data obeys a normal distribution based on historical data, and calculates the mean and standard deviation through a slider window.

[0007] Furthermore, the self-healing strategy generation module uses the fault root cause and system state as the environment state and the repair operation as the action space to generate the optimal self-healing strategy by defining an appropriate reward function. The appropriate reward function includes minimizing fault repair time and maximizing system availability. In conjunction with a decision tree algorithm, predefined repair rules are applied to common fault scenarios. When the RL model fails to generate an effective strategy, the decision tree rules are used.

[0008] Furthermore, the corresponding repair operations in the automated execution module specifically include: restarting services, adjusting configuration parameters, and expanding resources.

[0009] Furthermore, it also includes a model optimization module for real-time optimization and updating of relevant detection models in anomaly detection.

[0010] A method for self-healing operation and maintenance faults based on an AI algorithm includes the following steps: S1. collecting user system data and preprocessing the collected data; S2. performing anomaly detection on the preprocessed data information and performing root cause analysis on the detected anomaly data; S3. generating an optimal self-healing strategy based on the root cause of the fault and the user system data and executing the strategy.

[0011] Furthermore, the abnormality detection of the pre-processed data information in step S2 specifically includes the following sub-steps: Perform anomaly detection on log data: S2I. Access preprocessed log data, extract timestamps, and calculate log length as a feature. S2II. Count the frequency of special characters in the log data and perform vectorization on the log text. S2III. Configure detection samples and the ratio of abnormal data. S2IV. Detect whether the log data contains anomalies, calculate the anomaly score, and generate abnormal log output. Detect abnormalities in system operation status; S21. Access pre-processed system performance indicator data; S22. Set seasonal cycle parameters and configure trend types; S23. Customize anomaly thresholds and detect anomalies in the system performance indicator data based on the residual standard deviation.

[0012] The present invention provides an operation and maintenance fault self-healing system and method based on AI algorithm, which has the following beneficial effects: The present invention uses AI algorithms to automatically analyze monitoring data and automatically restart faulty applications, greatly improving operation and maintenance efficiency, significantly increasing fault location efficiency, significantly reducing fault recovery time, and significantly improving system availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0014] Figure 1 A schematic diagram of the system structure provided by the present invention; Figure 2 The present invention provides a flow chart of the method. DETAILED DESCRIPTION

[0015] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0016] The following describes the implementation method of the present invention in detail with reference to the accompanying drawings. The description only includes some embodiments, not all embodiments. For the purpose of clarity, representations and descriptions that are not related to the present invention are omitted in the drawings and descriptions.

[0017] In order to have a clearer understanding of the technical features, purposes and beneficial effects of the present invention, the technical solutions of the present invention are now described in detail below. Obviously, the implementation cases described are part of the embodiments of the present invention, not all of them, and should not be understood as limiting the scope of the implementation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, the present invention provides an operation and maintenance fault self-healing system based on AI algorithm, including a data acquisition module, a data preprocessing module, an anomaly detection module, a root cause analysis module, a self-healing strategy generation module, and an automated execution module.

[0019] The data collection module is used to collect user system data: system performance indicator data (CPU usage, memory usage, network bandwidth), business system log data (system log, application log), and call chain data.

[0020] Collection agents deployed on each node collect system performance metrics, log data, and call chain data in real time. System performance metrics are collected using open-source monitoring tools such as Prometheus; log data is collected using the Filebeat log collection tool; and call chain data is collected using the SkyWalking distributed tracing system. The collected data is transmitted to the data preprocessing module via the Kafka message queue.

[0021] The data preprocessing module is used to preprocess the collected data. It cleans the collected multi-source data to remove duplicate data, missing values, and outliers; normalizes the data to unify data of different ranges to the same scale; and uses feature extraction algorithms (such as principal component analysis and wavelet transform) to extract key features of the data to reduce data dimensionality and improve the efficiency of subsequent analysis.

[0022] The anomaly detection module is used to detect anomalies in the processed data and issue fault warnings for anomalies. It includes an anomaly detection submodule and an anomaly threshold determination submodule.

[0023] The anomaly detection submodule uses IForest to perform anomaly detection on the preprocessed log data, uses Holt-Winters to monitor the system operation status in real time, and detects whether there are abnormal calls in the call link through the call link topology structure and performance indicators.

[0024] In the abnormal threshold determination submodule, based on historical data, it is assumed that the data follows a normal distribution and the mean and standard deviation are calculated through a slider window.

[0025] The root cause analysis module analyzes detected anomaly data and locates the root cause of the fault. It uses a GNN (graph neural network) to construct a service dependency topology for the system. Using the anomaly data as input, the GNN model analyzes the relationships between nodes and locates possible faulty nodes. Furthermore, a Bayesian network is used to model the probabilistic relationship between fault symptoms and causes. This is combined with the anomaly data to update the probability distribution and further determine the root cause of the fault.

[0026] The self-healing strategy generation module generates an optimal self-healing strategy based on the root cause of the fault and the user's system status. This module uses the root cause and system status as the environment state, and the repair actions as the action space. The optimal self-healing strategy is generated by defining a suitable reward function. Suitable reward functions include minimizing fault repair time and maximizing system availability. In conjunction with a decision tree algorithm, predefined repair rules are defined for common fault scenarios. When the RL model fails to generate an effective strategy, the decision tree rules are used.

[0027] The automated execution module automatically executes the corresponding remediation actions (restarting services, adjusting configuration parameters, and scaling resources) based on the generated optimal self-healing measurements. Based on the generated self-healing strategy, it calls the automated tools of the container management platform (Kubernetes Operator) to automatically execute the remediation actions. During the execution process, the execution status of the operations and system feedback are monitored in real time to ensure the smooth progress of the remediation operations.

[0028] The system also includes a model optimization module for real-time optimization and updating of relevant detection models used in anomaly detection. Using online learning methods, it leverages newly acquired data to incrementally update the anomaly detection model, root cause analysis model, and self-healing strategy generation model to adapt to dynamic system changes. Furthermore, federated learning methods enable data sharing and model collaboration with similar systems, addressing the shortage of rare fault samples and further improving model performance.

[0029] An AI-based self-healing method for operation and maintenance faults, such as Figure 2 As shown, the following steps are included: S1. Collect user system data and preprocess the collected data.

[0030] S2. Perform anomaly detection on preprocessed data and conduct root cause analysis on detected anomalies. The IForest algorithm is used to detect anomalies in preprocessed log data. The Holt-Winters time series prediction algorithm is used to monitor the system's operating status in real time. The call chain topology and performance indicators are used to detect abnormal calls in the call chain. Fault warnings are triggered when anomalies are detected.

[0031] Anomaly detection on preprocessed data information specifically includes the following sub-steps: Anomaly detection on log data: S2I. Access preprocessed log data, extract timestamps, and calculate log length as a feature; S2II. Count the frequency of special characters in the log data and perform vectorization operations on the log text; S2III. Configure detection samples and the proportion of abnormal data; S2IV. Detect whether the log data contains anomalies, calculate the anomaly score, and generate abnormal log output; Detect anomalies in the system operating status; S21. Access pre-processed system performance indicator data; S22. Set seasonal cycle parameters (5-minute sampling) and configure the trend type; S23. Customize anomaly thresholds and detect anomalies in the system performance indicator data based on the residual standard deviation.

[0032] S3. Generate the optimal self-healing strategy based on the root cause of the fault and the user's system data and execute it. When generating the strategy, consider factors such as system resource limitations, business needs, and security requirements.

[0033] The present invention uses AI algorithms to automatically analyze monitoring data and automatically restart faulty applications, greatly improving operation and maintenance efficiency, significantly increasing fault location efficiency, significantly reducing fault recovery time, and significantly improving system availability.

[0034] The foregoing description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments and can be modified within the scope of the concept described herein through the above teachings or techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention are intended to be protected by the appended claims.

Claims

1. An AI-based operation and maintenance fault self-healing system, characterized by: It includes data acquisition module, data preprocessing module, anomaly detection module, root cause analysis module, self-healing strategy generation module, and automated execution module; The data acquisition module is used to collect user system data; The data preprocessing module is used to preprocess the collected data; The anomaly detection module is used to detect anomalies in the processed data and issue fault warnings for anomalies; The root cause analysis module is used to analyze the detected abnormal data and locate the root cause of the fault; The self-healing strategy generation module is used to generate an optimal self-healing strategy based on the root cause of the fault and the user system status; The automatic execution module is used to automatically execute corresponding repair operations based on the generated optimal self-healing measurement.

2. The AI ​​algorithm-based operation and maintenance fault self-healing system according to claim 1 is characterized in that: The collected user system data specifically includes: system performance indicator data, business system log data, and call chain data; wherein, the system performance indicator data includes: CPU usage, memory usage, and network bandwidth; the business system log data includes: system logs and application logs.

3. The AI ​​algorithm-based operation and maintenance fault self-healing system according to claim 1 is characterized in that: The anomaly detection module includes an anomaly detection submodule and an anomaly threshold determination submodule; The anomaly detection submodule uses IForest to perform anomaly detection on the pre-processed log data, uses Holt-Winters to monitor the system operation status in real time, and detects whether there are abnormal calls in the call link through the call link topology structure and performance indicators; The abnormal threshold determination submodule assumes that the data obeys a normal distribution based on historical data, and calculates the mean and standard deviation through a slider window.

4. The AI ​​algorithm-based operation and maintenance fault self-healing system according to claim 1 is characterized in that: The self-healing strategy generation module generates an optimal self-healing strategy by defining a suitable reward function by taking the root cause of the fault and the system state as the environmental state and the repair operation as the action space; wherein the suitable reward function includes: minimizing the fault repair time and maximizing the system availability.

5. The AI ​​algorithm-based operation and maintenance fault self-healing system according to claim 1 is characterized in that: The corresponding repair operations in the automated execution module specifically include: restarting services, adjusting configuration parameters, and expanding resources.

6. The AI ​​algorithm-based operation and maintenance fault self-healing system according to claim 1 is characterized in that: It also includes a model optimization module for real-time optimization and updating of relevant detection models in anomaly detection.

7. An AI algorithm-based operation and maintenance fault self-healing method, based on the AI ​​algorithm-based operation and maintenance fault self-healing system according to any one of claims 1 to 6, characterized in that: The following steps are involved: S1. Collect user system data and preprocess the collected data; S2. Perform anomaly detection on the preprocessed data information and perform root cause analysis on the detected anomaly data; S3. Generate the optimal self-healing strategy based on the root cause of the fault and user system data and execute it.

8. The AI ​​algorithm-based operation and maintenance fault self-healing method according to claim 7, characterized in that: The abnormality detection of the pre-processed data information in step S2 specifically includes the following sub-steps: Perform anomaly detection on log data: S2I. Access the preprocessed log data, extract the timestamp, and calculate the log length as a feature; S2II. Count the frequency of special characters in the log data and perform vectorization operations on the log text; S2III. Configure the ratio of detection samples and abnormal data; S2IV. Detect whether there are anomalies in the log data, calculate the anomaly score, and generate anomaly log output; Detect abnormalities in system operation status; S21. Access pre-processed system performance indicator data; S22. Set seasonal cycle parameters and configure trend type; S23. Customize anomaly thresholds and detect outliers in system performance indicator data based on residual standard deviations.

Images