An intelligent operation and maintenance monitoring system and method based on machine learning optimization

Abstract

The present invention proposes an intelligent operation and maintenance monitoring system and method based on machine learning optimization. The system includes: a data acquisition module, a data preprocessing module, a model evaluation module, a model interpretation module, a trend analysis module, a fluctuation analysis module, an alarm analysis module, an availability analysis module, an automated response module, and an anomaly detection and early warning module. The present invention optimizes the performance of the model and improves the prediction accuracy through systematic feature engineering and algorithm selection processes. The present invention introduces a model interpretation module to improve the transparency and credibility of the model, making it easier to understand the decision-making process of the model. The present invention can accurately predict long-term performance trends and short-term fluctuations, thereby more effectively performing system capacity planning and resource allocation. The availability analysis module of the present invention ensures the high availability of the cluster system by dynamically analyzing the status of cluster nodes. The present invention achieves rapid response and early warning to abnormal events, and improves the stability and reliability of the system.

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance technology, and in particular to an intelligent operation and maintenance monitoring system and method based on machine learning optimization. Background Art

[0002] With the rapid development of business, the number of server hosts continues to increase, and the complexity of operation and maintenance work has also increased. Intelligent inspection can automatically complete inspection tasks, understand the operation of software and hardware systems, and discover problems in a timely manner. The general intelligent operation and maintenance system consists of several modules: data acquisition, data preprocessing, trend analysis, fluctuation analysis, alarm analysis, and scoring reports. The indicator data is processed in a certain order, and scored according to the alarm status and level through machine learning algorithms, and finally the scoring results are output.

[0003] In the field of information technology and operation and maintenance monitoring, existing technologies face many challenges. First, with the increasing complexity of IT infrastructure, traditional operation and maintenance monitoring methods can no longer meet the needs in terms of real-time and accuracy. These methods often rely on manual intervention, which is inefficient and error-prone. At the same time, existing systems have limited capabilities in data processing and feature extraction, and it is difficult to cope with large-scale, high-dimensional data, which limits the analytical capabilities of monitoring systems. In addition, existing models have insufficient generalization capabilities in new environments or private domains, and lack effective model interpretation and visualization, making it difficult for operation and maintenance teams to understand and trust the decision-making process of the model. High resource consumption and insufficient security and privacy protection are also the main shortcomings of existing technologies. Summary of the invention

[0004] The technical problem to be solved by the present invention is how to improve the performance of an intelligent operation and maintenance monitoring system, and to provide an intelligent operation and maintenance monitoring system and method based on machine learning optimization.

[0005] The intelligent operation and maintenance monitoring system based on machine learning optimization according to an embodiment of the present invention includes:

[0006] The data acquisition module is used to obtain the original data of the server host cluster and convert it into a unified data format;

[0007] A data preprocessing module, used for preprocessing the raw data;

[0008] Model evaluation module, used to evaluate selected data features and machine learning algorithm models;

[0009] Model interpretation module, used for interpretation and visualization of the selected machine learning algorithm model;

[0010] The trend analysis module is used to perform trend analysis based on the acquired raw data through a machine learning algorithm model;

[0011] The fluctuation analysis module is used to analyze the fluctuation rate of the index data of each server host through an algorithm model based on the acquired raw data;

[0012] The alarm analysis module is used to calculate the score based on the calculation results of the trend analysis module and the fluctuation analysis module according to the preset scoring rules, and issue an alarm according to the scoring result;

[0013] Availability analysis module, used to analyze the availability of server host clusters according to the status of each server host;

[0014] An automated response module that detects anomalies or failures, automatically generates event notifications, and performs root cause analysis;

[0015] The anomaly detection and early warning module is used for real-time detection of abnormal events and early warning.

[0016] According to some embodiments of the present invention, the preprocessing performed by the data preprocessing module includes at least one of the following:

[0017] Use regular expression pattern matching to identify and correct raw data formatting errors;

[0018] Apply wavelet transform to denoise the raw data;

[0019] Use time series prediction model to predict and fill missing values ​​in the original data;

[0020] Use the Z-Score method and set the threshold θ to identify and process outliers in the original data;

[0021] The raw data is normalized to scale the data to a uniform range.

[0022] In some embodiments of the present invention, the model evaluation module includes: a feature engineering extraction module and a model algorithm selection module;

[0023] The feature engineering extraction module is used to:

[0024] Calculate and extract statistical features based on the time series of raw data;

[0025] Extract frequency domain features based on raw data using Fourier transform;

[0026] Based on the extracted statistical features and frequency domain features, a recursive feature elimination method is applied to select and determine machine learning features;

[0027] Use principal component analysis to perform feature dimensionality reduction on the identified machine learning features;

[0028] Convert the reduced-dimensional machine learning features into numerical features;

[0029] The algorithm selection module is used for:

[0030] The acquired raw data is randomly divided into training set, validation set and test set;

[0031] Build a library of various machine learning algorithms;

[0032] Evaluate the performance of different machine learning algorithms through cross-validation;

[0033] Use preset indicators to optimize the performance of machine learning algorithms;

[0034] The algorithm with the best performance on the validation set is selected as the final algorithm model.

[0035] According to some embodiments of the present invention, the model interpretation module uses sensitivity analysis LIME for local interpretation, applies SHAP value for global interpretation, uses tree model for feature importance assessment, and uses visualization tools to display the decision-making process of the model.

[0036] In some embodiments of the present invention, the trend analysis module uses an LSTM model to predict the long-term performance trend of the server host, applies CUSUM technology to detect significant changes in trends, and uses a time series decomposition method to separate trend, seasonality, and random components.

[0037] According to some embodiments of the present invention, the fluctuation analysis module deploys a real-time data stream processing framework to perform real-time monitoring of fluctuations, uses fast Fourier transform to identify the frequency components of fluctuations, combines machine learning clustering algorithms to analyze the sources and related factors of fluctuations, and constructs a fluctuation prediction model to predict future fluctuation trends.

[0038] In some embodiments of the present invention, the alarm analysis module is specifically used to:

[0039] Receiving output data from the trend analysis module and the fluctuation analysis module;

[0040] Design scoring thresholds and scoring rules for each indicator based on business needs, and calculate the score value for each server host;

[0041] The alarm level is determined based on the scoring results. When the score exceeds the set severe event threshold, a severe alarm is triggered; when the score exceeds the warning event threshold but does not exceed the severe event threshold, a warning event is triggered; when the score is within the normal range, it is determined to be a normal event.

[0042] According to some embodiments of the present invention, the availability analysis module performs availability analysis of the server host cluster through distance function calculation, cluster analysis calculation, and cluster availability formula calculation.

[0043] In some embodiments of the present invention, the automated response module uses control charts in statistical process control or anomaly score calculations based on machine learning algorithms to perform anomaly detection; and uses machine learning models to perform root cause analysis to quickly locate the source of the problem.

[0044] According to an intelligent operation and maintenance monitoring method based on machine learning optimization according to an embodiment of the present invention, the method adopts the intelligent operation and maintenance monitoring system based on machine learning optimization described above to perform intelligent operation and maintenance monitoring of a server host cluster, and the method includes:

[0045] S10, obtaining the original data of the server host cluster and converting it into a unified data format;

[0046] S20, preprocessing the original data;

[0047] S30, evaluate and select data features and machine learning algorithm models;

[0048] S40, explanation and visualization of the selected machine learning algorithm model;

[0049] S50, based on the acquired raw data, trend analysis is performed through an algorithm model;

[0050] S60, based on the acquired raw data, analyzing the volatility of the index data of each server host through a machine learning algorithm model;

[0051] S70, based on the calculation results of the trend analysis module and the fluctuation analysis module, calculate the score according to the preset scoring rules, and issue an alarm according to the scoring result;

[0052] S80, performing availability analysis of the server host cluster according to the status of each server host;

[0053] S90, detects anomalies or failures, automatically generates event notifications, and performs root cause analysis;

[0054] S100, real-time detection of abnormal events and early warning.

[0055] The present invention has the following beneficial effects:

[0056] The present invention optimizes the performance of the model and improves the prediction accuracy through systematic feature engineering and algorithm selection processes. The present invention introduces a model interpretation module to improve the transparency and credibility of the model, so that the operation and maintenance team can understand the decision-making process of the model. By combining trend analysis and fluctuation analysis, the present invention can accurately predict long-term performance trends and short-term fluctuations, thereby more effectively performing system capacity planning and resource allocation. The availability analysis module of the present invention ensures the high availability of the cluster system by dynamically analyzing the status of cluster nodes. The present invention combines the automatic response module of the intelligent operation and maintenance monitoring system optimized based on machine learning and the abnormal monitoring and early warning module to achieve rapid response and early warning to abnormal events, thereby improving the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flow chart of an intelligent operation and maintenance monitoring method based on machine learning optimization according to an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of a CPU usage scoring strategy of an alarm analysis module according to an embodiment of the present invention;

[0059] Figure 3 A schematic diagram of cluster analysis of an availability analysis module according to an embodiment of the present invention;

[0060] Figure 4 A schematic diagram showing that an event of a business task arriving at a cluster server conforms to a Poisson distribution according to an embodiment of the present invention;

[0061] Figure 5 Schematic diagram of server host performance and availability curve according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose, the present invention is described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0063] The description of the method flow in the specification of the present invention and the steps of the flowchart in the drawings of the specification of the present invention do not have to be strictly executed according to the step numbers, and the method steps can be executed in a different order. Moreover, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0064] The present invention aims to address the deficiencies of the above-mentioned prior art by integrating advanced machine learning algorithms and big data analysis technologies to improve the real-time performance and accuracy of the monitoring system. The system will reduce reliance on manual intervention, enhance automation capabilities, and thus improve operation and maintenance efficiency. By optimizing data processing and feature extraction, and adopting diversified training and domain adaptation technologies, the present invention will enhance the generalization ability and analytical performance of the model. At the same time, it will enhance the interpretability and visualization of the model, improve model transparency, enhance the understanding and trust of the operation and maintenance team, and provide an automatic response module and an abnormal monitoring and early warning mechanism to better maintain the stability of the system. In addition, the present invention will optimize the system architecture, reduce resource consumption, and ensure data security and privacy protection to meet the needs of efficient and intelligent operation and maintenance of modern IT infrastructure.

[0065] The present invention proposes an intelligent operation and maintenance monitoring system and method based on machine learning optimization. Figure 1 As shown, the system is divided into eight modules, namely data acquisition module, data preprocessing module, model evaluation module, model interpretation module, trend analysis module, fluctuation analysis module, alarm analysis module, availability analysis module, automated response module, and anomaly detection and warning module.

[0066] The following is a detailed explanation of each module:

[0067] Data acquisition module:

[0068] The data acquisition module is designed to acquire raw data, connect to the data source and capture raw data through the API so that the intelligent operation and maintenance monitoring system can obtain data from the server host, convert the acquired data into a unified format for subsequent processing, and store the acquired data in the data warehouse to ensure data accessibility.

[0069] Data preprocessing module:

[0070] The data preprocessing module loads the raw data transmitted from the data acquisition module, and cleans, denoises and standardizes the acquired data;

[0071] Step 2.1: Data cleaning: Use regular expression pattern matching to identify and correct data format errors.

[0072] Step 2.2: Data denoising. Apply wavelet transform W(x,a,b) for denoising:

[0073]

[0074] Where x is the time series data, a and b are the scale and translation parameters of the wavelet transform.

[0075] Step 2.3: Fill in missing values. Use a time series forecasting model, such as the autoregressive moving average model (ARMA), to predict missing values:

[0076]

[0077] Among them, x t is the value of the time series at time t, φ i is the autoregressive coefficient, θ j is the moving average coefficient, ∈ t is the error term.

[0078] Step 2.4: Outlier processing. Use the Z-Score method, set the threshold θ, identify and process outliers:

[0079]

[0080] Among them, x t is the observed value, μ is the mean, and σ is the standard deviation.

[0081] Step 2.5: Data normalization: Scale the data to a uniform range, such as 0 to 1.

[0082] Model evaluation module:

[0083] The model evaluation module provides a comprehensive approach to processing data, selecting and evaluating machine learning algorithms, and ensuring that the final model performs optimally. This module includes feature engineering, algorithm selection, and model evaluation, ensuring the integrity and efficiency of the entire machine learning process.

[0084] 3.1 Feature Engineering Extraction:

[0085] Step 3.1.1: Statistical feature extraction. Calculate the statistical features of the time series, such as mean and variance.

[0086] Step 3.1.2: Time series feature construction: Use Fourier transform to extract frequency domain features.

[0087] Step 3.1.3: Machine learning feature selection. Apply the recursive feature elimination (RFE) method to select the most informative features.

[0088] Step 3.1.4: Feature dimensionality reduction: Use principal component analysis (PCA) to perform feature dimensionality reduction.

[0089] Step 3.1.5: Feature encoding. Convert categorical features into numerical features.

[0090] 3.2 Algorithm selection:

[0091] Step 3.2.1: Dataset division: Randomly divide the dataset into training set, validation set and test set.

[0092] Step 3.2.2: Algorithm library construction. Build a library containing various machine learning algorithms.

[0093] Step 3.2.3: Automated algorithm evaluation. Evaluate the performance of different algorithms through cross-validation.

[0094] Step 3.2.4: Performance metric optimization: Use metrics such as precision, recall, and F1 score to optimize algorithm performance.

[0095] Step 3.2.5: Algorithm selection. Select the algorithm that performs best on the validation set as the final model.

[0096] 3.3 Model Performance Evaluation

[0097] Step 3.3.1: Cross-validation: Stratified K-fold cross-validation is used to ensure the fairness and accuracy of model evaluation.

[0098] Step 3.3.2: Calculate performance indicators. Calculate the model's performance indicators such as accuracy, recall, F1 score, and AUC on the test set.

[0099] Step 3.3.3: Model stability evaluation: Evaluate the stability of the model performance through multiple training and validation.

[0100] Step 3.3.4: Error analysis. Analyze cases where the model makes incorrect predictions and identify the model’s weaknesses.

[0101] Step 3.3.5: Model fusion. Consider using model fusion techniques, such as voting, stacking, etc., to further improve model performance.

[0102] Model interpretation module:

[0103] The model interpretation module is a crucial part of the machine learning workflow and is designed to improve the interpretability of the model.

[0104] Step 4.1: Local interpretation: Use Local Interpretable Model-Sensitivity Analysis (LIME).

[0105] Step 4.2: Global interpretation: Apply SHAP value analysis.

[0106] Step 4.3: Feature Importance Evaluation. Feature Importance Evaluation using Tree Model.

[0107] Step 4.4: Model visualization. Use visualization tools to show the decision-making process of the model.

[0108] Step 4.5: Explanation model selection. Select the most appropriate explanation model based on the clarity and accuracy of the explanation.

[0109] Trend analysis module:

[0110] This module effectively removes noise data through machine learning algorithms while retaining the trend information of the data, achieving a data smoothing effect and preventing false positives and negative negatives.

[0111] Step 5.1: Long-term trend prediction. Use the LSTM model to predict the long-term performance trend.

[0112]

[0113] Among them, σ is the activation function, W y is the weight matrix, h t-1 is the hidden state of the previous time step, x t is the input of the current time step, b y is the bias term.

[0114] Step 5.2: Trend change detection. Apply the CUSUM technique to detect significant changes in the trend.

[0115] Step 5.3: Seasonal Decomposition. Use time series decomposition methods to separate trend, seasonality, and random components.

[0116] Step 5.4: Trend model evaluation: Evaluate the accuracy and stability of the trend prediction model.

[0117] Step 5.5: Trend forecast application: Apply trend forecast results to system capacity planning and resource allocation.

[0118] Fluctuation analysis module:

[0119] This module uses probability distribution algorithm to analyze the volatility of server host indicator data.

[0120]

[0121] Among them, X(k) is the frequency domain representation, x n is the discrete value of the time series, and N is the number of data points.

[0122] Step 6.1: Real-time fluctuation monitoring. Deploy a real-time data stream processing framework to perform real-time monitoring of fluctuations.

[0123] Step 6.2: Frequency domain fluctuation analysis. Use Fast Fourier Transform (FFT) to identify the frequency components of the fluctuation.

[0124] Step 6.3: Locate the source of fluctuations. Analyze the source and related factors of fluctuations by combining machine learning clustering algorithms.

[0125] Step 6.4: Fluctuation prediction model: Construct a fluctuation prediction model to predict future fluctuation trends.

[0126] Step 6.5: Fluctuation response strategy: According to the fluctuation prediction results, formulate corresponding system adjustment and optimization strategies.

[0127] Alarm analysis module:

[0128] 7.1 Receive output data from the trend analysis module and the fluctuation analysis module.

[0129] 7.2 According to business needs, design scoring thresholds and scoring rules for each indicator. Each server host status will get a specific score value.

[0130] 7.3 The alarm level is determined based on the scoring results. When the score exceeds the set severe event threshold, a severe alarm is triggered; when the score exceeds the warning event threshold but does not exceed the severe event threshold, a warning event is triggered; when the score is within the normal range, it is determined to be a normal event.

[0131] Taking CPU usage as an example, the usage scoring strategy is based on Figure 2 The piecewise linear formula shown is obtained.

[0132] Usability Analysis Module:

[0133] The server host cluster contains multiple nodes. Each node is usually in a dynamic alternation between work and maintenance. Individual nodes are allowed to go offline for maintenance, but sometimes individual nodes may go offline due to failures. Therefore, it is necessary to solve the problems of "false positives" and "missing positives". This module defines:

[0134] Distance function -> Cluster analysis -> Cluster availability formula

[0135] Determine the overall availability of the cluster system. When individual nodes in the cluster are undergoing normal maintenance and upgrades and do not affect the availability and performance of the overall system, the module will not alarm, ensuring that each node in the cluster is in a "dynamically highly available" state, thereby avoiding false alarms. When a cluster node has maintenance or a failure that affects the availability and performance of the cluster, the module will give a serious event (alarm) or warning event (early warning) based on the cluster availability coefficient (R) level to avoid missed reports.

[0136] (1) Distance function:

[0137]

[0138] Among them, f distance is the distance function, C is the number of concurrent shutdowns (the number of server shutdowns within a similar time range), s1 and s2 are the host survival status sequences, and N is the sequence length.

[0139] (2) Cluster analysis:

[0140] Cluster analysis Figure 3 As shown, C = cluster node set, C1 = normal node set, C0 = inactive node set, C max_Maint = Maximum maintenance node set, C live =C-C0-C max_Maint .

[0141] Cluster node set: the set of all nodes in the server host cluster;

[0142] Normal node set: the set of hosts that are always alive during the statistical time period;

[0143] Inactive node set: the set of offline nodes within the statistical time period;

[0144] Maximum maintenance node set: the maximum classification set after DBSCAN clustering.

[0145] (3) Cluster performance indicators and availability indicators:

[0146] like Figure 4 As shown, the event of business tasks arriving at the cluster server conforms to the Poisson distribution:

[0147] X~Pois(λ);

[0148] The cluster task at the current time (t) will be in the range of available hosts in the cluster at the current time (the number is A t ) to obtain the task arrival rate of each host:

[0149]

[0150] Among them, λ is the task arrival rate Task t is the number of tasks per host in the cluster at time t, A t is the number of available hosts in the cluster at time t. Therefore, the number of available hosts in the cluster at time t has a huge impact on the parallel processing performance of cluster tasks.

[0151] When the number of available hosts in the cluster reaches the minimum at time t, the performance and availability curves are as follows: Figure 5 shown.

[0152]

[0153] Among them, P is the cluster performance index, A is the cluster availability index, α is the maintenance downtime tolerance coefficient, α∈[0,0.33];

[0154] f is the node availability function = (total time - downtime) / total time (based on full historical data);

[0155] len is the number of elements in the returned set, len(C live )∈[0,N], N is the total number of server host nodes.

[0156] (4) Cluster availability formula:

[0157]

[0158] Where R is the cluster availability coefficient, α is the maintenance downtime tolerance coefficient, α∈[0,0.33];

[0159] f is the node availability function = (total time - downtime) / total time (based on full historical data);

[0160] len is the number of elements in the returned set, len(C live )∈[0,N], N is the total number of server host nodes.

[0161] Automated response module:

[0162] Step 8.1: Event triggering: When the monitoring system detects an anomaly or failure, an event notification is automatically generated.

[0163] Anomaly detection can usually be done using control charts in statistical process control or anomaly score calculation based on machine learning algorithms.

[0164] Formula example: Anomaly score = observed value - expected value standard deviation Abnormal score = standard deviation observed value - expected value

[0165] Reasoning process: If the anomaly score exceeds the preset threshold, an event is triggered.

[0166] Step 8.2: Root cause analysis: Use machine learning models to perform root cause analysis and quickly locate the source of the problem.

[0167] Decision trees, Bayesian networks, or other causal inference models can be used to determine the root cause of the failure.

[0168] Formula example: P(cause|fault)=P(fault)P(fault|cause)·P(cause).

[0169] Reasoning process: Identify the most likely cause of failure by calculating conditional probabilities.

[0170] Step 8.3: Automated scripts. Develop automated scripts to perform common recovery operations.

[0171] Step 8.4: Response effectiveness evaluation. Evaluate the effectiveness of the automated response and collect feedback data for model optimization. For example, regression analysis can be used to evaluate the effectiveness of the response measures:

[0172] Formula example: Y = β0 + β1X1 + β2X2 + ... + β n X n +∈;

[0173] where Y is the response effect, X1, X2, ...X n is the influencing factor, is the coefficient, and is the error term.

[0174] Step 8.5: Response strategy optimization. Based on the response effect and feedback, continuously optimize the automated response strategy. For example, you can use strong learning to continuously optimize the response strategy.

[0175] Formula example: Q(s,a)←Q(s,a)+α[r+γmax a' Q(s',a')-Q(s,a)];

[0176] Where Q is the action-value function, s is the state, a is the action, r is the reward, α is the learning rate, and γ is the discount factor.

[0177] Abnormal monitoring and early warning module:

[0178] Step 9.1: Abnormal event detection. Monitor system data in real time and identify abnormal events. For example, anomaly detection can use Z-Score, IQR (interquartile range) or other statistical methods:

[0179] Formula example:

[0180] Reasoning process: If Z exceeds the threshold, an anomaly is considered detected.

[0181] Step 9.2: Early warning. Issue early warning notifications based on the set thresholds and model predictions. For example, the early warning system can use time series analysis, such as the ARIMA model, to predict future anomalies.

[0182] Formula Example: X t =c+φ1X t-1 +...+φ p X t-p +θ1∈ t-1 +...+θ q ∈ t-q +∈ t ;

[0183] Among them, X t is the current value, φ is the autoregressive coefficient, θ is the moving average coefficient, and ∈ is the error term

[0184] Step 9.3: Event recording: Record abnormal events and their handling process for subsequent analysis.

[0185] Step 9.4: Feedback mechanism: Collect feedback on early warning effects and optimize the early warning model.

[0186] Step 9.5: Report generation. Generate abnormal monitoring reports regularly for reference by the operation and maintenance team.

[0187] Compared with the prior art, the present invention has the following advantages:

[0188] 1. Systematization of feature engineering and algorithm selection: Through the systematic feature engineering and algorithm selection process, the performance of the model is optimized and the prediction accuracy is improved.

[0189] Model interpretability: The introduction of the model interpretation module improves the transparency and credibility of the model, enabling the operation and maintenance team to understand the decision-making process of the model.

[0190] Combination of trend and fluctuation analysis: By combining trend analysis and fluctuation analysis, the present invention can accurately predict long-term performance trends and short-term fluctuations, thereby more effectively performing system capacity planning and resource allocation.

[0191] 2. Dynamic availability analysis: The availability analysis module ensures high availability of the cluster system by dynamically analyzing the status of cluster nodes.

[0192] 3. The combination of the automated response module and the abnormal monitoring and early warning module of the intelligent operation and maintenance monitoring system based on machine learning optimization realizes rapid response and early warning to abnormal events, and improves the stability and reliability of the system.

[0193] Through the description of the specific implementation methods, a deeper and more specific understanding of the technical means and effects adopted by the present invention to achieve the predetermined purpose should be obtained. However, the accompanying drawings are only for reference and illustration purposes and are not intended to limit the present invention.

Claims

1. An intelligent operation and maintenance monitoring system based on machine learning optimization, characterized in that: include: The data acquisition module is used to obtain the original data of the server host cluster and convert it into a unified data format; A data preprocessing module, used for preprocessing the raw data; Model evaluation module, used to evaluate selected data features and machine learning algorithm models; Model interpretation module, used for interpretation and visualization of the selected machine learning algorithm model; The trend analysis module is used to perform trend analysis based on the acquired raw data through a machine learning algorithm model; The fluctuation analysis module is used to analyze the fluctuation rate of the index data of each server host through an algorithm model based on the acquired raw data; The alarm analysis module is used to calculate the score based on the calculation results of the trend analysis module and the fluctuation analysis module according to the preset scoring rules, and issue an alarm according to the scoring result; Availability analysis module, used to analyze the availability of server host clusters according to the status of each server host; An automated response module that detects anomalies or failures, automatically generates event notifications, and performs root cause analysis; The anomaly detection and early warning module is used for real-time detection of abnormal events and early warning.

2. The intelligent operation and maintenance monitoring system based on machine learning optimization according to claim 1 is characterized in that: The preprocessing performed by the data preprocessing module includes at least one of the following: Use regular expression pattern matching to identify and correct raw data formatting errors; Apply wavelet transform to denoise the raw data; Use time series prediction model to predict and fill missing values ​​in the original data; Use the Z-Score method and set the threshold θ to identify and process outliers in the original data; The raw data is normalized to scale the data to a uniform range.

3. The intelligent operation and maintenance monitoring system based on machine learning optimization according to claim 1 is characterized in that: The model evaluation module includes: a feature engineering extraction module and a model algorithm selection module; The feature engineering extraction module is used to: Calculate and extract statistical features based on the time series of raw data; Extract frequency domain features based on raw data using Fourier transform; Based on the extracted statistical features and frequency domain features, a recursive feature elimination method is applied to select and determine machine learning features; Use principal component analysis to perform feature dimensionality reduction on the identified machine learning features; Convert the reduced-dimensional machine learning features into numerical features; The algorithm selection module is used for: The acquired raw data is randomly divided into training set, validation set and test set; Build a library of various machine learning algorithms; Evaluate the performance of different machine learning algorithms through cross-validation; Use preset indicators to optimize the performance of machine learning algorithms; The algorithm with the best performance on the validation set is selected as the final algorithm model.

4. The intelligent operation and maintenance monitoring system based on machine learning optimization according to claim 1 is characterized in that: The model interpretation module uses sensitivity analysis LIME for local interpretation, applies SHAP value for global interpretation, uses tree model for feature importance assessment, and uses visualization tools to display the decision-making process of the model.

5. The intelligent operation and maintenance monitoring system based on machine learning optimization according to claim 1 is characterized in that: The trend analysis module uses the LSTM model to predict the long-term performance trend of the server host, applies the CUSUM technique to detect significant changes in the trend, and uses the time series decomposition method to separate the trend, seasonality and random components.

6. The intelligent operation and maintenance monitoring system based on machine learning optimization according to claim 1 is characterized in that: The fluctuation analysis module deploys a real-time data stream processing framework to perform real-time monitoring of fluctuations, uses fast Fourier transform to identify the frequency components of fluctuations, combines machine learning clustering algorithms to analyze the sources and related factors of fluctuations, and constructs a fluctuation prediction model to predict future fluctuation trends.

7. The intelligent operation and maintenance monitoring system based on machine learning optimization according to claim 1 is characterized in that: The alarm analysis module is specifically used for: Receiving output data from the trend analysis module and the fluctuation analysis module; Design scoring thresholds and scoring rules for each indicator based on business needs, and calculate the score value for each server host; The alarm level is determined based on the scoring results. When the score exceeds the set severe event threshold, a severe alarm is triggered. When the score exceeds the warning event threshold but does not exceed the severe event threshold, a warning event is triggered; when the score is within the normal range, it is judged as a normal event.

8. The intelligent operation and maintenance monitoring system based on machine learning optimization according to claim 1 is characterized in that: The availability analysis module performs availability analysis of the server host cluster through distance function calculation, cluster analysis calculation, and cluster availability formula calculation.

9. The intelligent operation and maintenance monitoring system based on machine learning optimization according to claim 1 is characterized in that: The automated response module uses control charts in statistical process control or anomaly score calculations based on machine learning algorithms to perform anomaly detection; and uses machine learning models to perform root cause analysis to quickly locate the source of the problem.

10. An intelligent operation and maintenance monitoring method based on machine learning optimization, characterized in that: The method adopts the above-mentioned intelligent operation and maintenance monitoring system based on machine learning optimization to perform intelligent operation and maintenance monitoring of the server host cluster, and the method includes: S10, obtaining the original data of the server host cluster and converting it into a unified data format; S20, preprocessing the original data; S30, evaluate and select data features and machine learning algorithm models; S40, explanation and visualization of the selected machine learning algorithm model; S50, based on the acquired raw data, trend analysis is performed through an algorithm model; S60, based on the acquired raw data, analyzing the volatility of the index data of each server host through a machine learning algorithm model; S70, based on the calculation results of the trend analysis module and the fluctuation analysis module, calculate the score according to the preset scoring rules, and issue an alarm according to the scoring result; S80, performing availability analysis of the server host cluster according to the status of each server host; S90, detects anomalies or failures, automatically generates event notifications, and performs root cause analysis; S100, real-time detection of abnormal events and early warning.

Images