Cloud platform intelligent monitoring and early warning method based on AI and Prometheus

Abstract

The invention relates to the technical field of data monitoring, in particular to a cloud platform intelligent monitoring and early warning method based on AI and Prometheus. According to the system, an artificial intelligence technology is used to improve a traditional Prometheus-based monitoring scheme, historical operation data of a cloud platform is trained and analyzed through a deep learning algorithm, and a model capable of automatically identifying and predicting abnormal conditions is constructed. Therefore, the monitoring system not only can detect the current running state in real time, but also can predict the future trend and potential problems, give an alarm in advance, and help an administrator to take measures in time.

Description

Technical Field

[0001] The present invention relates to the field of data monitoring technology, and in particular to a cloud platform intelligent monitoring and early warning method based on AI and Prometheus. Background Art

[0002] With the rapid advancement of cloud computing technology, cloud platforms have become an indispensable cornerstone for enterprise information construction. However, cloud platforms are inevitably affected by various factors during operation, leading to performance degradation, frequent failures and other problems. If these problems are not handled promptly and effectively, they will cause significant losses to enterprise operations. Therefore, it is particularly urgent to implement an efficient and intelligent cloud platform operation status monitoring and early warning system.

[0003] The use of Prometheus monitoring solutions has become a widely used practice in the industry. Prometheus provides comprehensive monitoring capabilities for cloud platforms by collecting, storing, and querying time series data. Its powerful data collection and visualization capabilities enable administrators to grasp the operating status of the cloud platform in real time and observe various indicators in detail.

[0004] However, the traditional Prometheus-based monitoring solution has obvious deficiencies in intelligent analysis and prediction. It mainly relies on fixed thresholds and rules for anomaly detection, which is insufficient in the face of the complexity and dynamics of the cloud platform environment. The state changes of the cloud platform often present a high degree of complexity and uncertainty, and fixed thresholds and rules are difficult to accurately capture all potential problems.

[0005] In addition, the manual configuration process of traditional solutions is cumbersome and inefficient. In the dynamic environment of cloud services that are constantly changing, administrators need to frequently update and maintain configuration files to ensure the accuracy and effectiveness of the monitoring system. This not only increases the workload of administrators, but also reduces the response speed and flexibility of the monitoring system.

[0006] Therefore, a more intelligent and efficient monitoring and alarm system is needed to solve these problems. Summary of the invention

[0007] In order to solve these problems, the present invention proposes a cloud platform intelligent monitoring and alarm system based on artificial intelligence and Prometheus. The system uses artificial intelligence technology to improve the traditional Prometheus-based monitoring solution, trains and analyzes the historical operation data of the cloud platform through a deep learning algorithm, and builds a model that can automatically identify and predict abnormal situations. In this way, the monitoring system can not only detect the current operating status in real time, but also predict future trends and potential problems, issue warnings in advance, and help administrators take timely measures.

[0008] At the same time, the system also simplifies the configuration process by automatically identifying new load data that is similar to historical anomaly patterns without manually setting fixed thresholds. Compared with the fixed threshold method, the present invention provides a more flexible and accurate anomaly detection mechanism, which is particularly suitable for complex and dynamically changing cloud platform environments. This not only reduces the workload of administrators, but also improves the configuration efficiency and accuracy of the monitoring system.

[0009] In summary, the intelligent monitoring and alarm system for cloud platform operation status based on artificial intelligence and Prometheus can overcome the shortcomings of traditional solutions and provide a more comprehensive, intelligent and efficient cloud platform monitoring solution. Through real-time detection, intelligent analysis and prediction, and simplified configuration process, it can timely discover and handle problems in the operation of the cloud platform, ensuring the stable operation and business continuity of cloud platform services.

[0010] The following technical solutions are used to solve technical problems:

[0011] An intelligent monitoring and early warning method for cloud platforms based on AI and Prometheus, which collects data through Prometheus and combines artificial intelligence technology to intelligently analyze and predict the operating status of the cloud platform to achieve intelligent monitoring and alarm; the artificial intelligence technology is an artificial intelligence model trained through a deep learning algorithm, which can automatically identify abnormal conditions during the operation of the cloud platform; the alarm is prompted by a preset threshold.

[0012] Further steps include:

[0013] Monitoring data collection:

[0014] First, you need to deploy Prometheus monitoring server or cluster and configure related exporters to collect key indicator data on the cloud platform, such as CPU usage, memory usage, network traffic, and disk I / O. Then configure Prometheus's pull interval and storage strategy to ensure that Prometheus can run stably and continuously collect data, monitor and alert for abnormal situations.

[0015] Data preprocessing:

[0016] Data preprocessing is a crucial step in the data analysis process. Its goal is to convert raw monitoring data into a structured, high-quality data set suitable for subsequent intelligent analysis.

[0017] Smart Analysis:

[0018] Use machine learning or deep learning algorithms to analyze the preprocessed monitoring data and use the training set to train the model of the selected algorithm;

[0019] Early warning and decision making:

[0020] Based on the results of intelligent analysis, when it is predicted that the cloud platform load may be too high or abnormal indicators are detected, the early warning mechanism will be automatically triggered. The early warning information includes load status, abnormality type, and impact range, and relevant personnel will be notified via email, SMS or internal notification system.

[0021] The main advantage of the present invention is that the historical operation data of the cloud platform is trained and analyzed through a deep learning algorithm to build a model that can automatically identify and predict abnormal situations. New load data similar to historical abnormal patterns can be automatically identified without manually setting fixed thresholds. In a complex and dynamically changing cloud platform environment, this not only reduces the workload of administrators, but also improves the configuration efficiency and accuracy of the monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Flowchart for model building and training;

[0023] Figure 2 This is the system intelligent monitoring and early warning flow chart. DETAILED DESCRIPTION

[0024] The overall process of the present invention is as follows Figure 1 , Figure 2 As shown, it mainly includes the following steps:

[0025] Monitoring data collection:

[0026] In order to collect various load data of the cloud platform in real time, this solution first needs to deploy Prometheus monitoring server or cluster, and configure relevant exporters (such as Node Exporter for collecting server resource data) to collect key indicator data on the cloud platform, such as CPU usage, memory usage, network traffic, disk I / O, etc. Then configure Prometheus's pull interval and storage strategy to ensure the accuracy and timeliness of the data. Ensure that Prometheus can run stably and continuously collect data, monitor and alarm for abnormal situations.

[0027] Data preprocessing:

[0028] Data preprocessing is a crucial link in the data analysis process. Its goal is to convert raw monitoring data into a structured, high-quality data set suitable for subsequent intelligent analysis.

[0029] Assume that we provide a simplified monitoring data set as shown in the following table, which contains 1,000 records, each of which has four features (timestamp, CPU usage, memory usage, network traffic) and a label (whether it is abnormal):

[0030]

[0031] In this scenario, data preprocessing includes the following steps.

[0032] Data cleaning: Remove obviously erroneous records, such as negative values ​​or extremely high values ​​(such as CPU usage exceeding 0-100%).

[0033] Feature Engineering: Extract features related to load analysis based on historical monitoring data. These features may include timestamp, CPU usage, memory usage, network bandwidth, disk I / O, task queue length, etc. Where x1 represents the timestamp, x2 represents the CPU usage, until x n Represents other relevant features.

[0034] Standardization: Normalize all feature values ​​to the same scale to facilitate model learning.

[0035] Normalize CPU usage, memory usage, and network traffic to the [0,1] range. For example, the CPU usage normalization formula is:

[0036]

[0037] Assuming that CPU usage ranges from 0% to 100%, the normalized CPU usage of 56% is 0.56.

[0038] Data labeling: Based on operation and maintenance records or set thresholds (e.g., CPU usage exceeding 90% for one hour is marked as abnormal), the data is labeled y (0 = normal, 1 = abnormal).

[0039] Data division: Divide the prepared data set into training set D train (70%), validation set D val (15%) and test set D test (15%), ensuring the model’s generalization ability on unseen data.

[0040] Intelligent analysis: Use machine learning or deep learning algorithms to analyze pre-processed monitoring data. Use the training set to train the selected algorithm model to detect anomalies in the cloud platform load.

[0041] Model construction and training: In this solution, supervised learning algorithm (logistic regression) is used to train the anomaly detection model. As a binary classification model, logistic regression is suitable for dealing with anomaly detection problems and predicting whether an event has occurred (yes / no).

[0042] The basic form of a logistic regression model is:

[0043]

[0044] Where P(y=1|x) is the probability of event y occurring under the condition of a given feature vector x, where y indicates whether it is abnormal (1 indicates abnormality and 0 indicates normality); β0 is the bias term; β1 to β n are the features x1 to x n The coefficient of ; e is the base of natural logarithms.

[0045] Initialization parameters: Set to model parameters (β1, β2, ..., β n ) is assigned an initial value, usually random or zero.

[0046] Loss function calculation: Use the log-likelihood loss function (a common loss function for logistic regression) to measure the difference between the model's predicted probability and the actual label:

[0047]

[0048] Where N is the number of samples, y i is the actual label of the i-th sample, p i is the probability that the model predicts that sample i is abnormal, that is, P(y=1|X i ).

[0049] Gradient ascent optimization: Gradient ascent method is used to gradually adjust the parameters to minimize the loss function. Calculate each parameter β j The gradient of (the partial derivative of the loss function with respect to the parameter) is calculated and the parameters are updated accordingly:

[0050]

[0051] Update formula:

[0052]

[0053] Among them, α is the learning rate, which controls the step size of each parameter update.

[0054] Iteration process: Repeat the steps until a certain stopping condition is met, such as the change in the loss function is less than a minimum threshold, or the preset number of iterations is reached. In each round of iteration, the predicted probability is calculated based on the current parameters, and the parameters are adjusted by comparing the actual labels to gradually approach the optimal solution.

[0055] Validation and Tuning: Evaluate the model on the validation set and tune the model parameters based on the performance.

[0056] Real-time load analysis: Data flow configuration

[0057] Goal – To ensure a continuous and steady flow of monitoring data to analytical models.

[0058] Data source: Prometheus monitoring component, pulls data from each monitoring node export (such as NodeExporter) every minute. Data flow pipeline: Use Kafka or other message queue technologies to establish a data transmission channel to ensure high availability and order of data. Data format: JSON format, including timestamp, CPU usage, memory usage, network traffic, disk I / O and other fields.

[0059] Example data:

[0060]

[0061]

[0062] Model deployment: Deploy the trained anomaly detection model to a service environment, such as using TensorFlow, Seldon Core, or Kubeflow, so that the model can be called by an API.

[0063] Real-time anomaly detection: Set an anomaly threshold (such as 95%). If the probability that the anomaly detection model detects a load anomaly exceeds this value, the current load state is considered abnormal.

[0064] Assume that at a certain point in time, the data points received by the model are {cpu_usage_percent:98,memory_usage_percent:86,network_traffic_mbps:528,disk_io_ops:300}, and the sample point X = [0.98,0.86,0.52,0], where the features correspond to CPU usage, memory usage, network traffic, and an additional feature (here assumed to be 0, which may represent additional information that is not used) set as model parameters (β1,β2,...,β n ) are assigned initial values ​​(β0=0,β1=1,β2=2,β3=3,β4=0),

[0065] Among them, β1 to β4 are the coefficients of the eigenvalues ​​CPU usage, memory usage, network traffic and an additional eigenvalue, and β0 is the bias term. According to the prediction formula of the logistic regression model:

[0066]

[0067] Substitute the given parameters and eigenvalues ​​into the formula to calculate:

[0068]

[0069] Using the natural logarithm calculation, we get e -4.26 ≈0.014, so

[0070]

[0071] P(y=1|x)≈0.986

[0072] Therefore, according to the given parameters and sample characteristics, the calculated abnormal probability P(y=1|x) is approximately 0.986, which indicates that the model predicts that the probability of the sample point being abnormal is very high, and the current load state is determined to be abnormal.

[0073] Real-time response: Based on the detection results, the automated response mechanism is immediately triggered to trigger corresponding warnings and decision-making operations, or to activate automatic resource scheduling rules to adjust cloud resources.

[0074] Early warning and decision-making: Based on the results of intelligent analysis, when it is predicted that the cloud platform load may be too high or abnormal indicators are detected, the early warning mechanism will be automatically triggered. The early warning information includes load status, abnormal type, impact range, etc., and relevant personnel are notified through email, SMS or internal notification system.

[0075] At the same time, the cloud platform's resource configuration or load distribution strategy can be automatically adjusted based on the early warning information to reduce load pressure and ensure the stable operation of the cloud platform. The specific steps are as follows:

[0076] Generation and sending of early warning information: When an early warning is triggered, early warning information is automatically generated, including the type of warning, scope of impact, recommended measures, etc. Early warning information is sent to relevant personnel through email, SMS or internal notification system.

[0077] Automatic decision-making and execution: Based on the early warning information and combined with the cloud platform's resource management and scheduling strategies, automatically adjust resource configuration or optimize load distribution. For example, increase the number of virtual machines, adjust task scheduling strategies, etc.

[0078] Visualization: Provides a friendly user interface to display the cloud platform's load status, monitoring indicator data, warning information, etc. in real time. Users can intuitively understand the cloud platform's load status through this module and make decisions and management.

[0079] The following are the specific steps to implement the visualization module:

[0080] 1. Data visualization: Use charts and dashboards to display the load status of the cloud platform, monitoring indicator data, early warning information, etc. Provide an intuitive interface for users to view and analyze.

[0081] 2. Interactive functions: Provide users with interactive functions, such as filtering monitoring indicators, adjusting warning thresholds, viewing historical data, etc. Through interactive functions, users can manage and monitor the cloud platform more flexibly.

[0082] 3. System testing and optimization: After the system is implemented, testing and optimization are required. The accuracy and stability of the system can be verified by simulating different load scenarios and abnormal situations. Based on the test results, the system can be optimized and improved to improve performance and user experience. Specific embodiments

[0084] 1. Background: A cloud service provider manages a large number of server clusters. It aims to predict the server CPU usage in the future through a logistic regression model, and detect in real time whether the server is in an abnormal state, so as to schedule resources in advance or respond to potential service interruptions.

[0085] 2. Monitoring Data Collection

[0086] Use the Prometheus monitoring system and configure Node Exporter to collect key indicator data of the cloud server. The data includes CPU usage, memory usage, network traffic, and disk I / O recorded every 5 minutes.

[0087] 3. Data Preprocessing

[0088] Assume that the cleaned and normalized dataset contains the following simplified records:

[0089]

[0090]

[0091] 4. Logistic regression model construction and training

[0092] Initialization parameters: Set the initial values ​​to (β0=0,β1=1,β2=2,β3=3). Loss function: Use the cross entropy loss function. Gradient ascent method: Adjust the parameters by gradient ascent method to minimize the loss function. Iteration: Iterate 100 times or until the loss function converges. Validation and adjustment: Adjust the model parameters on the validation set to improve the prediction accuracy.

[0093] 5. Model Application and Anomaly Detection

[0094] The trained logistic regression model is deployed in TensorFlow Serving. For new data points in the real-time data stream, for example: timestamp: 2024-05-0115:00; CPU usage: 96%; memory usage: 90%; network traffic: 700Mbps; the model will calculate the anomaly probability P(y=1|x) based on the input features. Assuming that after the model calculation, the anomaly probability of this data point is 0.94, according to the set threshold (such as 0.9), the model will determine that the cloud platform load at this moment is abnormal.

[0095] 6. Real-time response

[0096] Once the model detects an anomaly, the automated response mechanism is immediately triggered, such as sending an early warning message to the operation and maintenance team, including the following content: "2024-05-0115:00, an abnormal cloud platform load was detected, and the CPU utilization rate was 96%. It is recommended to check the server resource status immediately." At the same time, the system automatically adjusts resources, increases server resources or adjusts task allocation to alleviate excessive load.

[0097] Key technical points of the invention and corresponding beneficial effects:

[0098] 1. An intelligent monitoring and alarm system for the operation status of a cloud platform based on artificial intelligence and Prometheus, characterized in that the system collects data through Prometheus, combines artificial intelligence technology to perform intelligent analysis and prediction of the operation status of the cloud platform, and realizes intelligent monitoring and alarm.

[0099] 2. The system according to claim 1, wherein the artificial intelligence model is trained by a deep learning algorithm and can automatically identify abnormal conditions during the operation of the cloud platform.

[0100] 3. The system according to claim 1 or 2, wherein the alarm information is sent to the administrator through a preset alarm channel so that the administrator can take corresponding processing measures in time.

[0101] The practicality of this solution is reflected in the following key aspects:

[0102] 1. Automated monitoring and instant feedback: By deploying Prometheus monitoring servers and related exporters, we can realize automated real-time monitoring of various load indicators of the cloud platform, quickly capture any abnormal changes, reduce the need for manual monitoring, and improve efficiency and response speed.

[0103] 2. Intelligent analysis and anomaly detection: Anomaly detection models built using machine learning algorithms such as logistic regression can predict possible future load anomalies based on historical data and real-time monitoring information. This is crucial for preventive maintenance and resource allocation, reducing the risk of system interruptions caused by unexpected load peaks.

[0104] 3. Automation of early warning and decision-making: The system can not only automatically send early warning information, but also automatically adjust cloud resources based on the early warning information, such as dynamically adding virtual machines, optimizing task scheduling, etc., which reduces the workload of operation and maintenance personnel and improves the utilization rate and overall performance of cloud resources.

[0105] In summary, this solution achieves effective control of cloud platform load and rapid response to exceptions through automated and intelligent monitoring and management strategies, greatly improving operation and maintenance efficiency and the reliability of cloud services, and has high practicality and application value.

[0106] This technical solution may have a wide range of application scenarios in the future, covering multiple industries and fields. The following are some specific application examples:

[0107] 1. Data centers and cloud service providers: Cloud service providers and large data centers can use this solution to monitor the health of their infrastructure in real time, automatically adjust resources to cope with sudden traffic or avoid service interruptions, and ensure high availability and customer satisfaction.

[0108] 2. Financial Technology: Financial institutions rely on stable and high-performance IT systems to process a large number of transactions. This solution can help automatically detect and prevent system overloads, ensure the stable operation of transaction systems, reduce transaction delays, and enhance customer trust.

[0109] E-commerce: E-commerce platforms face huge traffic pressure during promotional activities. This solution can monitor server load in real time, automatically allocate resources to cope with instantaneous traffic peaks, avoid website crashes, and ensure the shopping experience.

[0110] 3. Online education and remote work platform: During the epidemic, the demand for online education and remote work has surged. This solution can effectively monitor the load of video conferencing and online teaching platforms to ensure a smooth interactive experience even during high concurrency periods.

[0111] 4. Medical and health information system: The medical information system needs to process a large amount of patient data and imaging materials. This solution helps to maintain system stability, prevent service interruptions due to excessive load, and ensure the continuity of medical services.

[0112] From the above application scenarios, it can be seen that this solution, with its powerful real-time monitoring, intelligent analysis, and automated response capabilities, can be widely used in any scenario that requires efficient resource management, fault prevention, and optimization of service performance, providing strong support for digital transformation and intelligent operations.

[0113] Glossary

[0114] AI (Artificial Intelligence), also known as artificial intelligence in Chinese, is an important driving force for a new round of scientific and technological revolution and industrial transformation. It is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0115] Prometheus is an open source system monitoring and alerting toolkit. It pulls indicator data from configured targets based on a centralized server or exposes this data through short-term HTTP queries. It is mainly used for container monitoring and microservice monitoring. It saves all monitoring data as time series data and uses the PromQL query language for querying.

Claims

1. A cloud platform intelligent monitoring and early warning method based on AI and Prometheus, characterized in that: Prometheus is used to collect data, and artificial intelligence technology is used to intelligently analyze and predict the operating status of the cloud platform, thus achieving intelligent monitoring and alarming. The artificial intelligence technology is an artificial intelligence model trained through deep learning algorithms, which can automatically identify abnormal conditions during the operation of the cloud platform; the alarm is prompted through preset thresholds.

2. According to claim 1, a cloud platform intelligent monitoring and early warning method based on AI and Prometheus is characterized in that: The steps include: Monitoring data collection: First, you need to deploy Prometheus to monitor the server or cluster and configure related exporters to collect key indicator data on the cloud platform, such as CPU usage, memory usage, network traffic, and disk I / O. Then, configure the pull interval and storage strategy of Prometheus to ensure that Prometheus can run stably and continuously collect data, monitor and alert for abnormal situations. Data preprocessing: Data preprocessing is a crucial step in the data analysis process. Its goal is to convert raw monitoring data into a structured, high-quality data set suitable for subsequent intelligent analysis. Smart Analysis: Use machine learning or deep learning algorithms to analyze the preprocessed monitoring data and use the training set to train the model of the selected algorithm; Early warning and decision making: Based on the results of intelligent analysis, when it is predicted that the cloud platform load may be too high or abnormal indicators are detected, the early warning mechanism will be automatically triggered. The early warning information includes load status, abnormality type, and impact range, and relevant personnel will be notified via email, SMS or internal notification system.

Images