Operation and maintenance risk management method for cloud platform

Abstract

The invention discloses an operation and maintenance risk management method for a cloud platform, and relates to the technical field of operation and maintenance risk management, and the method comprises the following steps: S1, the operation and maintenance risk management method is applied to the cloud platform of multiple service items, and the overall configuration related data of the cloud platform and the data of each index in the operation process of the cloud platform are collected; and log data generated in the running process of the cloud platform is synchronously collected. The invention provides an operation and maintenance risk management method for a cloud platform, the operation and maintenance risk management method is applied to the cloud platform with multiple service items, and various index data generated in the operation process of the cloud platform are collected and backed up to provide data support for subsequent anomaly detection of the cloud platform. The objective of the invention is to avoid a false alarm condition that operation abnormity of a cloud platform is detected through a conventional multi-dimensional time series data abnormity detection method due to addition and removal operations of nodes of the cloud platform.

Description

Technical Field

[0001] The present invention relates to the technical field of operation and maintenance risk management, and in particular to an operation and maintenance risk management method for a cloud platform. Background Art

[0002] In recent years, with the rapid development of artificial intelligence and big data technologies, intelligent operation and maintenance management of platforms has gradually become an important way for enterprises to improve efficiency and reduce costs. The Chinese patent with application number 202211584439.4 discloses "an operation and maintenance management method, an operation and maintenance management platform and a storage medium, wherein the method includes: point-to-point access or batch addition of new servers; receiving, storing and parsing software offline installation packages; uniformly modifying the configuration of the target server, and installing the software offline installation package; obtaining the process status of all software on the server with elastic frequency, and judging whether the software is running normally. The present invention supports point-to-point and batch addition of new servers, and can be clustered after parsing the software offline installation package, and then uniformly monitor whether the software is in a normal operating state, and can perform corresponding processing on software with abnormal conditions according to a pre-established processing mechanism; obviously, the present invention does not need to add servers one by one, and does not need to install software to the server one by one. When the software has abnormal conditions, it can not only issue an alarm, but also provide more diverse services and improve the convenience of operation and maintenance services. ";

[0003] This existing technology only solves the problem of visual display of operation and maintenance data, and also uses the collected data to actively interfere with the operation and maintenance equipment, which solves the problem of operation and maintenance convenience to a certain extent. It does not take into account that when performing operation and maintenance risk management on a multi-service project cloud platform, not only should the cloud platform operation anomalies be detected through conventional multi-dimensional time series data anomaly detection methods, but also when an abnormality is detected in the cloud platform operation status, the specific situation corresponding to the cloud platform operation anomaly data should be determined by setting cloud platform alarm rules, and the cloud platform alarm rules should be supplemented in combination with the cloud platform operation anomaly data to improve the efficiency of cloud platform anomaly detection. Summary of the invention

[0004] The purpose of the present invention is to provide an operation and maintenance risk management method for a cloud platform to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for operation and maintenance risk management of a cloud platform, comprising the following steps:

[0006] S1: An operation and maintenance risk management method is applied to a cloud platform with multiple service projects, collecting the overall configuration-related data of the cloud platform and various indicator data during the operation of the cloud platform, and synchronously collecting the log data generated during the operation of the cloud platform;

[0007] S2: Based on the regularly collected time series data of various indicators during the operation of the cloud platform, the collected time series data of various indicators during the operation of the cloud platform are processed and converted into three-channel two-dimensional images in order to avoid false alarms caused by the addition and removal of cloud platform nodes, which are detected by conventional multi-dimensional time series data anomaly detection methods.

[0008] S3: Perform anomaly detection on cloud platform operation and maintenance through cloud platform operation and maintenance anomaly detection means based on thresholds and models, and build a cloud platform operation anomaly detection model. Use a web crawler to crawl the cloud platform server time series monitoring status data as sample data, build a cloud platform operation anomaly detection dataset based on the sample data, and use the cloud platform operation anomaly detection dataset to train the cloud platform operation anomaly detection model and verify its accuracy;

[0009] S4: Through the set alarm rules, corresponding alarm notification information is issued based on the abnormal detection results of cloud platform operation and maintenance, and the alarm notification information is stored in the data storage library built into the cloud platform in the form of logs.

[0010] Preferably, step S1 also includes obtaining the configuration and performance data of the cloud platform as a whole and each server by manual entry by the cloud platform administrator, and the real-time status of the cloud platform is periodically collected through a scheduled task, including various operating indicator data of the cloud platform, including the operating status data of each server, the operating status data of each service project, and the operating status data of the cloud host, wherein the server operating status data includes the hardware operating status data of the server infrastructure, wherein the service project operating status data includes the occupancy of infrastructure resources by various services and the operating status data of the service project itself, wherein the cloud host operating status data includes the overall and independent operating status data of all cloud hosts running in the cloud platform, and synchronously and periodically collects the log data generated during the operation of the cloud platform.

[0011] Preferably, step S2 further includes backing up the collected time series data of various indicators during the operation of the cloud platform based on the time series data of various indicators during the operation of the cloud platform, and processing the collected time series data of various indicators during the operation of the cloud platform to convert them into three-channel two-dimensional images in order to avoid false alarms caused by adding and removing cloud platform nodes through conventional multi-dimensional time series data anomaly detection methods. The specific operation is to divide the collected time series data of various indicators during the operation of the cloud platform into standard unit time periods in units of days, scale the various indicator data in each standard unit time to the interval [-1,1] by the max-min normalization method, and convert the processed indicator data into time series data in polar coordinates. The specific calculation formula is as follows:

[0012]

[0013] Where φ represents the cosine of the angle, x i represents the normalized indicator time series data, r represents the radius, t i represents a timestamp, N represents a standard unit time series, N x A constant factor representing the span of the regularized polar coordinate system.

[0014] Preferably, step S2 also includes mapping the time series data of each indicator to the interval [0,256] based on the sum angle relationship and difference angle relationship of the time series data of each indicator in the standard unit time under polar coordinates in order to maintain the combination relationship between the time series data of each indicator in the original cloud platform operation process, obtaining the corresponding GASF diagram and GADF diagram, and converting them into RGB color images, combining the time series data of each indicator, and realizing the conversion of the collected time series data of each indicator into a series of color two-dimensional images in units of days.

[0015] Preferably, step S3 also includes performing anomaly detection on the cloud platform operation and maintenance by means of a cloud platform operation and maintenance anomaly detection method based on a threshold and a model, wherein the model-based cloud platform operation and maintenance anomaly detection method specifically comprises building a cloud platform operation anomaly detection model based on a basic CNN network, introducing a ResNet50 residual network structure into the basic CNN network, and training all Layers of the ResNet50 residual network structure through the ImageNet data set, and after the training of the ResNet50 residual network structure is completed, adding a maximum pooling layer and a fully connected layer at the end of the ResNet50 residual network structure, and when the cloud platform operation anomaly detection model is subsequently trained, the parameters of the trained ResNet50 residual network structure part are frozen, thereby avoiding modifying the weights of the trained ResNet50 residual network structure part during the model training of the cloud platform operation anomaly detection model, and at the same time, the cloud platform operation anomaly detection model can adapt to the training data set more quickly, and the cross entropy loss function is selected as the loss function of the cloud platform operation anomaly detection model.

[0016] Preferably, step S3 also includes crawling cloud platform server time series monitoring status data as sample data through a web crawler, constructing a cloud platform operation anomaly detection data set based on the sample data, performing model training operations on the cloud platform operation anomaly detection model through the cloud platform operation anomaly detection data set, and performing accuracy verification operations on the trained cloud platform operation anomaly detection model. The threshold-based cloud platform operation and maintenance anomaly detection means is specifically based on the thresholds of various operation indicator data initially set by the cloud platform administrator, and the cloud platform users can adjust the relevant thresholds of the operation status data of their own personal related service items according to their own circumstances when using it, so as to detect the existing threshold anomalies in combination with the various operation indicator data obtained during the operation of the cloud platform, and send relevant alarm information of the threshold anomalies to the cloud platform users through the alarm service.

[0017] Preferably, step S4 also includes the cloud platform administrator setting different alarm rules according to the operation status of the cloud platform, and the management of the cloud platform alarm rules includes the alarm rule type, alarm rule conditions and alarm frequency. The cloud platform alarm rule types are divided into threshold anomaly detection alarms and model anomaly detection alarms, which correspond to the cloud platform operation and maintenance anomaly detection means based on thresholds and models. By calling the cloud platform operation and maintenance anomaly detection means based on thresholds and models, combined with the backup of the time series data of various indicators in the operation process of the cloud platform and the color two-dimensional images corresponding to the time series data of various indicators in the operation process of the cloud platform, the cloud platform operation status is detected for anomalies.

[0018] Preferably, step S4 also includes, when an abnormality is detected in the operation of the cloud platform, when the abnormal operation data of the cloud platform meets the set alarm rules, issuing alarm notification information of the corresponding alarm rules through cloud platform site notification, email sending and SMS notification, and marking the corresponding cloud platform abnormal operation data; otherwise, only cloud platform abnormal operation information is issued, and the cloud platform alarm rules are supplemented based on the cloud platform abnormal operation information, and the corresponding cloud platform abnormal operation data is marked, and the alarm notification information or cloud platform abnormal operation information is stored in the form of logs in the data storage library built into the cloud platform.

[0019] Compared with the prior art, the beneficial effects of the present invention at least include: the present invention proposes an operation and maintenance risk management method for a cloud platform, an operation and maintenance risk management method applied to a cloud platform of multiple service projects, collects and backs up various indicator data generated during the operation of the cloud platform, so as to provide data support for subsequent abnormality detection of the cloud platform, in order to avoid the false alarm of abnormal operation of the cloud platform detected by the conventional multi-dimensional time series data abnormality detection method due to the addition and removal of cloud platform nodes, the collected time series data of various indicators during the operation of the cloud platform are processed and converted into a three-channel two-dimensional image, and the cloud platform operation and maintenance is detected by combining the backup time series data of various indicators during the operation of the cloud platform and the color two-dimensional image corresponding to the time series data of various indicators during the operation of the cloud platform through a cloud platform operation and maintenance abnormality detection method based on a threshold and a model, and when the cloud platform operation is detected to be abnormal, when the cloud platform operation abnormality data meets the set alarm rule, the alarm notification information of the corresponding alarm rule is issued, otherwise only the cloud platform operation abnormality information is issued, and the cloud platform alarm rule is supplemented based on the cloud platform operation abnormality information. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a flow chart of an operation and maintenance risk management method for a cloud platform provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] See also Figure 1 The present invention provides a technical solution: a method for operation and maintenance risk management of a cloud platform, comprising the following steps:

[0023] S1: An operation and maintenance risk management method is applied to a cloud platform with multiple service projects, collecting the overall configuration-related data of the cloud platform and various indicator data during the operation of the cloud platform, and synchronously collecting the log data generated during the operation of the cloud platform;

[0024] S2: Based on the regularly collected time series data of various indicators during the operation of the cloud platform, the collected time series data of various indicators during the operation of the cloud platform are processed and converted into three-channel two-dimensional images in order to avoid false alarms caused by the addition and removal of cloud platform nodes, which are detected by conventional multi-dimensional time series data anomaly detection methods.

[0025] S3: Perform anomaly detection on cloud platform operation and maintenance through cloud platform operation and maintenance anomaly detection means based on thresholds and models, and build a cloud platform operation anomaly detection model. Use a web crawler to crawl the cloud platform server time series monitoring status data as sample data, build a cloud platform operation anomaly detection dataset based on the sample data, and use the cloud platform operation anomaly detection dataset to train the cloud platform operation anomaly detection model and verify its accuracy;

[0026] S4: Through the set alarm rules, corresponding alarm notification information is issued based on the abnormal detection results of cloud platform operation and maintenance, and the alarm notification information is stored in the data storage library built into the cloud platform in the form of logs.

[0027] Step S1 also includes obtaining the configuration and performance data of the cloud platform as a whole and each server through manual entry by the cloud platform administrator, and periodically collecting various operation indicator data of the cloud platform including the operation status data of each server, the operation status data of each service project and the operation status data of the cloud host through a scheduled task for the real-time status of the cloud platform, wherein the server operation status data includes the hardware operation status data of the server infrastructure, wherein the service project operation status data includes the occupation of infrastructure resources by each service and the operation status data of the service project itself, wherein the cloud host operation status data includes the overall and independent operation status data of all cloud hosts running in the cloud platform, and synchronously and periodically collects the log data generated during the operation of the cloud platform;

[0028] Step S2 also includes backing up the collected time series data of various indicators during the operation of the cloud platform based on the time series data of various indicators during the operation of the cloud platform, and processing the collected time series data of various indicators during the operation of the cloud platform to convert them into three-channel two-dimensional images in order to avoid false alarms caused by adding and removing cloud platform nodes through conventional multi-dimensional time series data anomaly detection methods. The specific operation is to divide the collected time series data of various indicators during the operation of the cloud platform into standard unit time periods in units of days, scale the various indicator data in each standard unit time to the [-1,1] interval through the max-min normalization method, and convert the processed indicator data into time series data in polar coordinates. The specific calculation formula is as follows:

[0029]

[0030] Where φ represents the cosine of the angle, x i represents the normalized indicator time series data, r represents the radius, t i represents a timestamp, N represents a standard unit time series, N x A constant factor representing the span of the regularized polar coordinate system;

[0031] Step S2 also includes mapping the time series data of various indicators to the interval [0,256] by the Gram angular field method based on the sum angle relationship and difference angle relationship of the time series data of various indicators in the standard unit time under polar coordinates in order to maintain the combination relationship between the time series data of various indicators in the original cloud platform operation process, obtaining the corresponding GASF diagram and GADF diagram, and converting them into RGB color images, combining the time series data of various indicators, and realizing the conversion of the collected time series data of various indicators into a series of color two-dimensional images in units of days;

[0032] Step S3 also includes performing anomaly detection on the cloud platform operation and maintenance by using a cloud platform operation and maintenance anomaly detection method based on a threshold and a model, wherein the cloud platform operation and maintenance anomaly detection method based on the model is specifically to build a cloud platform operation anomaly detection model based on a basic CNN network, and introduce a ResNet50 residual network structure into the basic CNN network, and use the ImageNet data set to train all the layers of the ResNet50 residual network structure, and after the training of the ResNet50 residual network structure is completed, a maximum pooling layer and a fully connected layer are added to the end of the ResNet50 residual network structure, and when the cloud platform operation anomaly detection model is subsequently trained, the parameters of the trained ResNet50 residual network structure part are frozen, thereby avoiding modifying the weights of the trained ResNet50 residual network structure part during the model training of the cloud platform operation anomaly detection model, and at the same time, the cloud platform operation anomaly detection model can adapt to the training data set faster, and the cross entropy loss function is selected as the loss function of the cloud platform operation anomaly detection model;

[0033] Step S3 also includes crawling cloud platform server time series monitoring status data as sample data through a network crawler, building a cloud platform operation anomaly detection data set based on the sample data, performing a model training operation on the cloud platform operation anomaly detection model through the cloud platform operation anomaly detection data set, and performing an accuracy verification operation on the trained cloud platform operation anomaly detection model. The threshold-based cloud platform operation and maintenance anomaly detection means is specifically based on the thresholds of various operation indicator data initially set by the cloud platform administrator, and the cloud platform users can adjust the relevant thresholds of the operation status data of their own personal related service items according to their own circumstances when using it, so as to detect the existing threshold anomalies in combination with the various operation indicator data obtained during the operation of the cloud platform and send relevant alarm information of the threshold anomalies to the cloud platform users through the alarm service;

[0034] Step S4 also includes that the cloud platform administrator sets different alarm rules according to the operation status of the cloud platform, and the management of the cloud platform alarm rules includes the alarm rule type, alarm rule conditions and alarm frequency. The cloud platform alarm rule types are divided into threshold anomaly detection alarms and model anomaly detection alarms, which correspond to the cloud platform operation and maintenance anomaly detection means based on thresholds and models. By calling the cloud platform operation and maintenance anomaly detection means based on thresholds and models, combined with the backup of the time series data of various indicators in the operation process of the cloud platform and the color two-dimensional images corresponding to the time series data of various indicators in the operation process of the cloud platform, the cloud platform operation status is detected abnormally;

[0035] Step S4 also includes, when an abnormality is detected in the operation of the cloud platform, when the abnormal operation data of the cloud platform meets the set alarm rules, issuing alarm notification information of the corresponding alarm rules through cloud platform station notification, email sending and SMS notification, and marking the corresponding cloud platform operation abnormal data; otherwise, only cloud platform operation abnormal information is issued, and the cloud platform alarm rules are supplemented based on the cloud platform operation abnormal information, and the corresponding cloud platform operation abnormal data is marked, and the alarm notification information or cloud platform operation abnormal information is stored in the form of logs in the built-in data storage library of the cloud platform.

[0036] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0037] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for operation and maintenance risk management of a cloud platform, characterized in that The following steps are involved: S1: An operation and maintenance risk management method is applied to a cloud platform with multiple service projects, collecting the overall configuration-related data of the cloud platform and various indicator data during the operation of the cloud platform, and synchronously collecting the log data generated during the operation of the cloud platform; S2: Based on the regularly collected time series data of various indicators during the operation of the cloud platform, the collected time series data of various indicators during the operation of the cloud platform are processed and converted into three-channel two-dimensional images in order to avoid false alarms caused by the addition and removal of cloud platform nodes, which are detected by conventional multi-dimensional time series data anomaly detection methods. S3: Perform anomaly detection on cloud platform operation and maintenance through cloud platform operation and maintenance anomaly detection means based on thresholds and models, and build a cloud platform operation anomaly detection model. Use a web crawler to crawl the cloud platform server time series monitoring status data as sample data, build a cloud platform operation anomaly detection dataset based on the sample data, and use the cloud platform operation anomaly detection dataset to train the cloud platform operation anomaly detection model and verify its accuracy; S4: Through the set alarm rules, corresponding alarm notification information is issued based on the abnormal detection results of cloud platform operation and maintenance, and the alarm notification information is stored in the data storage library built into the cloud platform in the form of logs.

2. The operation and maintenance risk management method for a cloud platform according to claim 1, characterized in that: The step S1 also includes obtaining the configuration and performance data of the cloud platform as a whole and each server by manually entering the data into the cloud platform by the cloud platform administrator, and periodically collecting the various operating indicator data of the cloud platform including the operating status data of each server, the operating status data of each service project and the operating status data of the cloud host through a scheduled task for the real-time status of the cloud platform, wherein the server operating status data includes the hardware operating status data of the server infrastructure, wherein the service project operating status data includes the occupancy of infrastructure resources by each service and the operating status data of the service project itself, wherein the cloud host operating status data includes the overall and independent operating status data of all cloud hosts running in the cloud platform, and synchronously and periodically collects the log data generated during the operation of the cloud platform.

3. The operation and maintenance risk management method for a cloud platform according to claim 1, characterized in that: The step S2 also includes backing up the collected time series data of various indicators during the operation of the cloud platform based on the time series data of various indicators during the operation of the cloud platform, and processing the collected time series data of various indicators during the operation of the cloud platform to convert them into a three-channel two-dimensional image in order to avoid false alarms caused by the addition and removal of cloud platform nodes through conventional multi-dimensional time series data anomaly detection methods. The time series data of various indicators during the operation of the cloud platform are processed and converted into a three-channel two-dimensional image. The specific operation is to divide the collected time series data of various indicators during the operation of the cloud platform into standard unit time periods in units of days, scale the various indicator data in each standard unit time to the [-1,1] interval through the max-min normalization method, and convert the processed indicator data into time series data in polar coordinates.

4. The operation and maintenance risk management method for a cloud platform according to claim 1, characterized in that: The step S2 also includes mapping the time series data of each indicator to the interval [0,256] through the Gram angular field method based on the sum angle relationship and difference angle relationship of the time series data of each indicator in the standard unit time under polar coordinates in order to maintain the combination relationship between the time series data of each indicator in the original cloud platform operation process, obtaining the corresponding GASF diagram and GADF diagram, and converting them into RGB color images, combining the time series data of each indicator, and realizing the conversion of the collected time series data of each indicator into a series of color two-dimensional images in units of days.

5. The operation and maintenance risk management method for a cloud platform according to claim 1, characterized in that: The step S3 also includes performing anomaly detection on the cloud platform operation and maintenance by using a cloud platform operation and maintenance anomaly detection method based on a threshold and a model, wherein the model-based cloud platform operation and maintenance anomaly detection method specifically includes building a cloud platform operation anomaly detection model based on a basic CNN network, introducing a ResNet50 residual network structure into the basic CNN network, and training all Layers of the ResNet50 residual network structure through the ImageNet data set, and after the training of the ResNet50 residual network structure is completed, adding a maximum pooling layer and a fully connected layer at the end of the ResNet50 residual network structure, and when the cloud platform operation anomaly detection model is subsequently trained, the parameters of the trained ResNet50 residual network structure part are frozen, thereby avoiding modifying the weights of the trained ResNet50 residual network structure part during the model training of the cloud platform operation anomaly detection model, and at the same time, the cloud platform operation anomaly detection model can adapt to the training data set more quickly, and the cross entropy loss function is selected as the loss function of the cloud platform operation anomaly detection model.

6. The operation and maintenance risk management method for a cloud platform according to claim 1, characterized in that: The step S3 also includes crawling cloud platform server time series monitoring status data as sample data through a web crawler, constructing a cloud platform operation anomaly detection data set based on the sample data, performing model training operations on the cloud platform operation anomaly detection model through the cloud platform operation anomaly detection data set, and performing accuracy verification operations on the trained cloud platform operation anomaly detection model. The threshold-based cloud platform operation and maintenance anomaly detection method is specifically based on the thresholds of various operation indicator data initially set by the cloud platform administrator, and the cloud platform users can adjust the relevant thresholds of the operation status data of their own personal related service items according to their own circumstances when using it, so as to detect the existing threshold anomalies in combination with the various operation indicator data obtained during the operation of the cloud platform, and send relevant alarm information of the threshold anomalies to the cloud platform users through the alarm service.

7. The operation and maintenance risk management method for a cloud platform according to claim 1, characterized in that: The step S4 also includes that the cloud platform administrator sets different alarm rules according to the operation status of the cloud platform, and the management of the cloud platform alarm rules includes the alarm rule type, alarm rule conditions and alarm frequency. The cloud platform alarm rule types are divided into threshold anomaly detection alarms and model anomaly detection alarms, which correspond to the cloud platform operation and maintenance anomaly detection means based on thresholds and models. By calling the cloud platform operation and maintenance anomaly detection means based on thresholds and models, combined with the backup of the time series data of various indicators in the operation process of the cloud platform and the color two-dimensional images corresponding to the time series data of various indicators in the operation process of the cloud platform, the cloud platform operation status is detected for anomalies.

8. The operation and maintenance risk management method for a cloud platform according to claim 1, characterized in that: The step S4 also includes, when an abnormality is detected in the operation of the cloud platform, when the abnormal operation data of the cloud platform meets the set alarm rules, issuing an alarm notification information of the corresponding alarm rule through the cloud platform station notification, email sending and SMS notification, and marking the corresponding cloud platform operation abnormal data; otherwise, only the cloud platform operation abnormal information is issued, and the cloud platform alarm rules are supplemented based on the cloud platform operation abnormal information, and the corresponding cloud platform operation abnormal data is marked, and the alarm notification information or the cloud platform operation abnormal information is stored in the cloud platform built-in data storage library in the form of logs.

Images