Prometheus-based time sequence anomaly detection deployment method and device

Abstract

The invention provides a Prometheus-based time sequence anomaly detection deployment method and apparatus. The method comprises the steps of pulling real-time time sequence data, which conforms to joint features in a preset time period, of target equipment from a Prometheus system; and loading a model parameter file corresponding to the joint feature into a deep learning model, obtaining prediction data of the target equipment by using the real-time time sequence data and the deep learning model, and determining an abnormal time period of the target equipment by using the real-time time sequence data, the prediction data and a threshold value. According to the method, time step length self-adaption is carried out according to original data through a single model, data of different time step lengths are detected, the detection precision of abnormal data in the system is greatly improved, and in addition, through an efficient model parameter file version, the detection precision of the abnormal data in the system is greatly improved when the basic model is pre-trained. The method can achieve the quick fine adjustment of the model under the condition of few samples, and can achieve the elastic replacement of an anomaly detection algorithm based on a wide-availability deployment scheme.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of network technology, and in particular, to a method and device for deploying time series anomaly detection based on Prometheus. Background Art

[0002] With the development of network and information technology, the complexity and scale of computer systems are increasing. The abnormal warning and root cause location of key performance indicators of the system have become important tasks in operation and maintenance management. Traditional monitoring systems usually rely on simple threshold settings and cannot accurately identify complex system anomalies, especially in large-scale time series data processing.

[0003] As an open source time series data collection tool, Prometheus is widely used in various monitoring scenarios and can efficiently collect system performance indicators. However, Prometheus itself does not have complex anomaly detection functions. Its core function is to collect and display time series data, and it cannot directly provide high-precision anomaly detection algorithms.

[0004] Application Contents

[0005] This application describes a Prometheus-based time series anomaly detection deployment method and device, which can solve the above technical problems.

[0006] According to a first aspect, a Prometheus-based time series anomaly detection deployment method is provided, the method comprising:

[0007] Pull the real-time time series data of the target device that meets the joint feature within a preset time period from the Prometheus system, where the joint feature includes multiple independent data features, each of which includes a time step, which is a data time interval determined according to the length of the preset time period;

[0008] The model parameter file corresponding to the joint feature is loaded into the deep learning model, and the prediction data of the target device is obtained by using the real-time time series data and the deep learning model. The deep learning model parameter file is a model parameter file obtained by training the deep learning model using the real-time time series data when the deep learning model is trained for the first time. When it is not the first training, it is a parameter model file trained by the real-time time series data and the timing time series data collected by the timing task. The timing time series data is the operation data of the target device that meets the joint feature collected by the timing task;

[0009] The abnormal time period of the monitoring target is determined by using the real-time time series data, the predicted data and the threshold.

[0010] Based on the above further embodiment, the use of the real-time time series data and the deep learning model to obtain the prediction data of the target device and the prediction data of the monitoring indicator specifically includes:

[0011] Calculating statistical data on the timing time series data to obtain configuration item data;

[0012] The real-time time series data is processed using the configuration item data to obtain input data; the input data is input into the deep learning model to obtain predicted data of the target device.

[0013] Based on the above further embodiment, the use of the configuration item data to process the real-time time series data to obtain input data specifically includes:

[0014] Calculate the standard score of the real-time time series data at each time point by using the variance and mean in the configuration item data to obtain a standard score sequence in the input data;

[0015] The data difference between each adjacent time point in the real-time time series data is calculated to obtain a result sequence in the input data.

[0016] Based on the above further embodiment, the use of the real-time time series data, the predicted data and the threshold to determine the abnormal period of the monitoring target specifically includes:

[0017] Calculate the square difference between the real-time time series data and the predicted data at the same time point to obtain an abnormal score sequence;

[0018] Obtaining an abnormality mark sequence according to a comparison result between the abnormality score sequence and a threshold, wherein the threshold is calculated by using real-time time series data;

[0019] An abnormal time period of the target device is determined according to the abnormal mark sequence.

[0020] Based on the above further embodiment, the deep learning model parameter file is a model parameter file obtained by training the deep learning model with regular time series data, and specifically includes:

[0021] Calculating the standard scores of the timing time series data at each time point using the variance and mean in the configuration item data to obtain a standard score sequence of input data for training;

[0022] Calculating the data difference of each adjacent time point in the timing time series data to obtain a result sequence of input data for training;

[0023] The training input data is input into the deep learning model for training to obtain a deep learning model parameter file of the trained deep learning model, and the deep learning model parameter file is saved in the object storage according to the joint features to which it belongs.

[0024] Based on the above further embodiment, the threshold is calculated by real-time time series data, specifically including:

[0025] Inputting the training input data as real-time time series data into the deep learning model to obtain training prediction data;

[0026] Calculate the square difference between the training prediction data and the real-time time series data at the same time point to obtain a training anomaly score sequence;

[0027] The threshold is calculated according to the statistical data of the training anomaly score sequence.

[0028] Based on the above further embodiment, the method further includes:

[0029] The scheduled task is used to regularly collect scheduled time series data, regularly train the deep learning model, and update configuration item data.

[0030] According to a second aspect, a Prometheus-based time series anomaly detection deployment device is provided, including:

[0031] The acquisition module is used to pull the real-time time series data of the target device that meets the joint characteristics within a preset time period from the Prometheus system. The joint characteristics include multiple independent data characteristics, and the independent data characteristics include a time step, which is a data time interval determined according to the length of the preset time period;

[0032] A processing module, used for loading the model parameter file corresponding to the joint feature into the deep learning model, and obtaining the prediction data of the target device by using the real-time time series data and the deep learning model, wherein the deep learning model parameter file is a model parameter file obtained by training the deep learning model using the real-time time series data when the deep learning model is first trained, and when it is not the first training, it is a parameter model file trained by the real-time time series data and the timing time series data collected by the timing task, and the timing time series data is the operation data of the target device that meets the joint feature collected by the timing task;

[0033] The determination module is used to determine the abnormal time period of the monitoring target by using the real-time time series data, the predicted data and the threshold.

[0034] Based on the above further embodiment, the processing module is specifically used to calculate statistical data on the timing time series data to obtain configuration item data;

[0035] The real-time time series data is processed using the configuration item data to obtain input data; the input data is input into the deep learning model to obtain predicted data of the target device.

[0036] Based on the above further embodiment, the processing module is specifically used to calculate the standard score of the real-time time series data at each time point by using the variance and mean in the configuration item data to obtain the standard score sequence in the input data;

[0037] The data difference between each adjacent time point in the real-time time series data is calculated to obtain a result sequence in the input data.

[0038] Based on the above further embodiment, the determination module is specifically used to calculate the square difference between the real-time time series data and the predicted data at the same time point to obtain an abnormal score sequence;

[0039] Obtaining an abnormality mark sequence according to a comparison result between the abnormality score sequence and a threshold, wherein the threshold is calculated by using real-time time series data;

[0040] An abnormal time period of the target device is determined according to the abnormal mark sequence.

[0041] Based on the above further embodiment, the processing module is specifically used to calculate the standard score of the timing time series data at each time point by using the variance and mean in the configuration item data to obtain the standard score sequence of the input data for training;

[0042] Calculating the data difference of each adjacent time point in the timing time series data to obtain a result sequence of input data for training;

[0043] The training input data is input into the deep learning model for training to obtain a deep learning model parameter file of the trained deep learning model, and the deep learning model parameter file is saved in the object storage according to the joint features to which it belongs.

[0044] Based on the above further embodiment, the processing module is specifically used to input the training input data as real-time time series data into the deep learning model to obtain training prediction data;

[0045] Calculate the square difference between the training prediction data and the real-time time series data at the same time point to obtain a training anomaly score sequence;

[0046] The threshold is calculated according to the statistical data of the training anomaly score sequence.

[0047] Based on the above further embodiment, the processing module is specifically used for the timing task to regularly collect timing time series data, regularly train the deep learning model and update configuration item data.

[0048] According to a third aspect, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by one or more processors, the Prometheus-based time series anomaly detection deployment method as described in any one of the above technical solutions is implemented.

[0049] According to a fourth aspect, an electronic device is provided, comprising a memory and one or more processors, wherein a computer program is stored on the memory, and when the computer program is executed by the one or more processors, a Prometheus-based time series anomaly detection deployment method as described in any one of the above technical solutions is implemented.

[0050] In the above-mentioned system and method provided in the embodiments of this specification, a single model can be used to realize time step adaptation according to the original data, and detect data of different time steps, thereby greatly improving the detection accuracy of abnormal data in the system. In addition, through an efficient model parameter file version, when pre-training the basic model, the model can be quickly fine-tuned in the case of few samples. In addition, based on the deployment scheme with wide availability, flexible replacement of anomaly detection algorithms can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A system schematic diagram of a time series anomaly detection system based on Prometheus provided in an embodiment of this specification is shown;

[0053] Figure 2 A flow chart of a method for deploying time series anomaly detection based on Prometheus provided in an embodiment of this specification is shown;

[0054] Figure 3 A flow chart of a method for deploying time series anomaly detection based on Prometheus provided in an embodiment of this specification is shown;

[0055] Figure 4A flow chart of a method for deploying time series anomaly detection based on Prometheus provided in an embodiment of this specification is shown;

[0056] Figure 5 A schematic diagram of a Prometheus-based time series anomaly detection deployment device provided in an embodiment of this specification is shown. DETAILED DESCRIPTION

[0057] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0058] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0059] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0060] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "plurality" means two or more.

[0061] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0062] Prometheus is an open source monitoring and alarm system based on a time series database. It captures the status of the target device through the exporter interface and collects the performance indicator data of the target device. It cannot provide high-precision time series anomaly detection.

[0063] Figure 1The following is an architecture diagram of a time series anomaly detection system based on Prometheus provided in an embodiment of the present invention. The anomaly detection system is divided into six layers, wherein the operating environment layer is used to provide target devices, including host devices, virtual machines and servers; the data sampling layer is used to monitor the key business indicator data of each target device through Prometheus; the data storage layer is used to store key business data in a MySQL database, and to store trained model parameters through the object storage iHarbor system; the business layer is used to perform anomaly detection on key business indicator data and refresh model parameter files regularly; the algorithm layer is used to provide a deep learning network model; and the display layer is used to provide a visual display of anomaly detection results.

[0064] For example, Figure 2 and Figure 3 This is a flow chart of a method for deploying time series anomaly detection provided by an embodiment of the present application, which is as follows:

[0065] Users query KPI indicator data within a specified time period through IP, pull corresponding KPI indicator data through the Prometheus system, and calculate the time step.

[0066] Specifically, the user sets the monitoring time range of the target device according to business needs, and determines the time step corresponding to different monitoring time ranges according to preset rules. For example, if the monitoring time range is 1 hour, the corresponding time step can be 30 seconds, 1 minute or 2 minutes, etc. If the monitoring time range is 1 day, the corresponding time step can be 15 minutes, 30 minutes or 60 minutes.

[0067] Call the model parameter file of the corresponding KPI indicator to generate anomaly scores for each time point.

[0068] Specifically, when the model parameters corresponding to the KPI indicator are obtained for the first time, the obtained KPI indicator data is statistically processed to obtain the mean and variance of the KPI indicator data, and the mean and variance are used as configuration item data. The Z-Score operation and the Diff operation are used for the real-time time series data to obtain the Z-Score result and the Diff result set, where the Z-Score operation is as follows: z = (x-μ) / σ, z represents the z-score result, x is the KPI indicator data, μ is the mean Mean of the above KPI indicator data, and σ is the variance Std of the above KPI indicator data. The Diff operation is as follows: Res[i] = arr[i+1]-arr[i], Res is the result set after the Diff operation, and arr is the KPI indicator data set, that is, the data difference between adjacent time points in the KPI indicator data is used to obtain the result set Res[i].

[0069] In the subsequent training process, the scheduled task is used to obtain KPI indicator data regularly, and the configuration item data is automatically updated at a scheduled time. The updated configuration item data is used to train the deep learning network model to improve the accuracy of the model's subsequent predictions. The configuration item data is stored in MySQL, and the parameter model file is saved using object storage iHarbor to improve the efficiency of model calls during detection.

[0070] The business process processing of the present invention adopts independent data features and joint data features, wherein the independent data features represent a specific indicator corresponding to a single query, and the joint features are the integration of independent data features, which can uniquely determine a query information.

[0071] In this embodiment, the KPI indicator is one of the independent feature data in the business process, and Prometheus is used to pull the KPI indicator data of different time steps. The KPI indicator data is aligned according to the time step, so the time step is also one of the independent feature data in the business process. In the iHarbor object storage, the KPI indicator and the time step are stored as joint features of the model parameter file. When the user performs the first anomaly detection on the specified joint feature, MySQL and iHarbor do not match the model training parameter file corresponding to the joint feature, pull the business data that meets the joint feature in the specified time period, calculate the configuration item data of the KPI indicator data through data preprocessing Z-Score, store the configuration item data of the joint feature in MySQL, and save the generated model parameter file in the iHarbor object storage after the training.

[0072] In the above process, the specified time period may be too short, which may lead to inaccurate detection results. In this embodiment, the method of scheduled tasks is used to process. The first scheduled task includes using the historical time series data of the KPI indicators in the past 7 days to calculate the configuration item data and update it. The second scheduled task includes using the historical time series data of the KPI indicators and the configuration item data in the past 7 days to regularly train the model parameter file corresponding to the joint feature to ensure the accuracy of the model parameters and the accuracy of the anomaly detection results, and update it to the object storage for replacement according to the joint features.

[0073] The square difference formula is used to generate the abnormal score at each time point for the actual value of the KPI indicator data at each time point and the predicted value obtained through the algorithm model. Among them, score i is the abnormal score at time point i, y i is the actual value, is the predicted value.

[0074] The thresholds automatically calculated based on the anomaly scores at each time point are compared, and the time points exceeding the threshold are marked as abnormal.

[0075] Specifically, the automatic threshold calculation formula is threshold=μ_score+3*σ_score, where threshold is the threshold, μ_score is the mean of the anomaly score, and σ_score is the variance of the anomaly score. When the model parameters corresponding to the joint feature are not used for the first time, the mean of the anomaly score and the variance of the anomaly score are obtained from MysQL, where the mean of the anomaly score and the variance of the anomaly score are calculated by the KPI indicator data obtained by the scheduled task. When the threshold threadhold is -1, it means that the model parameters corresponding to the joint feature are used for the first time, and the threshold in the configuration item is not set. At this time, it can be configured manually or by the automatic threshold calculation formula. The manually set threshold is saved in the MySQL data table. If the automatic threshold calculation formula is used, the threshold is calculated using the automatic threshold calculation formula.

[0076] The anomaly score set is compared with the threshold to form a label sequence, where 0 in the label sequence represents no anomaly at that time point, and 1 represents an anomaly at that time point.

[0077] Abnormal situations are displayed in a visual form through the system front end.

[0078] According to the continuous abnormal marks in the abnormal mark sequence, different color intervals are used to represent different KPI sequence abnormalities in the ECharts line chart information of the front-end system, highlighting the abnormal time period information, so that operation and maintenance personnel can more intuitively perform abnormal screening and root cause location.

[0079] The present invention uses Prometheus to store and pull time series data, uses MySQL to store business data, and uses iHarbor objects to store model parameter files. By saving data from different processes separately, detection data and business data are isolated, and the system structure definition is clearer. At the same time, the coupling use of these mature technical components provides a unique solution for the system deployment of anomaly detection.

[0080] like Figure 4 The flowchart of a time series anomaly detection deployment method based on Prometheus is shown, including the following steps:

[0081] 110. Pull the real-time time series data of the target device that meets the joint characteristics within the preset time period from the Prometheus system. The joint characteristics include multiple independent data characteristics, and the independent data characteristics include a time step. The time step is a data time interval determined according to the length of the preset time period.

[0082] Specifically, the joint feature may also include multiple independent data features. The independent data feature may be a monitoring indicator. The monitoring indicator may be performance data of hardware or software, detection KPI type, and may also include the type of target device. The type of target device may be a host node, server or virtual machine, etc.

[0083] The time step is determined according to the length of the preset time period. According to the preset rules, the time step corresponding to different monitoring time ranges is determined. For example, if the monitoring time range is 1 hour, the corresponding time step can be 30 seconds, 1 minute or 2 minutes, etc. If the monitoring time range is 1 day, the corresponding time step can be 15 minutes, 30 minutes or 60 minutes.

[0084] 120. Load the model parameter file corresponding to the joint feature into the deep learning model, and use the real-time time series data and the deep learning model to obtain the prediction data of the target device. The deep learning model parameter file is the model parameter file obtained by training the deep learning model with the real-time time series data when the deep learning model is trained for the first time. When it is not the first training, it is a parameter model file trained with the real-time time series data and the timed time series data collected by the scheduled task. The timed time series data is the operating data of the target device that meets the joint features collected by the scheduled task.

[0085] Specifically, this embodiment does not limit the specific deep learning model.

[0086] Specifically, statistical data is calculated on the timing time series data to obtain configuration item data;

[0087] Use the configuration item data to process the real-time time series data to obtain input data;

[0088] Feed the input data into the deep learning model and get the predicted data of the target device.

[0089] Specifically, the configuration items mainly include business data needed in a query, such as historical mean variance data, query time range, joint feature data, threshold setting marks corresponding to joint features, etc.

[0090] Among them, the standard score of the real-time time series data at each time point is calculated by using the variance and mean in the configuration item data to obtain the standard score sequence in the input data;

[0091] Calculate the data difference between each adjacent time point in the real-time time series data to obtain the result sequence in the input data.

[0092] 130. Use real-time time series data, forecast data and thresholds to determine abnormal periods of target equipment.

[0093] Specifically, the square difference between the real-time time series data and the predicted data at the same time point is calculated to obtain the anomaly score sequence;

[0094] The abnormality mark sequence is obtained based on the comparison result between the abnormality score sequence and the threshold, and the threshold is calculated by the timing time series data;

[0095] According to the abnormal mark sequence, determine the abnormal period of the target device.

[0096] Specifically, the mean μ in the configuration item data and the variance σ in the configuration item data are input into the automated threshold calculation formula threshold=μ_score+3*σ_score to obtain the threshold threshold, wherein the mean μ in the configuration item data is obtained by calculating the mean of the timing series data, and the variance σ in the configuration item data is obtained by calculating the variance of the timing series data.

[0097] Specifically, the deep learning model parameter file is stored in the first database, where the first database is an object storage, such as iHarbor, and the configuration item data is stored in the second database, which can be MySQL. The second database is used to store data information of monitoring indicators, namely configuration item information.

[0098] Specifically, using the variance and mean in the configuration item data, the standard score of the real-time time series data at each time point is calculated to obtain a standard score sequence of the input data for training;

[0099] Calculate the data difference of each adjacent time point in the real-time time series data to obtain the result sequence of input data for training;

[0100] The training input data is input into the deep learning model for training, and a deep learning model parameter file of the trained deep learning model is obtained.

[0101] Specifically, the threshold is calculated through real-time time series data, including:

[0102] Inputting the training input data as real-time time series data into the deep learning model to obtain training prediction data;

[0103] Calculate the square difference between the training prediction data and the real-time time series data at the same time point to obtain a training anomaly score sequence;

[0104] The threshold is calculated according to the statistical data of the training anomaly score sequence.

[0105] Additionally, the thresholds can be manually configured by the user.

[0106] Specifically, the scheduled task is used to regularly collect scheduled time series data, regularly train the deep learning model, and update configuration item data.

[0107] In the above method provided in the embodiment of this specification, a single model can be used to adapt the time step according to the original data, detect data of different time steps, and greatly improve the detection accuracy of abnormal data in the system. In addition, through an efficient model parameter file version, when pre-training the basic model, the model can be quickly fine-tuned in the case of few samples, and the algorithm can be flexibly replaced. In addition, based on the deployment solution with wide availability, the elastic replacement of the anomaly detection algorithm can be achieved.

[0108] like Figure 5 A schematic diagram of a Prometheus-based time series anomaly detection deployment device is shown, including:

[0109] The acquisition module is used to pull the real-time time series data of the target device that meets the joint characteristics within a preset time period from the Prometheus system. The joint characteristics include multiple independent data characteristics, and the independent data characteristics include a time step, which is a data time interval determined according to the length of the preset time period;

[0110] A processing module, used for loading the model parameter file corresponding to the joint feature into the deep learning model, and obtaining the prediction data of the target device by using the real-time time series data and the deep learning model, wherein the deep learning model parameter file is a model parameter file obtained by training the deep learning model using the real-time time series data when the deep learning model is first trained, and when it is not the first training, it is a parameter model file trained by the real-time time series data and the timing time series data collected by the timing task, and the timing time series data is the operation data of the target device that meets the joint feature collected by the timing task;

[0111] The determination module is used to determine the abnormal time period of the target device by using the real-time time series data, the predicted data and a threshold.

[0112] Based on the above further embodiment, the processing module is specifically used to calculate statistical data on the timing time series data to obtain configuration item data;

[0113] Use configuration item data to process real-time time series data to obtain input data; input data into the deep learning model to obtain predicted data for the target device.

[0114] Based on the above further embodiment, the processing module is specifically used to calculate the standard score of the real-time time series data at each time point using the variance and mean in the configuration item data to obtain the standard score sequence in the input data;

[0115] The data difference between each adjacent time point in the real-time time series data is calculated to obtain a result sequence in the input data.

[0116] Based on the above further embodiment, the determination module is specifically used to determine the abnormal period of the monitoring target by using the real-time time series data and the prediction data, and specifically includes:

[0117] Calculate the square difference between the real-time time series data and the predicted data at the same time point to obtain the anomaly score sequence;

[0118] The abnormality mark sequence is obtained based on the comparison result between the abnormality score sequence and the threshold, and the threshold is calculated by real-time time series data;

[0119] According to the abnormal mark sequence, determine the abnormal period of the target device.

[0120] Based on the above further embodiment, the processing module is specifically used to input the training input data as real-time time series data into the deep learning model to obtain training prediction data;

[0121] Calculate the square difference between the training prediction data and the real-time time series data at the same time point to obtain a training anomaly score sequence;

[0122] The threshold is calculated according to the statistical data of the training anomaly score sequence.

[0123] Based on the above further embodiment, the processing module is specifically used to input the training input data into the deep learning model to obtain the training prediction data;

[0124] Calculate the square difference between the training prediction data and the timing time series data at the same time point to obtain the training anomaly score sequence;

[0125] The threshold is calculated according to the statistical data of the training anomaly score sequence.

[0126] Based on the above further embodiment, the processing module is specifically used for the scheduled task to regularly collect scheduled time series data, regularly train the deep learning model and update the configuration item data.

[0127] In the above system provided in the embodiment of the present specification, a single model can be used to realize time step adaptation according to the original data, and detect data of different time steps, which greatly improves the detection accuracy of abnormal data in the system. In addition, through an efficient model parameter file version, when pre-training the basic model, the model can be quickly fine-tuned in the case of few samples. In addition, based on the deployment solution with wide availability, flexible replacement of anomaly detection algorithms can be achieved.

[0128] The present invention also provides a computer storage medium, on which a computer program is stored. When the computer program is executed by one or more processors, the Prometheus-based time series anomaly detection deployment method as described in any one of the above technical solutions is implemented.

[0129] The present invention also provides an electronic device, comprising a memory and one or more processors, wherein a computer program is stored on the memory, and when the computer program is executed by the one or more processors, the Prometheus-based time series anomaly detection deployment method as described in any one of the above technical solutions is implemented.

[0130] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation methods of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the scope of protection of the present application.

Claims

1. A time series anomaly detection deployment method based on Prometheus, characterized in that: The method comprises: Pull the real-time time series data of the target device that meets the joint feature within a preset time period from the Prometheus system, where the joint feature includes multiple independent data features, each of which includes a time step, which is a data time interval determined according to the length of the preset time period; The model parameter file corresponding to the joint feature is loaded into the deep learning model, and the prediction data of the target device is obtained by using the real-time time series data and the deep learning model. The deep learning model parameter file is a model parameter file obtained by training the deep learning model using the real-time time series data when the deep learning model is trained for the first time. When it is not the first training, it is a parameter model file trained by the real-time time series data and the timing time series data collected by the timing task. The timing time series data is the operation data of the target device that meets the joint feature collected by the timing task; The abnormal time period of the monitoring target is determined by using the real-time time series data, the predicted data and the threshold.

2. The method according to claim 1, characterized in that The using the real-time time series data and the deep learning model to obtain the prediction data of the target device and the prediction data of the monitoring indicator specifically includes: Calculating statistical data on the timing time series data to obtain configuration item data; The real-time time series data is processed using the configuration item data to obtain input data; the input data is input into the deep learning model to obtain predicted data of the target device.

3. The method according to claim 2, characterized in that The using the configuration item data to process the real-time time series data to obtain input data specifically includes: Calculate the standard score of the real-time time series data at each time point by using the variance and mean in the configuration item data to obtain a standard score sequence in the input data; The data difference between each adjacent time point in the real-time time series data is calculated to obtain a result sequence in the input data.

4. The method according to claim 1, characterized in that The determining of the abnormal time period of the monitoring target by using the real-time time series data, the prediction data and the threshold value specifically includes: Calculate the square difference between the real-time time series data and the predicted data at the same time point to obtain an abnormal score sequence; According to the comparison result of the abnormal score sequence and the threshold, an abnormal mark sequence is obtained, which is calculated by real-time time series data; An abnormal time period of the target device is determined according to the abnormal mark sequence.

5. The method according to claim 2, characterized in that: The deep learning model parameter file is a model parameter file obtained by training the deep learning model with regular time series data, and specifically includes: Calculating the standard scores of the timing time series data at each time point using the variance and mean in the configuration item data to obtain a standard score sequence of input data for training; Calculating the data difference of each adjacent time point in the timing time series data to obtain a result sequence of input data for training; The training input data is input into the deep learning model for training to obtain a deep learning model parameter file of the trained deep learning model, and the deep learning model parameter file is saved in the object storage according to the joint features to which it belongs.

6. The method according to claim 4, characterized in that The threshold is calculated based on real-time time series data, including: Inputting the training input data as real-time time series data into the deep learning model to obtain training prediction data; Calculate the square difference between the training prediction data and the real-time time series data at the same time point to obtain a training anomaly score sequence; The threshold is calculated according to the statistical data of the training anomaly score sequence.

7. The method according to claim 2, characterized in that The method further comprises: The scheduled task is used to regularly collect scheduled time series data, regularly train the deep learning model, and update configuration item data.

8. A time series anomaly detection deployment device based on Prometheus, characterized in that: include: The acquisition module is used to pull the real-time time series data of the target device that meets the joint feature within a preset time period from the Prometheus system. The joint feature includes multiple independent data features, and the independent data features include a time step, which is a data time interval determined according to the length of the preset time period; A processing module, used for loading the model parameter file corresponding to the joint feature into the deep learning model, and obtaining the prediction data of the target device by using the real-time time series data and the deep learning model, wherein the deep learning model parameter file is a model parameter file obtained by training the deep learning model using the real-time time series data when the deep learning model is first trained, and when it is not the first training, it is a parameter model file trained by the real-time time series data and the timing time series data collected by the timing task, and the timing time series data is the operation data of the target device that meets the joint feature collected by the timing task; The determination module is used to determine the abnormal time period of the target device by using the real-time time series data, the predicted data and a threshold.

9. A computer storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the Prometheus-based time series anomaly detection deployment method as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The method comprises a memory and one or more processors, wherein a computer program is stored in the memory, and when the computer program is executed by the one or more processors, the method for deploying time series anomaly detection based on Prometheus is implemented as described in any one of claims 1 to 7.

Images