A prometheus-based time series anomaly detection deployment method and device

Abstract

The application provides a Prometheus-based time series anomaly detection deployment method and device, comprising: pulling real-time time series data of a target device meeting a joint feature in a preset time period from a Prometheus system; loading a model parameter file corresponding to the joint feature into a deep learning model, obtaining predicted data of the target device by using the real-time time series data and the deep learning model, and determining an abnormal time period of the target device by using the real-time time series data, the predicted data and a threshold value.The application realizes time step adaptation according to original data by a single model, detects data of different time steps, greatly improves the detection accuracy of abnormal data in the system, in addition, by using an efficient model parameter file version, the pre-training of the base model can be realized in a small sample situation, in addition, based on a widely available deployment scheme, the elastic replacement of the anomaly detection algorithm can be realized.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of network, and in particular to a Prometheus-based time series anomaly detection deployment method and device. BACKGROUND

[0002] With the development of network and information technology, the complexity and scale of computer systems are growing, and the abnormal early warning and root cause positioning of key performance indicators of the system have become an important task in operation and management. Traditional monitoring systems usually rely on simple threshold setting, which cannot accurately identify complex system anomalies, especially in large-scale time series data processing.

[0003] Prometheus, as an open-source time series data collection tool, is widely used in various monitoring scenarios and can efficiently collect system performance indicators. However, Prometheus itself does not have complex anomaly detection functions, and its core function is to collect and display time series data, which cannot directly provide high-precision anomaly detection algorithms.

[0004] SUMMARY

[0005] The present application describes a Prometheus-based time series anomaly detection deployment method and device, which can solve the above technical problems.

[0006] According to a first aspect, a Prometheus-based time series anomaly detection deployment method is provided, the method comprising:

[0007] Pulling real-time time series data of a target device in a preset time period from a Prometheus system, the real-time time series data meeting a joint feature, the joint feature comprising a plurality of independent data features, the independent data features including a time step, the time step being a data time interval determined according to the length of the preset time period;

[0008] Loading a model parameter file corresponding to the joint feature into a deep learning model, using the real-time time series data and the deep learning model to obtain predicted data of the target device, the deep learning model parameter file being a model parameter file obtained by training the deep learning model for the first time using the real-time time series data, and the non-first training being a parameter model file trained by the real-time time series data and the timing time series data collected by a timing task, the timing time series data being running data of the target device meeting the joint feature collected by the timing task;

[0009] Using the real-time time series data, the predicted data and a threshold value to determine an abnormal period of the monitoring target.

[0010] Based on the further embodiment, the predicted data of the target device is obtained by using the real-time time series data and the deep learning model, and the predicted data of the monitoring index is obtained, specifically including:

[0011] Statistical data of the timing time series data is calculated to obtain configuration item data;

[0012] The real-time time series data is processed by using the configuration item data to obtain input data, and the input data is input into the deep learning model to obtain the predicted data of the target device.

[0013] Based on the further embodiment, the real-time time series data is processed by using the configuration item data to obtain input data, specifically including:

[0014] The standard score of the real-time time series data at each time point is calculated by using the variance and mean in the configuration item data to obtain the standard score sequence in the input data;

[0015] The data difference value of each adjacent time point in the real-time time series data is calculated to obtain the result sequence in the input data.

[0016] Based on the further embodiment, the abnormal period of the monitoring target is determined by using the real-time time series data, the predicted data and a threshold, specifically including:

[0017] The squared difference of the real-time time series data and the predicted data at the same time point is calculated to obtain an abnormal score sequence;

[0018] An abnormal marker sequence is obtained according to the comparison result of the abnormal score sequence and a threshold, and the threshold is calculated by real-time time series data;

[0019] The abnormal period of the target device is determined according to the abnormal marker sequence.

[0020] Based on the further embodiment, the deep learning model parameter file is a model parameter file obtained by training the deep learning model by using the timing time series data, specifically including:

[0021] The standard score of the timing time series data at each time point is calculated by using the variance and mean in the configuration item data to obtain the standard score sequence of the training input data;

[0022] The data difference value of each adjacent time point in the timing time series data is calculated to obtain the result sequence of the training input data.

[0023] The training input data is input into the deep learning model for training, a deep learning model parameter file of the trained deep learning model is obtained, and the deep learning model parameter file is saved in object storage according to the joint feature to which the deep learning model parameter file belongs.

[0024] Based on the above further embodiments, the threshold is calculated based on real-time time series data, specifically including:

[0025] The training input data is input into the deep learning model as real-time time series data to obtain training prediction data;

[0026] The squared difference value of the training prediction data and the real-time time series data at the same time point is calculated to obtain a training anomaly score sequence;

[0027] The threshold is calculated according to the statistical data of the training anomaly score sequence.

[0028] Based on the above further embodiments, the method further includes:

[0029] The timing task is used for timing acquisition of timing time series data, timing training of the deep learning model, and updating of configuration item data.

[0030] According to a second aspect, a Prometheus-based time series anomaly detection deployment apparatus is provided, including:

[0031] The acquisition module is configured to pull real-time time series data of a target device in a preset time period from a Prometheus system, the real-time time series data meeting a joint feature, the joint feature including a plurality of independent data features, and the independent data features including a time step, the time step being a data time interval determined according to a length of the preset time period;

[0032] The processing module is configured to load a model parameter file corresponding to the joint feature into a deep learning model, and obtain prediction data of the target device by using the real-time time series data and the deep learning model, the model parameter file being a model parameter file obtained by training the deep learning model for the first time by using the real-time time series data, and the model parameter file being a parameter model file trained by the real-time time series data and timing time series data collected by a timing task when the training is not the first time, the timing time series data being running data of the target device meeting the joint feature collected by the timing task;

[0033] The determination module is configured to determine an abnormal period of the monitoring target by using the real-time time series data, the prediction data, and a threshold.

[0034] Based on the above further embodiments, the processing module is specifically configured to perform statistical data calculation on the timing time series data to obtain configuration item data.

[0035] The configuration item data is used to process the real-time time series data to obtain input data, and the input data is input into the deep learning model to obtain prediction data of the target device.

[0036] Based on the above further embodiments, the processing module is specifically configured to calculate the standard score of the real-time time series data at each time point using the variance and mean in the configuration item data to obtain a standard score sequence in the input data.

[0037] The data difference between adjacent time points in the real-time time series data is calculated to obtain a result sequence in the input data.

[0038] Based on the above further embodiments, the determination module is specifically configured to calculate the squared difference between the real-time time series data and the prediction data at the same time point to obtain an abnormal score sequence.

[0039] An abnormal marker sequence is obtained according to the comparison result of the abnormal score sequence and a threshold value, and the threshold value is calculated by real-time time series data.

[0040] According to the abnormal marker sequence, the abnormal period of the target device is determined.

[0041] Based on the above further embodiments, the processing module is specifically configured to calculate the standard score of the timing time series data at each time point using the variance and mean in the configuration item data to obtain a standard score sequence of training input data.

[0042] The data difference between adjacent time points in the timing time series data is calculated to obtain a result sequence of training input data.

[0043] The training input data is input into the deep learning model for training to obtain a deep learning model parameter file of the trained deep learning model, and the deep learning model parameter file is saved in the object storage according to the joint feature thereof.

[0044] Based on the above further embodiments, the processing module is specifically configured to input the training input data as real-time time series data into the deep learning model to obtain training prediction data.

[0045] The squared difference between the training prediction data and the real-time time series data at the same time point is calculated to obtain a training abnormal score sequence.

[0046] The threshold value is calculated according to the statistical data of the training abnormal score sequence.

[0047] Based on the above further embodiments, the processing module is specifically used for the timing task to collect timing time series data, train the deep learning model and update configuration item data.

[0048] According to a third aspect, a computer storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by one or more processors, the computer program implements the Prometheus-based time series anomaly detection deployment method according to any one of the above technical solutions.

[0049] According to a fourth aspect, an electronic device is provided, and the electronic device includes a memory and one or more processors. The memory stores a computer program, and the computer program is executed by the one or more processors to implement the Prometheus-based time series anomaly detection deployment method according to any one of the above technical solutions.

[0050] In the above system and method provided by the embodiments of the present specification, the time step adaptation according to the original data can be realized by a single model, the data of different time steps can be detected, and the detection accuracy of the abnormal data in the system is greatly improved. In addition, through the efficient model parameter file version, the model can be quickly fine-tuned in the case of few samples when pre-training the base model. In addition, based on the wide availability of the deployment scheme, the elastic replacement of the anomaly detection algorithm can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0052] Figure 1 The system schematic diagram of the Prometheus-based time series anomaly detection system provided by the embodiments of the present specification is shown.

[0053] Figure 2 The flowchart of the Prometheus-based time series anomaly detection deployment method provided by the embodiments of the present specification is shown.

[0054] Figure 3 The flowchart of the Prometheus-based time series anomaly detection deployment method provided by the embodiments of the present specification is shown.

[0055] Figure 4A flowchart of a Prometheus-based time series anomaly detection deployment method provided by an embodiment of the specification is shown.

[0056] Figure 5 A schematic diagram of a Prometheus-based time series anomaly detection deployment device provided by an embodiment of the specification is shown. DETAILED DESCRIPTION

[0057] The scheme provided by the specification will be described below with reference to the drawings.

[0058] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described below with reference to the drawings.

[0059] In the description of the embodiments of the present application, the words "exemplary", "for example", or "for instance" are used to mean serving as an example, instance or illustration. Any embodiment or design solution described as "exemplary", "for example" or "for instance" in the embodiments of the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design solutions. Rather, the words "exemplary", "for example" or "for instance" are used to present related concepts in a specific manner.

[0060] In the description of the embodiments of the present application, the term "and / or" is merely used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of existence of A alone, existence of B alone, and existence of A and B simultaneously. In addition, unless otherwise specified, the term "multiple" means two or more.

[0061] In addition, the terms "first", "second" are used for description purposes only, and should not be interpreted or implied to indicate or imply relative importance or implicitly indicate the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more features. The terms "include", "contain", "have" and their variants mean "include but are not limited to", unless otherwise specifically emphasized.

[0062] Prometheus is an open source monitoring and alarm system based on a time series database. It can collect performance index data of target devices by grabbing the status of target devices through an exporter interface, but cannot provide high-precision time series anomaly detection.

[0063] Figure 1A schematic diagram of a time series anomaly detection system based on Prometheus is provided for an embodiment of the present application. The anomaly detection system is divided into six layers, wherein the running environment layer is used to provide target devices including host devices, virtual machines and servers, the data sampling layer is used to monitor key business indicator data of each target device through Prometheus, the data storage layer is used to store key business data into a MySQL database, and trained model parameters are stored through an object storage iHarbor system, the business layer is used to perform anomaly detection on key business indicator data and refresh model parameter files at regular intervals; the algorithm layer is used to provide a deep learning network model, and the display layer is used to provide visual display for anomaly detection results.

[0064] Exemplarily, Figure 2 and Figure 3 is a flowchart of a time series anomaly detection deployment method provided by an embodiment of the present application, which is specifically as follows:

[0065] The user queries KPI indicator data in a specified time period through an IP, pulls corresponding KPI indicator data through a Prometheus system, and calculates a time step.

[0066] Specifically, the user sets a monitoring time range of a target device according to a business requirement, and determines a time step corresponding to different monitoring time ranges according to a preset rule, for example, the monitoring time range is 1 hour, and the corresponding time step can be 30 seconds, 1 minute or 2 minutes, etc., and for example, the monitoring time range is 1 day, and the corresponding time step can be 15 minutes, 30 minutes or 60 minutes.

[0067] The model parameter file of the corresponding KPI indicator is called to generate an anomaly score for each time point.

[0068] Specifically, when the model parameters corresponding to the KPI indicator are obtained for the first time, the obtained KPI indicator data is statistically processed to obtain the mean and variance of the KPI indicator data, and the mean and variance are taken as configuration item data. Z-Score operation and Diff operation are used on real-time time series data to obtain a Z-Score result and a Diff result set, wherein the Z-Score operation is as follows: z=(x-μ) / σ, z represents a z-score result, x is KPI indicator data, μ is the mean Mean of the KPI indicator data, and σ is the variance Std of the KPI indicator data. The Diff operation is as follows: Res[i]=arr[i+1]-arr[i], Res is the result set after Diff operation, and arr is a KPI indicator data set, that is, the difference between adjacent time points in the KPI indicator data is used to obtain the result set Res[i].

[0069] In the subsequent training process, the KPI index data is obtained by using a timing task at regular intervals, the configuration item data is automatically updated at regular intervals, the deep learning network model is trained using the updated configuration item data, and the accuracy of subsequent model prediction is improved. The configuration item data is stored in MySQL, and the parameter model file is saved in the object storage iHarbor, thereby improving the efficiency of model calling during detection.

[0070] In the service process of the application, independent data features and joint data features are adopted, wherein the independent data features represent a specific index corresponding to a single query, and the joint features are the integration of the independent data features, which can uniquely determine the query information.

[0071] In the embodiment, the KPI index is one of the independent feature data in the business process, and the KPI index data of different time steps is pulled by using Prometheus. The KPI index data is aligned according to the time steps, and therefore the time steps are also one of the independent feature data in the business process. In the iHarbor object storage, the KPI index and the time steps are stored as joint features of the model parameter file. When a user performs first-time anomaly detection on a specified joint feature, the MySQL and the iHarbor do not match the model training parameter file corresponding to the joint feature, the business data conforming to the joint feature in a specified time period is pulled, the configuration item data of the KPI index data is calculated through data preprocessing Z-Score, the configuration item data of the joint feature is stored in the MySQL, and the generated model parameter file is saved in the iHarbor object storage after training.

[0072] In the above process, the specified time period may be too short to cause inaccurate detection results, and in the embodiment, the method of the timing task is used for processing. The first timing task includes calculating and updating the configuration item data by using the historical time series data of the KPI index in the recent 7 days. The second timing task includes training the model parameter file corresponding to the joint feature by using the historical time series data of the KPI index in the recent 7 days and the configuration item data, ensuring the accuracy of the model parameters, ensuring the accuracy of the anomaly detection results, and updating the joint feature to the object storage for replacement.

[0073] The actual value and the predicted value of each time point in the KPI index data are used to generate the anomaly score of each time point by using the square difference formula. Wherein, score i is the anomaly score of the time point i, y i is the actual value, is the predicted value.

[0074] The time points exceeding the threshold are marked as abnormal states according to comparison of the threshold automatically calculated according to the abnormal scores of each time point.

[0075] Specifically, the automatic threshold calculation formula is threshold = mu_score + 3 * sigma_score, wherein threshold is the threshold, mu_score is the mean of the abnormal score Mean, and sigma_score is the variance of the abnormal score Std. When the model parameters corresponding to the joint features are not used for the first time, the mean of the abnormal score Mean and the variance of the abnormal score Std are obtained from MysQL, wherein the mean of the abnormal score Mean and the variance of the abnormal score Std are calculated by the KPI index data obtained by the timing task. When the threshold threadhold is -1, it indicates that the model parameters corresponding to the joint features are used for the first time, and the threshold in the configuration item is not set, at this time, the threshold can be configured manually or by the automatic threshold calculation formula. The manually set threshold is saved in the MySQL data table, and if the automatic threshold calculation formula is used, the threshold is calculated by the automatic threshold calculation formula.

[0076] The abnormal score set is compared with the threshold to form a marked sequence, wherein 0 in the marked sequence represents that the time point is not abnormal, and 1 represents that the time point is abnormal.

[0077] The abnormal condition is displayed in a visual form through the system front end.

[0078] According to the continuous abnormal marks in the abnormal mark sequence, different color intervals are used to represent different KPI sequence abnormalities in the ECharts line chart information of the front-end system, the abnormal time period information is highlighted, and the abnormal screening and root cause positioning are facilitated, so that the operation and maintenance personnel can more intuitively perform the abnormal screening and root cause positioning.

[0079] The application uses Prometheus for time series data storage and pulling, uses MySQL to store business data, and uses iHarbor object storage model parameter files. By saving the data of different processes respectively, the detection data and the business data are isolated, and the system structure definition is more clear. Meanwhile, the coupling between the several mature technical components provides a unique solution for the system deployment of the abnormal detection.

[0080] As Figure 4 A flowchart of a time series abnormality detection deployment method based on Prometheus is shown, which comprises the following steps:

[0081] 110, pulling real-time time series data of the target device meeting the joint feature in the preset time period from the Prometheus system, the joint feature comprising a plurality of independent data features, the independent data features comprising a time step, the time step being a data time interval determined according to the length of the preset time period.

[0082] Specifically, the joint feature can further comprise a plurality of independent data features, the independent data features can be a monitoring index, the monitoring index can be performance data of hardware or software, a detection KPI type, and can further comprise a type of the target device, the type of the target device can be a host node, a server, or a virtual machine, etc.

[0083] The time step is determined according to the length of the preset time period, and the time step corresponding to different monitoring time ranges is determined according to a preset rule, for example, the monitoring time range is 1 hour, and the corresponding time step can be 30 seconds, 1 minute, or 2 minutes, etc., and for example, the monitoring time range is 1 day, and the corresponding time step can be 15 minutes, 30 minutes, or 60 minutes.

[0084] 120, loading the model parameter file corresponding to the joint feature into the deep learning model, obtaining the prediction data of the target device by using the real-time time series data and the deep learning model, the deep learning model parameter file being a model parameter file obtained by training the deep learning model for the first time by using the real-time time series data, and the model parameter file not being trained for the first time being a parameter model file trained by the real-time time series data and the timing time series data collected by the timing task, the timing time series data being running data of the target device meeting the joint feature collected by the timing task.

[0085] Specifically, the deep learning model is not limited in the embodiment.

[0086] Specifically, the timing time series data is subjected to statistical data calculation to obtain configuration item data.

[0087] The real-time time series data is processed by using the configuration item data to obtain input data.

[0088] The input data is input into the deep learning model to obtain the prediction data of the target device.

[0089] Specifically, the configuration item mainly comprises business data required in one query, such as historical mean variance data, a time range of the query, joint feature data, and a threshold setting mark corresponding to the joint feature, etc.

[0090] The standard score sequence in the input data is obtained by using the variance and the mean in the configuration item data to calculate the standard score of the real-time time series data at each time point.

[0091] The data difference of each adjacent time point in the real-time time series data is calculated to obtain a result sequence in the input data.

[0092] 130. An abnormal period of the target device is determined by using the real-time time series data, the prediction data and the threshold value.

[0093] Specifically, the squared difference of the real-time time series data and the prediction data at the same time point is calculated to obtain an abnormal score sequence;

[0094] According to the comparison result of the abnormal score sequence and the threshold value, an abnormal mark sequence is obtained, and the threshold value is calculated by the timing time series data;

[0095] According to the abnormal mark sequence, the abnormal period of the target device is determined.

[0096] Specifically, the mean value μ in the configuration item data and the variance σ in the configuration item data are input into the automatic threshold calculation formula threshold = μ_score + 3 * σ_score to obtain the threshold value threshold, wherein the mean value μ in the configuration item data is obtained by calculating the mean value of the timing time series data, and the variance σ in the configuration item data is obtained by calculating the variance of the timing time series data.

[0097] Specifically, the deep learning model parameter file is stored in the first database, wherein the first database is an object storage such as iHarbor, and the configuration item data is stored in the second database, and the second database can be MySQL, and the second database is used to store the data information of the monitoring indicators, that is, the configuration item information.

[0098] Specifically, the variance and the mean value in the configuration item data are used to calculate the standard score of the real-time time series data at each time point to obtain a standard score sequence of the training input data;

[0099] The data difference of each adjacent time point in the real-time time series data is calculated to obtain a result sequence in the input data.

[0100] The training input data is input into the deep learning model for training to obtain a deep learning model parameter file of the trained deep learning model.

[0101] Specifically, the threshold value is calculated by the real-time time series data, and specifically includes:

[0102] The training input data is input into the deep learning model as the real-time time series data to obtain training prediction data;

[0103] The squared difference of the training prediction data and the real-time time series data at the same time point is calculated to obtain a training abnormal score sequence;

[0104] The threshold is calculated according to statistical data of the abnormal score sequence for training.

[0105] In addition, the threshold can also be manually configured by a user.

[0106] Specifically, the timing task is used to collect timing time series data at a timing, train the deep learning model at a timing, and update the configuration item data.

[0107] In the above method provided by the embodiments of the present specification, time step adaptation according to original data can be realized by a single model, data at different time steps can be detected, and the detection accuracy of abnormal data in the system can be greatly improved. In addition, through the efficient model parameter file version, the model can be quickly fine-tuned in a few sample cases when pre-training the base model, and the algorithm can be flexibly replaced. In addition, based on the wide availability of the deployment scheme, the elastic replacement of the anomaly detection algorithm can be realized.

[0108] As Figure 5 A schematic diagram of a Prometheus-based time series anomaly detection deployment device is shown, comprising:

[0109] The collection module is configured to pull real-time time series data of a target device in a preset time period from a Prometheus system, wherein the real-time time series data meets a joint feature, the joint feature comprises a plurality of independent data features, and the independent data features comprise a time step, and the time step is a data time interval determined according to the length of the preset time period;

[0110] The processing module is configured to load a model parameter file corresponding to the joint feature into a deep learning model, and obtain predicted data of the target device by using the real-time time series data and the deep learning model, wherein the deep learning model parameter file is a model parameter file obtained by training the deep learning model using the real-time time series data when the deep learning model is trained for the first time, and when the deep learning model is trained for the second time, the model parameter file is a parameter model file trained by the real-time time series data and timing time series data collected by a timing task, and the timing time series data is running data of the target device meeting the joint feature collected by the timing task;

[0111] The determination module is configured to determine an abnormal period of the target device by using the real-time time series data, the predicted data, and a threshold.

[0112] Based on the above further embodiments, the processing module is specifically configured to perform statistical data calculation on the timing time series data to obtain configuration item data.

[0113] The configuration item data is used to process the real-time time series data to obtain input data, and the input data is input into the deep learning model to obtain predicted data of the target device.

[0114] Based on the above further embodiment, the processing module is specifically configured to calculate the standard scores of the real-time time series data at each time point by using the variance and the mean in the configuration item data, to obtain the standard score sequence in the input data;

[0115] The processing module is further configured to calculate the data difference value of each adjacent time point in the real-time time series data, to obtain the result sequence in the input data.

[0116] Based on the above further embodiment, the determining module is specifically configured to determine the abnormal period of the monitoring target by using the real-time time series data and the prediction data, and specifically includes:

[0117] The determining module is further configured to calculate the squared difference of the real-time time series data and the prediction data at the same time point, to obtain an abnormal score sequence;

[0118] The determining module is further configured to obtain an abnormal marker sequence according to the comparison result of the abnormal score sequence and a threshold value, the threshold value being calculated by the real-time time series data;

[0119] The determining module is further configured to determine the abnormal period of the target device according to the abnormal marker sequence.

[0120] Based on the above further embodiment, the processing module is specifically configured to input the training input data as the real-time time series data into the deep learning model, to obtain the training prediction data;

[0121] The processing module is further configured to calculate the squared difference of the training prediction data and the real-time time series data at the same time point, to obtain a training abnormal score sequence;

[0122] The processing module is further configured to calculate the threshold value according to the statistical data of the training abnormal score sequence.

[0123] Based on the above further embodiment, the processing module is specifically configured to input the training input data into the deep learning model, to obtain the training prediction data;

[0124] The processing module is further configured to calculate the squared difference of the training prediction data and the real-time time series data at the same time point, to obtain a training abnormal score sequence;

[0125] The processing module is further configured to calculate the threshold value according to the statistical data of the training abnormal score sequence.

[0126] Based on the above further embodiment, the processing module is specifically configured to perform the timing task for collecting the timing time series data, training the deep learning model and updating the configuration item data.

[0127] In the system provided by the embodiments of the present specification, time step adaptation according to original data can be realized by a single model, data of different time steps can be detected, and the detection accuracy of abnormal data in the system is greatly improved. In addition, through the efficient model parameter file version, the model can be quickly fine-tuned in the case of few samples when pre-training the base model. In addition, based on the wide availability of the deployment scheme, the elastic replacement of the anomaly detection algorithm can be realized.

[0128] The present application also provides a computer storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by one or more processors to realize the Prometheus-based time series anomaly detection deployment method in any of the above technical solutions.

[0129] The present application also provides an electronic device, including a memory and one or more processors, the memory stores a computer program, and the computer program is executed by the one or more processors to realize the Prometheus-based time series anomaly detection deployment method in any of the above technical solutions.

[0130] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application, and it should be understood that the above is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application shall be included in the protection scope of the present application.

Claims

1. A deployment method for time-series anomaly detection based on Prometheus, characterized in that, The method includes: The system retrieves real-time time-series data of the target device that conforms to a joint feature within a preset time period from the Prometheus system. The joint feature includes multiple independent data features, and the independent data features include a time step, which is a data time interval determined according to the length of the preset time period. The model parameter file corresponding to the joint features is loaded into the deep learning model. The prediction data of the target device is obtained by using the real-time time series data and the deep learning model. The deep learning model parameter file is the model parameter file obtained by training the deep learning model using the real-time time series data when it is trained for the first time. When it is not trained for the first time, it is the parameter model file trained by the real-time time series data and the time-series data collected by the time-series task. The time-series data is the running data of the target device that conforms to the joint features collected by the time-series task. Using the real-time time series data, the predicted data, and the threshold, abnormal time periods of the monitored target are determined; The step of obtaining the prediction data for the target device using the real-time time series data and the deep learning model specifically includes: Statistical data calculations are performed on the timing sequence data to obtain configuration item data; Using the configuration item data, the real-time time series data is processed to obtain input data; the input data is then input into the deep learning model to obtain the prediction data for the target device. The deep learning model parameter file is the model parameter file obtained by training the deep learning model with time-series data, and specifically includes: Using the variance and mean in the configuration item data, the standard scores of the timing sequence data at each time point are calculated to obtain the standard score sequence of the training input data. Calculate the data difference between adjacent time points in the timing sequence data to obtain the result sequence of the input data for training. The training input data is input into the deep learning model for training, and the deep learning model parameter file of the trained deep learning model is obtained. The deep learning model parameter file is then saved in object storage according to its joint features.

2. The method according to claim 1, characterized in that, The step of determining the abnormal time period of the monitored target using the real-time time series data, the predicted data, and the threshold specifically includes: Calculate the squared difference between the real-time time series data and the predicted data at the same time point to obtain the anomaly score sequence; An anomaly labeling sequence is obtained based on the comparison result between the anomaly score sequence and the threshold, wherein the threshold is calculated using real-time time series data; The abnormal time period of the target device is determined based on the abnormal marker sequence.

3. The method according to claim 2, characterized in that, The threshold is calculated using real-time time-series data, specifically including: The training input data is used as real-time time series data and input into the deep learning model to obtain the training prediction data. Calculate the squared difference between the training prediction data and the real-time time series data at the same time point to obtain the training anomaly score sequence; The threshold is calculated based on the statistical data of the abnormal score sequences used in training.

4. The method according to claim 1, characterized in that, The method further includes: The scheduled task is used to periodically collect time-series data, periodically train the deep learning model, and periodically update configuration data.

5. A deployment device for timing anomaly detection based on Prometheus, characterized in that, include: The acquisition module is used to pull real-time time-series data of the target device that conforms to joint features within a preset time period from the Prometheus system. The joint features include multiple independent data features, and the independent data features include a time step, which is a data time interval determined according to the length of the preset time period. The processing module is used to load the model parameter file corresponding to the joint features into the deep learning model, and use the real-time time series data and the deep learning model to obtain the prediction data of the target device. The deep learning model parameter file is the model parameter file obtained when the deep learning model is trained for the first time using the real-time time series data. When it is not trained for the first time, it is the parameter model file trained by the real-time time series data and the timed time series data collected by the timed task. The timed time series data is the running data of the target device that conforms to the joint features collected by the timed task. The determination module is used to determine the abnormal time period of the monitored target by using the real-time time series data, the predicted data and the threshold; The processing module is used for: Statistical data calculations are performed on the timing sequence data to obtain configuration item data; Using the configuration item data, the real-time time series data is processed to obtain input data; the input data is then input into the deep learning model to obtain the prediction data for the target device. The processing module is used for: Using the variance and mean in the configuration item data, the standard scores of the timing sequence data at each time point are calculated to obtain the standard score sequence of the training input data. Calculate the data difference between adjacent time points in the timing sequence data to obtain the result sequence of the input data for training. The training input data is input into the deep learning model for training, and the deep learning model parameter file of the trained deep learning model is obtained. The deep learning model parameter file is then saved in object storage according to its joint features.

6. A computer storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by one or more processors, implements the Prometheus-based time-series anomaly detection deployment method as described in any one of claims 1 to 4.

7. An electronic device, characterized in that, It includes a memory and one or more processors, wherein the memory stores a computer program that, when executed by the one or more processors, implements the Prometheus-based timing anomaly detection deployment method as described in any one of claims 1 to 4.

Images