Abnormity detection method and device, product, equipment and medium

Abstract

The invention discloses an anomaly detection method and device, a product, equipment and a medium. The anomaly detection method comprises the steps of obtaining historical time sequence data corresponding to a monitoring index of a target platform in a preset historical time period; inputting the historical time sequence data into a data prediction model for data prediction to obtain prediction data of the monitoring index in a target time period; determining a data normal threshold range corresponding to the monitoring index in the target time period based on the prediction data; and if any item of data in the actual data of the monitoring index in the target time period exceeds the data normal threshold range, determining that the any item of data is abnormal data. In this way, the accuracy and flexibility of anomaly detection can be improved.

Description

Technical Field

[0001] The present application relates to the field of anomaly detection technology, and in particular to an anomaly detection method, device, product, equipment and medium. Background Art

[0002] In order to ensure the stable operation of the business platform, it is often necessary to monitor the business platform and detect whether the time series data is abnormal. At present, the existing anomaly detection solutions for time series data usually use the method of year-on-year and month-on-month comparison of historical data and current data for anomaly detection. This is not only complicated, but also has a high misjudgment rate and is inflexible in adding indicators. Therefore, how to improve the accuracy and flexibility of anomaly detection is a problem that needs to be solved at present. Summary of the invention

[0003] In view of this, the purpose of this application is to provide an anomaly detection method, device, product, equipment and medium, which can improve the accuracy and flexibility of anomaly detection. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses an anomaly detection method, comprising:

[0005] Obtain the historical time series data corresponding to the monitoring indicators of the target platform in the preset historical period;

[0006] Inputting the historical time series data into a data prediction model for data prediction to obtain the prediction data of the monitoring indicator in the target time period;

[0007] Determine, based on the predicted data, a normal threshold range of data corresponding to the monitoring indicator in the target time period;

[0008] If any item of data in the actual data of the monitoring indicator in the target time period exceeds the normal data threshold range, the any item of data is determined to be abnormal data.

[0009] Optionally, the data prediction model is a model obtained by training a preset machine learning model using a training sample set, wherein the training sample set includes training samples and labels corresponding to the training samples, the training samples are time series data corresponding to monitoring indicators, and the labels are predicted data for the next time period corresponding to the training samples.

[0010] Optionally, also include:

[0011] Acquire time series data based on the actual data to obtain target time series data;

[0012] Eliminating abnormal data in the target time series data to obtain preprocessed data, or eliminating abnormal data in the target time series data and filling in missing data to obtain preprocessed data;

[0013] The data prediction model is trained using the preprocessed data to obtain an updated data prediction model.

[0014] Optionally, when there are multiple different data prediction models, determining the normal data threshold range corresponding to the monitoring indicator in the target time period based on the prediction data includes:

[0015] Based on the prediction data output by multiple different data prediction models, the multiple data normal threshold ranges corresponding to the monitoring indicators in the target time period are determined.

[0016] Correspondingly, if any item of data in the actual data of the monitoring indicator in the target time period exceeds the normal data threshold range, determining that the any item of data is abnormal data includes:

[0017] If any item of data in the actual data of the monitoring indicator in the target time period exceeds multiple data normal threshold ranges, the any item of data is determined to be abnormal data.

[0018] Optionally, determining a normal threshold range of data corresponding to the monitoring indicator in the target time period based on the predicted data includes:

[0019] If the target time period is a specific time period, a first range corresponding to the specific time period is used to determine the parameter, and a normal threshold range of data corresponding to the monitoring indicator in the target time period is determined based on the predicted data, wherein the specific time period is a time period determined based on historical data of the monitoring indicator;

[0020] If the target time period is a non-specific time period, a second range corresponding to the non-specific time period is used to determine the parameter, and a normal threshold range of data corresponding to the monitoring indicator in the target time period is determined based on the predicted data.

[0021] Optionally, also include:

[0022] Determine whether the warning conditions are met based on abnormal data;

[0023] If the warning condition is met, the warning information is sent to the maintenance personnel terminal through a preset communication channel;

[0024] In response to a clearing operation of the maintenance personnel terminal, the abnormal data is modified into normal data.

[0025] In a second aspect, the present application discloses an abnormality detection device, comprising:

[0026] A data acquisition module is used to obtain historical time series data corresponding to the monitoring indicators of the target platform in a preset historical period;

[0027] A data prediction module is used to input the historical time series data into a data prediction model to perform data prediction and obtain the prediction data of the monitoring indicator in the target time period;

[0028] A threshold determination module, used to determine a normal threshold range of data corresponding to the monitoring indicator in the target time period based on the predicted data;

[0029] The anomaly detection module is used to determine that any item of data in the actual data of the monitoring indicator in the target time period is abnormal data if the data exceeds the normal threshold range of the data.

[0030] In a third aspect, the present application discloses a computer program product, which implements the aforementioned anomaly detection method when executed.

[0031] In a fourth aspect, the present application discloses an electronic device, including a memory and a processor, wherein:

[0032] The memory is used to store the computer program;

[0033] The processor is used to execute the computer program to implement the above-mentioned abnormality detection method.

[0034] In a fifth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program implements the aforementioned anomaly detection method when executed by a processor.

[0035] It can be seen from the above scheme that the present application provides an anomaly detection method, including: obtaining historical time series data corresponding to the monitoring indicators of the target platform in a preset historical time period; inputting the historical time series data into a data prediction model for data prediction to obtain the predicted data of the monitoring indicators in the target time period; determining the normal data threshold range corresponding to the monitoring indicators in the target time period based on the predicted data; if any item of data in the actual data of the monitoring indicators in the target time period exceeds the normal data threshold range, then determining the any item of data as abnormal data.

[0036] It can be seen that the beneficial effects of the present application are: utilizing the historical time series data of monitoring indicators and combining it with the model for data prediction, and being able to more accurately determine the normal threshold range of the data based on the predicted data, and comparing it with the actual data to determine whether the data is abnormal, which can improve the accuracy of anomaly detection. When adding indicators, the historical time series data corresponding to the added monitoring indicators is obtained, and prediction and judgment can be performed together with the original indicator data, which facilitates the addition of indicators and can improve the flexibility of anomaly detection.

[0037] Correspondingly, an abnormality detection device, product, equipment and readable storage medium provided by the present application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 A flow chart of an anomaly detection method provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of anomaly detection provided in an embodiment of the present application;

[0041] Figure 3 A prediction schematic diagram provided for an embodiment of the present application;

[0042] Figure 4 A detection schematic diagram provided for an embodiment of the present application;

[0043] Figure 5 A warning diagram provided for an embodiment of the present application;

[0044] Figure 6 A schematic diagram of an anomaly detection scenario provided in an embodiment of the present application;

[0045] Figure 7 A schematic diagram of the structure of an abnormality detection device provided in an embodiment of the present application;

[0046] Figure 8 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] The existing method of performing year-on-year and month-on-month anomaly detection on time series data is not only complex, has a high misjudgment rate, and is inflexible in adding indicators, but also has a large resource consumption based on the memory cache calculation principle. To this end, the embodiment of the present application provides an anomaly detection solution with higher accuracy and better flexibility.

[0049] See also Figure 1 As shown, the embodiment of the present application discloses an anomaly detection method, including:

[0050] Step S11: Obtain historical time series data corresponding to the monitoring indicators of the target platform in a preset historical period.

[0051] Among them, the target platform can be any business platform, that is, a digital platform that supports enterprises or organizations to conduct business operations and management activities. For example, a charging service business platform. The preset historical period can be a historical period set based on business needs, such as the past 7 days. The monitoring indicator is the indicator that needs to be detected for anomaly detection, and can be based on the indicators that need to be detected in the business settings. Historical time series data refers to the time series data of the monitoring indicator in the preset historical period. The historical time series data is the time series data composed of the data within the preset time granularity corresponding to the monitoring indicator. For example, the preset time granularity is minutes. In the past 7 days, the time series data composed of the data of each minute corresponding to the monitoring indicator is the historical time series data corresponding to the monitoring indicator. One minute of data is one item of data in the historical time series data of the monitoring indicator.

[0052] In the embodiment of the present application, the monitoring indicator may be one or more. Different monitoring indicators may correspond to different normal data threshold ranges, and the data of the monitoring indicator is the access data of the monitoring indicator, and the access data may be the call volume corresponding to the monitoring indicator in the target platform. The target platform may include multiple background services after the front-end program, and the monitoring indicators may include services, specified functional modules in the services (some important functional modules may be specified), methods called during internal implementation of the services, and functional modules in the front-end programs. For example, the call volume for a background service in one minute, that is, the access volume in one minute, or the call volume of a method in one minute.

[0053] The embodiment of the present application can manually mark abnormal data of some monitoring indicators so that they do not go through the data prediction model, avoiding unnecessary model prediction processes.

[0054] Step S12: inputting the historical time series data into a data prediction model for data prediction to obtain the prediction data of the monitoring indicator in the target time period.

[0055] In the embodiment of the present application, before the historical time series data is input into the data prediction model, preprocessing may be performed, which may include removing abnormal data and supplementing missing data, and the missing data includes: missing data and removed abnormal data. If the model can automatically supplement the missing data, it is not necessary to supplement the missing data before inputting the model.

[0056] In an optional embodiment, the data prediction model is a model obtained by training a preset machine learning model using a training sample set, wherein the training sample set includes training samples and labels corresponding to the training samples, the training samples are time series data corresponding to monitoring indicators, and the labels are predicted data for the next time period corresponding to the training samples.

[0057] For example, the training sample is the 7-day time series data corresponding to the monitoring indicator, and the label is the 8th day's time series data, both of which are one data per minute. During the training process, the loss is calculated based on the output results and labels of the model, and the model is updated based on the loss until the model converges.

[0058] The embodiments of the present application can train multiple preset machine learning models to obtain multiple data prediction models, for example, a DNN (Deep Neural Network) model, a Prophet (prophet) model, and a LihgtGBM (Light Gradient Boosting Machine, a framework based on gradient boosting decision trees) model.

[0059] Further, in an optional implementation, the training sample may be time series data corresponding to the monitoring indicator and feature columns corresponding to the time series data, such as a result set obtained by summing according to a preset time length as a feature column, and / or a lag set calculated based on the time series data as a feature column. For example, the preset time length is 5 minutes or 10 minutes.

[0060] Step S13: Determine the normal threshold range of data corresponding to the monitoring indicator in the target time period based on the predicted data.

[0061] This embodiment can determine the data normal threshold range corresponding to the target period according to the preset time granularity based on business needs. For example, the preset historical period is from February 1 to February 7, and the target period is February 8, that is, the target period is the next day corresponding to the historical time series data, and the preset time granularity is minutes. The embodiment of the present application can predict the data of the monitoring indicator every minute in the next day. The data corresponding to one minute is one item of data, and the data normal threshold range corresponding to the target period includes the data normal threshold range corresponding to each item of data.

[0062] In an optional embodiment, if the target time period is a specific time period, the parameters are determined using a first range corresponding to the specific time period, and the data normal threshold range corresponding to the monitoring indicator in the target time period is determined based on the predicted data, wherein the specific time period is a time period determined based on historical data of the monitoring indicator; if the target time period is a non-specific time period, the parameters are determined using a second range corresponding to the non-specific time period, and the data normal threshold range corresponding to the monitoring indicator in the target time period is determined based on the predicted data.

[0063] For example, the specific time period is a holiday, evening rush hour, etc., the first range parameter and the second range parameter can be a proportional coefficient or an adjustment value, and the first range parameter and the second range parameter can be different, that is, holidays and ordinary times can correspond to different threshold ranges. The proportional coefficient for determining the upper limit of the normal data threshold range and the proportional coefficient for the lower limit can be the same or different, and the adjustment value for determining the upper limit of the normal data threshold range and the adjustment value for the lower limit can be the same or different. For example, the predicted data of a certain data item is M, and the proportional coefficient for determining the upper limit of the normal data threshold range and the proportional coefficient for the lower limit are 1.2 and 0.8 respectively, and the normal data threshold range corresponding to the data item is 0.8M to 1.2M. If the range parameter is an adjustment value, the adjustment value is added or subtracted based on the predicted data to obtain the normal data threshold range. The first range parameter and the second range parameter can be the same, and false alarms in specific time periods can be avoided through subsequent different warning conditions.

[0064] Step S14: If any item of data in the actual data of the monitoring indicator in the target time period exceeds the normal threshold range of the data, the any item of data is determined to be abnormal data.

[0065] Among them, exceeding the normal data threshold range means being greater than the upper limit of the normal data threshold range or less than the lower limit of the normal data threshold range. Each data item is compared with the normal data threshold range corresponding to the data item. If any data item exceeds the normal data threshold range corresponding to the data item, the data item is determined to be abnormal data.

[0066] In an optional implementation, multiple data normal threshold ranges corresponding to the monitoring indicators in the target time period can be determined based on the prediction data respectively output by multiple different data prediction models; accordingly, if any item of data in the actual data of the monitoring indicator in the target time period exceeds the data normal threshold range, then the arbitrary item of data is determined to be abnormal data, including: if any item of data in the actual data of the monitoring indicator in the target time period exceeds multiple data normal threshold ranges, then the arbitrary item of data is determined to be abnormal data. For example, three models predict data, and three data normal threshold ranges corresponding to the target time period are determined. If one item of data corresponds to the three data normal threshold ranges and exceeds the data normal threshold ranges corresponding to 2 or 3 models, then the data is determined to be abnormal data. In this way, multiple models collaborate to improve accuracy.

[0067] Furthermore, in an optional implementation, it is also possible to obtain time series data based on the actual data to obtain target time series data; eliminate abnormal data in the target time series data to obtain preprocessed data, or eliminate abnormal data in the target time series data and fill in missing data to obtain preprocessed data; use the preprocessed data to train the data prediction model to obtain an updated data prediction model.

[0068] That is, the embodiment of the present application can update the data prediction model based on actual data. Before updating the data prediction model, the abnormal data can be removed first, or the abnormal data can be removed first and then filled to obtain training data for training, thereby improving the model performance. The model can be updated regularly based on a preset cycle. When updating, the actual data in the latest cycle is used as training data after preprocessing, so that the model is continuously optimized.

[0069] Furthermore, the embodiment of the present application can also determine whether the warning conditions are met based on the abnormal data; if the warning conditions are met, the warning information is sent to the maintenance personnel terminal through a preset communication channel; in response to the elimination operation of the maintenance personnel terminal, the abnormal data is modified to normal data.

[0070] Among them, the early warning conditions may include that the abnormal data ratio reaches a preset ratio threshold and / or the abnormal data exceeds the early warning threshold, and the abnormal data ratio is the ratio between the number of abnormal data in the actual data of any monitoring indicator and the number of all data. The early warning threshold is different from the normal data threshold range, and can be higher than the upper limit of the normal data threshold range, or lower than the lower limit of the normal threshold range. And the early warning conditions corresponding to different monitoring indicators can be set separately, and the monitoring indicators can correspond to different early warning levels, and different early warning levels correspond to different early warning conditions.

[0071] The preset message channel may be an email address, real-time messaging software, etc. The embodiment of the present application may further improve the accuracy of abnormal data detection by manually verifying abnormal data.

[0072] It can be seen that the embodiment of the present application utilizes the historical time series data of monitoring indicators and combines it with the model to perform data prediction. Based on the predicted data, the normal threshold range of the data can be determined more accurately, and compared with the actual data to determine whether the data is abnormal, which can improve the accuracy of anomaly detection. When adding indicators, the historical time series data corresponding to the added monitoring indicators is obtained, and prediction and judgment can be performed together with the original indicator data, which facilitates the addition of indicators and can improve the flexibility of anomaly detection.

[0073] For further information, see Figure 2 As shown, Figure 2 A schematic diagram of anomaly detection provided for an embodiment of the present application. The following steps may be included: prediction: training a model, and using the trained model to generate prediction data; detection: identifying anomalies (i.e., abnormal data) based on the prediction data; warning: sending anomaly detection warnings. Among them, the model may include models corresponding to the DNN algorithm, the Prophet algorithm, and the LihgtGBM algorithm, i.e., the DNN model, the Prophet model, and the LihgtGBM model. Identifying anomalies may include automatically detecting anomalies by a scheduled task, i.e., identifying anomalies based on the normal threshold range of the data determined by the prediction data, and manually marking anomalies for special data. In addition, anomaly detection warnings may be triggered based on the warning verification plug-in, and anomaly detection logs may be generated. That is, the entire process is divided into three stages: prediction, detection, and warning.

[0074] In the embodiment of the present application, prediction is to predict the time series data based on the algorithm of the prediction data, and support the removal of abnormal points during model training to improve the accuracy of model training. Monitoring indicators can be divided into the following categories:

[0075] 1) General monitoring metadata (e.g. StartChargeSuccess): direct query without grouping and aggregation, etc. 2) Special monitoring metadata (e.g. BSS.RRO.ReceiveOrder): aggregated by minute, with the condition being a custom dimension. The monitoring indicators in 1) and 2) are important functional modules and data in the service. Some data has been aggregated, while some has not yet been aggregated. The unaggregated data needs to be aggregated. The custom dimension is the client type, such as APP, mini program, etc. 3) SG metadata (TeldSG.Invoke): aggregated by minute, with the condition being ServiceID (service identifier), that is, the monitoring indicator is the service. 4) HSF metadata (Teld.XXX.XXX.XXXService): aggregated by minute, with the condition being Method (method), that is, the monitoring indicator is the method called by the service. 5) Front-end reporting indicator (FrontendMonitor): aggregated by minute, with the condition having multiple dimensions, that is, the monitoring indicator is the functional module in the front-end program, and different dimensions can correspond to different functional modules.

[0076] For monitoring indicators, the splicing expression is too complicated, so the expression assembled in the anomaly detection indicator is used. That is, the expression for querying the monitoring indicator data is pre-assembled, and when querying the monitoring indicator data, the query is directly based on the expression. Each indicator has its own particularity. You can write a complete expression directly on the page, and the background program can directly parse the complete expression, which simplifies the background logic splicing processing and facilitates the subsequent flexible adjustment of individual indicators.

[0077] See also Figure 3 As shown, Figure 3A prediction schematic diagram provided for an embodiment of the present application. The original data (data for the last N days), i.e., the historical time series data corresponding to the monitoring indicators, are respectively input into the DNN model, the Prophet model, and the LihgtGBM model for prediction to obtain the predicted data, and the predicted data is stored. In addition, the model can also be trained based on the original data to update the model. Among them, before inputting the DNN model or the LihgtGBM model, the missing data needs to be supplemented. The missing data includes the missing data of the original data and the abnormal point data marked manually or automatically. The Prophet model can use the existing data points to estimate the missing data points by interpolation. The abnormal point data marked manually or automatically can be removed before inputting the Prophet model. Whether it is training or prediction, the data is preprocessed as described above. During training, the result set of the sum of 5 minutes and 10 minutes of the original data can also be added as a feature column, and the lag set calculated for the original data can be added as a feature column. When the algorithm makes a prediction, the embodiment of the present application ensures the accuracy of the prediction, and the missing data is supplemented. The missing data includes the missing data of the monitoring data and the abnormal points marked by the identification. The embodiments of the present application can implement a prediction program to perform model training and data prediction.

[0078] Further, detection: automatic identification of abnormal points by comparing actual data with predicted data, and manual labeling of abnormal points by human intervention. A Check (detection) program can be implemented to traverse the abnormal detection indicator table, calculate the abnormal points based on the predicted data, actual data, and deviation threshold, and write them into the abnormal point table, where the deviation threshold is the normal threshold range of the data, which is determined based on the predicted data. The abnormal point table stores the abnormal data of each indicator in the time series data. On the one hand, it is used to check abnormal points in the first stage of predicted data and enhance the accuracy of predicted data. On the other hand, it is used as the basic table for the third stage early warning program to issue early warnings and query abnormal data. See Figure 4 As shown, Figure 4 A detection schematic diagram provided for an embodiment of the present application. The embodiment of the present application can detect the upper and lower thresholds predicted by three algorithms. If all three algorithms exceed the upper and lower bounds, they are automatically identified as abnormal and the abnormal state is marked. Anomalies can be marked manually, and manual inspections can be automatically marked as error marks, and the marks can be discarded. The automatic identification and manual marking of abnormal points can be flexibly adjusted to ensure the accuracy of the original data. The data state can be characterized based on Auto, Manual, Discard, and Normal, where Auto represents automatic abnormal points calculated by the algorithm; Manual represents manually marked abnormal points; Discard represents manually cancelled abnormal points; Normal represents normal data: unmarked (Normal). That is, the data state of the monitoring indicator is characterized by different marks, including automatic abnormal points calculated by the algorithm, manual standard abnormal points, manually cancelled abnormal points, and normal data.

[0079] Further, early warning: Based on the preset early warning platform, abnormal points are identified and early warnings are sent. Figure 5 As shown, Figure 5 A warning schematic diagram provided for an embodiment of the present application. The embodiment of the present application can issue an early warning through a trend detection and verification plug-in (i.e., a preset early warning platform). The plug-in includes an anomaly detection engine, which performs data preprocessing on actual time series data, and may include statistical abnormal data ratio and / or abnormal data acquisition, etc., dynamic threshold calculation, i.e., comparison of abnormal data ratio with ratio threshold and / or comparison of abnormal data with early warning threshold, etc. When the threshold is triggered, an early warning log is created, and an early warning message is sent through a preset message channel. The embodiment of the present application can implement an early warning program, which can query the abnormal points of the indicator, and flexibly judge whether the indicator damage meets the configuration threshold and trigger the threshold warning through the personalized configuration (warning level) and holiday configuration (holiday traffic change floating value) of each indicator.

[0080] In addition, the embodiment of the present application supports the configuration of the holiday effect deviation rate to reduce the misjudgment rate of abnormal detection. When predicting the upper and lower bounds of the trend, a fixed value is configured, and the holiday deviation is relatively large. The overall increase or decrease in holiday effect traffic causes the prediction to break through the upper or lower bound for a long time. When predicting the upper and lower bounds of the trend, if the fixed value of a normal date is used, the holiday deviation is relatively large, resulting in a higher probability that the actual value exceeds the upper and lower bound thresholds, and more abnormal points are determined, resulting in the triggering of an early warning. For example: From the perspective of a substantial increase in charging volume during holidays, the corresponding predicted core indicator traffic will increase as a whole, especially the traffic on the eve of holidays will increase the most. In order to solve the increase and decrease in traffic caused by external factors during holidays and more accurate prediction data, the embodiment of the present application can appropriately adjust the upper and lower bounds of holidays (i.e., the normal threshold range of data) and / or the thresholds in the early warning conditions.

[0081] For further information, see Figure 6 As shown, Figure 6 A schematic diagram of an anomaly detection scenario provided for an embodiment of the present application. Usage scenario: Manual configuration function is required. Anomaly detection maintenance personnel define business impairment indicators (i.e., monitoring indicators), set corresponding warning conditions, manually mark abnormal points, and perform special configurations on holidays. Anomaly detection users can view business impairment indicators and business means indicator warnings through the monitoring indicator dashboard.

[0082] In this way, a solution with better accuracy and flexibility is achieved based on machine learning combined with actual business scenarios, including the use of multiple machine learning algorithms, the elimination of abnormal points (automatic identification + manual labeling), flexible configuration of indicators, flexible configuration of statutory holidays, and personalized configuration of business indicators. The misjudgment rate is reduced by combining multiple machine learning prediction algorithms. The personalized configuration of indicators solves the personalized configuration of indicators with different traffic sizes and different trend changes, improves the accuracy of early warning, and the flexible configuration of holidays reduces the impact of holiday traffic changes. After experimental verification, the accuracy is relatively high, and basically the damage of indicators can be detected when a fault occurs online and the corresponding early warning information is issued.

[0083] For further information, see Figure 7 As shown, the embodiment of the present application discloses an abnormality detection device, including:

[0084] The data acquisition module 11 is used to acquire the historical time series data corresponding to the monitoring indicators of the target platform in a preset historical period;

[0085] A data prediction module 12 is used to input the historical time series data into a data prediction model to perform data prediction and obtain the prediction data of the monitoring indicator in the target time period;

[0086] A threshold determination module 13, configured to determine a normal threshold range of data corresponding to the monitoring indicator in the target time period based on the predicted data;

[0087] The anomaly detection module 14 is configured to determine that any item of data in the actual data of the monitoring indicator in the target time period is abnormal data if the item exceeds the normal threshold range of the data.

[0088] In an optional embodiment, the data prediction model is a model obtained by training a preset machine learning model using a training sample set, wherein the training sample set includes training samples and labels corresponding to the training samples, the training samples are time series data corresponding to monitoring indicators, and the labels are predicted data for the next time period corresponding to the training samples.

[0089] In an optional embodiment, the device also includes a model updating module, which is used to obtain time series data based on the actual data to obtain target time series data; eliminate abnormal data in the target time series data to obtain preprocessed data, or eliminate abnormal data in the target time series data and fill in missing data to obtain preprocessed data; use the preprocessed data to train the data prediction model to obtain an updated data prediction model.

[0090] In an optional embodiment, the threshold determination module 13 is specifically used to: when there are multiple different data prediction models, the determination of the data normal threshold range corresponding to the monitoring indicator in the target time period based on the prediction data includes: determining multiple data normal threshold ranges corresponding to the monitoring indicator in the target time period based on the prediction data output by multiple different data prediction models. Correspondingly, the anomaly detection module 14 is specifically used to determine that any data item is abnormal data if any data item in the actual data of the monitoring indicator in the target time period exceeds multiple data normal threshold ranges.

[0091] In an optional embodiment, the threshold determination module 13 is used to: if the target time period is a specific time period, determine the parameters using the first range corresponding to the specific time period, and determine the normal data threshold range of the monitoring indicator corresponding to the target time period based on the predicted data, wherein the specific time period is a time period determined based on the historical data of the monitoring indicator; if the target time period is a non-specific time period, determine the parameters using the second range corresponding to the non-specific time period, and determine the normal data threshold range of the monitoring indicator corresponding to the target time period based on the predicted data.

[0092] In an optional embodiment, the device may further include:

[0093] An early warning module is used to determine whether the early warning condition is met based on abnormal data; if the early warning condition is met, the early warning information is sent to the maintenance personnel terminal through a preset communication channel;

[0094] The modification module is used to modify the abnormal data into normal data in response to the elimination operation of the maintenance personnel terminal.

[0095] It can be seen that the embodiment of the present application utilizes the historical time series data of monitoring indicators and combines it with the model to perform data prediction. Based on the predicted data, the normal threshold range of the data can be determined more accurately, and compared with the actual data to determine whether the data is abnormal, which can improve the accuracy of anomaly detection. When adding indicators, the historical time series data corresponding to the added monitoring indicators is obtained, and prediction and judgment can be performed together with the original indicator data, which facilitates the addition of indicators and can improve the flexibility of anomaly detection.

[0096] The embodiment of the present application discloses a computer program product, which implements the anomaly detection method disclosed in the above embodiment when executed.

[0097] For the specific process of the above-mentioned abnormality detection method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0098] See also Figure 8As shown, an embodiment of the present application discloses an electronic device 20, including a processor 21 and a memory 22; wherein the memory 22 is used to store a computer program; the processor 21 is used to execute the computer program, the anomaly detection method disclosed in the above embodiment.

[0099] For the specific process of the above-mentioned abnormality detection method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0100] Furthermore, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the storage method may be temporary storage or permanent storage.

[0101] In addition, the electronic device 20 also includes a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26; wherein the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input / output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0102] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the anomaly detection method disclosed in the aforementioned embodiment.

[0103] For the specific process of the above-mentioned abnormality detection method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0104] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0105] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0106] The above is a detailed introduction to an anomaly detection method, device, product, equipment and medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An anomaly detection method, characterized in that: include: Obtain the historical time series data corresponding to the monitoring indicators of the target platform in the preset historical period; Inputting the historical time series data into a data prediction model for data prediction to obtain the prediction data of the monitoring indicator in the target time period; Determine, based on the predicted data, a normal threshold range of data corresponding to the monitoring indicator in the target time period; If any item of data in the actual data of the monitoring indicator in the target time period exceeds the normal data threshold range, the any item of data is determined to be abnormal data.

2. The anomaly detection method according to claim 1, characterized in that: The data prediction model is a model obtained by training a preset machine learning model using a training sample set, wherein the training sample set includes training samples and labels corresponding to the training samples, the training samples are time series data corresponding to monitoring indicators, and the labels are predicted data for the next time period corresponding to the training samples.

3. The anomaly detection method according to claim 2, characterized in that: Also includes: Acquire time series data based on the actual data to obtain target time series data; Eliminating abnormal data in the target time series data to obtain preprocessed data, or eliminating abnormal data in the target time series data and filling in missing data to obtain preprocessed data; The data prediction model is trained using the preprocessed data to obtain an updated data prediction model.

4. The anomaly detection method according to claim 1, characterized in that: When there are multiple different data prediction models, determining the normal data threshold range corresponding to the monitoring indicator in the target time period based on the prediction data includes: Determine multiple data normal threshold ranges corresponding to the monitoring indicators in the target time period based on the prediction data respectively output by multiple different data prediction models; Correspondingly, if any item of data in the actual data of the monitoring indicator in the target time period exceeds the normal data threshold range, determining that the any item of data is abnormal data includes: If any item of data in the actual data of the monitoring indicator in the target time period exceeds multiple data normal threshold ranges, the any item of data is determined to be abnormal data.

5. The anomaly detection method according to claim 1, characterized in that: Determining the normal threshold range of data corresponding to the monitoring indicator in the target time period based on the predicted data includes: If the target time period is a specific time period, a first range corresponding to the specific time period is used to determine the parameter, and a normal threshold range of data corresponding to the monitoring indicator in the target time period is determined based on the predicted data, wherein the specific time period is a time period determined based on historical data of the monitoring indicator; If the target time period is a non-specific time period, a second range corresponding to the non-specific time period is used to determine the parameter, and a normal threshold range of data corresponding to the monitoring indicator in the target time period is determined based on the predicted data.

6. The abnormality detection method according to any one of claims 1 to 5, characterized in that: Also includes: Determine whether the warning conditions are met based on abnormal data; If the warning condition is met, the warning information is sent to the maintenance personnel terminal through a preset communication channel; In response to a clearing operation of the maintenance personnel terminal, the abnormal data is modified into normal data.

7. An abnormality detection device, characterized in that: include: A data acquisition module is used to obtain historical time series data corresponding to the monitoring indicators of the target platform in a preset historical period; A data prediction module is used to input the historical time series data into a data prediction model to perform data prediction and obtain the prediction data of the monitoring indicator in the target time period; A threshold determination module, used to determine a normal threshold range of data corresponding to the monitoring indicator in the target time period based on the predicted data; The anomaly detection module is used to determine that any item of data in the actual data of the monitoring indicator in the target time period is abnormal data if the data exceeds the normal threshold range of the data.

8. A computer program product, characterized in that When the computer program product is executed, the anomaly detection method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store the computer program; The processor is used to execute the computer program to implement the anomaly detection method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the anomaly detection method according to any one of claims 1 to 6 is implemented.

Images