Streaming data anomaly detection method and device and cluster

Abstract

A streaming data anomaly detection method can be applied to a detection device, data features of streaming data are stored in the detection device, the streaming data comprise data points generated according to a time sequence, and the method comprises the following steps: obtaining a first data point of the streaming data; using an anomaly detection model to detect whether the first data point is abnormal data, the anomaly detection model being obtained based on the data features; under the condition that the first data point is not abnormal data, updating data features based on the first data point; updating an anomaly detection model based on the updated data features; acquiring a second data point of the streaming data; and detecting whether the second data point is abnormal data or not by using the updated anomaly detection model. According to the method, the anomaly detection model can adapt to the continuously changing data characteristics of the streaming data under the condition that the performance of a database or a memory is consumed less, so that the detection accuracy of the anomaly detection model on subsequent data points is guaranteed.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on July 7, 2023, with application number 202310833375.5 and application name “A Real-time Anomaly Detection Operator System”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of data processing technology, and in particular to a streaming data anomaly detection method, device and cluster. Background Art

[0003] Many application scenarios generate streaming data. By performing anomaly detection on streaming data, anomalies can be discovered in a timely manner. For example, in the manufacturing process scenario, anomaly detection on streaming data that reflects the operating status of machine tools can detect machine tool anomalies in a timely manner. Another example is anomaly detection on streaming data that reflects patients' vital signs, which can detect anomalies in patients' vital signs in a timely manner.

[0004] As time goes by, the data characteristics of streaming data may change. Therefore, it is difficult to use fixed standards or models to detect whether the data generated in different time periods are abnormal. At present, the most recently generated data is detected using the full historical data or recent historical data. Since data is continuously generated over time, and the data volume of the full historical data or recent historical data is large, data anomaly detection requires frequent and large-scale query operations on the database or memory, which seriously consumes the performance of the database or memory. Summary of the invention

[0005] The embodiments of the present application propose a method, device and cluster for detecting anomalies in streaming data, which can enable the anomaly detection model to adapt to the ever-changing data characteristics of streaming data while consuming less database or memory performance, thereby ensuring the accuracy of the anomaly detection model in detecting subsequent data points.

[0006] In a first aspect, a method for detecting anomalies in streaming data is provided. The method is applied to a detection device, the detection device stores data features of streaming data, the streaming data includes data points generated in chronological order, the method includes: obtaining a first data point of the streaming data; using an anomaly detection model, detecting whether the first data point is abnormal data, wherein the anomaly detection model is obtained based on the data features; if the first data point is not abnormal data, updating the data features based on the first data point; updating the anomaly detection model based on the updated data features; obtaining a second data point of the streaming data, the generation time of the second data point is after the generation time of the first data point; using the updated anomaly detection model, detecting whether the second data point is abnormal data.

[0007] In this way, whenever a normal data point is detected, the data features of the streaming data can be updated using the normal data point, and the anomaly detection model can be updated based on the updated data features. In other words, the anomaly detection model can be continuously updated as data points are detected, so that the update of the anomaly detection model and the update of the data features of the streaming data are synchronized, that is, the anomaly detection model and the new data features of the streaming data are adapted to each other. This makes it possible to adapt the anomaly detection model to the new data features of the streaming data without querying and saving the full amount of historical data points and the data points in a fixed time window.

[0008] After the anomaly detection model is updated, the updated anomaly detection model can be used to detect subsequently generated data points. As described above, the updated anomaly detection model is adapted to the new data features of the streaming data. Using the updated anomaly detection model to detect subsequently generated data points can improve the accuracy of detection and ensure the detection effect.

[0009] In one possible implementation, the detection device stores M peak data points of streaming data, and the data feature includes K peak data points among the M peak data points, where K is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to K; based on the first data point, the data feature is updated, including: replacing the third data point among the M peak data points with the first data point to obtain the updated M peak data points; and updating the K peak data points based on the updated M peak data points.

[0010] In this implementation, K data points used as data features of the streaming data are obtained from M peak data points, where M can be greater than K. In this way, it can be ensured that the updated K data points can accurately reflect the actual data features of the streaming data.

[0011] In a possible implementation, replacing a third data point among the M peak data points with the first data point includes: when the life cycle of the third data point times out, replacing the third data point with the first data point.

[0012] In this implementation, a life cycle can be set for the peak data points stored in the detection device. When the life cycle of a peak data point times out, the peak data point with the timed out life cycle is replaced by the most recently detected normal data point. In this way, it can be ensured that the peak data points stored in the detection device can reflect the new data characteristics of the streaming data.

[0013] In one possible implementation, replacing the third data point among the M peak data points with the first data point includes: when the peak data point is the maximum peak data point and the first data point is greater than the third data point, replacing the third data point with the first data point.

[0014] In this implementation, the first data point is greater than the third data point, indicating that the first data point can more accurately reflect the data characteristics of the streaming data than the third data point. In this way, the third data point is replaced by the first data point, so that the peak data point stored in the detection device can more accurately reflect the actual data characteristics of the streaming data.

[0015] In one possible implementation, replacing a third data point among the M peak data points with the first data point includes: when the peak data point is a minimum peak data point and the first data point is smaller than the third data point, replacing the third data point with the first data point.

[0016] In this implementation, the first data point is smaller than the third data point, indicating that the first data point can more accurately reflect the data characteristics of the streaming data than the third data point. In this way, the third data point is replaced by the first data point, so that the peak data point stored in the detection device can more accurately reflect the actual data characteristics of the streaming data.

[0017] In a possible implementation, the method further includes: when the first data point is abnormal data, outputting a detection result of the first data point, where the detection result is used to indicate that the first data point is abnormal data.

[0018] In this implementation, when an abnormal data point is detected, the detection result is output so that the user knows that an abnormality has occurred in the streaming data.

[0019] In a possible implementation, the data feature is obtained based on N data points of the streaming data; wherein the generation time of the N data points is earlier than the generation time of the first data point, and N is a positive integer greater than 1.

[0020] In this implementation, data features of the streaming data can be obtained based on multiple data points of the streaming data. The detection device stores the data features of the streaming data instead of storing the N data points, thereby saving storage space of the detection device.

[0021] In a second aspect, a streaming data anomaly detection device is provided, characterized in that the device stores data features of streaming data, the streaming data includes data points generated in chronological order, and the device includes: an acquisition module, used to acquire a first data point of the streaming data; a detection module, used to detect whether the first data point is abnormal data using an anomaly detection model, wherein the anomaly detection model is obtained based on the data features; a first update module, used to update the data features based on the first data point when the first data point is not abnormal data; and a second update module, used to update the anomaly detection model based on the updated data features.

[0022] In one possible implementation, the detection device stores M peak data points of streaming data, and the data features include K peak data points among the M peak data points, where K is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to K; the first update module is used to: replace the third data point among the M peak data points with the first data point to obtain the updated M peak data points; and update the K peak data points based on the updated M peak data points.

[0023] In one possible implementation, the first update module is used to: when the life cycle of the third data point times out, replace the third data point with the first data point; or, when the peak data point is the maximum peak data point and the first data point is greater than the third data point, replace the third data point with the first data point; or, when the peak data point is the minimum peak data point and the first data point is less than the third data point, replace the third data point with the first data point.

[0024] In a possible implementation, the device further includes: an output module, configured to output a detection result of the first data point when the first data point is abnormal data, wherein the detection result is used to indicate that the first data point is abnormal data.

[0025] In a possible implementation, the data feature is obtained based on N data points of the streaming data; wherein the generation time of the N data points is earlier than the generation time of the first data point, and N is a positive integer greater than 1.

[0026] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method provided in the first aspect.

[0027] In a fourth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method provided in the first aspect.

[0028] In a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computer device cluster, the computer device cluster executes the method provided in the first aspect.

[0029] The beneficial effects of the second to fifth aspects can be referred to the above introduction to the beneficial effects of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A structural diagram of a streaming data anomaly detection system provided in an embodiment of the present application;

[0031] Figure 2 A schematic diagram of the structure of a detection device provided in an embodiment of the present application;

[0032] Figure 3 A data feature diagram provided for an embodiment of the present application;

[0033] Figure 4 A flow chart of a method for detecting anomalies in streaming data provided by an embodiment of the present application;

[0034] Figure 5 A schematic diagram of a data feature update solution provided in an embodiment of the present application;

[0035] Figure 6 A schematic diagram of a data feature update solution provided in an embodiment of the present application;

[0036] Figure 7 A schematic diagram of a data feature update solution provided in an embodiment of the present application;

[0037] Figure 8 A schematic diagram of the structure of a streaming data anomaly detection device provided in an embodiment of the present application;

[0038] Fig. 9 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0039] Fig.10 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0040] Fig.11 A schematic diagram of a structure of a computing device cluster connected via a network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The scheme provided by the embodiment of the present application will be described below in conjunction with the accompanying drawings. In the embodiment of the present application, "multiple" means two or more, and "multiple" means two or more. "First", "second", etc. are only used to distinguish similar objects and are not necessarily used to describe a specific order or number of objects.

[0042] To facilitate understanding of the solutions provided by the embodiments of the present application, the technical terms that may be involved in the embodiments of the present application are first introduced.

[0043] Streaming data: also known as time series data, refers to data generated in chronological order, that is, data that is continuously generated over time. Streaming data consists of data points generated in chronological order, and different data points are generated at different times. In other words, a time point or time period can generate one or a piece of data, which can be called a data point. Multiple data points generated at consecutive time points or time periods constitute streaming data. Streaming data often has the characteristics of large data volume, short data point intervals, and continuous arrival of data points.

[0044] Data features of streaming data: refers to relevant information that can reflect the characteristics of streaming data, such as peak data points of streaming data, average values ​​of data points, trends of data points, cycles of data points, etc. Among them, peak data points can be divided into maximum peak data points and minimum peak data points. The maximum peak data point refers to the largest data point among the multiple data points that make up the streaming data. The minimum peak data point refers to the smallest data point among the multiple data points that make up the streaming data.

[0045] Anomaly detection for streaming data has a wide range of applications, such as in manufacturing processes, finance, and healthcare detection. This requires anomaly detection solutions for streaming data to assist users in detecting and analyzing streaming data. Among them, the characteristics of data points generated at different times in streaming data may change, which requires the standard or model used to determine whether streaming data is abnormal to also change.

[0046] In one solution, by analyzing the full amount of historical data, the characteristic changes of streaming data are obtained, and then whether the currently detected data point is abnormal data is determined. This requires querying the full amount of historical data from the database or memory. Data points are constantly generated, which requires frequent querying of the full amount of historical data from the database or memory, and the amount of full historical data is large. Therefore, this solution seriously consumes the performance of the database or memory. Therefore, storing the full amount of historical data in memory consumes a large amount of memory storage space, and even causes memory overflow, making the memory unable to work.

[0047] In another solution, a sliding time window is used to divide historical data. The data points generated in the sliding time window are used to detect the current data points. In order to ensure the accuracy of the detection, the sliding time window must be long enough, that is, the number of data points must be sufficient. This results in querying a large amount of data from the database or memory. This solution seriously consumes the performance of the database or memory, and may cause memory overflow, making the memory unable to work.

[0048] In another solution, the model is used to detect the data points of the streaming data one by one. In order to cope with the characteristic changes of the streaming data, the model is retrained before each detection, so that the current data points can be detected using the retrained model. Among them, the retraining of the model requires the full amount of historical data or a large amount of recent historical data. This also requires querying a large amount of data from the database or memory. Therefore, this solution seriously consumes the performance of the database or memory, and may cause memory overflow, making the memory unable to work.

[0049] An embodiment of the present application provides a method for detecting anomalies in streaming data. The method can use an anomaly detection model to detect whether a data point of streaming data is abnormal. Among them, the anomaly detection model is obtained based on the data features of the streaming data. When it is detected that a data point is not abnormal data, the data features of the streaming data can be updated based on the data point, and the anomaly detection model can be updated based on the updated data features. The updated anomaly detection model is used to detect the next data point of the data point. That is to say, in this method, the anomaly detection model is updated using the currently detected non-abnormal data points, so that the anomaly detection model can adapt to the changes in the features of the data points, and can thus be used to detect subsequent data points. In this way, without having to obtain historical data, the update of the anomaly detection model is achieved, so that the anomaly detection model adapts to the constantly changing data features of the streaming data, thereby eliminating or reducing the consumption or impact of anomaly detection of streaming data on the database or memory.

[0050] Next, the streaming data anomaly detection method provided in the embodiment of the present application is described.

[0051] Figure 1 A streaming data anomaly detection system is shown. The system includes a data source 100 and a detection device 200. The data source 100 generates streaming data. The streaming data is composed of data points generated in time sequence. The detection device 200 can detect whether the data points of the streaming data are abnormal data. In some embodiments, Figure 1 As shown, the system further includes a database. The database can receive the streaming data sent by the data source 100 and store the streaming data. The database can also send the data points of the recently received streaming data to the detection device 200, so that the detection device 200 can detect the data points of the streaming data.

[0052] The data source 100 may continuously generate data points in a time sequence. The generated data points constitute streaming data. Exemplarily, the data source 100 may generate data points regularly, for example, one data point is generated every 1 second. Exemplarily, the data source 100 may generate data points irregularly, for example, the data source 100 may generate corresponding data for sensing changes in the environment or the collection object. The changes in the environment or the collection object may be irregular, and therefore, the data source 100 generates data points irregularly.

[0053] In some embodiments, the data source 100 may be a sensor, such as a pressure sensor, a temperature sensor, a flow sensor, or a heart rate sensor. In some embodiments, the data source 100 may be a monitor, such as a machine operation indicator monitor, etc. The embodiments of the present application do not limit the specific implementation form of the data source 100.

[0054] The detection device 200 may be any device, equipment, platform or cluster having a data processing function. The embodiment of the present application does not limit the specific implementation form of the detection device 200.

[0055] Figure 2 FIG. 2 shows a possible structure of the detection device 200. Figure 2 As shown, the detection device 200 includes a storage module 210 , an input module 220 , a detection module 230 , a feature updating module 240 and a model updating module 250 .

[0056] The storage module 210 is a module or device with data storage capability. In some embodiments, the storage module 210 may be a cache in the detection device 200. In some embodiments, the storage module 210 may be a memory or a hard disk in the detection device 200. In some embodiments, the storage module 210 may be implemented as a database. Figure 1 The database shown may be integrated into the detection device 200 and used as the storage module 210 .

[0057] like Figure 2 As shown, the storage module stores data features of streaming data. The data features of streaming data are obtained based on N consecutive data points of streaming data, that is, the N consecutive data points of streaming data can be abstracted to obtain the data features of streaming data. Among them, the N data points are not abnormal data, and N is a positive integer greater than 1. Among them, the generation time of the N data points is less than the generation time of the currently detected data point. That is to say, the N data points are historical data points. It is easy to understand that the data features of streaming data are obtained by abstracting the data points of streaming data, and the amount of data is much smaller than the amount of data points. Therefore, storing the data features of streaming data will not consume too many storage resources. In addition, data feature queries have little impact on the performance of the storage module 210.

[0058] like Figure 2 As shown, the storage module 210 also stores an anomaly detection model. Specifically, the storage module storing the anomaly detection model may store parameters of the anomaly detection model, such as a threshold value for determining whether a data point is an abnormal data. The anomaly detection model may be used to detect whether a data point of the streaming data is an abnormal data. For example, it may be determined whether a data point exceeds a threshold value. If the threshold value is exceeded, the data point may be confirmed to be an abnormal value.

[0059] Continue reading Figure 2 The input module 220 is used to obtain the data point to be detected and send the obtained data point to the detection module 230. The detection module 230 can use the anomaly detection model to detect whether the data point is abnormal data. Specifically, the detection module 230 can obtain the anomaly detection model from the storage module 210, and use the anomaly detection model to detect the data point to determine whether the data point is abnormal data.

[0060] If the data point is abnormal data, the detection module 230 may output a detection result, where the detection result is used to indicate that the data point is abnormal data.

[0061] If the data point is not abnormal data, or in other words, the data point is normal data, the detection module 2230 can send the data point to the feature update module 240. The feature update module 240 can update the data feature of the streaming data to which the data point belongs based on the data point. The specific process of updating the data feature will be introduced below and will not be repeated here.

[0062] The feature update module 240 may send the updated data features to the storage module 210. After receiving the updated data features, the storage module 210 may store the updated data features in the storage module 210. In some embodiments, the storage module 210 may also delete the original data features, that is, the data features before the update. In other words, the storage module 210 only stores the latest data features of the streaming data.

[0063] The model update module 250 may obtain the updated data features. In some embodiments, the feature update module 240 may also send the updated data features to the model update module 250. In some embodiments, after the updated data features are stored in the storage module 210, the feature update module 240 sends an indication message to the model update module 250, which may instruct the model update module 250 to obtain the updated data features from the storage module 210.

[0064] After obtaining the updated data features, the model update module 250 can update the anomaly detection model based on the updated data features. Among them, the model update module 250 can obtain the anomaly detection model from the storage module 210, and update the anomaly detection model based on the updated data features. Then, the model update module 250 sends the updated anomaly detection model to the storage module 210. After receiving the updated anomaly detection model, the storage module 210 can store the updated anomaly detection model in the storage module 210. In some embodiments, the storage module 210 can also delete the original anomaly detection model, that is, the anomaly detection model before the update. That is to say, the storage module 210 only stores the latest anomaly detection model of the streaming data. Among them, the specific process of updating the anomaly detection model will be introduced below and will not be repeated here.

[0065] In some embodiments, the functional modules such as the input module 220, the detection module 230, the feature update module 240 or the model update module 250 can be implemented by hardware. For example, the functional module can be specifically one or more servers. The hardware device can run relevant program codes to implement the functions of the functional module.

[0066] In some embodiments, these functional modules can also be implemented by software. For example, the functional module can be implemented as a virtual device such as a virtual machine (VM) or a container. The virtual device can run related program codes to implement the functions of the functional module. For another example, the functional module can be implemented as a process, etc. Among them, different functional modules can run on different devices or on the same device.

[0067] The above example introduces the streaming data anomaly detection system provided by the embodiment of the present application. Next, in conjunction with the streaming data anomaly detection system, the streaming data anomaly detection method provided by the embodiment of the present application is introduced.

[0068] The method can be performed by the detection device 200. The storage module in the detection device 200 stores the data features of the streaming data. Before implementing the method, the data features of the streaming data can be obtained through an initialization step. In some embodiments, the data features of the streaming data include one or a combination of any multiple of the peak data point, the average value of the data point, the trend of the data point, and the period of the data point. The data features of the streaming data can be obtained in the following manner.

[0069] In some embodiments, N data points of the streaming data can be obtained, where N is a positive integer greater than 1. Exemplarily, the N data points can be obtained manually or in other ways. The N data points are all normal data points, that is, the multiple data points are not abnormal data. In addition, the N data points are adjacent in sequence in the streaming data. Data point adjacency means that the data points are adjacent in the streaming data. More specifically, if two data points are adjacent, then in the streaming data, there are no other data points between the two data points. In other words, no other data points are generated between the generation times of the two data points.

[0070] Based on the N data points, data features of the streaming data are obtained. The following is an example of how to obtain the data features.

[0071] In some embodiments, the data feature of the streaming data is K peak data points, where K is a positive integer greater than or equal to 1, and K is less than N. K peak data points can be obtained from the N data points.

[0072] When the peak data point is the maximum peak data point, the largest K data points among the N data points are used as the K peak data points, that is, the data features of the streaming data. The largest K data points are also called K maximum data points, and each data point among the K maximum data points is greater than the data points among the N data points except the K maximum data points.

[0073] When the peak data point is the minimum peak data point, the smallest K data points among the N data points are used as the K peak data points, that is, the data features of the streaming data. The smallest K data points are also called K minimum data points, and each data point among the K minimum data points is smaller than the data points among the N data points except the K minimum data points.

[0074] In this way, K peak data points can be obtained. The K peak data points can be used as data features of the streaming data and stored in the storage module 210.

[0075] In an example of this embodiment, M peak data points among N data points can be stored in the storage module 210. When the peak data point is the maximum peak data point, the M peak data points are the M maximum data among the N data points. When the peak data point is the minimum peak data point, the M peak data points are the M minimum data among the N data points. M is a positive integer greater than or equal to K, and M is less than N. The M peak data points include the above-mentioned K peak data points, that is, the K peak data points are a subset of the M peak data points. The M peak data points can be called coarse-grained features, and the K peak data points can be called fine-grained features. Fine-grained features are obtained from coarse-grained features. Fine-grained features are used to obtain or update anomaly detection models.

[0076] In some embodiments, Figure 3 As shown, M is greater than K, that is, the K peak data points are a proper subset of the M peak data points. When the peak data point is the maximum data point, each of the K peak data points is greater than the data points of the M peak data points except the K peak data points. When the peak data point is the minimum data point, each of the K peak data points is less than the data points of the M peak data points except the K peak data points.

[0077] In some embodiments, the data feature of the streaming data may be an average value of the data points of the streaming data. The average value of the N data points may be calculated, and the calculated result may be used as the average value of the data points of the streaming data. Exemplarily, the average value of the N data points may be the arithmetic mean of the N data points.

[0078] In some embodiments, the data feature of the streaming data may be a trend of a data point, such as an upward trend, a downward trend, etc. The trend of the data point of the streaming data may be obtained based on the N data points. Specifically, if the N data points increase sequentially in time sequence, the trend of the data point of the streaming data is an upward trend. If the N data points decrease sequentially in time sequence, the trend of the data point of the streaming data is a downward trend.

[0079] An anomaly detection model can be obtained based on the data features of the streaming data. The anomaly detection model is composed of parameters and algorithms. The algorithm of the anomaly detection model can be fixed. Common algorithms of anomaly detection models include univariate time series streaming anomaly detection algorithms based on extreme value theory (for example, stream peaks-over-threshold (spot) algorithm). Obtaining an anomaly detection model can refer to obtaining parameters of the anomaly detection model. The parameters of the anomaly detection model combined with the algorithm of the anomaly detection model can be used to determine whether a data point is abnormal data. When the algorithm of the anomaly detection model is a univariate time series streaming anomaly detection algorithm based on extreme value theory, the parameters of the anomaly detection model can be a threshold value. The threshold value is also called an extreme value. When a data point exceeds the threshold value, the data point can be considered as abnormal data. Otherwise, the data point is not abnormal data.

[0080] As described above, the storage module 210 stores the characteristic data of the streaming data, and the characteristic data can be used to obtain the parameters of the anomaly detection model. Next, an example is given for introduction.

[0081] In some embodiments, the data features of the streaming data may be K peak data points, and the algorithm of the anomaly detection model is a univariate time series streaming anomaly detection algorithm based on extreme value theory. Then, the parameters of the anomaly detection model include a threshold value. The threshold value may be obtained based on the K peak data points. Among them, the threshold value may be obtained using the limit theory.

[0082] In an example of this embodiment, the threshold value can be obtained based on the K peak data points using the generalized Pareto distribution parameters. The K peak data points can be set as the K maximum peak data points, and the threshold value can be obtained by the following formula.

[0083] First, the generalized Pareto distribution parameters (γ, σ) are estimated based on K peak data points using formula (1).

[0084]

[0085] Wherein, t is the threshold, X is the peak data point, x is the difference between the peak data point and the threshold, τ=sup{x∈R, F(x)<1}, and R is a real number.

[0086] Then, the threshold value z is calculated by formula (2): q .

[0087]

[0088] N t is the number of peak data points, q is the abnormal probability, n is the number of normal data points observed, represents the calculated δ, represents the calculated γ.

[0089] In the above, the data features of streaming data are K peak data points, and the algorithm of the anomaly detection model is a univariate time series streaming anomaly detection algorithm based on extreme value theory. As an example, the parameters of the anomaly detection model are introduced by way of example, which does not constitute a limitation. The data features of streaming data can be data features in other forms, and the algorithm of the anomaly detection model can also adopt other algorithms. The parameters of the anomaly detection model can be set or calculated according to the corresponding data features and the algorithm of the anomaly detection model. For details, please refer to the introduction of the prior art, which will not be described one by one here.

[0090] When the anomaly detection model is obtained, the anomaly detection model may be stored in the storage module 210 .

[0091] Thus, the initialization can be completed. After the initialization is completed, the abnormality detection method provided in the embodiment of the present application can be implemented. Figure 4 , the method may include the following steps.

[0092] Step 401: Acquire the first data point of the streaming data, wherein the first data point may be one data point or two or more data points.

[0093] Step 401 only requires obtaining the first data point to be detected, without obtaining other data points, such as data points generated before the first data point.

[0094] In addition, if the data features of the streaming data currently stored in the storage module 210 are obtained based on the N data points introduced above, the generation time of the first data point is later than the generation time of each of the N data points.

[0095] Step 401 may be performed by the input module 220. The input module 220 may monitor the generation or storage of a certain data point of the streaming data or data points. Whenever a data point is generated or stored, the input module 220 may obtain the data point.

[0096] In some embodiments, the input module 220 can obtain the data point to be detected from the database. Specifically, the input module 220 can monitor whether there is a data point of the streaming data written to the database. Whenever a data point is written to the database, the input module 220 can obtain the data point from the database. In one example, the input module 220 can obtain the data point by executing the following structured query language (SQL) statement.

[0097]

[0098] The statement specifies the cpu_usage variable in the table named cpu in the database named mysql, and the variable satisfies the label instanceId = id1. The input module 220 can specify the data point to be detected through the statement and obtain the data point to be detected.

[0099] In some embodiments, the input module 220 may receive data points sent by the data source 100. The input module 220 may copy the received data points. The input module 220 may send a copy of the data points as the data points to be detected to the detection module 230, and send another copy of the data points to the database to store the data points in the database.

[0100] Step 402: Detect whether the first data point is abnormal data by using an anomaly detection model, wherein the anomaly detection model is obtained based on the data features.

[0101] Step 402 may be performed by the detection module 230. The detection module 230 may obtain the anomaly detection model from the storage module 210, and use the anomaly detection model to detect the first data point to obtain a detection result.

[0102] If the first data point is abnormal data, that is, the detection indicates that the first data point is abnormal data, the detection result can be output. In some embodiments, the detection result can be written to a database. Exemplarily, the detection module 230 can write the detection result to the database through the input module 220. Specifically, the detection module 230 can send the detection result to the input module 220, and the input module 220 can call the SQL statement received above to write the detection result to the database.

[0103] If the first data point is not abnormal data, that is, the first data point is normal data, step 403 may be performed to update the data feature using the first data point. The data point that is not abnormal data may also be referred to as a normal data point.

[0104] Step 403 may be performed by the feature updating module 240. When the first data point is not abnormal data, the detection module 230 sends the first data point to the feature updating module 240. The feature updating module 240 may update the data features stored in the storage module 210 based on the first data point.

[0105] In some embodiments, as described above, the storage module 210 stores M peak data, and K peak data points in the M peak data are used as data features of streaming data, that is, used to obtain or update the anomaly detection model. In step 403, the first data point can be used to replace the data point in the M peak data points, and the replaced data point can be set as the third data point. That is, in step 403, the third data point in the M peak data points is replaced with the first data point to obtain the updated M peak data points. Among them, the number of data points included in the third data point is the same as the number of data points included in the first data point. The K peak data points can be updated based on the updated M peak data points. The updated K peak data points can be used to update the anomaly detection model.

[0106] In order to ensure that the K peak data points as the data features of the streaming data can timely reflect the latest features of the streaming data, it is necessary to age the peak data points stored in the storage module 210. The peak data points stored in the storage module 210 may have a life cycle, and when the life cycle of the data point expires, the data point is aged. Data point aging may refer to deleting the data point from the storage module 210. The length of the life cycle of the peak data point may be preset, for example, the life cycle of the peak data point may be set to the time required to generate P data points, where P is a positive integer greater than or equal to M.

[0107] In the case where the peak data points stored in the storage module 210 have an aging time, M is greater than K. This is because if M=K, there are data points that are aged among the M peak data points, and when the M peak data points are updated, the K peak data points that are the data features of the streaming data are directly aged and updated, which may cause the updated K peak data points to be difficult to accurately reflect the features of the streaming data, that is, the features of the streaming data reflected by the K peak data points are greatly deviated from the actual features of the streaming data.

[0108] In one example, if Figure 5 As shown, at time T1, the streaming data includes data points of size 8, data points of size 9, data points of size 6, data points of size 10, data points of size 5, data points of size 6, data points of size 7, data points of size 4, data points of size 7, data points of size 6, data points of size 2, and data points of size 5, which are arranged in time order. M=K and M=3 can be set, and the peak data point is the maximum peak data point. Then at time T1, the M peak data points (i.e., K peak data points) stored by the storage module 210 are: data points of size 8, data points of size 9, and data points of size 10. These three data points can accurately reflect the characteristics of the streaming data. At time T2 after time T1, the data points of size 8 and size 9 of the streaming data are aged, and the normal data points detected recently are data points of size 1 and size 4. In the case of M=K, the K peak data points as the data features of the streaming data are updated to: data points of size 10, data points of size 1, and data points of size 4. Obviously, the updated K peak data points cannot accurately reflect the data features of the streaming data, and a great degree of deviation occurs.

[0109] To avoid the above situation, when the peak data points stored in the storage module 210 have an aging time, M is set to be greater than K. Among them, M peak data points can be called coarse-grained features, and K peak data points can be called fine-grained features. Coarse-grained features are downsampling of data within a fixed-length time window (i.e., the life cycle of a data point). Compared with fine-grained features, coarse-grained features save more streaming data information, thereby reflecting the data characteristics of streaming data as much as possible. Fine-grained features are obtained from coarse-grained features, and therefore can more accurately reflect the data characteristics of streaming data.

[0110] In the case where there are data points aging among the peak data points stored in the storage module 210, the coarse-grained features are updated, and then the fine-grained features, that is, the K peak data points, are updated based on the updated coarse-grained features.

[0111] like Figure 6 As shown, the feature update module 230 can obtain the coarse-grained features (i.e., M peak data points) from the storage module, and receive the first data point sent by the detection module 220. If the life cycle of the third data point among the M peak data points times out, the third data point is aged, that is, the third data point is deleted from the coarse-grained features, and the first data point is added to the coarse-grained features. In other words, the third data point among the M peak data points is replaced with the first data point to update the M peak data points, that is, to update the coarse-grained features. The feature update module 230 can store the updated coarse-grained features in the storage module 210.

[0112] When the coarse-grained features are updated, the feature update module 230 updates the fine-grained features based on the updated coarse-grained features, that is, updates the K peak data points. Then, the feature update module 230 sends the updated K peak data points to the model update module 250. The model update module 250 updates the anomaly detection model based on the updated K peak data points.

[0113] The above solution does not directly age and update fine-grained features, but ages and updates coarse-grained features, which can minimize or even avoid the deviation caused by aging and updating, and maintain the real-time and accuracy of the data features used to update the anomaly detection model. This avoids the situation where the updated K peak data points cannot accurately reflect the data characteristics of the streaming data, resulting in a large degree of deviation, and further avoids the situation where the updated anomaly detection model is not suitable for the detection of streaming data. The following example introduces the effect of this technology.

[0114] In an example, Figure 7As shown, at time T1, the streaming data includes data points of size 8, data points of size 9, data points of size 6, data points of size 10, data points of size 5, data points of size 6, data points of size 7, data points of size 4, data points of size 7, data points of size 6, data points of size 2, and data points of size 5, which are arranged in time order. M=5, K=3 can be set, and the peak data point is the maximum peak data point. Then at time T1, the M peak data points stored by the storage module 210 are: data points of size 8, data points of size 9, data points of size 10, and two data points of size 7. Based on these 5 data points, K peak data points are obtained, which are data points of size 8, data points of size 9, and data points of size 10. At time T2 after time T1, the data points of size 8 and size 9 of the streaming data are aged, and the normal data points detected recently are data points of size 1 and size 4. Then the M peak data points stored in the storage module 210 are updated to: a data point of size 10, two data points of size 7, a data point of size 1, and a data point of size 4. Based on these five data points, three peak data points of size 8 and two data points of size 7 are obtained. Obviously, the updated fine-grained features (i.e., K peak data points) can accurately reflect the data characteristics of the streaming data.

[0115] In some embodiments, when the first data point can more accurately reflect the data characteristics of the streaming data, the first data point is used to update the M peak data points regardless of whether the current M peak data points have data points with their life cycles ended.

[0116] In one example of this embodiment, the peak data point is the maximum peak data point, and the third data point is the minimum peak data point among the M peak data points. The minimum peak data point may also be referred to as a coarse-grained feature threshold tm. If the first data point is greater than the third data point, the third data point is replaced by the first data point to update the M peak data points.

[0117] In an example of this embodiment, the peak data point is the minimum peak data point, and the third data point is the maximum peak data point among the M peak data points. The maximum peak data point may also be referred to as a coarse-grained feature threshold tm. If the first data point is less than the third data point, the third data point is replaced by the first data point to update the M peak data points.

[0118] In this way, the update of M peak data points can be achieved. When M is equal to K, the M peak data points are updated, that is, the K peak data points are updated. When M is greater than K, the K peak data points can be updated based on the updated M peak data points.

[0119] In some embodiments, as described above, the data feature of the streaming data may be an average value of the data points of the streaming data. Then, in step 403, an average value of the first data point and the average value may be calculated to update the average value. The updated average value is the updated data feature of the streaming data.

[0120] In other embodiments, the data features of the streaming data may also be in other forms. For data features in different forms, corresponding methods may be used to update the data features using the first data point, which will not be described in detail here.

[0121] The above solution can be used to update the data features, and the updated data features can be stored in the storage module 210 .

[0122] Continue reading Figure 4 After the data features are updated, step 404 may be performed to update the anomaly detection model based on the updated data features.

[0123] Step 404 may be performed by the model updating module 250. In step 404, the model updating module 250 may obtain updated data features, for example, from the storage module 210. The model updating module 250 may update the anomaly detection model based on the updated data features.

[0124] As described above, the anomaly detection model consists of an algorithm and parameters. Among them, the algorithm is usually unchanged. In step 404, updating the anomaly detection model specifically refers to updating the parameters of the anomaly detection model. Taking the algorithm of the anomaly detection model as a univariate time series streaming anomaly detection algorithm based on extreme value theory as an example, in step 404, the parameters required by the algorithm, such as the threshold value, can be updated. Among them, the updated data features can be used to recalculate the threshold value. For example, the updated data features can be input into the formula (1) and formula (2) described above to obtain the updated threshold value.

[0125] After the update of the anomaly detection model is completed, the updated anomaly detection model can be stored in the storage module 210. When the next data point of the first data point arrives, the updated anomaly detection model can be used to detect whether the data point is abnormal data, and if the data point is not abnormal data, step 403 and step 404 are performed for the data point. The next data point of the first data point is a data point whose generation time is after the generation time of the first data point.

[0126] Next, an example is given by taking the next data point of the first data point as the second data point.

[0127] like Figure 4As shown, in step 405, a second data point is obtained, and the generation time of the second data point is after the generation time of the first data point. The specific method of obtaining the second data point can be implemented by referring to the introduction of step 401 above, and will not be repeated here.

[0128] In step 406, the updated anomaly detection model is used to detect whether the second data point is an abnormal data. The method of detecting whether the data point is an abnormal data can be implemented by referring to the introduction of step 402 above, which will not be repeated here.

[0129] Among them, when the second data point is normal data, the second data point can be used to update the data features stored in the storage module 210 again. After the data features are updated, the updated data features can be used again to update the anomaly detection model. For details, please refer to the above introduction to step 403 and step 404, which will not be repeated here.

[0130] In this way, the data features and anomaly detection models of streaming data can be continuously updated as data points are detected, making it possible to adapt the anomaly detection model to the new features of streaming data without having to query and save all historical data points and data points in a fixed time window. Therefore, by using the updated anomaly detection model to detect subsequent data points, the accuracy of detection can be improved and the detection effect can be guaranteed.

[0131] In particular, the univariate time series streaming anomaly detection algorithm based on extreme value theory is a commonly used algorithm for streaming data detection. In the traditional scheme of using this algorithm for streaming data detection, it is necessary to store and query the full amount of peak data points detected in sequence from the first detection to the latest detection to calculate the threshold value. As time flows, there will be more and more peak data points, which consumes a lot of storage space and the performance of the database or memory. Through the streaming data anomaly detection method provided by the embodiment of the present application, it is only necessary to save the data features of the streaming data, reducing the consumption of storage space. Moreover, the data features are continuously updated as the detection proceeds, so that the latest data features of the streaming data can be accurately reflected. With the update of the data features, the anomaly detection model is also continuously updated, which ensures the adaptability of the anomaly detection model to the new characteristics of the streaming data, thereby ensuring the accuracy of the detection.

[0132] The embodiment of the present application also provides a streaming data anomaly detection device 800. The device 800 stores data features of streaming data, where the streaming data includes data points generated in time sequence. Figure 8 As shown, the device 800 includes:

[0133] An acquisition module 810, configured to acquire a first data point of the streaming data;

[0134] A detection module 820, configured to detect whether the first data point is abnormal data by using an abnormality detection model, wherein the abnormality detection model is obtained based on the data feature;

[0135] A first updating module 830, configured to update the data feature based on the first data point when the first data point is not abnormal data;

[0136] A second updating module 840, configured to update the anomaly detection model based on the updated data features;

[0137] The acquisition module 810 is further used to acquire a second data point of the streaming data, where the generation time of the second data point is after the generation time of the first data point;

[0138] The detection module 820 is further configured to detect whether the second data point is abnormal data by using the updated abnormality detection model.

[0139] In some embodiments, the device 800 stores M peak data points of the streaming data, and the data features include K peak data points among the M peak data points, K is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to K; the first update module 830 is used to: replace the third data point among the M peak data points with the first data point to obtain updated M peak data points; and update the K peak data points based on the updated M peak data points.

[0140] In an example of this embodiment, the first update module 830 is used to: when the life cycle of the third data point times out, replace the third data point with the first data point; or, when the peak data point is the maximum peak data point and the first data point is greater than the third data point, replace the third data point with the first data point; or, when the peak data point is the minimum peak data point and the first data point is less than the third data point, replace the third data point with the first data point.

[0141] In some embodiments, the apparatus 800 further includes: an output module (not shown), configured to output a detection result of the first data point when the first data point is abnormal data, wherein the detection result is used to indicate that the first data point is abnormal data.

[0142] In some embodiments, the data feature is obtained based on N data points of the streaming data; wherein, the generation time of the N data points is earlier than the generation time of the first data point, and N is a positive integer greater than 1.

[0143] Among them, the acquisition module 810, the detection module 820, the first update module 830 and the second update module 840 can all be implemented by software, or can be implemented by hardware. Exemplarily, the following takes the acquisition module 810 as an example to introduce the implementation of the acquisition module 810. Similarly, the implementation of the detection module 820, the first update module 830 and the second update module 840 can refer to the implementation of the acquisition module 810.

[0144] As an example of a software functional unit, the acquisition module 810 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the acquisition module 810 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0145] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0146] As an example of a hardware functional unit, the acquisition module 810 may include at least one computing device, such as a server, etc. Alternatively, the acquisition module 810 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0147] The multiple computing devices included in the acquisition module 810 can be distributed in the same region or in different regions. The multiple computing devices included in the acquisition module 810 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the acquisition module 810 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0148] It should be noted that, in other embodiments, the acquisition module 810 may be used to execute Figure 4 In any step of the method shown, the detection module 820 can be used to perform Figure 4 In any step of the method shown, the first update module 820 can be used to perform Figure 4 In any step of the method shown, the second update module 840 can be used to perform Figure 4 The steps implemented by the acquisition module 810, the detection module 820, the first update module 830 and the second update module 840 can be specified as needed, and the steps implemented by the acquisition module 810, the detection module 820, the first update module 830 and the second update module 840 can be specified as needed. Figure 4 The different steps in the method shown are used to implement the overall functions of the apparatus 800 .

[0149] The present application also provides a computing device 900. Fig. 9As shown, the computing device 900 includes: a bus 902, a processor 904, a memory 906, and a communication interface 908. The processor 904, the memory 906, and the communication interface 908 communicate through the bus 902. The computing device 900 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 900.

[0150] The bus 902 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 The bus 902 may include a path for transmitting information between various components of the computing device 900 (eg, the memory 906, the processor 904, and the communication interface 908).

[0151] The processor 904 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0152] The memory 906 may include a volatile memory, such as a random access memory (RAM). The memory 906 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0153] The memory 906 stores executable program codes, and the processor 904 executes the executable program codes to respectively implement the functions of the acquisition module 810, the detection module 820, the first update module 830 and the second update module 840, thereby implementing Figure 4 That is, the memory 906 stores the method for executing Figure 4 Instructions for the method shown.

[0154] The communication interface 908 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 900 and other devices or communication networks.

[0155] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0156] like Fig.10 As shown, the computing device cluster includes at least one computing device 900. The memory 906 in one or more computing devices 900 in the computing device cluster may store the same Figure 4 Instructions for the method shown.

[0157] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store a program for executing Figure 4 In other words, the combination of one or more computing devices 900 can jointly execute instructions for executing Figure 4 Instructions for the method shown.

[0158] It should be noted that the memory 906 in different computing devices 900 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the apparatus 800. That is, the instructions stored in the memory 906 in different computing devices 900 may implement the functions of one or more of the acquisition module 810, the detection module 820, the first update module 830, and the second update module 840.

[0159] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.11 A possible implementation is shown. Fig.11 As shown, two computing devices 900A and 900B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 906 in the computing device 900A stores instructions for executing the functions of the acquisition module 810 and the first update module 830. At the same time, the memory 906 in the computing device 900B stores instructions for executing the functions of the detection module 820 and the second update module 840.

[0160] It should be understood that Fig.11The functions of the computing device 900A shown in FIG. 9A may also be completed by multiple computing devices 900. Similarly, the functions of the computing device 900B may also be completed by multiple computing devices 900.

[0161] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig.10 and Fig.11 The difference is that the memory 906 in one or more computing devices 900 in the computing device cluster may store the same memory for executing Figure 4 Instructions for the method shown.

[0162] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store a program for executing Figure 4 In other words, the combination of one or more computing devices 900 can jointly execute instructions for executing Figure 4 Instructions for the method shown.

[0163] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes Figure 4 The method shown.

[0164] The present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a host migration device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute Figure 4 The method shown.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A streaming data anomaly detection method, characterized in that: The method is applied to a detection device, the detection device stores data features of streaming data, the streaming data includes data points generated in time sequence, and the method includes: Acquire a first data point of the streaming data; Using an anomaly detection model, detecting whether the first data point is abnormal data, wherein the anomaly detection model is obtained based on the data features; When the first data point is not abnormal data, updating the data feature based on the first data point; Based on the updated data features, updating the anomaly detection model; Acquire a second data point of the streaming data, where a time point at which the second data point is generated is later than a time point at which the first data point is generated; The updated anomaly detection model is used to detect whether the second data point is abnormal data.

2. The method according to claim 1, characterized in that The detection device stores M peak data points of the streaming data, the data feature includes K peak data points among the M peak data points, K is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to K; The updating of the data feature based on the first data point includes: Replacing a third data point among the M peak data points with the first data point to obtain updated M peak data points; Based on the updated M peak data points, the K peak data points are updated.

3. The method according to claim 2, characterized in that The step of replacing a third data point among the M peak data points with the first data point comprises: When the life cycle of the third data point times out, the third data point is replaced by the first data point; or, When the peak data point is a maximum peak data point and the first data point is greater than the third data point, the third data point is replaced by the first data point; or, When the peak data point is a minimum peak data point and the first data point is smaller than the third data point, the third data point is replaced by the first data point.

4. The method according to any one of claims 1 to 3, characterized in that The method further includes: in a case where the first data point is abnormal data, outputting a detection result of the first data point, wherein the detection result is used to indicate that the first data point is abnormal data.

5. The method according to any one of claims 1 to 4, characterized in that The data feature is obtained based on N data points of the streaming data; wherein, the generation time of the N data points is earlier than the generation time of the first data point, and N is a positive integer greater than 1.

6. A streaming data anomaly detection device, characterized in that: The device stores data features of streaming data, the streaming data includes data points generated in time sequence, and the device includes: An acquisition module, used for acquiring a first data point of the streaming data; A detection module, configured to detect whether the first data point is abnormal data by using an abnormality detection model, wherein the abnormality detection model is obtained based on the data feature; A first updating module, configured to update the data feature based on the first data point when the first data point is not abnormal data; A second updating module, used for updating the anomaly detection model based on the updated data features; The acquisition module is further used to acquire a second data point of the streaming data, where the generation time of the second data point is after the generation time of the first data point; The detection module is also used to detect whether the second data point is abnormal data using the updated abnormality detection model.

7. The device according to claim 6, characterized in that The device stores M peak data points of the streaming data, the data feature includes K peak data points of the M peak data points, K is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to K; The first update module is used for: Replacing a third data point among the M peak data points with the first data point to obtain updated M peak data points; Based on the updated M peak data points, the K peak data points are updated.

8. The device according to claim 7, characterized in that The first update module is used for: When the life cycle of the third data point times out, the third data point is replaced by the first data point; or, When the peak data point is a maximum peak data point and the first data point is greater than the third data point, the third data point is replaced by the first data point; or, When the peak data point is a minimum peak data point and the first data point is smaller than the third data point, the third data point is replaced by the first data point.

9. The device according to any one of claims 6 to 8, characterized in that The device further includes: an output module, configured to output a detection result of the first data point when the first data point is abnormal data, wherein the detection result is used to indicate that the first data point is abnormal data.

10. The device according to any one of claims 6 to 9, characterized in that: The data feature is obtained based on N data points of the streaming data; wherein, the generation time of the N data points is earlier than the generation time of the first data point, and N is a positive integer greater than 1.

11. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 5.

13. A computer program product comprising instructions, characterized in that When the instructions are executed by a computer device cluster, the computer device cluster is caused to perform the method according to any one of claims 1 to 5.

Images