Abnormal Detection Device and Abnormal Detection Method

Abstract

An anomaly detection device and an anomaly detection method are provided. The anomaly detection method includes: receiving a historical sensing data set and a current sensing data set; generating a ratio of a training data set and a test data set based on the historical sensing data set according to an automated machine learning algorithm; obtaining a first subset of sensing data from the current sensing data set, and splitting the first subset of sensing data into a first training data subset and a first test data subset according to the ratio; training a first anomaly detection model according to the first training data subset; determining whether an anomaly event occurs according to the first anomaly detection model and the first test data subset to generate a first determination result; and outputting the first determination result. The present invention can generate an accurate determination result of an anomaly event according to a concept drift data set.

Description

Technical Field

[0001] The present invention relates to an anomaly detection device and an anomaly detection method.

Background Art

[0002] Anomaly detection technology can analyze signals to determine whether an abnormal event has occurred. Anomaly detection technology can be applied to a wide range of fields, including the financial management field or the industrial field, etc. For example, anomaly detection technology can be used to detect abnormal transaction events or to detect machine failures, etc. Generally, the training of an anomaly detection model requires the use of a sensed data set representing normal conditions. However, the sensed data sets generated in actual application scenarios (for example, the data obtained by sensors used to monitor the condition of machines in a factory) often have the property of concept drifting. That is to say, the natural wear of the machine in the normal use state may cause the sensor to obtain a gradually drifting data set. The anomaly detection model trained using the gradually drifting data set cannot correctly detect the occurrence of an abnormal event.

[0003] Therefore, how to train a good anomaly detection model in the case where the sensed data set has the property of concept drifting is one of the goals that those skilled in the art strive for.

Summary of the Invention

[0004] The present invention provides an anomaly detection device and an anomaly detection method, which can generate an accurate judgment result of an abnormal event according to a concept-drifting data set.

[0005] An anomaly detection device of the present invention includes a processor and a transceiver. The processor is coupled to the transceiver, and the processor is configured to perform: receiving a historical sensed data set and a current sensed data set through the transceiver; generating a ratio of a training data set and a test data set according to the historical sensed data set based on an automated machine learning algorithm; obtaining a first sensed data subset from a first period of the current sensed data set, and dividing the first sensed data subset into a first training data subset and a first test data subset according to the ratio; obtaining a second sensed data subset different from the first sensed data subset from a second period of the current sensed data set, and dividing the second sensed data subset into a second training data subset and a second test data subset according to the ratio, wherein the first period is different from the second period; training a first anomaly detection model according to the first training data subset; judging whether an abnormal event has occurred according to the first anomaly detection model and the first test data subset to generate a first judgment result; outputting the first judgment result through the transceiver; training a second anomaly detection model according to the second training data subset; judging whether an abnormal event has occurred according to the second anomaly detection model and the second test data subset to generate a second judgment result; and outputting the second judgment result through the transceiver.

[0006] In one embodiment of the present invention, the above-mentioned processor is further configured to execute: obtaining a first sensed data subset and a second sensed data subset from the current sensed data set according to a window function.

[0007] In one embodiment of the present invention, the above-mentioned historical sensed data set and the current sensed data set correspond to a concept drift data set.

[0008] In one embodiment of the present invention, the above-mentioned processor trains a first anomaly detection model based on one of the following: one-class support vector method, isolation forest method, and autoencoder neural network.

[0009] In one embodiment of the present invention, the above-mentioned automated machine learning algorithm includes one of the following: reinforcement learning algorithm, grid search algorithm, Bayesian optimization algorithm, and random search algorithm.

[0010] In one embodiment of the present invention, the above-mentioned processor inputs a first test data subset into the first anomaly detection model to calculate an outlier, and determines that an anomaly event has occurred and generates a first judgment result in response to the outlier being greater than a threshold value.

[0011] An anomaly detection method of the present invention includes: receiving a historical sensed data set and a current sensed data set; generating a ratio of a training data set and a test data set based on the historical sensed data set by an automated machine learning algorithm; obtaining a first sensed data subset from a first time period of the current sensed data set, and dividing the first sensed data subset into a first training data subset and a first test data subset according to the ratio; obtaining a second sensed data subset different from the first sensed data subset from a second time period of the current sensed data set, and dividing the second sensed data subset into a second training data subset and a second test data subset according to the ratio, wherein the first time period is different from the second time period; training a first anomaly detection model according to the first training data subset; determining whether an anomaly event has occurred according to the first anomaly detection model and the first test data subset to generate a first judgment result; outputting the first judgment result; training a second anomaly detection model according to the second training data subset; determining whether an anomaly event has occurred according to the second anomaly detection model and the second test data subset to generate a second judgment result; and outputting the second judgment result.

[0012] In one embodiment of the present invention, the above-mentioned anomaly detection method further includes: obtaining a first sensed data subset and a second sensed data subset from the current sensed data set according to a window function.

[0013] In one embodiment of the present invention, the above-mentioned historical sensed data set and the current sensed data set correspond to a concept drift data set.

[0014] In an embodiment of the present invention, the step of training the first anomaly detection model according to the first training data subset includes: training the first anomaly detection model based on one of the following: one-class support vector method, isolation forest method, and autoencoder neural network.

[0015] In an embodiment of the present invention, the above-mentioned automated machine learning algorithm includes one of the following: reinforcement learning algorithm, grid search algorithm, Bayesian optimization algorithm, and random search algorithm.

[0016] In an embodiment of the present invention, the step of determining whether an anomaly event occurs to generate a first determination result includes: inputting the first test data subset into the first anomaly detection model to calculate an anomaly value; and determining that an anomaly event occurs to generate a first determination result in response to the anomaly value being greater than a threshold.

[0017] Based on the above, the present invention can automatically generate an anomaly detection model applicable to a concept drift dataset. Therefore, the present invention can correctly determine whether an anomaly event occurs without manual intervention by personnel to adjust parameters.

Description of the Drawings

[0018] Figure 1 A schematic diagram of an anomaly detection device is illustrated according to an embodiment of the present invention.

[0019] Figure 2 A schematic diagram of the current sensed dataset is illustrated according to an embodiment of the present invention.

[0020] Figure 3 A schematic diagram of the test data subset and the anomaly value is illustrated according to an embodiment of the present invention.

[0021] Figure 4 A flowchart of an anomaly detection method is illustrated according to an embodiment of the present invention.

[0022]

Symbol Description

[0023] 10: Current sensed data

[0024] 100: Anomaly detection device

[0025] 110: Processor

[0026] 120: Storage medium

[0027] 130: Transceiver

[0028] 20: Threshold

[0029] 30, 40: Sensed data subset

[0030] 31, 41: Training data subset

[0031] 32, 42: Subsets of test data

[0032] S401, S402, S403, S404, S405, S406: Steps

[0033] T: Window function

[0034] t0, t1, t2, t3: Time points.

Detailed implementation manners

[0035] In order to make the content of the present invention more easily understood, the following specific examples are given as examples that the present invention can actually be implemented according to. In addition, wherever possible, elements / components / steps with the same reference numerals are used in the drawings and detailed implementation manners to represent the same or similar components.

[0036] Figure 1 According to an embodiment of the present invention, a schematic diagram of an anomaly detection device 100 is illustrated. The anomaly detection device 100 can be used to detect whether an anomaly event occurs in a dataset with concept drift characteristics. The anomaly detection device 100 may include a processor 110, a storage medium 120, and a transceiver 130.

[0037] The processor 110 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose micro control units (MCUs), microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), graphics processing units (GPUs), image signal processors (ISPs), image processing units (IPUs), arithmetic logic units (ALUs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other similar elements or combinations of the above elements. The processor 110 can be coupled to the storage medium 120 and the transceiver 130, and access and execute multiple modules and various application programs stored in the storage medium 120 to perform the functions of the anomaly detection device 100.

[0038] The storage medium 120 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar components, or a combination of the above components, and is used to store multiple modules or various application programs that can be executed by the processor 110.

[0039] The transceiver 130 transmits and receives signals in a wireless or wired manner. The transceiver 130 can also perform operations such as low-noise amplification, impedance matching, mixing, up or down frequency conversion, filtering, amplification, and similar operations.

[0040] Figure 2 A schematic diagram illustrating the current sensing data set 10 according to an embodiment of the present invention. The processor 110 can obtain the current sensing data set 10 from the sensor through the transceiver 130, where the sensor can include various types of sensors such as a vibration sensor, a temperature sensor, a humidity sensor, or a pressure sensor, and the present invention is not limited thereto. For the convenience of description, in the following embodiments, it is assumed that the sensor is a vibration sensor, and the current sensing data set 10 is a data set representing the current vibration state of the machine. The current sensing data set 10 can have the property of concept drift. For example, the current sensing data set 10 can include a sudden drifting data set, a gradual drifting data set, an incremental drifting data set, or a recurring drifting data set, but the present invention is not limited thereto.

[0041] The processor 110 can obtain one or more sensing data subsets from the current sensing data set 10. For example, the processor 110 can extract the sensing data subset 30 from the time point t1 to the time point t2 of the current sensing data set 10 according to the window function T. The processor 110 can extract the sensing data subset 40 from the time point t2 to the time point t3 of the current sensing data set 10 according to the window function T. In one embodiment, the size of the window function T can be customized by the user.

[0042] The processor 110 can divide the sensing data subset into a training data subset and a test data subset according to a preset ratio. For example, the processor 110 can divide the sensing data subset 30 into a training data subset 31 and a test data subset 32 according to a preset ratio. The processor 110 can divide the sensing data subset 40 into a training data subset 41 and a test data subset 42 according to a preset ratio.

[0043] The processor 110 may generate a preset ratio based on an automated machine learning algorithm (AutoML). For example, the processor 110 may obtain a historical sensing data set from sensors through the transceiver 130. Similar to the current sensing data set 10, the historical sensing data set may have the property of concept drift. The processor 110 may find the default ratio of the best training data subset and test data subset based on the automated machine learning algorithm according to the historical sensing data set, so that the anomaly detection model generated according to the training data subset has the best performance (for example: the accuracy rate of the anomaly detection model detecting anomaly events in the test data subset can reach the highest). The automated machine learning algorithm may include but is not limited to reinforcement learning algorithm, grid search algorithm, Bayesian optimization algorithm or random search algorithm.

[0044] The processor 110 may train a corresponding anomaly detection model according to the training data subset. The anomaly detection model may be used to detect whether the test data subset corresponding to the training data subset is related to an anomaly event. For example, the processor 110 may train an anomaly detection model (hereinafter referred to as the "first anomaly detection model") for detecting the test data subset 32 according to the training data subset 31, and may train an anomaly detection model (hereinafter referred to as the "second anomaly detection model") for detecting the test data subset 42 according to the training data subset 41. That is, the processor 110 may automatically generate a new anomaly detection model according to the period relative to the window function to detect data in different time periods. Therefore, even if the current sensing data is affected by concept drift, the processor 110 can still accurately detect anomaly events according to the sensing data through the latest anomaly detection model. The algorithms for training the anomaly detection model may include but are not limited to one-class support vector machine (one-class SVM), isolated forest (iForest) or autoencoder.

[0045] Specifically, the processor 110 may input the test data subset into the anomaly detection model. The anomaly detection model may output anomaly values corresponding to the data in the test data subset. The processor 110 may generate a judgment result according to the output of the anomaly detection model. For example, the processor 110 may judge that an anomaly event has occurred based on the anomaly value output by the anomaly detection model being greater than the threshold. The processor 110 may output the judgment result through the transceiver 130 to alert the user of the occurrence of the anomaly event.

[0046] Taking the test data subset 42 as an example, Figure 3 The schematic diagram of the test data subset 42 and the outliers is illustrated according to an embodiment of the present invention. The processor 110 can input the test data subset 42 into the second anomaly detection model to generate outliers corresponding to the data in the test data subset 42. As can be seen from Figure 3 It can be seen that at time point t0, the machine tool had abnormal vibrations, resulting in the sensor detecting abnormal amplitude changes. The processor 110 can determine that the outlier at time point t0 is greater than the threshold 20 according to the outlier output by the second anomaly detection model. Therefore, the processor 110 can determine that the data corresponding to time point t0 in the test data subset 42 represents the occurrence of an abnormal event. The processor 110 can output the judgment result through the transceiver 130 to alert the user that an abnormal event occurred at time point t0.

[0047] Based on the above, even if the current sensing data set 10 is affected by concept drift, resulting in a large difference between the sensing data subset 30 and the sensing data subset 40. The anomaly detection device 100 will not determine the occurrence of an abnormal event based on the difference. And the abnormal event that occurred at time point t0 can still be accurately detected by the anomaly detection device 100.

[0048] Figure 4 The flowchart of an anomaly detection method is illustrated according to an embodiment of the present invention, where the anomaly detection method can be implemented by the anomaly detection device 100 as shown in Figure 1 In step S401, a historical sensing data set and a current sensing data set are received. In step S402, based on an automated machine learning algorithm, the ratio of the training data set and the test data set is generated according to the historical sensing data set. In step S403, a first sensing data subset is obtained from the current sensing data set, and the first sensing data subset is divided into a first training data subset and a first test data subset according to the ratio. In step S404, a first anomaly detection model is trained according to the first training data subset. In step S405, it is determined whether an abnormal event occurs according to the first anomaly detection model and the first test data subset to generate a first judgment result. In step S406, the first judgment result is output.

[0049] In summary, the present invention can automatically generate an anomaly detection model applicable to concept drift datasets over time, without the need for manual intervention by personnel to perform tasks such as adjusting parameters used in the algorithm (e.g., sampling time or training dataset size, etc.) or removing abnormal data. The anomaly detection model generated by the present invention can retain the ability to detect abnormal events without generating false alarms. The present invention is applicable to various different application scenarios, such as detecting abnormal parameters such as vibrations, environmental humidity, temperature, or pressure, thereby comprehensively reducing the damage caused by various abnormal events.

[0050] The above description is only a preferred embodiment of the present invention, and the scope of implementation of the present invention cannot be limited thereby. That is, all simple equivalent changes and modifications made according to the claims and the content of the specification of the present invention still fall within the scope covered by the patent of the present invention. In addition, any embodiment or claim of the present invention does not have to achieve all the purposes, advantages, or features disclosed by the present invention. In addition, the abstract part and the title are only used to assist in the search of patent documents and do not limit the scope of rights of the present invention. In addition, the terms "first", "second", etc. mentioned in this specification or the scope of the patent application are only used to name elements or distinguish different embodiments or scopes, and do not limit the upper or lower limits of the number of elements.

Claims

1. An anomaly detection device, comprising a transceiver and a processor, wherein, the processor is coupled to the transceiver, and the processor is configured to perform: receiving a historical sensing data set and a current sensing data set through the transceiver; generating a ratio of a training data set and a test data set based on the historical sensing data set according to an automated machine learning algorithm; obtaining a first subset of sensing data from a first time period of the current sensing data set, and splitting the first subset of sensing data into a first training data subset and a first test data subset according to the ratio; obtaining a second subset of sensing data different from the first subset of sensing data from a second time period of the current sensing data set, and splitting the second subset of sensing data into a second training data subset and a second test data subset according to the ratio, wherein the first time period is different from the second time period; training a first anomaly detection model according to the first training data subset; judging whether an anomaly event occurs according to the first anomaly detection model and the first test data subset to generate a first judgment result; outputting the first judgment result through the transceiver; training a second anomaly detection model according to the second training data subset; judging whether the anomaly event occurs according to the second anomaly detection model and the second test data subset to generate a second judgment result; and outputting the second judgment result through the transceiver.

2. The anomaly detection device according to claim 1, wherein, the processor is further configured to perform: obtaining the first subset of sensing data and the second subset of sensing data from the current sensing data set according to a window function.

3. The anomaly detection device according to claim 1, wherein, the historical sensing data set and the current sensing data set correspond to a concept drift data set.

4. The anomaly detection device according to claim 1, wherein, the processor trains the first anomaly detection model based on one of the following: one-class support vector method, isolation forest method, and autoencoder neural network.

5. The anomaly detection device according to claim 1, wherein, the automated machine learning algorithm includes one of the following: reinforcement learning algorithm, grid search algorithm, Bayesian optimization algorithm, and random search algorithm.

6. The anomaly detection device according to claim 1, wherein, the processor inputs the first test data subset into the first anomaly detection model to calculate an anomaly value, and judges that the anomaly event occurs in response to the anomaly value being greater than a threshold value to generate the first judgment result.

7. An anomaly detection method, comprising: receiving a historical sensing data set and a current sensing data set; generating a ratio of a training data set and a test data set based on the historical sensing data set according to an automated machine learning algorithm; obtaining a first subset of sensing data from a first time period of the current sensing data set, and splitting the first subset of sensing data into a first training data subset and a first test data subset according to the ratio; Obtain a second subset of sensed data different from the first subset of sensed data from a second time period of the current sensed data set, and divide the second subset of sensed data into a second training data subset and a second test data subset according to the ratio, wherein the first time period is different from the second time period; Train a first anomaly detection model according to the first training data subset; Determine whether an anomaly event occurs according to the first anomaly detection model and the first test data subset to generate a first determination result; Output the first determination result; Train a second anomaly detection model according to the second training data subset; Determine whether the anomaly event occurs according to the second anomaly detection model and the second test data subset to generate a second determination result; and Output the second determination result.

8. The anomaly detection method according to claim 7, further comprising: Obtain the first subset of sensed data and the second subset of sensed data from the current sensed data set according to a window function.

9. The anomaly detection method according to claim 7, wherein, the historical sensed data set and the current sensed data set correspond to a concept drift data set.

10. The anomaly detection method according to claim 7, wherein, the step of training the first anomaly detection model according to the first training data subset includes: training the first anomaly detection model based on one of the following: One-class support vector method, isolation forest method, and autoencoder neural network.

11. The anomaly detection method according to claim 7, wherein, the automated machine learning algorithm includes one of the following: Reinforcement learning algorithm, grid search algorithm, Bayesian optimization algorithm, and random search algorithm.

12. The anomaly detection method according to claim 7, wherein, the step of determining whether the anomaly event occurs to generate the first determination result includes: Input the first test data subset into the first anomaly detection model to calculate an anomaly value; and Determine that the anomaly event occurs to generate the first determination result in response to the anomaly value being greater than a threshold.

Images