Anomaly detection device, anomaly detection method, and anomaly detection program

Abstract

In this abnormality detection device (10), an acquisition unit (15a) acquires and causes past traffic data to be stored in a buffer area (14a) of a storage unit (14). A calculation unit (15b) applies the traffic data in the buffer area (14a) to a predetermined recurrence formula, thereby calculating the average and variance of the traffic data. An analysis unit (15c) uses the calculated average and variance to calculate a Z-score representative of the deviation of target traffic data from the average, and uses the Z-score to analyze whether this traffic data is abnormal or not.

Description

Technical Field

[0001] The present invention relates to an abnormality detection device, an abnormality detection method, and an abnormality detection program.

Background Art

[0002] In order for a user to stably use services such as so-called OTT (Over-The-Top) services provided through a network, it is essential for network operators to detect abnormalities in the traffic in the network and address the causes.

[0003] Conventionally, daily traffic data is classified into patterns using clustering or the like and stored, a similar pattern is selected from newly input data, and when the difference from that pattern is large, an abnormality detection technique for traffic that determines it as an abnormality is known (see Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, according to the prior art, the cost for abnormality detection may become extremely high. For example, the more traffic data there is for each stored pattern, the more data access and calculations for comparison occur, and thus there is a problem that it requires time and computational cost.

[0006] The present invention has been made in view of the above, and an object thereof is to perform abnormality detection of traffic while suppressing costs.

Means for Solving the Problems

[0007] To solve the above-mentioned problems and achieve the objective, the anomaly detection device according to the present invention is characterized by comprising: an acquisition unit that acquires past traffic data and stores it in a buffer area of ​​a storage unit; a calculation unit that calculates the mean and variance of the traffic data by applying the traffic data in the buffer area to a predetermined recurrence relation; and an analysis unit that uses the calculated mean and variance to calculate a Z score representing the deviation of the traffic data to be processed from the mean, and uses the Z score to analyze whether the traffic data is anomaly or not. [Effects of the Invention]

[0008] According to the present invention, it is possible to detect anomalies in traffic while keeping costs down. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a diagram illustrating the overview of the anomaly detection device of this embodiment. [Figure 2] Figure 2 is a diagram illustrating the overview of the anomaly detection device of this embodiment. [Figure 3] Figure 3 is a diagram illustrating the overview of the anomaly detection device of this embodiment. [Figure 4] Figure 4 is a diagram illustrating the overview of the anomaly detection device of this embodiment. [Figure 5] Figure 5 is a schematic diagram illustrating the general configuration of the anomaly detection device of this embodiment. [Figure 6] Figure 6 is a diagram illustrating the processing of the calculation unit. [Figure 7] Figure 7 is a flowchart showing the anomaly detection processing procedure. [Figure 8] Figure 8 shows an example of a computer running an anomaly detection program. [Modes for carrying out the invention]

[0010] Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to this embodiment. Furthermore, in the drawings, the same parts are denoted by the same reference numerals.

[0011] [Overview of Anomaly Detection Device] Figures 1 to 4 are diagrams illustrating the overview of the anomaly detection device of this embodiment. The anomaly detection device of this embodiment compares the value of the traffic data to be analyzed for anomaly with a past set of normal traffic data, and determines the data to be anomaly if the deviation from the mean of the traffic data set is greater than a predetermined threshold Th. Specifically, the anomaly detection device uses the mean μ and standard deviation σ within the range indicated by the double arrows in Figure 1 as statistics of the past set of normal traffic data, and calculates a Z score z representing the degree of deviation from the mean of the latest traffic data x, as shown in equation (1) below.

[0012]

number

[0013] In the example shown in Figure 1, data from the same time period on past weekdays is selected as the comparison target for the latest data x from Thursday. In this way, users can arbitrarily set the days of the week to be compared using a predetermined configuration file.

[0014] The anomaly detection device then uses the calculated Z-score z as an indicator for analyzing whether the traffic data x is anomaly or not. In other words, the anomaly detection device detects anomalies in the traffic data x when z, normalized by the mean and variance as shown in equation (1) above, is greater than a predetermined threshold Th, as a larger absolute value of the Z-score indicates an anomaly that deviates from the mean. Since the standard deviation σ is defined as the square root √v of the variance V, the Z-score will be expressed in terms of the variance V below.

[0015] Thus, the Z-score of the traffic data to be analyzed is represented by the mean and variance of the past normal traffic data. For example, as illustrated in FIG. 2, the traffic data x at time t on day n for which analysis is performed i at ti,n (the ti subscript of "t i ") has a Z-score z ti,n which is expressed as in the following equation (2) using the mean E i and variance V ti,n-1 at the same time t on the previous day n-1. Here, the previous day in this case does not necessarily have to be the day before in terms of the date. ti,n-1

[0016]

Equation

[0017] When the Z-score z ti,n obtained here is greater than a predetermined threshold Th, abnormal detection of the traffic data x at that time t i becomes possible. ti,n

[0018] Here, as shown in FIG. 3, in order to calculate the Z-score at time t on the next day n+1, the abnormal detection device uses the mean E i and variance V i at time t of the traffic data group on a certain day n ti,n ti,n ​​​We calculate the mean and variance. Here, the mean and variance, which are basic statistics of a data set, can be obtained from a certain amount of data, and the more data there is, the higher the generality and reliability of the statistics. In the example shown in Figure 3, it is considered preferable to obtain the mean and variance from the data set in the range enclosed by the solid line (2), which goes further back in time and has a larger quantity, rather than from the data set in the range enclosed by the dashed line (1), because the generality and reliability of the obtained mean and variance will be higher. On the other hand, obtaining the mean and variance from the data set in the range enclosed by the solid line (2) rather than from the data set in the range enclosed by the dashed line (1) increases the number of accesses to the memory that holds past traffic data and the number of calculations, thus increasing the cost.

[0019] Therefore, the anomaly detection device stores traffic data for at least w+1 points with a window width w in the past direction from the current time (the timing of calculation of the mean and variance) in a buffer area such as main memory that can be accessed at high speed. In this case, as illustrated in Figure 4, time t i In +w / 2, the buffer area contains time t i -w / 2 to time t i Traffic data for w+1 points with a window width w up to +w / 2 is stored. As a result, the anomaly detection device detects time t i In +w / 2, it becomes possible to obtain n data for the day simply by accessing the buffer area at high speed without accessing the memory unit.

[0020] Furthermore, the anomaly detection device detects time t on the day n. i Average E ti,n and variance V ti,n When calculating, the time t calculated on the previous day n-1 i Average E ti,n-1 and variance V ti,n-1 It utilizes the calculated average E. ti,n and variance V ti,n It is stored in the memory unit.

[0021] Thus, the anomaly detection device detects a time t on a certain day n. i Average Eti,n and variance V ti,n When calculating the average E, at least n traffic data for the current day can be obtained from the buffer area without accessing the storage unit. This allows the anomaly detection device to efficiently obtain a portion of past traffic data faster than obtaining all of the past traffic data from the storage unit, thereby averaging E ti,n and variance V ti,n It becomes possible to calculate this.

[0022] Furthermore, the anomaly detection device detects the average E of n-1 from the previous day. ti,n-1 and variance V ti,n-1 Using the mean E ti,n and variance V ti,n By calculating this, it becomes possible to calculate a more general and likely mean and variance. Therefore, anomaly detection devices can perform anomaly detection using the Z-score of traffic data with high efficiency and accuracy.

[0023] [Configuration of the anomaly detection device] Figure 5 is a schematic diagram illustrating the general configuration of the anomaly detection device of this embodiment. As illustrated in Figure 5, the anomaly detection device 10 of this embodiment is implemented using a general-purpose computer such as a personal computer, and comprises an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.

[0024] The input unit 11 is implemented using an input device such as a keyboard or mouse, and in response to input operations by the operator, it inputs various instruction information, such as processing start, to the control unit 15. The output unit 12 is implemented using a display device such as a liquid crystal display, a printing device such as a printer, etc. For example, the output unit 12 displays the results of the abnormality detection process described later.

[0025] The communication control unit 13 is implemented using a NIC (Network Interface Card) or the like, and controls communication between the control unit 15 and external devices via telecommunication lines such as a LAN (Local Area Network) or the Internet. For example, the communication control unit 13 controls communication between the control unit 15 and a management device that manages various types of information.

[0026] The storage unit 14 is implemented by semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks and optical discs. The storage unit 14 pre-stores processing programs for operating the anomaly detection device 10, and data used during the execution of the processing programs, or temporarily stores them each time processing is performed. The storage unit 14 may also be configured to communicate with the control unit 15 via the communication control unit 13. In this embodiment, the storage unit 14 has a buffer area 14a. It also stores the calculation results 14b of the mean, variance, Z score, etc., in the anomaly detection processing described later.

[0027] The control unit 15 is implemented using a CPU (Central Processing Unit) or the like, and executes a processing program stored in memory. As a result, the control unit 15 functions as an acquisition unit 15a, a calculation unit 15b, and an analysis unit 15c, as illustrated in Figure 5, and performs anomaly detection processing. Note that each or part of these functional units may be implemented on different hardware. For example, the analysis unit 15c may be implemented on a different device from the other functional units. Furthermore, the control unit 15 may also include other functional units.

[0028] The acquisition unit 15a acquires past traffic data and stores it in the buffer area 14a of the storage unit 14. For example, the acquisition unit 15a acquires traffic data from a management device that manages past traffic data collected from the network, via the input unit 11 or the communication control unit 13. Then, the acquisition unit 15a stores the acquired traffic data in the buffer area 14a. Specifically, the acquisition unit 15a acquires the traffic data at time t on day n, as illustrated in Figure 4.i -w / 2 to time t i Traffic data for w+1 points within a predetermined window width w up to +w / 2 is stored in buffer area 14a. The window width w can be set arbitrarily by the user.

[0029] The calculation unit 15b calculates the mean and variance of the traffic data in the buffer area 14a by applying a predetermined recurrence relation.

[0030] Furthermore, the calculation unit 15b stores the calculated mean and variance in the calculation result 14b of the storage unit 14. In this case, the calculation unit 15b may store the calculation result 14b in an area other than the buffer area 14a.

[0031] Furthermore, when the calculation unit 15b calculates the mean and variance of the next acquired traffic data, it uses the mean and variance of the previous day's calculation result 14b, which was calculated earlier.

[0032] Here, Figure 6 is a diagram illustrating the processing of the calculation unit. Specifically, as shown in Figure 6, the calculation unit 15b uses the traffic data acquired and stored in the buffer area 14a and the average E of the previous day n-1 of the calculation result 14b. ti,n-1 and variance V ti,n-1 Apply the following equation (3) to the recurrence relation. In this way, the calculation unit 15b calculates the average E exponentially. ti,n and variance V ti,n We seek.

[0033]

number

[0034] In equation (3) above, the calculation process of the variance v j The average calculation process e j The following is used. Note that the initial value e -1 =E ti,n-1 ,v -1 =V ti,n-1 Let's assume that the last term e w , v wUsing E ti,n =e w ,V ti,n =v w Here, the mean and variance (≠0) of the previous day on the first day of traffic data collection can be arbitrarily set by the user; for example, mean = 0 and variance = 1.

[0035] The recurrence relation in equation (3) above is, for example, when time t i Average E ti,n Expanding on this, it can be expressed as follows: (4)

[0036]

number

[0037] Here, the parameter α is any predetermined value in the range 0 < α < 1, and the larger the value of α, the less it contributes to the calculation result of past traffic data. As illustrated in Figure 6, the coefficient including the parameter α in equation (4) above becomes smaller as the traffic data goes further back in time, and the contribution to the calculation result decreases.

[0038] In this way, the calculation unit 15b applies historical traffic data to a recurrence relation that is weighted such that the contribution of traffic data decreases as you go further back in time. This makes it possible to efficiently and quickly calculate the mean and variance using a small amount of historical traffic data covering a wide range of areas.

[0039] Returning to the explanation of Figure 5, the analysis unit 15c calculates a Z-score representing the deviation of the traffic data to be processed from the mean using the calculated mean and variance, and uses this Z-score to analyze whether the traffic data is abnormal or not. Specifically, the analysis unit 15c determines that the traffic data to be processed is abnormal if the calculated Z-score is greater than a predetermined threshold Th.

[0040] In this way, the anomaly detection device 10 can efficiently calculate the Z score using a highly general and reliable mean and variance, thereby enabling highly accurate anomaly detection using the Z score of traffic data.

[0041] [Anomaly detection processing] Next, with reference to Figure 7, the anomaly detection process by the anomaly detection device 10 according to this embodiment will be described. Figure 7 is a flowchart of the anomaly detection process procedure. The flowchart in Figure 7 shows, for example, the time t on a certain day n. i The process starts at the moment the user inputs an action to initiate it, with the current time being set.

[0042] First, the calculation unit 15b acquires w+1 traffic data points from the buffer area 14a up to the current time. The calculation unit 15b also acquires the time t of the n-1 day prior to the calculation result 14b of the storage unit 14 for Z-score calculation. i Average E ti,n-1 and variance V ti,n-1 Furthermore, the calculation unit 15b obtains the time t of the previous day n-1 of the calculation result 14b from the storage unit 14 for calculating the mean and variance of n days. i -Average E in w / 2 ti―w / 2,n-1 and variance V ti-w / 2,n-1 Obtain the following (Step S1).

[0043] Then, the calculation unit 15b calculates the traffic data of w+1 items in the acquired buffer area and E ti-w / 2,n-1 and variance V ti-w / 2,n-1 By applying and to a predetermined recurrence relation, the average E ti-w / 2,n and variance V ti-w / 2,n The calculation unit 15b calculates the average E ti-w / 2,n and variance V ti-w / 2,n The result 14b of the memory unit 14 is stored (step S3). The average E calculated here ti-w / 2,n and variance V ti-w / 2,n This is the time t on day n+1. i -w / 2 is used for anomaly detection.

[0044] Furthermore, the analysis unit 15c analyzes the current traffic data x ti And the average E for n-1 days was calculated. ti,n-1 and variance V ti,n-1 Using and, the Z score z ti The Z score z is calculated (step S4). Then, the analysis unit 15c calculates the Z score z ti The system uses this to analyze whether the traffic data at the current time is abnormal or not (step S5). Specifically, the analysis unit 15c calculates the Z score z ti If the value is greater than a predetermined threshold Th, the traffic data to be processed is determined to be abnormal. This completes the series of abnormality detection processes.

[0045] [effect] As described above, in the anomaly detection device 10 of this embodiment, the acquisition unit 15a acquires past traffic data and stores it in the buffer area 14a of the storage unit 14. The calculation unit 15b calculates the mean and variance of the traffic data by applying the traffic data in the buffer area 14a to a predetermined recurrence relation and stores the calculated mean and variance in the storage unit 14. The analysis unit 15c uses the mean and variance calculated the previous day for the same time on the previous day to calculate a Z score that represents the deviation of the traffic data to be processed from the mean, and uses the Z score to analyze whether the traffic data is anomaly or not.

[0046] Specifically, the analysis unit 15c determines that the traffic data to be processed is abnormal if the calculated Z score is greater than a predetermined threshold.

[0047] Thus, the anomaly detection device 10 detects a certain day n at time t i Average E ti,n and variance V ti,n When calculating the average E, at least n traffic data for the current day can be obtained from the buffer area without accessing the storage unit. As a result, the anomaly detection device 10 can efficiently obtain past traffic data faster than obtaining all past traffic data from the storage unit, thereby averaging E ti,n and variance V ti,nThis makes it possible to calculate the Z-score. Therefore, the anomaly detection device 10 can efficiently perform anomaly detection using the Z-score of traffic data while reducing the time and computational costs.

[0048] Furthermore, when the calculation unit 15b calculates the mean and variance of the next acquired traffic data, it uses the mean and variance of the previous day that it has further calculated. Specifically, each time new traffic data is acquired, the calculation unit 15b calculates the mean E of the time w / 2 times prior to that time. ti-w / 2,n , variance V ti-w / 2,n This is calculated using the mean and variance of the same time on the previous day. In this way, the anomaly detection device 10 can calculate a more general and reliable mean and variance by using the mean and variance of the previous day (n-1) to calculate the mean and variance of the current day used when calculating the Z score for the next day. Therefore, the anomaly detection device 10 can perform anomaly detection using the Z score of traffic data with high accuracy.

[0049] Furthermore, the calculation unit 15b applies historical traffic data to a recurrence relation that is weighted such that the contribution of traffic data decreases as you go further back in time. This makes it possible to efficiently calculate the mean and variance using a small amount of historical traffic data covering a wide range of periods.

[0050] [program] It is also possible to create a program that describes the processing performed by the anomaly detection device 10 according to the above embodiment in a language that can be executed by a computer. In one embodiment, the anomaly detection device 10 can be implemented by installing an anomaly detection program that performs the above anomaly detection processing as packaged software or online software on a desired computer. For example, by having the above anomaly detection program run on an information processing device, the information processing device can be made to function as the anomaly detection device 10. The information processing device referred to here includes general-purpose computers such as server computers, and desktop or notebook personal computers. In addition, the information processing device also includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System), and slate terminals such as PDA (Personal Digital Assistant). Furthermore, the functions of the anomaly detection device 10 may be implemented on a cloud server.

[0051] Figure 8 shows an example of a computer running an anomaly detection program. Computer 1000 includes, for example, memory 1010, CPU 1020, hard disk drive interface 1030, disk drive interface 1040, serial port interface 1050, video adapter 1060, and network interface 1070. These components are connected by bus 1080.

[0052] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031. The disk drive interface 1040 is connected to the disk drive 1041. A removable storage medium, such as a magnetic disk or optical disk, is inserted into the disk drive 1041. A serial port interface 1050 is connected to, for example, a mouse 1051 and a keyboard 1052. A video adapter 1060 is connected to, for example, a display 1061.

[0053] Here, the hard disk drive 1031 stores, for example, the OS (Operating System) 1091, the application program 1092, the program module 1093, and the program data 1094. The information described in the above embodiment is stored, for example, in the hard disk drive 1031 or the memory 1010.

[0054] Furthermore, the anomaly detection program is stored in the hard disk drive 1031 as a program module 1093 containing instructions to be executed by the computer 1000, for example. Specifically, the program module 1093 containing instructions for each process executed by the anomaly detection device 10 described in the above embodiment is stored in the hard disk drive 1031.

[0055] Furthermore, the data used for information processing by the anomaly detection program is stored as program data 1094, for example, in the hard disk drive 1031. The CPU 1020 then reads the program module 1093 and program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as needed and executes the procedures described above.

[0056] Furthermore, the program module 1093 and program data 1094 related to the anomaly detection program are not limited to being stored on the hard disk drive 1031; for example, they may be stored on a removable storage medium and read by the CPU 1020 via a disk drive 1041 or the like. Alternatively, the program module 1093 and program data 1094 related to the anomaly detection program may be stored on another computer connected via a network such as a LAN or WAN (Wide Area Network) and read by the CPU 1020 via a network interface 1070.

[0057] Although embodiments applying the invention made by the present inventors have been described above, the present invention is not limited by the descriptions and drawings that constitute part of the disclosure of the present invention in this embodiment. That is, all other embodiments, examples, and operational techniques made by those skilled in the art based on this embodiment are included in the scope of the present invention. [Explanation of Symbols]

[0058] 10 Anomaly detection device 11 Input section 12 Output section 13 Communication Control Unit 14 Storage section 14a Buffer area 14b Calculation results 15 Control Unit 15a Acquisition part 15b Calculation part 15c Analysis Department

Claims

1. An acquisition unit that acquires past traffic data and stores it in the buffer area of ​​the memory unit, A calculation unit calculates the mean and variance of the traffic data by applying the traffic data of the buffer area to a predetermined recurrence equation. An analysis unit calculates a Z-score representing the deviation of the traffic data to be processed from the average, using the average and variance of the traffic data for the same time on the previous day, from among the multiple averages and multiple variances calculated from the traffic data for each time of day, and analyzes whether the traffic data to be processed is abnormal or not using the Z-score. An anomaly detection device characterized by having the following features.

2. The anomaly detection device according to claim 1, characterized in that the calculation unit uses the calculated mean and variance when calculating the mean and variance of the traffic data acquired next.

3. The anomaly detection device according to claim 1, characterized in that the calculation unit applies past traffic data to the recurrence relation which is weighted such that the contribution of traffic data decreases as you go further back in time.

4. The anomaly detection device according to claim 1, characterized in that the analysis unit determines the traffic data to be processed to be abnormal when the calculated Z score is greater than a predetermined threshold.

5. An anomaly detection method performed by an anomaly detection device, The abnormality detection device has a storage unit, The acquisition step involves acquiring past traffic data and storing it in the buffer area of ​​the storage unit, A calculation step of calculating the mean and variance of the traffic data by applying the traffic data of the buffer area to a predetermined recurrence relation, An analysis step which involves calculating a Z-score representing the deviation of the traffic data to be processed from the average, using the average and variance of the traffic data for the same time on the previous day from among the multiple averages and multiple variances calculated from the traffic data for each time of day, and using the Z-score to analyze whether the traffic data to be processed is abnormal or not. An anomaly detection method characterized by including

6. The acquisition step involves obtaining past traffic data and storing it in the buffer area of ​​the memory unit, A calculation step of calculating the mean and variance of the traffic data by applying the traffic data of the buffer area to a predetermined recurrence relation, An analysis step which involves calculating a Z-score representing the deviation of the traffic data to be processed from the average, using the average and variance of the traffic data for the same time on the previous day from among the multiple averages and multiple variances calculated from the traffic data for each time of day, and using the Z-score to analyze whether the traffic data to be processed is abnormal or not. An anomaly detection program that causes a computer to execute a command.

Images