Traffic anomaly detection device and traffic anomaly detection method

Abstract

The purpose is to more reliably detect traffic anomalies. [Solution] The traffic anomaly detection device 1 includes a learning unit 11 configured to construct a multivariate probabilistic model using the traffic volume at each node of a plurality of nodes 3 that can communicate with each other, and to generate a matrix from the multivariate probabilistic model that represents a conditional dependency between any two nodes among the plurality of nodes, and a judgment unit 12 configured to judge traffic anomalies based on the values ​​of components that represent the conditional dependency between two nodes in the matrix generated by the learning unit 11.

Description

[Technical Field]

[0001] The present invention relates to a traffic anomaly detection device and a traffic anomaly detection method. [Background technology]

[0002] Internet service providers (ISPs) around the world interconnect their routers at locations commonly known as Internet exchanges (IXs). An example of an IX is JPIX in Japan. Content providers such as social networking sites, cloud services, and data centers also connect to IXs via OSPs.

[0003] In this way, the routers of ISPs around the world are configured in a nearly full mesh (although some ISPs do not have interconnections), and traffic is exchanged daily using this configuration.

[0004] In order to determine whether traffic is abnormal, there is a technique for comparing traffic information of a target IP with a traffic threshold, which may be a preset value (see Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-150531 Summary of the Invention [Problem to be solved by the invention]

[0006] However, because normal traffic volume differs for each ISP, it has been difficult to accurately detect traffic abnormalities using the method of comparing traffic volume with a threshold, as in Patent Document 1. Furthermore, with conventional technology, it has been difficult to identify ISPs experiencing abnormal traffic due to DDoS (Distributed Denial of Service) attacks, etc., due to the complex network configuration between ISPs, which is close to a full mesh.

[0007] The present invention has been made to solve the above-mentioned problems, and has an object to more reliably detect traffic anomalies. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the traffic anomaly detection device of the present invention includes a learning unit configured to construct a multivariate probabilistic model using traffic volumes at each node of a plurality of nodes that can communicate with each other, and to generate a matrix representing a conditional dependency between any two nodes among the plurality of nodes from the multivariate probabilistic model; and a determination unit configured to determine a traffic anomaly based on the values ​​of components in the matrix generated by the learning unit that represent the conditional dependency between two nodes.

[0009] In addition, in the traffic anomaly detection device according to the present invention, the learning unit may be further configured to construct a multivariate normal distribution model with a sample mean of 0 as the multivariate probability model, using the traffic volume at each node for each period of an observation period including a plurality of periods, and to generate a precision matrix of the multivariate normal distribution model as the matrix.

[0010] In addition, in the traffic anomaly detection device according to the present invention, the determination unit may be further configured to determine, based on the precision matrix generated for a certain observation period and the precision matrix generated for an observation period prior to the certain observation period, at least one of the following: a value of a component indicating the conditional dependency between two nodes has changed from 0 to a value other than 0; a value of a component indicating the conditional dependency between two nodes has changed from a value other than 0 to 0; and a value of a component indicating the conditional dependency between two nodes has exceeded a threshold value.

[0011] The traffic abnormality detection device according to the present invention may further include an alarm unit configured to generate an alarm based on the determination of a traffic abnormality made by the determination unit.

[0012] Furthermore, the traffic abnormality detection device according to the present invention may be provided in a system equipped with a traffic control device, and the alarm may include information indicating the node related to the determination of a traffic abnormality, and the traffic control device may be configured to instruct the node related to the determination of a traffic abnormality to impose a bandwidth restriction based on the received alarm.

[0013] In order to solve the above-mentioned problems, the traffic anomaly detection method of the present invention includes a learning step of constructing a multivariate probabilistic model using traffic volumes at each node of a plurality of nodes that can communicate with each other, and generating a matrix from the multivariate probabilistic model that indicates a conditional dependency between any two nodes among the plurality of nodes, and a determination step of determining whether there is a traffic anomaly based on the values ​​of components that indicate the conditional dependency between two nodes in the matrix generated in the learning step.

[0014] In addition, in the traffic anomaly detection method according to the present invention, the learning step may include a step of constructing a multivariate normal distribution model with a sample mean of 0 as the multivariate probability model using the traffic volume at each node in each period of an observation period including a plurality of periods, and generating a precision matrix of the multivariate normal distribution model as the matrix.

[0015] In addition, in the traffic anomaly detection method according to the present invention, the determining step may include a step of determining, based on the precision matrix generated for a certain observation period and the precision matrix generated for an observation period prior to the certain observation period, whether at least one of the following occurs: a value of a component indicating the conditional dependency between two nodes has changed from 0 to a value other than 0; a value of a component indicating the conditional dependency between two nodes has changed from a value other than 0 to 0; or a value of a component indicating the conditional dependency between two nodes has exceeded a threshold value.

[0016] The traffic abnormality detection method according to the present invention may further comprise the step of generating an alarm based on the determination of the traffic abnormality in the determining step.

[0017] In addition, in the traffic anomaly detection method according to the present invention, the alarm may include information indicating a node related to the determination of a traffic anomaly, and the method may further include a step of instructing the node related to the determination of a traffic anomaly to impose a bandwidth restriction based on the alarm. [Effects of the Invention]

[0018] According to the present invention, a multivariate probabilistic model is constructed using the traffic volume at each of a plurality of nodes that can communicate with each other, and a matrix indicating the conditional dependency between any two of the plurality of nodes is generated from the multivariate probabilistic model, thereby enabling more reliable detection of traffic anomalies. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a block diagram showing the configuration of a system including a traffic anomaly detection device and a traffic control device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram for explaining the configuration of a learning unit included in the traffic anomaly detection device according to this embodiment. [Figure 3A] FIG. 3A is a schematic diagram for explaining the configuration of a learning unit included in the traffic anomaly detection device according to this embodiment. [Figure 3B] FIG. 3B is a schematic diagram for explaining the configuration of the learning unit included in the traffic anomaly detection device according to this embodiment. [Figure 4] FIG. 4 is a block diagram showing the hardware configuration of a traffic anomaly detection device according to this embodiment. [Figure 5] FIG. 5 is a sequence diagram showing the operation of a system including a traffic anomaly detection device and a traffic control device according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] A preferred embodiment of the present invention will be described in detail below with reference to FIGS.

[0021] [System Configuration] First, referring to Fig. 1, an overview of a system including a traffic anomaly detection device 1 according to an embodiment of the present invention and a traffic control device 2 will be described. Note that a plurality of nodes 3 may be considered to be included or not included in this system.

[0022] In this system, the traffic anomaly detection device 1 and the traffic control device 2 are connected via a network NW. The network NW can be considered to be composed of one or more networks. The traffic anomaly detection device 1 and the traffic control device 2 are connected so that at least the traffic control device 2 can receive an alarm generated by the traffic anomaly detection device 1. Note that in FIG. 1, the traffic anomaly detection device 1 and the traffic control device 2 are shown as separate devices, but this is not limiting and the traffic anomaly detection device 1 and the traffic control device 2 may be a single integrated device. Furthermore, the traffic control device 2 is not essential to this system.

[0023] In this system, the traffic abnormality detection device 1 and the multiple nodes 3 are connected via a network NW. The traffic abnormality detection device 1 and the multiple nodes 3 are connected at least so that the traffic abnormality detection device 1 can collect the traffic volume at each of the multiple nodes 3. Note that the traffic abnormality detection device 1 may also be connected directly to any one of the multiple nodes 3 without going through the network NW.

[0024] In this system, the traffic control device 2 and the multiple nodes 3 are connected via a network NW. The traffic control device 2 and the multiple nodes 3 are connected at least so that the traffic control device 2 can instruct each of the multiple nodes to limit bandwidth. The traffic control device 2 may also function as an operation center that issues arbitrary instructions to the multiple nodes 3. The traffic control device 2 may also be directly connected to any one of the multiple nodes 3 without going through the network NW.

[0025] In this system, the plurality of nodes 3 are connected to each other via a network NW so as to be able to communicate with each other. Note that at least one of the plurality of nodes 3 may be directly connected to another of the plurality of nodes 3 without going through the network NW.

[0026] An example of a node among the plurality of nodes 3 is a router, for example, a router owned by an ISP. Another example of a node among the plurality of nodes 3 is the ISP itself. Yet another example of a node among the plurality of nodes 3 is a content delivery network (CDN), a cloud, or a data center. The nodes among the plurality of nodes 3 are not limited to these examples, and may be any entities that can communicate with each other.

[0027] The number of the multiple nodes 3 is arbitrary. Hereinafter, the number of the multiple nodes 3 is assumed to be M. Furthermore, hereinafter, it is assumed that the nodes are assigned unique numbers (1st, 2nd, ..., Mth) represented by natural numbers so that one node can be identified from the multiple nodes 3. An IP address, a MAC address, etc. can be used as the number to identify each node.

[0028] [Functional block of traffic anomaly detection device] Next, functional blocks of the traffic anomaly detection device 1 according to this embodiment will be described with reference to the block diagram of Fig. 1. As shown in Fig. 1, the traffic anomaly detection device 1 includes a preparation unit 10, a learning unit 11, a determination unit 12, an alarm unit 13, and a storage unit 15.

[0029] The preparation unit 10 prepares data on the traffic volume at each of the plurality of nodes 3, which is used by the learning unit 11. The preparation unit 10 may prepare data on the traffic volume at each node for each period of an observation period including multiple periods. The traffic volume data at each node for one period of an observation period including multiple periods is expressed as an observation value x=(x1, x2, . . . , x M ) where each element x of one observation x m(m=1, 2, . . . , M) is the traffic volume in one cycle at the m-th node among the multiple nodes 3. An example of the length of one cycle is one minute, but this is not limited to this. The traffic volume in one cycle at a node may be the amount of data received by the node in that cycle (number of packets, number of bytes, number of bits, etc.; the same applies below), the amount of data transmitted by the node, the sum of the amount of data received and transmitted by the node, or any of these divided by the length of the cycle (e.g., bps), but is not limited to these.

[0030] The preparation unit 10 uses any method to collect the traffic volume at each node of the multiple nodes 3. For example, if the node periodically measures and publishes the traffic volume, the preparation unit 10 may simply acquire the traffic volume from the node. The preparation unit 10 may also measure the traffic volume at the node using any technique. The method by which the preparation unit 10 collects the traffic volume at each node is not limited to these. The preparation unit 10 may also calculate the traffic volume at each node for each period based on the collected traffic volume.

[0031] The preparation unit 10 collects the observed values ​​x for each period for each observation period to generate a data set D=(x (1) ,x (2) ,···,x (N) ) can be prepared. Here, each element x of a data set D (n) (n=1, 2, . . . , N) is the observed value x of the nth cycle in one observation period. An example of the length of the observation period is 5 hours, but it is not limited to this. If the length of the observation period is 5 hours and the length of the cycle is 1 minute, then N=5 (hours) × 60 (minutes) / 1 (minute) = 300. One data set D corresponds to the traffic volume in each cycle of one observation period at each node of multiple nodes 3 that can communicate with each other.

[0032] The learning unit 11 constructs a multivariate probability model using the traffic volume at each node of the plurality of nodes 3 that can communicate with each other, and generates a matrix from this multivariate probability model that indicates the conditional dependency between any two nodes in the plurality of nodes 3. The learning unit 11 may construct a multivariate normal distribution model with a sample mean of 0 as the multivariate probability model using the traffic volume at each node of the plurality of nodes 3 for each period of an observation period that includes multiple periods, i.e., a dataset D, and generate a precision matrix of the multivariate normal distribution model as a matrix that indicates the conditional independence (dependence) between any two nodes in the plurality of nodes. In this case, each element x of the observation value x included in the dataset D m are the variables of the multivariate data for the multivariate normal distribution model.

[0033] The learning unit 11 uses the traffic volume at each of the multiple nodes 3 as a variable to learn the essential dependencies between variables. Figures 2(a) and (b) show a directed graph used by the learning unit 11. Each node a, b, and c represents a variable and the traffic volume at each node. Each edge represents a direct probability dependency. In the directed graph of Figure 2(a), the value of node c is unobserved. In this case, the joint probability distribution ρ(a, b, c) of variables a, b, and c is expressed by the following equation (1).

number

[0034] By marginalizing the variable c, it can be expressed as the following equation (2).

number

[0035] In the above equation (2), the two variables a and b are not independent, since they cannot generally be expressed as ρ(a)ρ(b). On the other hand, in the directed graph of Figure 2(b), variable c is observed. The joint probability ρ(a, b|c) when variable c is observed is expressed by the following equation (3).

number

[0036] In equation (3) above, when the value of the observed variable c, which is a common condition, is taken into consideration, it can be seen that variables a and b do not affect each other, i.e., they are independent. This is called conditional independence. By taking conditional independence into consideration, it is possible to extract the essential or true relationship between variables. For example, even if there appears to be a correlation between the traffic volume between a certain node a and another node b, when the traffic volume at yet another node c is also taken into consideration, this corresponds to the case where the traffic volumes between nodes a and b do not affect each other.

[0037] As mentioned above, the traffic volume in one period at each node of the plurality of nodes 3 is expressed as an M-dimensional observation value x=(x1, x2, . . . , x M ) and a data set D consisting of N observations x is expressed as D={x (1) ,x (2) ,···,x (N) In this case, the multivariate normal distribution model is expressed by the following equation (4).

number

[0038] In the above equation (4), μ is the sample mean when the observed value x is used as a sample, Σ is the covariance matrix, and |·| is the determinant. The learning unit 11 performs maximum likelihood estimation to find μ and Σ, which are parameters of the multivariate normal distribution model, from a dataset D of observed data. The logarithmic likelihood L(μ,Σ|D) of the dataset D is expressed by the following equation (5).

number

[0039] Substituting the above equation (4) into the above equation (5) gives the following equation (6).

number

[0040] The parameters μ and Σ that maximize the log likelihood L(μ,Σ|D) in the above equation (6) are estimated as the most likely solution. For the most likely solution of the parameters μ and Σ, μ and Σ are respectively -1 Differentiating with and setting it to 0, the maximum likelihood solutions of the parameters μ and Σ are expressed by the following equations (7) and (8), respectively.

number

number

[0041] Inverse matrix Σ of parameters Σ -1 is called the precision matrix Λ. The precision matrix Λ can be obtained by calculating the inverse matrix of the above formula (8). Note that in the maximum likelihood estimation shown in the above formulas (7) and (8), if regularization is not effective and there is a risk of overlearning, the learning unit 11 can also perform estimation by applying maximum a posteriori estimation (MAP estimation) to the parameters μ and Σ.

[0042] Here, the correlation structure in a multivariate normal distribution is expressed by a graph model. Such a graph model is called a Gaussian graphic model. Below, we will explain how to calculate conditional probability when applying a multivariate normal distribution model to graph theory. In the multivariate normal distribution model of the above formula (4), the inverse matrix Σ of the parameter Σ is -1 When the precision matrix Λ is used and the parameter μ is set to 0, it is expressed by the following equation (9).

number

[0043] The ρ(x) in the above equation (9) is called a Gaussian graphic model. Under a multivariate normal distribution, the conditional probability ρ(x1,x2|x3,...,x M ) is expressed by the following equation (10).

number

[0044] The function of variables x1 and x2 in the above equation (10) is ρ(x) in the above equation (9), that is, N(x|0,Λ -1 ), so if we extract all the parts of the above equation (9) related to the variables x1 and x2, we obtain the relationship in the following equation (11).

number

[0045] where the conditional independence ρ(x1|x3, ,x M )ρ(x2|x3, ,x M The condition for satisfying this is given by the following equation (12):

number

[0046] The above equation (12) indicates that the values ​​of the first and second components of the precision matrix Λ are zero, i.e., there is no edge between variables x1 and x2, as in the case of variables a and b in the directed graph of Figure 2. Figures 3A and 3B are schematic diagrams showing the relationship between the correlation between variables on the graph of a Gaussian graphical model and the precision matrix Λ. Each node in Figures 3A and 3B corresponds to each of multiple nodes 3.

[0047] Figure 3A shows that there is a correlation between the traffic volume between the second node and the ninth node, and between the third node and the thirteenth node during a certain observation period. In this case, the precision matrix Λ has at least two, nine components Λ in addition to the diagonal components. 2,9 , and ,3,13 components Λ 3,13 In Figures 3A and 3B, non-zero values ​​are represented by "*".

[0048] On the other hand, Figure 3B shows that in the observation period after the above-mentioned certain observation period, the correlation of the traffic volume between the second node and the ninth node disappears, but the correlation of the traffic volume between the third node and the thirteenth node is maintained, and furthermore, a new correlation of the traffic volume between the second node and the M-1th node appears. At this time, the 2nd and 9th components Λ of the precision matrix Λ2,9 changes from a non-zero value to a zero value, and the 3, 13 component Λ 3,13 maintains non-zero values, and furthermore, 2,M-1 components Λ 2,M-1 will change from a zero value to a non-zero value.

[0049] If the traffic volume between two nodes is normally uncorrelated, a change in the value of the corresponding element of the precision matrix Λ from zero to non-zero would indicate a traffic anomaly. A subsequent change in the value of the element from non-zero to zero would indicate that the traffic anomaly has been resolved. If the traffic volume between two nodes is normally correlated, a change in the value of the corresponding element of the precision matrix Λ from non-zero to zero would indicate a traffic anomaly. If a specific element of the precision matrix Λ has a value of zero, this means that the two corresponding variables are conditionally independent. Therefore, the precision matrix Λ makes it possible to determine traffic anomalies by looking only at the direct relationship between variables, without being affected by noise or indirect relationships.

[0050] In order to be able to use the above formula (9), the learning unit 11 may construct a multivariate normal distribution model after setting the sample mean μ to 0. Therefore, the learning unit 11 may construct a multivariate normal distribution model after setting the n-th element x (n) (n=1 N) (observation value) m-th element (m=1 M)x m (n) Normalization transformation for (traffic volume at node)

number

number

[0051] 1, the determination unit 12 determines whether an anomaly exists based on the value of a component that indicates conditional independence (dependence) between two of the multiple nodes 3 in the matrix generated by the learning unit 11. The traffic anomalies determined by the determination unit 12 include anomalies due to attack traffic as well as anomalies due to faults or incorrect settings.

[0052] The following describes a case where the learning unit 11 uses a precision matrix Λ of a multivariate normal distribution model constructed with the sample mean set to 0. Based on the precision matrix Λ generated for a certain observation period and the precision matrix Λ generated for an observation period prior to the certain observation period, the determination unit 12 may determine whether the value of a component in the precision matrix Λ indicating a conditional dependency between two nodes has changed from zero to non-zero (hereinafter referred to as "Case 1"). As described above, a change from zero to non-zero of the value of a component indicating a conditional dependency between two nodes that are normally uncorrelated can be considered a traffic anomaly. Therefore, the determination unit 12 may determine a traffic anomaly when at least one of one or more components in the precision matrix Λ indicating a conditional dependency between two nodes that are normally uncorrelated corresponds to Case 1.

[0053] Based on the precision matrix Λ generated for a given observation period and the precision matrix Λ generated for an observation period prior to the given observation period, the determination unit 12 may determine whether the value of a component in the precision matrix Λ indicating a conditional dependency between two nodes has changed from non-zero to zero (hereinafter referred to as "Case 2"). A change in the value of a component indicating a conditional dependency between two nodes that are normally correlated from non-zero to zero can be considered a traffic anomaly. For example, Case 2 includes a case where the traffic of two nodes that previously fluctuated simultaneously no longer fluctuates simultaneously in a subsequent observation period. Furthermore, a phenomenon in which a relationship in which one node is congested and another node is also congested under normal circumstances disappears may be due to a route change caused by some kind of communication anomaly. Furthermore, if a specific communication is filtered or rerouted due to an attack or failure such as a DDoS attack, the dependency between nodes 3 may disappear. Therefore, such a phenomenon is subject to determination as a traffic anomaly in Case 2.

[0054] Therefore, the determination unit 12 may determine that a traffic abnormality has occurred when at least one of one or more components in the precision matrix Λ that indicates a conditional dependency between two nodes that are normally correlated corresponds to Case 2. Furthermore, the determination unit 12 may determine that the traffic abnormality has been resolved when a component that previously corresponded to Case 1 (or Case 3, which will be described later) now corresponds to Case 2.

[0055] The determination unit 12 may perform the determination after setting a threshold value, taking into consideration that component values ​​in the precision matrix Λ that are essentially zero may take on slightly non-zero values ​​due to noise, sampling error, etc. Therefore, the determination unit 12 may determine that the value of a component in the precision matrix Λ that indicates a conditional dependency between two nodes exceeds a threshold value (hereinafter referred to as "Case 3"), based on the precision matrix Λ generated for a certain observation period and the precision matrix Λ generated for an observation period prior to the certain observation period. The determination unit 12 may determine that a traffic anomaly has occurred when at least one of one or more components in the precision matrix Λ that indicates a conditional dependency between two nodes that are normally not correlated corresponds to Case 3.

[0056] The threshold may be a predetermined threshold (fixed value) or a value that depends on the value of a past component. An example of a threshold that depends on the value of a past component is a value obtained by multiplying the value of a component in the precision matrix Λ generated by the learning unit 11 for an observation period prior to a certain observation period by a predetermined value. In other words, when the component in the precision matrix Λ generated by the learning unit 11 for a certain observation period is multiplied by Λ, i,j current , the components in the precision matrix Λ generated by the learning unit 11 for an observation period prior to a certain observation period are denoted by Λ i,j old Then, the decision unit 12 determines Λ i,j current / Λ i,j old >T (T is a predetermined value).

[0057] Therefore, the determining unit 12 may determine at least one of Cases 1 to 3 based on the precision matrix Λ generated for a certain observation period and the precision matrix Λ generated for an observation period prior to the certain observation period.

[0058] The determination unit 12 may determine that a DDoS attack has occurred based on the number of nodes related to the determination of a traffic abnormality. The nodes related to the determination of a traffic abnormality may be nodes corresponding to the component that is the basis for the determination of a traffic abnormality. For example, i,j and component Λi,k If it is determined that a traffic abnormality has occurred because it corresponds to Case 3, the nodes related to the determination of the abnormality may be the i-th node, the j-th node, and the k-th node. Note that when it is determined that a DDoS attack has occurred, the nodes related to the determination of the abnormality may be identified as the nodes where a traffic abnormality has occurred due to the DDoS attack.

[0059] For example, if a DDoS attack occurs on a server, the attacking terminal that carries out the DDoS attack will continuously inject traffic into a router (an example of a node, hereinafter referred to as the "attacked node") connected to that server via multiple other routers (hereinafter referred to as "other nodes") for a certain period of time. This means that the traffic volume not only at the attacked node but also at other nodes on the route will increase simultaneously, and a correlation will occur in the traffic volume between the attacked node and other nodes, causing at least some of the components of the precision matrix Λ to change to a value other than 0.

[0060] Furthermore, when the correlation that existed between the attacked node and other nodes disappears, some components of the precision matrix Λ change from a non-zero value to 0 when traffic due to a DDoS attack is concentrated on the attacked node and the route or bandwidth of the attacked node is blocked. Similarly, when there is a certain correlation in the traffic volume between the attacked node and other nodes, if that correlation changes further, some components of the precision matrix Λ will exceed a threshold. Therefore, the determination unit 12 may determine that a DDoS attack has occurred (or that a DDoS attack is at least likely to have occurred) when, for example, the number of nodes determined to be abnormal exceeds a predetermined threshold.

[0061] The warning unit 13 generates an alarm based on the determination of a traffic abnormality by the determination unit 12. The warning unit 13 may generate an alarm when the determination unit 12 determines that there is a traffic abnormality. The alarm may include information indicating the node related to the determination of the abnormality.

[0062] As described above, the determination unit 12 determines whether a traffic abnormality exists, by determining which of Cases 1 to 3 it is. The warning unit 13 may generate an alarm when one or more elements of the precision matrix Λ correspond to any of Cases 1 to 3, rather than when the determination unit 12 determines whether a traffic abnormality exists. The alarm may include information indicating which of Cases 1 to 3 it corresponds to.

[0063] As time passes, the preparation unit 10 sequentially generates a dataset D for a different observation period, the learning unit 11 sequentially generates a precision matrix Λ using the different datasets D, and the determination unit 12 sequentially determines traffic anomalies in the different precision matrices Λ. Therefore, the warning unit 13 may generate an alarm when a component corresponding to any of Cases 1 to 3 is found multiple times in succession in the sequentially determined precision matrix Λ. For example, the warning unit 13 may generate an alarm when a component corresponding to Case 3 is found 10 times in succession in the sequentially determined precision matrix Λ.

[0064] The storage unit 15 may store parameters of the multivariate probability model constructed by the learning unit 11 (for example, parameters μ, Σ of the multivariate normal distribution model) and a matrix (for example, precision matrix Λ) generated by the learning unit 11. The learning unit 11 may also store the results of the determinations made by the determination unit 12 for cases 1 to 3. The information stored by the learning unit 11 is not limited to these.

[0065] [Traffic control device functions] Next, the function of the traffic control device 2 according to this embodiment will be described. The traffic control device 2 receives an alarm issued by the alarm unit 13 of the traffic abnormality detection device 1, and instructs the node determined to be abnormal to impose a bandwidth restriction based on the received alarm. As described above, the alarm may include information indicating the node determined to be abnormal. The traffic control device 2 may instruct the node determined to be abnormal to impose a bandwidth restriction. Note that the method for instructing the bandwidth restriction is arbitrary.

[0066] As described above, the alarm can include information indicating which of Cases 1 to 3 has occurred. Therefore, the traffic control device 2 may issue a command to limit the bandwidth when any of Cases 1 to 3 has occurred. For example, the traffic control device 2 may issue a command to limit the bandwidth when Case 3 has occurred.

[0067] [Hardware configuration of traffic anomaly detection device and traffic control device] Next, an example of a hardware configuration for realizing the traffic anomaly detection device 1 and the traffic control device 2 having the above-described functions will be described with reference to FIG.

[0068] 4, traffic anomaly detection device 1 can be realized by, for example, a computer including a processor 102, a main memory device 103, a communication interface 104, an auxiliary memory device 105, and an input / output (I / O) device 106, all connected via a bus 101, and a program for controlling these hardware resources. Furthermore, traffic anomaly detection device 1 includes a display device 107.

[0069] The processor 102 is realized by a CPU, a GPU, an FPGA, an ASIC, or the like.

[0070] The main memory device 103 pre-stores programs for the processor 102 to perform various controls and calculations. The processor 102 and the main memory device 103 implement the functions of the traffic anomaly detection device 1, such as the preparation unit 10, learning unit 11, determination unit 12, and alarm unit 13 shown in FIG.

[0071] The communication interface 104 is an interface circuit for connecting the traffic anomaly detection device 1 to various external electronic devices via a network. The communication interface 104 realizes at least a part of the configuration of the preparation unit 10 and the alarm unit 13.

[0072] The auxiliary storage device 105 is composed of a readable / writable storage medium and a drive for reading and writing various information such as programs and data from and to the storage medium. The auxiliary storage device 105 can use a semiconductor memory such as a hard disk or flash memory as the storage medium.

[0073] The auxiliary storage device 105 has a program storage area for storing a traffic anomaly detection program. The auxiliary storage device 105 also has a program storage area for storing a learning program that estimates parameters using a multivariate normal distribution model executed by the traffic anomaly detection device 1 and generates a precision matrix. The auxiliary storage device 105 realizes the storage unit 15 described in FIG. 1. Furthermore, the auxiliary storage device 105 may have, for example, a backup area for backing up the data and programs described above.

[0074] The input / output I / O 106 is an input / output device that inputs signals from external devices and outputs signals to external devices.

[0075] The display device 107 is configured by an organic EL display, a liquid crystal display, etc. The display device 107 can display on the screen information such as the fact that an abnormality has been determined.

[0076] The traffic control device 2 may also have the same hardware configuration as the traffic anomaly detection device 1. That is, the processor 102 and main memory device 103 of the traffic control device 2 realize the above-mentioned functions of the traffic control device 2. Furthermore, the communication interface 104 of the traffic control device 2 realizes at least a part of the configuration of the above-mentioned functions. The auxiliary memory device 105 of the traffic control device 2 has a program storage area for storing a control program for instructing the nodes to impose bandwidth restrictions. The display device 107 of the traffic control device 2 can display the status of bandwidth restrictions of the nodes on a screen.

[0077] [System Operation] Next, the operation of a system including the traffic anomaly detection device 1 and traffic control device 2 having the above-described configuration will be described with reference to the sequence in Fig. 5. Note that Fig. 5 shows a sequence for one operation that is repeated in this system. Therefore, it should be noted that steps S1 to S10, which will be described later, are repeatedly executed multiple times.

[0078] As shown in FIG. 5, first, the preparation unit 10 of the traffic abnormality detection device 1 collects the traffic volume at each of the plurality of nodes 3 (step S1).

[0079] Next, the preparation unit 10 prepares data on the traffic volume at each node from the traffic volume at each node collected in step S1 (step S2). In step S2, the preparation unit 10 may prepare one data set D of observed values ​​x from the traffic volume at each node for each cycle of one observation period. That is, the preparation unit 10 prepares a set of M-dimensional observed values ​​x=(x1, x2, . . . , x M ) based on the data set D={x (1) ,x (2) ,···,x (N)} may be prepared.

[0080] Next, the learning unit 11 constructs a multivariate probability model using the traffic volume at each node (step S3). In step S3, the multivariate probability model may be a multivariate normal distribution model. That is, the learning unit 11 may estimate, by maximum likelihood estimation, the maximum likelihood solution of parameters μ and Σ of the multivariate normal distribution model for a dataset D, which is multivariate data with the traffic volume at each node as a variable. In step S3, the learning unit 11 may use the above equation (6) to estimate the parameters μ and Σ that maximize the log likelihood L(μ,Σ|D) as the maximum likelihood solution (the above equations (7) and (8)).

[0081] In step S3, the maximum likelihood solution for the parameters of the multivariate normal distribution model may be estimated twice. That is, using the maximum likelihood solution for the parameters μ and Σ estimated for the original data set D so that the sample mean μ when each observed value x is a sample is 0, standardization transformation may be performed on the data set D using the above equation (13), and the maximum likelihood solution for the parameters μ and Σ may be estimated again for the data set D after standardization transformation. However, since the maximum likelihood solution for the parameter μ estimated the second time should be 0, the second estimation of the parameter μ may be omitted. Therefore, in step S3, the learning unit 11 may construct a multivariate normal distribution model with a sample mean of 0 as a multivariate probability model using the traffic volume at each node for each period of an observation period including multiple periods, i.e., the data set D.

[0082] In addition, in the maximum likelihood estimation shown in the above equations (7) and (8), if there is a risk that regularization is not effective and overlearning may occur, in step S3, the learning unit 11 can also perform estimation by applying maximum a posteriori estimation (MAP estimation) to the parameters μ and Σ.

[0083] Next, the learning unit 11 generates a matrix representing the conditional dependency between any two nodes among the plurality of nodes 3 from the constructed multivariate probability model (step S4). In step S4, the learning unit 11 may generate the precision matrix Λ of the multivariate normal distribution model as a matrix indicating the conditional independence between any two nodes among the plurality of nodes 3. In this case, in step S4, the learning unit 11 generates the inverse matrix Σ of the parameters Σ estimated in step S3 -1 The learning unit 11 may store the generated precision matrix Λ in the storage unit 15.

[0084] Next, the determination unit 12 determines whether there is a traffic abnormality based on the values ​​of components indicating the conditional dependency between two nodes in the matrix generated in step S4 (step S5). In step S5, the determination unit 12 may determine at least one of the above-mentioned cases 1 to 3 based on the precision matrix generated in the immediately preceding step S4 and the precision matrix generated in the step S4 executed immediately before the immediately preceding step S4. The determination unit 12 may acquire the precision matrix Λ used in step S5 from the storage unit 15. Furthermore, the determination unit 12 may determine whether there is a traffic abnormality after step S4 has been executed a predetermined number of times, rather than every time step S4 is executed. Therefore, the judgment unit 12 may judge one of cases 1 to 3 based on the precision matrix generated for a certain observation period (for example, the precision matrix generated in the step S4 executed immediately before the aforementioned step) and the precision matrix generated for an observation period prior to the certain observation period (for example, the precision matrix generated in the step S4 executed immediately before the step S4 executed immediately before the immediately preceding step).

[0085] In step S6, the determination unit 12 may determine that a DDoS attack has occurred based on the number of nodes determined to be abnormal.

[0086] Next, the warning unit 13 determines whether to issue an alarm based on the determination of the traffic abnormality by the determination unit 12 (step S6). In step S6, the warning unit 13 may determine that an alarm should be issued when the determination unit 12 determines that there is a traffic abnormality. Alternatively, in step S6, the warning unit 13 may determine that an alarm should be issued when one or more elements of the precision matrix Λ correspond to any of Cases 1 to 3. Alternatively, in step S6, the warning unit 13 may determine that an alarm should be issued when the precision matrix Λ, which is determined sequentially, has an element that corresponds to any of Cases 1 to 3 multiple times in succession. If it is determined that an alarm should be issued in step S6 (step S6: YES), the process proceeds to step S7. If not (step S6: NO), the process ends, and the process proceeds to step S1 in the next sequence (not shown).

[0087] If it is determined that an alarm should be issued (step S6: YES), the alarm unit 13 issues an alarm (step S7). That is, in step S7, the alarm unit 13 issues an alarm based on the determination of a traffic abnormality made by the determination unit 12. The alarm may be received by at least the traffic control device 2. As described above, the alarm may include information indicating which of Cases 1 to 3 has occurred. The alarm may also include information indicating the node related to the determination of an abnormality.

[0088] Thereafter, the traffic control device 2 that has received the alarm determines whether or not to impose a bandwidth restriction based on the received alarm (step S8). In step S8, the traffic control device 2 may refer to the received alarm and determine that a bandwidth restriction is necessary if any of cases 1 to 3 applies. If it is determined that a bandwidth restriction is necessary in step S8 (step S8: YES), the process proceeds to step S9; if not (step S8: NO), the sequence ends and the process proceeds to step S1 in the next sequence (not shown).

[0089] If the traffic control device 2 determines that bandwidth should be limited (step S8: YES), it instructs the node determined to be abnormal to limit the bandwidth (step S9). That is, in step S9, the traffic control device 2 instructs the node determined to be abnormal to limit the bandwidth based on the received alarm. The traffic control device 2 may instruct the node determined to be abnormal to limit the bandwidth.

[0090] Thereafter, the node that has received the instruction executes the bandwidth limitation (step S10). In Fig. 5, the traffic control device 2 has instructed the first node and the second node to impose a bandwidth limitation, and the bandwidth limitation between the first node and the second node is being executed.

[0091] As described above, the traffic anomaly detection device 1 according to this embodiment focuses on the fact that when an anomaly occurs in the frequency of communication or data volume between two nodes, an increase or decrease in traffic volume is observed simultaneously at both nodes. Then, a multivariate model is constructed from the traffic volume at each node, and a matrix representing the conditional dependency relationship between nodes for traffic volume, such as a precision matrix Λ, is calculated to determine correlation anomalies between nodes, and traffic anomalies are detected based on the results. This allows for more reliable detection of traffic anomalies.

[0092] Furthermore, the traffic abnormality detection device 1 according to this embodiment determines a traffic abnormality based on the traffic volume at each node and generates an alarm, thereby making it possible to notify of a traffic abnormality more quickly.

[0093] Furthermore, a system including the traffic abnormality detection device 1 and traffic control device 2 according to this embodiment determines whether a traffic abnormality exists based on the traffic volume at each node, identifies the node where the abnormal traffic is occurring, and then instructs the node to impose a bandwidth restriction, thereby enabling more effective countermeasures to be taken against traffic abnormalities.

[0094] Furthermore, the abnormal traffic detection device 1 according to this embodiment determines the occurrence of a traffic abnormality based on the traffic volume at each node using a multivariate probability model, and determines the occurrence of a DDoS attack based on the number of nodes involved in the traffic abnormality, thereby enabling more effective detection of the occurrence of a DDoS attack.

[0095] In the embodiment described above, the learning unit 11 employs a Gaussian graphic model to analyze conditional independence through a precision matrix, which is the inverse matrix of the covariance matrix. However, the algorithm employed by the learning unit 11 is not limited to the Gaussian graphic model. For example, in sparse estimation such as Graphical Lasso, it can be employed in combination with Maximum A Posteriori Estimation (MAP). Alternatively, Bayesian network structure learning can be employed.

[0096] The above describes the embodiments of the anomalous traffic detection device and anomalous traffic detection method of the present invention. However, the present invention is not limited to the described embodiments, and various modifications that can be imagined by those skilled in the art can be made within the scope of the invention described in the claims. [Explanation of symbols]

[0097] 1...traffic anomaly detection device, 2...traffic control device, 3...node, 10...preparation unit, 11...learning unit, 12...judgment unit, 13...alarm unit, 15...memory unit, 101...bus, 102...processor, 103...main memory device, 104...communication interface, 105...auxiliary memory device, 106...input / output I / O, 107...display device, NW...network.

Claims

1. a learning unit configured to generate a matrix representing a conditional dependency between traffic volumes at any two nodes among the plurality of nodes, taking into account traffic volumes at nodes other than the two nodes, by estimating parameters of a multivariate probability model representing a probability density for a random variable including a plurality of components each corresponding to the traffic volume at each node, using traffic volumes at each node among the plurality of nodes that can communicate with each other; a determination unit configured to determine a traffic anomaly based on a value of a component in the matrix generated by the learning unit that indicates a conditional dependency between traffic volumes of two nodes, taking into account traffic volumes of nodes other than the two nodes; A traffic anomaly detection device comprising:

2. 2. The traffic anomaly detection device according to claim 1, the multivariate probability model is a multivariate normal distribution model; The learning unit is further configured to generate a precision matrix of the multivariate normal distribution model as the matrix by estimating parameters of the multivariate normal distribution model with a sample mean set to 0 using a traffic volume in each period of an observation period including a plurality of periods at each node. A traffic anomaly detection device comprising:

3. 3. The traffic anomaly detection device according to claim 2, The determination unit determines, based on the precision matrix generated for a certain observation period and the precision matrix generated for an observation period prior to the certain observation period, The value of the component indicating the conditional dependency between two nodes changes from 0 to a value other than 0; The value of the component indicating the conditional dependency between two nodes changes from non-zero to zero; and The value of the component indicating the conditional dependency between the two nodes exceeds a threshold value; and further configured to determine at least one of A traffic anomaly detection device comprising:

4. 2. The traffic anomaly detection device according to claim 1, The system further includes an alarm unit configured to generate an alarm based on the determination of the traffic abnormality by the determination unit. A traffic anomaly detection device comprising:

5. The traffic anomaly detection device according to claim 4; A traffic control device A system comprising: the alarm includes information indicating a node related to a determination of a traffic abnormality; The traffic control device is configured to instruct the node related to the determination of the traffic abnormality to limit bandwidth based on the received alarm. A system characterized by:

6. A learning step in which a computer uses the traffic volume at each node of a plurality of nodes that can communicate with each other to estimate parameters of a multivariate probability model that represents the probability density for a random variable that includes multiple components, each of which corresponds to the traffic volume at each node, to generate a matrix that shows the conditional dependency between the traffic volumes at any two nodes of the plurality of nodes, taking into account the traffic volumes at nodes other than those two nodes; a determination step in which the computer determines whether there is a traffic abnormality based on the value of a component in the matrix generated in the learning step that indicates a conditional dependency between the traffic volumes of two nodes, taking into account the traffic volumes of nodes other than the two nodes; A traffic anomaly detection method comprising:

7. 7. The traffic anomaly detection method according to claim 6, the multivariate probability model is a multivariate normal distribution model; The learning step includes a step of estimating parameters of the multivariate normal distribution model with a sample mean of 0 using traffic volume in each period of an observation period including a plurality of periods at each node, thereby generating a precision matrix of the multivariate normal distribution model as the matrix. A traffic anomaly detection method comprising:

8. The traffic anomaly detection method according to claim 7, The determining step is based on the precision matrix generated for a certain observation period and the precision matrix generated for an observation period prior to the certain observation period, The value of the component indicating the conditional dependency between two nodes changes from 0 to a value other than 0; The value of the component indicating the conditional dependency between two nodes changes from non-zero to zero; and The value of the component indicating the conditional dependency between the two nodes exceeds a threshold value; determining at least one of A traffic anomaly detection method comprising:

9. 7. The traffic anomaly detection method according to claim 6, The method further comprises a step of generating an alarm by the computer based on the determination of the traffic abnormality in the determining step. A traffic anomaly detection method comprising:

10. The traffic anomaly detection method according to claim 9, the alarm includes information indicating a node related to a determination of a traffic abnormality; The method further comprises a step of instructing the node related to the determination of the traffic abnormality to limit bandwidth based on the alarm. A traffic anomaly detection method comprising:

Images