Apparatus and method for detecting abnormal correlation between applications

Abstract

The purpose is to more reliably detect abnormal correlations between applications. [Solution] The abnormal correlation detection device 1 includes a learning unit 11 configured to construct a multivariate probabilistic model using traffic volumes related to each of a plurality of web applications and generate a matrix representing a conditional dependency between any two of the plurality of web applications from the multivariate probabilistic model, and a determination unit 12 configured to determine an abnormal correlation between the two web applications based on the values ​​of components in the matrix generated by the learning unit 11 that represent the conditional dependency between the two web applications.

Description

[Technical Field]

[0001] The present invention relates to an apparatus and method for detecting abnormal correlation between applications. [Background technology]

[0002] In recent years, traffic from Internet applications (web applications) such as Netflix and YouTube (registered trademark) has been increasing explosively. A burst of traffic from one web application can affect the communications of other web applications. As a measure of communication quality, a communication device has been proposed that measures the amount of communication delay and fluctuation on a packet-by-packet or application-by-application basis in a communication system from a mobile device to a communication device for Ethernet packets sent from equipment connected to the mobile device (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7395073 Summary of the Invention [Problem to be solved by the invention]

[0004] However, simply measuring communication delays, etc., on an application (or web application) basis, as in Patent Document 1, makes it difficult to identify which web applications affect the communications of other web applications, in other words, which web applications are correlated with other web applications.

[0005] The present invention has been made to solve the above-mentioned problems, and has an object to more reliably detect abnormal correlations between web applications. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the abnormal correlation detection device of the present invention includes a learning unit configured to construct a multivariate probabilistic model using traffic volumes related to each web application of a plurality of web applications and generate a matrix representing a conditional dependency between any two web applications from the multivariate probabilistic model, and a determination unit configured to determine an abnormal correlation between the two web applications based on values ​​of components in the matrix generated by the learning unit that represent the conditional dependency between the two web applications.

[0007] Furthermore, in the abnormal correlation detection device according to the present invention, the learning unit may be further configured to construct, as the multivariate probability model, a multivariate normal distribution model with a sample mean of 0, using traffic volume for each period of an observation period including multiple periods, for each web application, and to generate, as the matrix, a precision matrix of the multivariate normal distribution model.

[0008] Furthermore, in the abnormal correlation detection device according to the present invention, the determination unit may be further configured to determine, based on the precision matrix generated for a certain observation period and the precision matrix generated for an observation period prior to the certain observation period, at least one of the following: a value of a component indicating the conditional dependency between two web applications has changed from 0 to a value other than 0; a value of a component indicating the conditional dependency between two web applications has changed from a value other than 0 to 0; or a value of a component indicating the conditional dependency between two web applications has exceeded a threshold value.

[0009] Furthermore, in the abnormal correlation detection device according to the present invention, the determination unit may be further configured to identify, for two web applications determined to have an abnormal correlation, at least one of a web application that is affecting communication and a web application that is affected by communication, based on the traffic volume related to each web application.

[0010] In addition, the abnormal correlation detection device of the present invention may further include a preparation unit configured to acquire traffic volume related to each web application of the plurality of web applications by performing deep packet inspection on one or both of communications from one or more UPFs to the Internet and communications from the Internet to the one or more UPFs.

[0011] Furthermore, in the abnormal correlation detection device according to the present invention, each of the one or more UPFs may be connected to the Internet via a router, and the abnormal correlation detection device may further include an instruction unit configured to instruct the router to impose a bandwidth limit on communications related to at least one of the two web applications for which an abnormal correlation has been determined.

[0012] In order to solve the above-mentioned problems, the abnormal correlation detection method of the present invention includes a learning step of constructing a multivariate probabilistic model using traffic volume related to each web application of a plurality of web applications and generating a matrix indicating a conditional dependency between any two web applications among the plurality of web applications from the multivariate probabilistic model, and a determination step of determining an abnormal correlation between the two web applications based on the values ​​of components indicating the conditional dependency between the two web applications in the matrix generated in the learning step.

[0013] In addition, in the abnormal correlation detection method according to the present invention, the learning step may include a step of constructing a multivariate normal distribution model with a sample mean of 0 as the multivariate probability model using traffic volume for each web application in each period of an observation period including a plurality of periods, and generating a precision matrix of the multivariate normal distribution model as the matrix.

[0014] In addition, in the abnormal correlation detection method according to the present invention, the determining step may include a step of determining, based on the precision matrix generated for a certain observation period and the precision matrix generated for an observation period prior to the certain observation period, whether at least one of the following occurs: a value of a component indicating the conditional dependency between two web applications has changed from 0 to a value other than 0; a value of a component indicating the conditional dependency between two web applications has changed from a value other than 0 to 0; or a value of a component indicating the conditional dependency between two web applications has exceeded a threshold value.

[0015] In addition, the abnormal correlation detection method of the present invention may further include a step of identifying at least one of the web application that is affecting communication and the web application that is affected by communication, based on the traffic volume related to each web application, for two web applications that are determined to have an abnormal correlation.

[0016] In addition, the abnormal correlation detection method of the present invention may further include a step of acquiring traffic volume related to each web application of the plurality of web applications by performing deep packet inspection on one or both of communications from one or more UPFs to the Internet and communications from the Internet to the one or more UPFs.

[0017] In addition, in the abnormal correlation detection method of the present invention, each of the one or more UPFs may be connected to the Internet via a router, and the method may further include a step of instructing the router to impose bandwidth restrictions on communications related to at least one of the two web applications for which an abnormal correlation has been determined. [Effects of the Invention]

[0018] According to the present invention, a multivariate probabilistic model is constructed using the traffic volume for each of a plurality of web applications, and a matrix indicating the conditional dependency between any two of the plurality of web applications is generated from the multivariate probabilistic model, thereby enabling more reliable detection of abnormal correlations between web applications. [Brief explanation of the drawings]

[0019] [Figure 1A] FIG. 1A is a block diagram showing the configuration of a system including an abnormal correlation detection device according to an embodiment of the present invention. [Figure 1B] FIG. 1B is a block diagram showing the configuration of a core network according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram for explaining the traffic volume of a web application acquired by the abnormal correlation detection device according to this embodiment. [Figure 3] FIG. 3 is a schematic diagram for explaining the configuration of the learning unit included in the abnormal correlation detection device according to this embodiment. [Figure 4A] FIG. 4A is a schematic diagram for explaining the configuration of a learning unit included in the abnormal correlation detection device according to this embodiment. [Figure 4B] FIG. 4B is a schematic diagram for explaining the configuration of the learning unit included in the abnormal correlation detection device according to this embodiment. [Figure 5] FIG. 5 is a block diagram showing the hardware configuration of the abnormal correlation detection device according to this embodiment. [Figure 6]FIG. 6 is a sequence diagram showing the operation of a system including an abnormal correlation detection device according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to FIGS. 1A to 6. FIG.

[0021] [System Configuration] Hereinafter, an overview of a system including an abnormal correlation detection device 1 according to an embodiment of the present invention will be described with reference to FIGS. 1A and 1B.

[0022] Referring to FIG. 1A, an abnormal correlation detection device 1 according to this embodiment is provided in association with a mobile communication network compatible with the 5G communication standard, for example. The mobile communication network includes a core network 4. The core network 4 includes UPFs (User Plane Functions) 44 as its functional nodes. Although only three UPFs 44 are depicted in FIG. 1A, the number of UPFs 44 included in the core network 4 may be any number equal to or greater than one. Note that functional nodes included in the core network 4 other than the UPFs 44 are omitted in FIG. 1A. The core network 4 will be described later in association with FIG. 1B.

[0023] Each UPF 44 is connected to a subordinate router 5. Although only one subordinate router 5 is depicted for each UPF 44 in FIG. 1A, the number of subordinate routers 5 connected to each UPF 44 may be any number equal to or greater than one.

[0024] Each router 5 is connected to the network NW via an abnormal correlation detection device 1. The abnormal correlation detection device 1 is connected to each router 5 and the network NW so as to perform deep packet inspection (DPI) on either or both of communications from each router 5 to the network NW and communications from the network NW to each router 5. In FIG. 1A, each router 5 is depicted as being connected to only one abnormal correlation detection device 1. However, the abnormal correlation detection device 1 may be configured to include multiple different abnormal correlation detection devices, and each router 5 may be connected to a different abnormal correlation detection device. Also, in FIG. 1A, only one path from the abnormal correlation detection device 1 to the network NW is depicted, but the number of paths from the abnormal correlation detection device 1 to the network NW may be any number greater than or equal to one.

[0025] The network NW is a data network such as the Internet. One or more networks (not shown) may exist between each router 5 and the abnormal correlation detection device 1, and between the abnormal correlation detection device 1 and the network NW. Servers that provide web applications, clouds, data centers, etc. are connected to the network NW.

[0026] The core network 4 will be described below with reference to Fig. 1B. Each user terminal 2 is realized as a mobile communication terminal such as a smartphone, a tablet computer, a laptop computer, or the like. Each user terminal 2 is equipped with a SIM, and the SIM contract profile stores the user's subscriber identification information, including identifier information such as the subscriber identification number (IMSI: International Mobile Subscriber Identity) assigned to the mobile phone line contract, the subscriber's telephone number (MSISDN: Mobile Subscriber International Subscriber Directory Number), and the SIM card number (ICCID: Integrated Circuit Card Identifier). The user terminal 2 is uniquely identified by the IMSI.

[0027] Each user terminal 2 is also assigned a terminal IP address that uniquely identifies the terminal. The IP address is assigned to the user terminal 2 via the SMF 42 after a session is established. In this embodiment, there are two or more user terminals 2. Each user terminal 2 is located in the communication area of ​​a different base station 3. Each user terminal 2 accesses the network NW from a designated UPF 44 via the base station 3 in which it is located, and via a subordinate router 5. Note that in Figure 1B, the configuration beyond the router 5 as seen from the UPF 44 is omitted; for the network NW, please refer to Figure 1A.

[0028] Each user terminal 2 transmits a location registration request signal to the core network 4 via the base station 3 when moving within a communication area, for periodic location updates, or when powered on. The location registration request signal transmitted by the user terminal 2 includes the IMSI.

[0029] Each base station 3 is composed of a wireless base station compatible with the 5G system, and relays communications between user terminals 2 located within the communication area and the core network 4. Each base station 3 is connected to the core network 4 via a network such as a backhaul link. Two or more base stations 3 are provided. Each base station 3 is uniquely identified by a base station ID. The base station ID also makes it possible to ascertain the geographical location of the base station 3.

[0030] The core network 4 further includes an Access and Mobility Management Function (AMF) 40, a Unified Data Management (UDM) / Unified Data Repository (UDR) 41, a Session Management Function (SMF) 42, and a Policy Control Function (PCF) 43, which are nodes in the C-plane. The UPF 44 included in the core network 4 is a node in the U-plane. Functional nodes in the U-plane and C-plane other than those mentioned above that the core network 4 includes are not shown in the figure.

[0031] The AMF 40 is an access and mobility management device that manages the registration and wireless connection of the user terminal 2 that moves to each communication area.

[0032] The UDM / UDR 41 manages subscriber profiles, performs authentication, and manages mobility.

[0033] The SMF 42 is a session management function that establishes, modifies, and releases PDU (Packet Data Unit) sessions between the user terminal 2 and a data network such as the Internet. The SMF 42 sets an appropriate communication path for data communication between the user terminal 2 and the UPF 44 based on a PCC (Policy and Charging Control) policy from the PCF 43.

[0034] The PCF 43 determines QoS and policies and provides them to the SMF 42. The PCF 43 applies PCC rules according to the 3GPP (registered trademark) specifications and creates a PCC policy for setting a communication path for the UPF 44 with which the user terminal 2 communicates.

[0035] The UPF 44 is a user plane function that processes packets between the base station 3 and the network NW (a data network such as the Internet). The UPF 44 functions as a gateway between the core network 4 and an external network NW. A plurality of UPFs 44 are provided in the core network 4. As shown in FIG. 1B, each UPF 44 is directly connected to each base station 3, i.e., a so-called full mesh connection. In this embodiment, each UPF 44 transmits packets from the user terminal 2 to the network NW via a subordinate router 5. Each UPF 44 also forwards packets transmitted from the network NW to the user terminal 2. Each UPF 44 has an IP address, which allows the UPF 44 to be uniquely identified.

[0036] [Function block of the abnormal correlation detection device] Next, the functional blocks of the abnormal correlation detection device 1 according to this embodiment will be described with reference to the block diagram of Fig. 1A. As shown in Fig. 1A, the abnormal correlation detection device 1 includes a preparation unit 10, a learning unit 11, a determination unit 12, an instruction unit 13, and a storage unit 15.

[0037] The preparation unit 10 prepares data on the amount of traffic related to each of a plurality of web applications, which is used by the learning unit 11.

[0038] In this embodiment, a web application refers to application software used on a website via a network such as the Internet, and may be, but is not limited to, Netflix or YouTube. The user terminal 2 uses the web application via a web browser such as Safari or Google Chrome. The user terminal 2 may also use a web application (e.g., YouTube) using an application other than a web browser (e.g., a native YouTube app installed on a smartphone). The user terminal 2 corresponds to a computer.

[0039] In this embodiment, the traffic volume related to a web application is the traffic volume generated by a user terminal 2 (computer) using the web application. The traffic volume related to a web application may be the overall traffic volume generated by one or more user terminals 2 (computers) using the web application, but is not limited to this. For example, this does not exclude the use of the traffic volume generated at a certain user terminal 2 when the certain user terminal 2 uses the web application as the traffic volume related to a web application.

[0040] The number of multiple web applications is arbitrary. Hereinafter, the number of multiple web applications will be assumed to be M. Furthermore, hereinafter, it will be assumed that each web application is assigned a unique number (1st, 2nd, ..., Mth) expressed as a natural number so that one web application can be identified from multiple web applications. The number to identify each web application can be the IP address or URL of the server that provides the web application.

[0041] The preparation unit 10 may prepare data on the traffic volume for each web application in each period of an observation period including multiple periods. The traffic volume data for each web application in one period of an observation period including multiple periods is expressed as an observation value x=(x1, x2, . . . , x M ) where each element x of one observation x m (m=1, 2, . . . , M) is the traffic volume for one cycle related to the m-th web application among multiple web applications. An example of the length of one cycle is one minute, but this is not limited to this. The traffic volume for one cycle related to a web application may be the total amount of data (number of packets, number of bytes, number of bits, etc.; the same applies below) received by one or more terminals 2 (computers) during that cycle by using the web application, the total amount of data sent by one or more terminals 2 (computers), the sum of the total amount of data received and the total amount of data sent by one or more terminals 2 (computers), or any of these divided by the length of the cycle (e.g., bps), but is not limited to these.

[0042] The preparation unit 10 may be configured to obtain traffic volume related to each web application of multiple web applications by performing deep packet inspection on one or both of communications from UPF 44 (there may be one or more) to the network NW (Internet) and communications from the network NW (Internet) to UPF 44.

[0043] Deep packet inspection refers to the inspection of the data portion of a packet (e.g., but not limited to, the payload of a TCP packet) in communication in the form of a packet. Deep packet inspection may include the inspection of the header portion of the packet (e.g., but not limited to, the header of a TCP packet or an IP packet). The specific method of deep packet inspection is arbitrary and may include, for example, but not limited to, inspecting communication signatures specific to a web application or inspecting communication patterns specific to a web application.

[0044] FIG. 2 illustrates a table 420 that the preparation unit 10 can use in the process of preparing data on the traffic volume associated with each of multiple web applications. The preparation unit 10 may perform deep packet inspection on either or both of communications from each UPF 44 to the network NW (Internet) and communications from the network NW (Internet) to each UPF 44 to identify the traffic originating from each web application in the communications (either or both of the data received by the user terminal 2 and the data transmitted by the user terminal 2 using the web application) and measure and acquire the volume of the traffic (the value in each row of the "traffic volume" column in table 420). Each UPF 44 adds its own UPF number to the data portion of the packet, and the preparation unit 10 sets the UPF number in table 420 by performing deep packet inspection. The preparation unit 10 also associates the UPF number with the source IP address (router) of the packet to which the UPF number is added. The value ("2.1"), for example, in the first row of the "Traffic Volume" column of table 420 corresponds to the amount of data received and / or transmitted by a user terminal 2 communicating via the first UPF 44 using the first web application. Furthermore, the preparation unit 10 may calculate and acquire the traffic volume (the value in each row of the "Total" column of table 420) for each web application by summing the values ​​in each row of the "Traffic Volume" column of table 420 across the UPFs 44 for each web application. The value ("11.3"), for example, in the first row of the "Total" column of table 420 corresponds to the total amount of data received and transmitted by one or more user terminals 2 (including the user terminal 2 communicating via the first UPF 44, the user terminal 2 communicating via the second UPF 44, ...) using the first web application. The preparation unit 10 may perform the above process in each period of an observation period that includes multiple periods. The value in each row of the “Total” column of table 420 obtained in one period of an observation period including multiple periods corresponds to the traffic volume, i.e., one observation value x, for each web application in one period of an observation period including multiple periods.

[0045] The preparation unit 10 collects the observed values ​​x for each period for each observation period to generate a data set D=(x (1) ,x (2) ,···,x (N) ) can be prepared. Here, each element x of a data set D (n) (n=1,2,...,N) is the observed value x of the nth cycle in one observation period. An example of the length of the observation period is 5 hours, but it is not limited to this. If the length of the observation period is 5 hours and the length of the cycle is 1 minute, then N=5(hours) x 60(minutes) / 1(minute)=300. One dataset D corresponds to the traffic volume for each cycle in one observation period for each web application of multiple web applications.

[0046] The learning unit 11 constructs a multivariate probability model using the traffic volume for each web application of the multiple web applications, and generates a matrix indicating the conditional dependency between any two web applications of the multiple web applications from this multivariate probability model. The learning unit 11 may construct a multivariate normal distribution model with a sample mean of 0 as the multivariate probability model using the traffic volume for each web application of the multiple web applications in each period of an observation period including multiple periods, i.e., a dataset D, and generate a precision matrix of the multivariate normal distribution model as a matrix indicating the conditional independence (dependence) between any two web applications of the multiple web applications. In this case, each element x of the observation value x included in the dataset D m are the variables of the multivariate data for the multivariate normal distribution model.

[0047] The learning unit 11 learns the essential dependencies between variables using the traffic volume related to each of multiple web applications as a variable. Figures 3(a) and (b) show the directed graph used by the learning unit 11. Each node a, b, and c indicates a variable corresponding to a web application and indicates the traffic volume related to the web application. Each edge indicates a direct probability dependency. In the directed graph of Figure 3(a), the value of node c is unobserved. In this case, the joint probability distribution ρ(a, b, c) of variables a, b, and c is expressed by the following equation (1).

number

[0048] By marginalizing the variable c, it can be expressed as the following equation (2).

number

[0049] In the above equation (2), the two variables a and b are not independent, since they cannot generally be expressed as ρ(a)ρ(b). On the other hand, in the directed graph of Figure 2(b), variable c is observed. The joint probability ρ(a, b|c) when variable c is observed is expressed by the following equation (3).

number

[0050] In equation (3) above, when the value of the observed variable c, which is a common condition, is taken into consideration, it can be seen that variables a and b do not affect each other, i.e., they are independent. This is called conditional independence. By taking conditional independence into consideration, it becomes possible to extract the essential or true relationship between variables. For example, even if the values ​​(traffic volume) of one node a and another node b appear to be related, when the value (traffic volume) of yet another node c is taken into consideration, this corresponds to a case in which the values ​​(traffic volume) of node a and node b do not affect each other.

[0051] As described above, the traffic volume for each of the multiple web applications in one period is expressed as an M-dimensional observation value x=(x1, x2, . . . , x M ) and a data set D consisting of N observations x is expressed as D={x (1) ,x (2) ,···,x (N) In this case, the multivariate normal distribution model is expressed by the following equation (4).

number

[0052] In the above equation (4), μ is the sample mean when the observed value x is used as a sample, Σ is the covariance matrix, and |·| is the determinant. The learning unit 11 performs maximum likelihood estimation to find μ and Σ, which are parameters of the multivariate normal distribution model, from a dataset D of observed data. The log likelihood L(μ,Σ|D) of the dataset D is expressed by the following equation (5).

number

[0053] Substituting the above equation (4) into the above equation (5) gives the following equation (6).

number

[0054] The parameters μ and Σ that maximize the log likelihood L(μ,Σ|D) in the above equation (6) are estimated as the most likely solution. For the most likely solution of the parameters μ and Σ, μ and Σ are respectively -1 Differentiating with and setting it to 0, the maximum likelihood solutions of the parameters μ and Σ are expressed by the following equations (7) and (8), respectively.

number

number

[0055] Inverse matrix Σ of parameters Σ -1 is called the precision matrix Λ. The precision matrix Λ can be obtained by calculating the inverse matrix of the above formula (8). Note that in the maximum likelihood estimation shown in the above formulas (7) and (8), if regularization is not effective and there is a risk of overlearning, the learning unit 11 can also perform estimation by applying maximum a posteriori estimation (MAP estimation) to the parameters μ and Σ.

[0056] Here, the correlation structure in a multivariate normal distribution is expressed by a graph model. Such a graph model is called a Gaussian graphic model. Below, we will explain how to calculate conditional probability when applying a multivariate normal distribution model to graph theory. In the multivariate normal distribution model of the above formula (4), the inverse matrix Σ of the parameter Σ is -1 When the precision matrix Λ is used and the parameter μ is set to 0, it is expressed by the following equation (9).

number

[0057] The ρ(x) in the above equation (9) is called a Gaussian graphic model. Under a multivariate normal distribution, the conditional probability ρ(x1,x2|x3,...,x M ) is expressed by the following equation (10).

number

[0058] The function of variables x1 and x2 in the above equation (10) is ρ(x) in the above equation (9), that is, N(x|0,Λ -1 ), so if we extract all the parts of the above equation (9) related to the variables x1 and x2, we obtain the relationship in the following equation (11).

number

[0059] where the conditional independence ρ(x1|x3, ,x M )ρ(x2|x3, ,x M The condition for satisfying this is given by the following equation (12):

number

[0060] The above equation (12) indicates that the values ​​of the first and second components of the precision matrix Λ are zero, i.e., there is no edge between variables x1 and x2, as in the case of variables a and b in the directed graph of Figure 3. Figures 4A and 4B are schematic diagrams showing the relationship between the correlation between variables on a Gaussian graphical model graph and the precision matrix Λ. Each node in Figures 4A and 4B corresponds to a web application among multiple web applications.

[0061] Figure 4A shows that there is a correlation between the traffic volume between the second node (web application) and the ninth node (web application), and between the third node (web application) and the thirteenth node (web application) during a certain observation period. In this case, the precision matrix Λ has at least two 9 components Λ in addition to the diagonal components.2,9 , and ,3,13 components Λ 3,13 In Figures 4A and 4B, non-zero values ​​are represented by "*".

[0062] On the other hand, Figure 4B shows that in the observation period after the above-mentioned certain observation period, the correlation in traffic volume between the second node (web application) and the ninth node (web application) has disappeared, but the correlation in traffic volume between the third node (web application) and the thirteenth node (web application) has been maintained, and furthermore, a new correlation in traffic volume has appeared between the second node (web application) and the M-1th node (web application). At this time, the 2nd and 9th components Λ of the precision matrix Λ 2,9 changes from non-zero to zero, and the 3, 13 component Λ 3,13 maintains non-zero values, and furthermore, 2,M-1 components Λ 2,M-1 will change from a zero value to a non-zero value.

[0063] If there is no normal correlation in the traffic volumes between two nodes (web applications), a change in the value of the corresponding element of the precision matrix Λ from zero to non-zero can be considered to indicate an anomalous correlation in the traffic volumes. Furthermore, a subsequent change in the value of the element from non-zero to zero can be considered to indicate that the anomalous correlation has been resolved. If there is a normal correlation in the traffic volumes between two nodes (web applications), a change in the value of the corresponding element of the precision matrix Λ from non-zero to zero can be considered to indicate an anomalous correlation in the traffic volumes. If a specific element of the precision matrix Λ has a value of zero, this means that the two corresponding variables are conditionally independent. Therefore, the precision matrix Λ makes it possible to determine anomalous correlation in the traffic volumes by looking only at the direct relationship between variables, without being affected by noise or indirect relationships.

[0064] In order to be able to use the above formula (9), the learning unit 11 may construct a multivariate normal distribution model after setting the sample mean μ to 0. Therefore, the learning unit 11 may construct a multivariate normal distribution model after setting the n-th element x (n)(n=1 N) (observation value) m-th element (m=1 M)x m (n) Normalized conversion for (traffic volume related to web applications)

number

number

[0065] 1A, the determination unit 12 determines whether there is an abnormal correlation between two web applications among the plurality of web applications, based on the values ​​of components indicating conditional independence (dependence) between the two web applications in the matrix generated by the learning unit 11. The abnormal correlation determined by the determination unit 12 includes an abnormal correlation caused by a burst increase in traffic related to a certain web application affecting the communications of other web applications.

[0066] The following describes a case where the learning unit 11 uses a precision matrix Λ of a multivariate normal distribution model constructed with the sample mean set to 0. Based on the precision matrix Λ generated for a certain observation period and the precision matrix Λ generated for an observation period prior to the certain observation period, the determining unit 12 may determine whether the value of a component in the precision matrix Λ indicating a conditional dependency between two web applications has changed from zero to non-zero (hereinafter referred to as "Case 1"). A change from zero to non-zero in the value of a component indicating a conditional dependency between two web applications that are normally uncorrelated in terms of traffic volume can be considered an anomalous correlation. Therefore, the determining unit 12 may determine an anomalous correlation when at least one of one or more components in the precision matrix Λ indicating a conditional dependency between two web applications that are normally uncorrelated in terms of traffic volume corresponds to Case 1.

[0067] The determination unit 12 may determine whether the value of a component in the precision matrix Λ that indicates a conditional dependency between two web applications in the precision matrix Λ has changed from non-zero to zero (hereinafter referred to as "Case 2") based on the precision matrix Λ generated for a given observation period and the precision matrix Λ generated for an observation period prior to the given observation period. A change from non-zero to zero in the value of a component that indicates a conditional dependency between two web applications that are normally correlated in terms of traffic volume can be considered an anomalous correlation. For example, Case 2 includes a case where the traffic volumes of two web applications that previously fluctuated simultaneously no longer fluctuate simultaneously in a subsequent observation period. In addition, a phenomenon in which a relationship in which an increase or decrease in traffic volume related to one application normally leads to an increase or decrease in traffic volume related to another application disappears may indicate some kind of abnormality. For example, if an attack such as a DDoS attack causes abnormal requests to be concentrated on one web application, or if a network route is changed due to a change in network settings, or if a system failure causes processing to be stalled in a certain part of the system, preventing data from flowing normally from one web application to another, the dependency between web applications in terms of traffic volume may disappear, and such phenomena may be subject to judgment as an abnormal correlation in traffic volume in Case 2.

[0068] Therefore, the determination unit 12 may determine that there is an anomalous correlation when at least one of one or more components in the precision matrix Λ that indicates a conditional dependency between two applications that are normally correlated in terms of traffic volume corresponds to Case 2. Furthermore, the determination unit 12 may determine that the anomalous correlation has been resolved when a component that previously corresponded to Case 1 (or Case 3, which will be described later) now corresponds to Case 2.

[0069] The determination unit 12 may make a determination after setting a threshold value, taking into consideration that component values ​​in the precision matrix Λ that are essentially zero may take on slightly non-zero values ​​due to noise, sampling error, etc. Therefore, the determination unit 12 may determine that the value of a component in the precision matrix Λ that indicates a conditional dependency between two web applications exceeds a threshold value (hereinafter referred to as "Case 3"), based on the precision matrix Λ generated for a certain observation period and the precision matrix Λ generated for an observation period prior to the certain observation period. The determination unit 12 may determine that an abnormal correlation exists when at least one of one or more components in the precision matrix Λ that indicates a conditional dependency between two web applications that are normally not correlated in terms of traffic corresponds to Case 3.

[0070] The threshold may be a predetermined threshold (fixed value) or a value that depends on the value of a past component. An example of a threshold that depends on the value of a past component is a value obtained by multiplying the value of a component in the precision matrix Λ generated by the learning unit 11 for an observation period prior to a certain observation period by a predetermined value. In other words, when the component in the precision matrix Λ generated by the learning unit 11 for a certain observation period is multiplied by Λ, i,j current , the components in the precision matrix Λ generated by the learning unit 11 for an observation period prior to a certain observation period are denoted by Λ i,j old Then, the decision unit 12 determines Λ i,j current / Λ i,j old >T (T is a predetermined value).

[0071] Therefore, the determining unit 12 may determine at least one of Cases 1 to 3 based on the precision matrix Λ generated for a certain observation period and the precision matrix Λ generated for an observation period prior to the certain observation period.

[0072] The determination unit 12 may determine that an abnormality has occurred when the precision matrix Λ, which is determined sequentially, has a component that falls into any of Cases 1 to 3 multiple times in a row. For example, the determination unit 12 may determine that an abnormality has occurred when the precision matrix Λ, which is determined sequentially, has a component that falls into Case 3 10 times in a row.

[0073] The determination unit 12 may identify at least one of the web application affecting communication and the web application affected by communication for two web applications determined to have an abnormal correlation based on the traffic volume for each web application. For example, for two web applications determined to have an abnormal correlation, the one with the larger traffic volume may be identified as the web application affecting communication, and the one with the smaller traffic volume may be identified as the web application affected by communication. Alternatively, for two web applications determined to have an abnormal correlation, the one with an increasing traffic volume may be identified as the web application affecting communication, and the one with a decreasing traffic volume may be identified as the web application affected by communication, based on past traffic volume.

[0074] The instruction unit 13 instructs the router 5 to limit the bandwidth of communications related to at least one of the two web applications for which an abnormal correlation has been determined. The instruction unit 13 may instruct all of the one or more routers 5 to limit the bandwidth of communications related to at least one of the two web applications for which an abnormal correlation has been determined. Alternatively, the instruction unit 13 may instruct some of the one or more routers 5 to limit the bandwidth of communications related to at least one of the two web applications for which an abnormal correlation has been determined. The instruction unit 13 may instruct only communications related to at least one of the two web applications for which an abnormal correlation has been determined, which is affecting communications of other web applications.

[0075] The two web applications for which an abnormal correlation has been determined may be web applications corresponding to elements of the precision matrix Λ that have been determined by the determination unit 12 to fall into any one of the above-mentioned cases 1 to 3. Note that there may be multiple sets of two web applications for which an abnormal correlation has been determined. For example, when the determination unit 12 determines that the Λ i,j and Λ i,k If it is determined that the abnormal correlation applies to one of the above-mentioned cases 1 to 3, the two web applications determined to have an abnormal correlation are the i-th web application and the j-th web application, and the i-th web application and the k-th web application.

[0076] Alternatively, the two web applications determined to have an abnormal correlation may be web applications corresponding to elements of the precision matrix Λ that have been determined by the determination unit 12 to fall into a specific one of the above-described cases 1 to 3. For example, two web applications corresponding to elements of the precision matrix Λ that have been determined by the determination unit 12 to fall into case 3 may be defined as two web applications determined to have an abnormal correlation, while two web applications corresponding to elements of the precision matrix Λ that have been determined by the determination unit 12 to fall into case 1 or 2 may not be defined as two web applications determined to have an abnormal correlation.

[0077] As time passes, the preparation unit 10 sequentially generates datasets D for different observation periods, the learning unit 11 sequentially generates precision matrices Λ using the different datasets D, and the determination unit 12 sequentially determines whether there is an abnormal correlation in the different precision matrices Λ. Therefore, the instruction unit 13 may determine, as two web applications determined to have an abnormal correlation, two web applications corresponding to components of the precision matrix Λ that fall into any one of Cases 1 to 3 or a specific one multiple times consecutively in the sequentially determined precision matrix Λ. For example, the instruction unit 13 may determine, as two web applications determined to have an abnormal correlation, two web applications corresponding to components determined to fall into Case 3 10 times consecutively in the sequentially determined precision matrix Λ.

[0078] The storage unit 15 may store parameters of the multivariate probability model constructed by the learning unit 11 (for example, parameters μ, Σ of the multivariate normal distribution model) and a matrix (for example, precision matrix Λ) generated by the learning unit 11. The learning unit 11 may also store the results of the determinations made by the determination unit 12 for cases 1 to 3. The information stored by the learning unit 11 is not limited to these.

[0079] [Hardware configuration of the abnormal correlation detection device] Next, an example of a hardware configuration for realizing the abnormal correlation detection device 1 having the above-described functions will be described with reference to FIG.

[0080] 5, the abnormal correlation detection device 1 can be realized by, for example, a computer including a processor 102, a main memory device 103, a communication interface 104, an auxiliary memory device 105, and an input / output (I / O) 106, which are connected via a bus 101, and a program that controls these hardware resources. Furthermore, the abnormal correlation detection device 1 includes a display device 107.

[0081] The processor 102 is realized by a CPU, a GPU, an FPGA, an ASIC, or the like.

[0082] The main memory device 103 pre-stores programs for the processor 102 to perform various controls and calculations. The processor 102 and the main memory device 103 implement the functions of the abnormal correlation detection device 1, such as the preparation unit 10, learning unit 11, determination unit 12, and instruction unit 13 shown in FIG.

[0083] The communication interface 104 is an interface circuit for connecting the anomaly correlation device 1 and various external electronic devices via a network. The communication interface 104 realizes at least a part of the configuration of the preparation unit 10 and the instruction unit 13.

[0084] The auxiliary storage device 105 is composed of a readable / writable storage medium and a drive for reading and writing various information such as programs and data from and to the storage medium. The auxiliary storage device 105 can use a semiconductor memory such as a hard disk or flash memory as the storage medium.

[0085] The auxiliary storage device 105 has a program storage area for storing the abnormal correlation detection program. The auxiliary storage device 105 also has a program storage area for storing a learning program that estimates parameters using a multivariate normal distribution model executed by the abnormal correlation detection device 1 and generates a precision matrix. The auxiliary storage device 105 realizes the storage unit 15 described in FIG. 1A. Furthermore, the auxiliary storage device 105 may have, for example, a backup area for backing up the above-mentioned data, programs, etc.

[0086] The input / output I / O 106 is an input / output device that inputs signals from external devices and outputs signals to external devices.

[0087] The display device 107 is configured by an organic EL display, a liquid crystal display, etc. The display device 107 can display on the screen information such as the fact that an abnormal correlation has been determined.

[0088] [System Operation] Next, the operation of a system including the abnormal correlation detection device 1 having the above-described configuration will be described with reference to the sequence in Fig. 6. Note that Fig. 6 shows a sequence for one operation that is repeated in this system. Therefore, it should be noted that steps S1 to S8, which will be described later, are repeatedly executed multiple times.

[0089] As shown in FIG. 6, first, the preparation unit 10 of the abnormal correlation detection device 1 acquires the traffic volume related to each of a plurality of web applications (step S1).

[0090] Next, the preparation unit 10 prepares data on the traffic volume for each web application from the traffic volume for each web application acquired in step S1 (step S2). In step S2, the preparation unit 10 may prepare one data set D of observed values ​​x from the traffic volume for each cycle of one observation period for each web application. That is, the preparation unit 10 prepares a set of M-dimensional observed values ​​x=(x1, x2, . . . , x M ) based on the data set D={x (1) ,x (2) ,···,x (N)} may be prepared.

[0091] Next, the learning unit 11 constructs a multivariate probability model using the traffic volume related to each web application (step S3). In step S3, the multivariate probability model may be a multivariate normal distribution model. That is, the learning unit 11 may estimate, by maximum likelihood estimation, the maximum likelihood solution of parameters μ and Σ of the multivariate normal distribution model for dataset D, which is multivariate data with the traffic volume related to each web application as a variable. In step S3, the learning unit 11 may use the above equation (6) to estimate parameters μ and Σ that maximize the log likelihood L(μ,Σ|D) as the maximum likelihood solution (the above equations (7) and (8)).

[0092] In step S3, the maximum likelihood solution for the parameters of the multivariate normal distribution model may be estimated twice. That is, using the maximum likelihood solution for the parameters μ and Σ estimated for the original dataset D so that the sample mean μ when each observed value x is a sample is 0, standardization transformation may be performed on the dataset D using the above equation (13), and the maximum likelihood solution for the parameters μ and Σ may be estimated again for the dataset D after standardization transformation. However, since the maximum likelihood solution for the parameter μ estimated the second time should be 0, the second estimation of the parameter μ may be omitted. Therefore, in step S3, the learning unit 11 may construct a multivariate normal distribution model with a sample mean of 0 as a multivariate probability model using the traffic volume for each web application, i.e., the dataset D, for each period of an observation period including multiple periods.

[0093] In addition, in the maximum likelihood estimation shown in the above equations (7) and (8), if there is a risk that regularization is not effective and overlearning may occur, in step S3, the learning unit 11 can also perform estimation by applying maximum a posteriori estimation (MAP estimation) to the parameters μ and Σ.

[0094] Next, the learning unit 11 generates a matrix representing the conditional dependency between any two web applications from the constructed multivariate probability model (step S4). In step S4, the learning unit 11 may generate the precision matrix Λ of the multivariate normal distribution model as a matrix indicating the conditional independence between any two web applications from the multiple web applications. In this case, in step S4, the learning unit 11 generates the inverse matrix Σ of the parameters Σ estimated in step S3. -1 The learning unit 11 may store the generated precision matrix Λ in the storage unit 15.

[0095] Next, the determination unit 12 determines whether there is an anomalous correlation between the two web applications based on the values ​​of components indicating a conditional dependency between the two web applications in the matrix generated in step S4 (step S5). In step S5, the determination unit 12 may determine at least one of the above-mentioned cases 1 to 3 based on the precision matrix generated in the immediately preceding step S4 and the precision matrix generated in the step S4 executed immediately before the immediately preceding step S4. The determination unit 12 may acquire the precision matrix Λ used in step S5 from the storage unit 15. Furthermore, the determination unit 12 may determine whether there is an anomalous correlation in the traffic volume after step S4 has been executed a predetermined number of times, rather than determining whether there is an anomalous correlation in the traffic volume every time step S4 is executed. Therefore, the judgment unit 12 may judge one of cases 1 to 3 based on the precision matrix generated for a certain observation period (for example, the precision matrix generated in the step S4 executed immediately before the above-mentioned) and the precision matrix generated for an observation period prior to the certain observation period (for example, the precision matrix generated in the step S4 executed immediately before the step S4 executed immediately before the last step).

[0096] In step S5, the judgment unit 12 may identify at least one of the web application that has affected communication and the web application that is affected by communication for the two web applications that have been determined to have an abnormal correlation, based on the traffic volume related to each web application.

[0097] Next, the instruction unit 13 determines whether to instruct the router 5 to perform band limitation based on the determination of the anomalous correlation by the determination unit 12 (step S6). In step S6, the instruction unit 13 may determine that the router 5 should be instructed to perform band limitation when one or more elements of the precision matrix Λ correspond to any one of cases 1 to 3. Alternatively, in step S6, the instruction unit 13 may determine that the router 5 should be instructed to perform band limitation when one or more elements of the precision matrix Λ correspond to a specific one of cases 1 to 3. Alternatively, in step S6, the instruction unit 13 may determine that the router 5 should be instructed to perform band limitation when the precision matrix Λ, which is sequentially determined, has an element that corresponds to a specific one of cases 1 to 3 multiple times in a row. If it is determined in step S6 that the router 5 should be instructed to perform band limitation (step S6: YES), the process proceeds to step S7. If not (step S6: NO), the process ends this sequence, and the process proceeds to step S1 in the next sequence (not shown).

[0098] If the instruction unit 13 determines that the router 5 should be instructed to limit the bandwidth (step S6: YES), the instruction unit 13 instructs the router 5 to limit the bandwidth for communication related to at least one of the two web applications for which an abnormal correlation has been determined (step S7).

[0099] Thereafter, the router 5 that has received the instruction executes the bandwidth limitation (step S8). In Fig. 6, the instruction unit 13 instructs all of the one or more existing routers 5 to impose the bandwidth limitation, but this is not limitative.

[0100] As described above, the abnormal correlation detection device 1 according to this embodiment focuses on the occurrence of a correlation between the traffic volumes of two web applications when, for example, a bursty increase in traffic related to one web application affects the communication of another web application. Then, a multivariate model is constructed from the traffic volumes of each web application, and a matrix (e.g., precision matrix Λ) representing the conditional dependency between the web applications in terms of traffic volume is calculated to determine the abnormal correlation between the two web applications, and two web applications in which an abnormal correlation has occurred are detected based on the results. Furthermore, from the two web applications in which an abnormal correlation has occurred, it is possible to identify the web application that is affecting communication and the web application whose communication is affected by the abnormal correlation.

[0101] Furthermore, a system including the abnormal correlation detection device 1 according to this embodiment determines an abnormal correlation between two web applications based on the traffic volume related to each web application, and instructs the router 5 to limit the bandwidth of communications related to at least one of the two web applications for which an abnormal correlation has occurred. This makes it possible to more effectively prevent traffic generated by one web application from affecting communications of other web applications.

[0102] In the embodiment described above, the learning unit 11 employs a Gaussian graphic model to analyze conditional independence through a precision matrix, which is the inverse matrix of the covariance matrix. However, the algorithm employed by the learning unit 11 is not limited to the Gaussian graphic model. For example, in sparse estimation such as Graphical Lasso, it can be employed in combination with Maximum A Posteriori Estimation (MAP). Alternatively, Bayesian network structure learning can be employed.

[0103] The above describes embodiments of the abnormal correlation detection device and the abnormal correlation detection method of the present invention. However, the present invention is not limited to the described embodiments, and various modifications that can be conceived by those skilled in the art can be made within the scope of the invention described in the claims. [Explanation of symbols]

[0104] 1...Abnormal correlation detection device, 2...User terminal, 3...Base station, 4...Core network, 10...Preparation unit, 11...Learning unit, 12...Decision unit, 13...Instruction unit, 15...Memory unit, 40...AMF, 41...UDM / UDR, 42...SMF, 43...PCF, 44...UPF, 5...Router, 101...Bus, 102...Processor, 103...Main memory device, 104...Communication interface, 105...Auxiliary memory device, 106...Input / output I / O, 107...Display device, NW...Network.

Claims

1. a learning unit configured to construct a multivariate probabilistic model using traffic volume related to each of a plurality of web applications, and to generate, from the multivariate probabilistic model, a matrix representing a conditional dependency relationship between any two of the plurality of web applications; a determining unit configured to determine an abnormal correlation between the two web applications based on values ​​of components indicative of the conditional dependency between the two web applications in the matrix generated by the learning unit; and An abnormal correlation detection device comprising:

2. 2. The abnormal correlation detection device according to claim 1, The learning unit is further configured to construct, as the multivariate probability model, a multivariate normal distribution model with a sample mean of 0, using traffic volume for each period of an observation period including a plurality of periods, related to each web application, and to generate, as the matrix, a precision matrix of the multivariate normal distribution model. An abnormal correlation detection device characterized by:

3. 3. The abnormal correlation detection device according to claim 2, The determination unit determines, based on the precision matrix generated for a certain observation period and the precision matrix generated for an observation period prior to the certain observation period, The value of the component indicating the conditional dependency between the two web applications changes from 0 to a value other than 0; and The value of the component indicating the conditional dependency between the two web applications changes from non-zero to zero; and that a value of a component indicating the conditional dependency between two web applications exceeds a threshold; and and further configured to determine at least one of An abnormal correlation detection device characterized by:

4. 2. The abnormal correlation detection device according to claim 1, Furthermore, the determination unit is configured to identify at least one of a web application that is affecting communication and a web application that is affected by communication, based on the traffic volume related to each of the two web applications determined to have an abnormal correlation. An abnormal correlation detection device characterized by:

5. 2. The abnormal correlation detection device according to claim 1, Further, a preparation unit configured to acquire a traffic volume for each of the plurality of web applications by performing deep packet inspection on one or both of communications from one or more UPFs to the Internet and communications from the Internet to the one or more UPFs. An abnormal correlation detection device characterized by:

6. 6. The abnormal correlation detection device according to claim 5, each of the one or more UPFs is connected to the Internet via a router; The abnormal correlation detection device further includes an instruction unit configured to instruct the router to limit bandwidth for communication related to at least one of the two web applications for which an abnormal correlation has been determined. An abnormal correlation detection device characterized by:

7. A learning step in which a computer constructs a multivariate probabilistic model using traffic volumes associated with each of a plurality of web applications, and generates, from the multivariate probabilistic model, a matrix indicating conditional dependencies between any two of the plurality of web applications; a determining step in which the computer determines an abnormal correlation between the two web applications based on values ​​of components in the matrix generated in the learning step that indicate the conditional dependency between the two web applications; 1. An abnormal correlation detection method comprising:

8. The abnormal correlation detection method according to claim 7, The learning step includes a step of constructing a multivariate normal distribution model with a sample mean of 0 as the multivariate probability model using traffic volume for each period of an observation period including a plurality of periods for each web application, and generating a precision matrix of the multivariate normal distribution model as the matrix.

10. An abnormal correlation detection method comprising:

9. 9. The abnormal correlation detection method according to claim 8, The determining step is based on the precision matrix generated for a certain observation period and the precision matrix generated for an observation period prior to the certain observation period, The value of the component indicating the conditional dependency between the two web applications changes from 0 to a value other than 0; and The value of the component indicating the conditional dependency between the two web applications changes from non-zero to zero; and that a value of a component indicating the conditional dependency between two web applications exceeds a threshold; and determining at least one of 10. An abnormal correlation detection method comprising:

10. The abnormal correlation detection method according to claim 7, The method further comprises a step of identifying, by the computer, at least one of a web application that is affecting communication and a web application that is affected by communication, based on the traffic volume related to each of the two web applications determined to have an abnormal correlation.

10. An abnormal correlation detection method comprising:

11. The abnormal correlation detection method according to claim 7, The method further includes a step of acquiring a traffic volume for each of the plurality of web applications by performing deep packet inspection on one or both of communications from one or more UPFs to the Internet and communications from the Internet to the one or more UPFs by the computer.

10. An abnormal correlation detection method comprising:

12. The abnormal correlation detection method according to claim 11, each of the one or more UPFs is connected to the Internet via a router; The method further includes a step of instructing the router to limit bandwidth for communication related to at least one of the two web applications for which an abnormal correlation has been determined.

10. An abnormal correlation detection method comprising:

Images