Anomaly detection device and anomaly detection method

Abstract

The purpose is to more easily detect abnormalities in program operation. [Solution] The anomaly detection device 1 comprises a learning unit 11 configured to construct a multivariate probabilistic model using each of a plurality of processing resource usages used in the execution of processes corresponding to a plurality of process IDs observed in each time period within an observation period, and to generate a matrix representing the conditional dependency between a plurality of processing resource usages corresponding to a plurality of process IDs by estimating parameters of the constructed multivariate probabilistic model; and a judgment unit 12 configured to judge an operational anomaly of processes corresponding to two process IDs based on values ​​indicating the conditional dependency between the processing resource usages used in the execution of processes corresponding to any two of the plurality of process IDs in the matrix generated by the learning unit 11.

Description

[Technical Field]

[0001] The present invention relates to an abnormality detection device and an abnormality detection method. [Background technology]

[0002] In recent years, the demands on software have become more sophisticated and complex in order to provide high-performance systems and services. The software implemented in high-performance systems consists of programs with huge amounts of source code, and the execution of these programs is becoming increasingly complex.

[0003] When a process failure or an abnormal operation occurs in an operating system, identifying the cause of the program failure or abnormal operation is complicated, time-consuming, and not easy. For example, Patent Document 1 discloses a method for using a core file to retroactively identify the location of the error and the variable values ​​at the time.

[0004] However, the technology disclosed in Patent Document 1 requires analysis of the source code because it is not possible to identify the cause or location of an error from log information. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-301570 Summary of the Invention [Problem to be solved by the invention]

[0006] As described above, conventional techniques have not been able to easily detect abnormal program operation.

[0007] The present invention has been made to solve the above-mentioned problems, and has an object to more easily detect abnormalities in program operation. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the anomaly detection device of the present invention comprises a learning unit configured to construct a multivariate probabilistic model using each of a plurality of processing resource usages used in the execution of processes corresponding to a plurality of process IDs observed in each time period within an observation period, and to generate a matrix representing the conditional dependency between the plurality of processing resource usages corresponding to the plurality of process IDs by estimating parameters of the constructed multivariate probabilistic model; and a judgment unit configured to judge an operational abnormality of processes corresponding to any two process IDs among the plurality of process IDs based on values ​​indicating the conditional dependency between the processing resource usages used in the execution of processes corresponding to the two process IDs in the matrix generated by the learning unit.

[0009] Furthermore, in the anomaly detection device according to the present invention, the learning unit may estimate, by maximum likelihood estimation, the parameters of a multivariate normal distribution model having a sample mean of 0 for multivariate data in which each of the plurality of processing resource usages used in executing the processes corresponding to the plurality of process IDs is a variable, thereby generating a precision matrix of the multivariate normal distribution model as the matrix indicating the conditional dependency between the plurality of processing resource usages corresponding to the plurality of process IDs.

[0010] In addition, in the anomaly detection device according to the present invention, the learning unit may generate a first matrix, which is the matrix during a first observation period, and a second matrix, which is the matrix during a second observation period after the first observation period; and the determination unit may be further configured to determine an operational anomaly of the process based on at least one of the following conditions: when a value of a component in the first matrix indicating the conditional dependency between amounts of processing resources used in the execution of the processes corresponding to the two process IDs is compared with a value of a corresponding component in the second matrix, the value changes from a zero value to a non-zero value; when a value of a component in the first matrix indicating the conditional dependency between amounts of processing resources used in the execution of the processes corresponding to the two process IDs is compared with a value of the corresponding component in the second matrix, the value changes from a non-zero value to a zero value; or when a value of the corresponding component in the second matrix exceeds a threshold value.

[0011] In addition, the abnormality detection device according to the present invention may further include an acquisition unit configured to acquire usage amounts of the plurality of processing resources used in executing processes corresponding to the plurality of process IDs.

[0012] Furthermore, the abnormality detection device according to the present invention may further include an instruction unit configured to, when the determination unit determines that an operational abnormality exists in the process, send an instruction to execute a predetermined response to resolve the operational abnormality in the process.

[0013] In order to solve the above-mentioned problems, the anomaly detection method of the present invention comprises a learning step of constructing a multivariate probabilistic model using each of a plurality of processing resource usages used in the execution of processes corresponding to a plurality of process IDs observed in each time period within an observation period, and estimating parameters of the constructed multivariate probabilistic model to generate a matrix representing the conditional dependency between the plurality of processing resource usages corresponding to the plurality of process IDs; and a determination step of determining an operational anomaly of processes corresponding to any two of the plurality of process IDs based on values ​​indicating the conditional dependency between the processing resource usages used in the execution of processes corresponding to the two process IDs in the matrix generated in the learning step.

[0014] Furthermore, in the anomaly detection method according to the present invention, the learning step may generate a precision matrix of the multivariate normal distribution model as the matrix indicating the conditional dependency between the multiple processing resource usages corresponding to the multiple process IDs by estimating the parameters of a multivariate normal distribution model with a sample mean of 0 for multivariate data having each of the multiple processing resource usages used in executing the processes corresponding to the multiple process IDs as a variable, by maximum likelihood estimation.

[0015] In addition, in the anomaly detection method according to the present invention, the learning step may generate a first matrix, which is the matrix during a first observation period, and a second matrix, which is the matrix during a second observation period after the first observation period, and the determination step may further determine an operational anomaly of the process based on at least one of the following conditions: when a value of a component in the first matrix indicating the conditional dependency between amounts of processing resources used in the execution of the processes corresponding to the two process IDs is compared with a value of a corresponding component in the second matrix, the value changes from a zero value to a non-zero value; when a value of a component in the first matrix indicating the conditional dependency between amounts of processing resources used in the execution of the processes corresponding to the two process IDs is compared with a value of the corresponding component in the second matrix, the value changes from a non-zero value to a zero value; or when a value of the corresponding component in the second matrix exceeds a threshold value.

[0016] The anomaly detection method according to the present invention may further comprise an acquisition step of acquiring usage amounts of the plurality of processing resources used in executing the processes corresponding to the plurality of process IDs.

[0017] Furthermore, the anomaly detection method according to the present invention may further include an instruction step of sending an instruction to execute a predetermined response to resolve the operational abnormality of the process when the determination step determines that an operational abnormality of the process exists. [Effects of the Invention]

[0018] According to the present invention, an operational anomaly of processes corresponding to any two process IDs among a plurality of process IDs is determined based on a value indicating a conditional dependency between resource usage amounts corresponding to these two process IDs in a matrix generated by a learning unit, thereby making it possible to more easily detect an operational anomaly of a program. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a block diagram showing the configuration of an abnormality detection system including an abnormality detection device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram for explaining the configuration of the learning unit included in the abnormality detection device according to this embodiment. [Figure 3] FIG. 3 is a schematic diagram for explaining the configuration of the learning unit included in the abnormality detection device according to this embodiment. [Figure 4] FIG. 4 is a schematic diagram for explaining the configuration of the learning unit included in the abnormality detection device according to this embodiment. [Figure 5] FIG. 5 is a block diagram showing the hardware configuration of the abnormality detection device according to this embodiment. [Figure 6] FIG. 6 is a sequence diagram showing the operation of an abnormality detection system including an abnormality detection device according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to FIGS.

[0021] [Configuration of anomaly detection system] First, with reference to FIG. 1, an overview of an abnormality detection system including an abnormality detection device 1 according to an embodiment of the present invention will be described.

[0022] The anomaly detection system includes an anomaly detection device 1 and an information processing device 2. The anomaly detection device 1 and the information processing device 2 are connected via a network NW. The anomaly detection system according to this embodiment learns a multivariate model from observed data of processing resource usage for each process ID in a normal operating state, and detects a relationship between processing resource usage for a process ID that deviates from the pattern of processing resource usage used by the process ID under normal conditions, thereby detecting an abnormality in process operation.

[0023] The network NW includes, for example, wired networks such as LAN, WAN, the Internet, and ISDN, as well as wireless networks such as wireless LAN and mobile communication networks using LTE / 4G, 5G, and 6G wireless communication systems, but the scope of the present invention is not limited to these.

[0024] The information processing device 2 can be realized as a server, a gateway, a desktop computer, an embedded device, a mobile communication terminal such as a smartphone, a tablet computer, a laptop computer, etc. In this embodiment, the information processing device 2 is not limited to one device, but includes the case of multiple devices. In the case of multiple devices, each information processing device 2 executes the same process generated by the same application or the same executable file.

[0025] The information processing device 2 is uniquely identified by network identification information such as an IP address or a MAC address, or a device ID assigned by an anomaly detection system. The information processing device 2 can be realized by a computer equipped with a processor, a main memory device, a communication interface, an auxiliary memory device, and an input / output (I / O), and a program that controls these hardware resources.

[0026] The information processing device 2 executes one or more applications (programs) on the OS, and each application runs as one or more processes. Each process is assigned a unique process ID (PID) by the OS. A process ID is assigned to each execution unit of a process, and even the same application may have different process IDs when multiple processes are generated, depending on the execution environment and timing. The information processing device 2 calculates CPU utilization as the amount of processing resources used for each process ID corresponding to the running process, and records the calculated CPU utilization in memory. If the configurations of the CPUs of the multiple information processing devices 2, such as the number of cores and clock speed, are not the same, the CPU utilization recorded by each information processing device 2 is normalized or corrected using each CPU benchmark, etc. The normalization or correction process may be performed by the anomaly detection device 1.

[0027] In this embodiment, the CPU usage rate for each process ID is used as the data on the processing resource usage, but the processing resource usage may also be the memory usage rate for each process ID, the I / O wait time for each process ID, etc. Abnormal values ​​of the CPU usage rate are caused by abnormal operation of the process executed by the information processing device 2.

[0028] A state in which an operational abnormality occurs in a process executed by the information processing device 2 refers to a state in which the process deviates from the normal load range, such as excessive consumption of processing resources. For example, this state applies when the CPU usage rate for each process ID exceeds a preset threshold, or when a normal CPU usage rate range is defined according to the attributes of the process and the CPU usage rate outside that range deviates from the normal load range. Even if the CPU usage rate does not exceed the threshold, a predefined CPU usage rate pattern can be used as a load state pattern when an operational abnormality occurs in the process, and can be used as an abnormal CPU usage rate that deviates from the normal CPU usage rate range.

[0029] In this way, when a process is operating abnormally, it is assumed that abnormal processing such as an infinite loop or excessive recursive calls is occurring in the execution of the process linked to that process ID.On the other hand, normal CPU usage means that no abnormal processing is occurring in the execution of the process linked to each process ID, and CPU resource consumption is within a normal range.

[0030] [Function block of the anomaly detection device] Next, functional blocks of the abnormality detection device 1 according to this embodiment will be described with reference to the block diagram of Fig. 1. As shown in Fig. 1, the abnormality detection device 1 includes an acquisition unit 10, a learning unit 11, a determination unit 12, an instruction unit 13, and a storage unit 14.

[0031] The acquisition unit 10 collects, via the network NW, CPU utilization rates, which are normal processing resource usage rates used in executing processes corresponding to each of a plurality of process IDs, recorded in the information processing device 2. The acquisition unit 10 can collect CPU utilization rates from a plurality of information processing devices 2. The CPU utilization rate data is associated with the process ID. The acquisition unit 10 acquires, from the information processing device 2, CPU utilization rates corresponding to a plurality of process IDs for each set time period (e.g., one-minute cycle) over a set observation period (e.g., five hours). The acquired CPU utilization rates include CPU utilization rates for each of M process IDs. The acquired CPU utilization rates are also linked to the identification information of the source information processing device 2 and the time of transmission.

[0032] Here, if the CPU usage rate of each process ID is obtained at one-minute intervals, the CPU usage rate of each process ID from 1 to M is, for example, x=(x1,x2,x3,x4,...,x M) = (35.2, 42.7, 95.8, 38.9, , 57.1). This shows that for one minute, process ID: p1 had a CPU usage rate of 35.2% and process ID: p2 had a CPU usage rate of 42.7%. For example, process ID: p3 had a CPU usage rate of 95.8%, indicating that an abnormality in the process operation is occurring.

[0033] The learning unit 11 constructs a multivariate probability model using each of a plurality of CPU utilization rates used in the execution of processes corresponding to a plurality of process IDs observed in each time period within the observation period, and generates a matrix representing the conditional independence (dependence) between the plurality of CPU utilization rates corresponding to the plurality of process IDs by estimating parameters of the constructed multivariate probability model. More specifically, the learning unit 11 generates a precision matrix of the multivariate normal distribution model as a matrix representing the conditional independence between the plurality of CPU utilization rates corresponding to the plurality of process IDs by estimating, by maximum likelihood estimation, the parameters of the multivariate normal distribution model in which the sample mean is set to 0 for multivariate data having each of the plurality of CPU utilization rates used in the execution of processes corresponding to a plurality of process IDs recorded by the information processing device 2 as a variable.

[0034] The learning unit 11 generates a first matrix Λ, which is a precision matrix for a first observation period, and a second matrix Λ, which is a precision matrix for a second observation period after the first observation period. The observation periods are monitoring periods for anomaly detection, and the first observation period is, for example, the first five hours, and the second observation period is, for example, the five hours immediately following the first observation period.

[0035] As described above, the learning unit 11 learns the essential dependencies between variables using the patterns of CPU usage used in the execution of processes corresponding to multiple process IDs as variables. Here, (a) and (b) of Figure 2 show the directed graph used by the learning unit 11. Each node a, b, and c represents a variable and indicates the CPU usage for each process ID. Each edge represents a direct probability dependency. In the directed graph of (a) of Figure 2, the value of node c is unobserved. In this case, the joint probability distribution ρ(a, b, c) of variables a, b, and c is expressed by the following equation (1).

number

[0036] By marginalizing the variable c, it can be expressed as the following equation (2).

number

[0037] In the above equation (2), the two variables a and b are not independent, since they cannot generally be expressed as ρ(a)ρ(b). On the other hand, in the directed graph of Figure 2(b), variable c is observed. The joint probability ρ(a, b|c) when variable c is observed is expressed by the following equation (3).

number

[0038] In equation (3) above, when the value of the observed variable c, which is a common condition, is taken into consideration, we can see that variables a and b do not affect each other, i.e., they are independent. This is called conditional independence. By taking conditional independence into consideration, it becomes possible to extract the essential or true relationship between variables. For example, even if there appears to be a correlation between the CPU usage rate corresponding to one process ID and the CPU usage rate corresponding to another process ID, when the CPU usage rate corresponding to yet another process ID is taken into consideration, this would be the case where the CPU usage rates for these multiple process IDs do not affect each other.

[0039] Here, as mentioned above, the CPU usage for each process ID is expressed as an M-dimensional observation x=(x1, x2, , x M ) A data set D consisting of N observations x is expressed as D={x (1) ,x (2) ,···,x (M)}, this multivariate normal distribution model is expressed by the following equation (4).

number

[0040] In the above equation (4), μ is the sample mean when the observed value x is a sample, Σ is the covariance matrix, and |·| is the determinant. The learning unit 11 performs maximum likelihood estimation to find μ and Σ, which are parameters of the multivariate normal distribution model, from a dataset D of observed data. The log likelihood L(μ,Σ|D) of the dataset D is expressed by the following equation (5).

number

[0041] Substituting the above equation (4) into the above equation (5) gives the following equation (6).

number

[0042] The parameters μ and Σ that maximize the log likelihood L(μ,Σ|D) in the above equation (6) are estimated as the most likely solution. For the most likely solution of the parameters μ and Σ, μ and Σ are respectively -1 Differentiating with and setting it to 0, the maximum likelihood solutions of the parameters μ and Σ are expressed by the following equations (7) and (8), respectively.

number

number

[0043] Inverse matrix Σ of parameters Σ -1is called the precision matrix Λ. The precision matrix Λ can be obtained by calculating the inverse matrix of the above formula (8). Note that in the maximum likelihood estimation shown in the above formulas (7) and (8), if regularization is not effective and there is a risk of overlearning, the learning unit 11 can also perform estimation by applying maximum a posteriori estimation (MAP estimation) to the parameters μ and Σ.

[0044] Here, the correlation structure in a multivariate normal distribution is expressed by a graph model. Such a graph model is called a Gaussian graphic model. Below, we will explain how to calculate conditional probability when applying a multivariate normal distribution model to graph theory. In the multivariate normal distribution model of the above formula (4), the inverse matrix Σ of the parameter Σ is -1 When the precision matrix Λ is used and the parameter μ is set to 0, it is expressed by the following equation (9).

number

[0045] The ρ(x) in the above equation (9) is called a Gaussian graphic model. Under a multivariate normal distribution, the conditional probability ρ(x1,x2|x3,...,x M ) is expressed by the following equation (10).

number

[0046] The function of variables x1 and x2 in the above equation (10) is ρ(x) in the above equation (9), that is, N(x|0,Λ -1 ), so if we extract all the parts of the above equation (9) related to the variables x1 and x2, we obtain the relationship in the following equation (11).

number

[0047] where the conditional independence ρ(x1|x3, ,x M )ρ(x2|x3, ,x M The condition for satisfying this is given by the following equation (12):

number

[0048] Equation (12) above indicates that the values ​​of the first and second components of the precision matrix Λ are 0, i.e., there is no edge between variables x1 and x2, as in the directed graph of Figure 2. Figures 3 and 4 (a) and (b) are schematic diagrams showing the relationship between the correlation between variables on the Gaussian graphical model graph and the precision matrix Λ. The nodes in Figure 3 (a) and Figure 4 (a) correspond to process IDs 1 through M and represent the CPU usage observed for each process ID. Figure 3 (a) also shows the correlation between variables based on the CPU usage for multiple process IDs, which are observed values ​​during the first observation period. Figure 4 (a) shows the correlation between variables based on the CPU usage for multiple process IDs, which are observed values ​​during the second observation period following the first observation period.

[0049] Figure 3(a) shows that there is a correlation between the CPU utilization rates used in the execution of processes corresponding to these two process IDs during the first observation period between the second node and the ninth node, and between the third node and the thirteenth node (solid line edges). Also, in the precision matrix Λ shown in Figure 3(b), in addition to the diagonal elements, there are at least two, nine elements Λ. 2,9 , and ,3,13 components Λ 3,13 In the precision matrix Λ in Figure 3(b) and Figure 4(b), non-zero values ​​are represented by "*".

[0050] On the other hand, Figure 4(a) shows that in the second observation period, which follows the first observation period, the correlation between node 3 and node 13, i.e., the correlation between the CPU utilization rates of the two process IDs corresponding to these nodes, has disappeared (the dotted edge represents the disappeared edge). Furthermore, Figure 4(a) shows that in the second observation period, the correlation between the CPU utilization rates of the two process IDs corresponding to node 2 and node 9 is maintained (the solid edge). It also shows that in the second observation period, a new correlation has emerged between the CPU utilization rates of the two process IDs corresponding to node 3 and node M-3 (the thick edge). At this time, the 3rd and 13th components Λ of the precision matrix Λ 3,13 changes from non-zero to zero, and the 2,9 component Λ 2,9 maintains non-zero values, and furthermore, the 3,M-3 component Λ 3,M-3 changes from a zero value to a non-zero value.

[0051] When there is no correlation between the CPU usage rates corresponding to two process IDs between two nodes under normal conditions where no abnormalities are occurring in the signals, a change in the value of the corresponding element of the precision matrix Λ from zero to a non-zero value indicates that the relationship between specific process IDs has been broken. The emergence of a new correlation between process IDs that are originally independent indicates the occurrence of unexpected dependencies or conflicts between these processes, that is, the possibility that an abnormal load on one process is affecting the other.

[0052] For example, this occurs when a deadlock occurs, causing processes to wait for each other's locks, resulting in a simultaneous increase in CPU usage, or when one process generates excessive threads, causing excessive thread generation that increases simultaneously with the CPU usage of the other process.If the value of the component subsequently changes from a non-zero value to a zero value (returns), this indicates that the dependency or load interlock that occurred previously has been resolved and the processes have returned to an independent state, i.e., the abnormal state has been resolved.

[0053] When there is a correlation between CPU utilization rates corresponding to two process IDs between two nodes under normal conditions where no process abnormalities are occurring, a change in the value of a corresponding element of the precision matrix Λ from a non-zero value to a zero value can be said to indicate that the relationship between specific process IDs that were originally correlated has been disrupted. For example, this indicates that cooperative operation between these processes has been disrupted. Specifically, this includes a state in which one process stops or crashes, resulting in the loss of interlocking operation; a state in which normal synchronization processing is no longer performed due to an inter-node communication failure; or a state in which one process is skipped due to an overload or error, leaving the other process running alone. Thus, in this embodiment, the disappearance of the correlation between process IDs that are originally correlated is considered to indicate the occurrence of a process abnormality.

[0054] If the value of a specific component of the precision matrix Λ is zero, it means that the two corresponding variables are conditionally independent. Therefore, it is possible to determine anomalies in a signal by looking only at the direct relationship between the variables in the precision matrix Λ, without being affected by noise or indirect relationships. Note that, in order to be able to use the above formula (9), the learning unit 11 may construct a multivariate normal distribution model after setting the sample mean μ to 0. In this case, the learning unit 11 calculates the n-th element x of the data set D. (n) (n=1 N) (observation value) m-th element (m=1 M)x m (n) After performing the standardization transformation expressed by the following equation (13) on (CPU usage rate for each process ID), the observed value x (n) A multivariate normal distribution model can be constructed using the following as a sample.

number

number

[0055] 1, the determination unit 12 determines an operational abnormality of the processes corresponding to any two process IDs among multiple process IDs based on the values ​​of components indicating a conditional independence relationship between the CPU utilization rates corresponding to these two process IDs in the precision matrix Λ (matrix) of the multivariate normal distribution model constructed by the learning unit 11 with the sample mean set to 0. The determination unit 12 determines a change in the correlation between the CPU utilization rates of these two process IDs as an operational abnormality of the processes.

[0056] More specifically, the determination unit 12 compares the first matrix Λ for the first observation period with the second matrix Λ for the second observation period, and determines that an abnormal correlation has occurred between the CPU utilization rates corresponding to these two process IDs and that an operational abnormality has occurred in the processes corresponding to these process IDs if at least one of conditions 1 to 3 is satisfied, when a change in the value of the component indicating conditional independence between the CPU utilization rates corresponding to two process IDs satisfies the following condition 1 or condition 2, or when the value of the component indicating conditional independence of the second matrix Λ for the second observation period satisfies condition 3. The determination unit 12 can determine that an operational abnormality has occurred in the process if at least one of conditions 1 to 3 is satisfied.

[0057] More specifically, when comparing the value of a component in the first matrix Λ for the first observation period that indicates conditional independence between CPU utilization rates corresponding to two process IDs with the value of the corresponding component in the second matrix Λ for the second observation period, if the component value changes from zero (0) to a non-zero value (other than 0), the determination unit 12 determines that an operational anomaly has occurred in the processes corresponding to these two process IDs (Condition 1). As shown in (a) of FIG. 3, the 3,M-3 component Λ of the precision matrix Λ (first matrix Λ) for the first observation period 3,M-3 was zero, whereas the 3 and M-3 components Λ of the precision matrix Λ (second matrix Λ) in the second observation period in Figure 4(a) 3,M-3has changed to a non-zero value, satisfying condition 1. In condition 1, the CPU usage of process ID: p3 and process ID: p M-3 The emergence of a new correlation between the CPU usage rate and the process is detected as an abnormality in the process behavior.

[0058] Furthermore, when the value of a component indicating conditional independence between CPU utilization rates corresponding to two process IDs in the first matrix Λ for the first observation period is compared with the value of the corresponding component of the second matrix Λ for the second observation period, if the component value changes from a non-zero value (other than 0) to a zero value (0), the determination unit 12 determines that an operational abnormality has occurred in the processes corresponding to these two process IDs (Condition 2). 3,13 was a non-zero value, whereas the 3rd and 13th components Λ of the precision matrix Λ (second matrix Λ) in the second observation period in Figure 4(a) 3,13 has changed to zero, which corresponds to condition 2. In condition 2, the CPU usage of process ID: p3 and the CPU usage of process ID: p 13 The loss of correlation between process IDs, which are normally correlated, and the CPU usage of the process is detected as an abnormality in process operation.

[0059] Furthermore, when the value of the component indicating the conditional independence between the CPU utilization rates corresponding to two process IDs in the first matrix Λ for the first observation period is compared with the value of the corresponding component of the second matrix Λ for the second observation period, if the value of the corresponding component of the second matrix Λ for the second observation period exceeds a set threshold, the determination unit 12 determines that an operational abnormality has occurred in the process corresponding to these process IDs (condition 3). As shown in (a) of Figure 3, the 2nd and 9th components of the precision matrix Λ (first matrix Λ) for the first observation period are 2,9 , and the 2nd and 9th components Λ of the precision matrix Λ (second matrix Λ) in the second observation period in Figure 4(a). 2,9 Even if both are non-zero, the second and ninth components Λ 2,9exceeds the threshold, condition 3 is met. That is, the third condition is determined by using the value of the component in the first observation period as a reference and determining whether the value of the corresponding component in the second observation period exceeds the threshold.

[0060] The determination unit 12 may, for example, set the conditional expression of the third condition as follows: i,j ) / (first observation period component Λ i,j ) ≧ 2 can be used. The threshold value in the condition expression can be set arbitrarily. In condition 3, a stronger correlation between the CPU usage rate of process ID: p2 and the CPU usage rate of process ID: p9 is detected as a process operation abnormality. In determining based on condition 3, condition 3 can be set to be met when the CPU usage rates corresponding to the same two process IDs match a certain number of times in succession (for example, 10 times in succession).

[0061] In this way, the determination unit 12 focuses on non-zero elements that are sparse representations of the precision matrix Λ and determines whether or not there is an operational abnormality between the processes IDs. Note that the determination unit 12 can perform the determination process taking into consideration the cases where the values ​​of components other than the diagonal components of the precision matrix Λ are non-zero, as well as the possibility that component values ​​that are essentially zero may take on slightly non-zero values ​​due to noise, sampling error, etc.

[0062] When the determination unit 12 determines that an abnormality in the process operation has occurred, the instruction unit 13 sends an instruction to execute a predetermined response to resolve the abnormality in the process operation. Specifically, the instruction unit 13 can identify two process IDs in which an abnormal CPU usage rate has occurred, and send an instruction to the information processing device 2 via the network NW to terminate and restart the corresponding processes.

[0063] The storage unit 14 stores the parameters μ and Σ of the multivariate normal distribution model estimated by the learning unit 11 through learning, and the precision matrix Λ generated by the learning unit 11 .

[0064] [Hardware configuration of the anomaly detection device] Next, an example of a hardware configuration for realizing the abnormality detection device 1 having the above-described functions will be described with reference to FIG.

[0065] 5, the abnormality detection device 1 can be realized by, for example, a computer including a processor 102, a main memory device 103, a communication interface 104, an auxiliary memory device 105, and an input / output (I / O) device 106, all of which are connected via a bus 101, and a program that controls these hardware resources. The abnormality detection device 1 further includes a display device 107.

[0066] The processor 102 is a circuit or device that performs arithmetic processing, and is realized by, for example, a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc. Alternatively, the processor 102 may be configured by combining some or all of these.

[0067] The main memory device 103 pre-stores programs for the processor 102 to perform various controls and calculations. The processor 102 and the main memory device 103 implement the functions of the anomaly detection device 1, such as the acquisition unit 10, learning unit 11, determination unit 12, and instruction unit 13 shown in FIG.

[0068] The communication interface 104 is an interface circuit for connecting the abnormality detection device 1 to various external electronic devices via a network. The communication interface 104 realizes at least a part of the configuration of the instruction unit 13.

[0069] The auxiliary storage device 105 is composed of a readable / writable storage medium and a drive for reading and writing various information such as programs and data from and to the storage medium. The auxiliary storage device 105 can use a semiconductor memory such as a hard disk or flash memory as the storage medium.

[0070] The auxiliary storage device 105 has a program storage area for storing the anomaly detection program. The auxiliary storage device 105 also has a program storage area for storing a learning program that estimates parameters using a multivariate normal distribution model executed by the anomaly detection device 1 and generates a precision matrix. The auxiliary storage device 105 realizes the storage unit 14 described in FIG. 1. Furthermore, the auxiliary storage device 105 may have, for example, a backup area for backing up the above-mentioned data, programs, etc.

[0071] The input / output I / O 106 is an input / output device that inputs signals from external devices and outputs signals to external devices.

[0072] The display device 107 is configured by an organic EL display, a liquid crystal display, etc. The display device 107 can display on the screen information such as the process ID of the process in which an abnormality has occurred and the CPU utilization rate.

[0073] [Operation of the anomaly detection device] Next, the operation of the abnormality detection system including the abnormality detection device 1 having the above-described configuration will be described with reference to the sequence of FIG.

[0074] 6, first, the information processing device 2 periodically transmits the CPU utilization rate used in executing the processes corresponding to the process IDs recorded in the information processing device 2 to the error detection device 1 (step S1). Next, the acquisition unit 10 of the error detection device 1 acquires the CPU utilization rates used in executing the processes corresponding to the multiple process IDs received in step S1 (step S2).

[0075] In step S2, the acquiring unit 10 acquires CPU utilization rates corresponding to 1M process IDs for each set time period (e.g., every one minute) over a set observation period (e.g., five hours). The acquiring unit 10 acquires the CPU utilization rate for each process ID for each observation period, with the first five hours being a first observation period and the immediately following five hours being a second observation period. Next, the acquiring unit 10 creates a data set D of observed values ​​x from the CPU utilization rates corresponding to 1 to M process IDs for each time period acquired in step S2 (step S3).

[0076] In step S3, the acquisition unit 10 acquires the CPU utilization rate corresponding to each process ID for each time period (1-minute cycle) over an observation period (5 hours), and calculates the M-dimensional observed value x=(x1, x2, . . . , x M ) based on the data set D consisting of N observations x, let D={x (1) ,x (2) ,···,x (M) For example, if you obtain the CPU usage rate for each process ID in one-minute increments over a five-hour observation period, you will obtain observed values ​​x, which are the CPU usage rates corresponding to N=300 process IDs from 1 to M.

[0077] Next, the learning unit 11 estimates the parameters μ and Σ of a multivariate normal distribution model with the sample mean set to 0 for a dataset D, which is multivariate data with CPU usage rates corresponding to multiple process IDs as variables, by maximum likelihood estimation (step S4). In step S4, the learning unit 11 uses the above equation (6) to estimate the parameters μ and Σ that maximize the log likelihood L(μ,Σ|D) as the maximum likelihood solution (the above equations (7) and (8)).

[0078] Next, the learning unit 11 generates a precision matrix Λ of the multivariate normal distribution model as a matrix indicating the conditional independence between the CPU utilization rates corresponding to the two process IDs (step S5). In step S5, the learning unit 11 generates an inverse matrix Σ of the parameters Σ estimated in step S4. -1Since steps S4 and S5 are processes performed for each dataset D created for each observation period, a precision matrix Λ is generated for each observation period, and here a first matrix Λ is generated for the dataset D of the first observation period, and a second matrix Λ is generated for the dataset D of the second observation period.

[0079] Next, the determination unit 12 compares the first matrix Λ, which is the precision matrix Λ for the first observation period generated in step S5, with the second matrix Λ, which is the precision matrix Λ for the second observation period, and determines that an operational abnormality has occurred in the process corresponding to the process ID whose correlation has changed if at least one of conditions 1 to 3 is satisfied (step S6). In step S6, a determination process is performed for each pair of process IDs represented by each component of the precision matrix Λ.

[0080] In step S6, when the value of a component indicating conditional independence between CPU utilization rates corresponding to two process IDs in the first matrix Λ for the first observation period is compared with the value of the corresponding component of the second matrix Λ for the second observation period, if the value changes from 0 to a value other than 0, the judgment unit 12 judges that an abnormal operation has occurred in the processes corresponding to the two process IDs whose correlation has changed (condition 1).

[0081] Furthermore, in step S6, when the value of a component indicating conditional independence between CPU utilization rates corresponding to two process IDs in the first matrix Λ for the first observation period is compared with the value of the corresponding component in the second matrix Λ for the second observation period, if the value changes from non-zero to 0, the judgment unit 12 judges that an abnormal operation has occurred in the processes corresponding to the two process IDs whose correlation has changed (condition 2).

[0082] Furthermore, in step S6, the determination unit 12 compares the value of a component in the first matrix Λ for the first observation period that indicates conditional independence between CPU utilization rates corresponding to two process IDs with the value of the corresponding component of the second matrix Λ for the second observation period, and if the value of the corresponding component of the second matrix Λ for the second observation period exceeds a threshold value, the determination unit 12 determines that an operational anomaly has occurred in the processes corresponding to the two process IDs whose correlation has changed (condition 3). For example, the determination unit 12 sets the conditional expression for the third condition as follows: i,j ) / (first observation period component Λ i,j ) ≧2 can be used.

[0083] Next, if it is determined in step S6 that an operational abnormality has occurred in a process, the instruction unit 13 identifies two process IDs corresponding to the processes in which the operational abnormality has occurred, and sends an instruction to terminate and restart the processes to the information processing device 2 via the network NW (step S7). In step S7, the instruction is sent if any one of conditions 1 to 3 is satisfied in the determination process in step S6. Furthermore, in the case of condition 3, if the CPU utilization rates corresponding to the same two process IDs exceed a threshold a certain number of times in succession (for example, 10 times in succession), the process can be subject to a restart instruction.

[0084] As explained in Figs. 3 and 4, the 3rd and 13th components Λ of the first matrix Λ and the second matrix Λ 3,13 has changed from a zero value to a non-zero value, satisfying the condition 1. Therefore, in step S7, the instruction unit 13 M-3 and sends an instruction to the information processing device 2 to terminate and restart the corresponding process.

[0085] On the other hand, if the components of neither the first matrix Λ nor the second matrix Λ satisfy conditions 1 to 3, no process operation abnormality has occurred during the observation period to be determined, so no restart instruction is issued (step S7: NO), and the process returns to step S2. In this way, the precision matrix Λ is generated based on the observation values ​​during the observation period that are sequentially acquired over time, and a determination is made as to whether or not there is a process operation abnormality for each observation period.

[0086] Next, when the information processing device 2 receives the instruction, it executes the process ID: p3, p M-3 In this way, the anomaly detection system sequentially repeats steps S2 to S8 each time a precision matrix Λ is generated for each observation period, and monitors operational anomalies in the processes executed by the information processing device 2.

[0087] As described above, the anomaly detection device 1 according to this embodiment constructs a multivariate model based on the CPU utilization rates of processes corresponding to process IDs for each observation period, and calculates a precision matrix Λ that represents the conditional independence between the CPU utilization rates of each of multiple process IDs. This determines whether an abnormal correlation has occurred between the CPU utilization rates of any two process IDs that should normally be independent, and detects process operational anomalies based on the results. Therefore, process operational anomalies can be detected even when there is little abnormal CPU utilization data.

[0088] Furthermore, the anomaly detection device 1 according to this embodiment performs a time-series comparison of each component of the precision matrix Λ generated for each monitoring period in response to the time-dependent acquisition of CPU utilization rates resulting from the execution of processes corresponding to process IDs, and detects process operational anomalies based on changes in correlation between CPU utilization rates corresponding to multiple process IDs. This makes it possible to detect process operational anomalies in real time in response to CPU utilization rates that change in real time as processes are executed.

[0089] Furthermore, according to the anomaly detection device 1 of this embodiment, even if there is little observation data related to abnormal CPU utilization, learning can be performed without using abnormal CPU utilization data by using only the observation data of CPU utilization for each process ID in a normal state in which no abnormality has occurred as the learning target and modeling the statistical relationship (conditional dependency) between CPU utilization for each process ID in normal times. As a result, when a change in the correlation of CPU utilization corresponding to a process ID that is different from normal is observed, it becomes possible to detect abnormal process operation from the inconsistency with the model of the CPU utilization pattern corresponding to the process ID in the normal state.

[0090] Furthermore, according to the abnormality detection device 1 of this embodiment, an operational abnormality of a process is determined for each pair of process IDs, so that the process ID of the process in which an operational abnormality has occurred can be easily identified and measures can be taken to resolve the abnormality.

[0091] In the embodiment described above, the learning unit 11 employs a Gaussian graphic model to analyze conditional independence through a precision matrix, which is the inverse matrix of the covariance matrix. However, the algorithm employed by the learning unit 11 is not limited to the Gaussian graphic model. For example, in sparse estimation such as Graphical Lasso, it can be employed in combination with Maximum A Posteriori Estimation (MAP). Alternatively, Bayesian network structure learning can be employed.

[0092] The above describes embodiments of the anomaly detection device and anomaly detection method of the present invention, but the present invention is not limited to the described embodiments, and various modifications that a person skilled in the art can conceive are possible within the scope of the invention described in the claims. [Explanation of symbols]

[0093] 1...anomaly detection device, 2...information processing device, 10...acquisition unit, 11...learning unit, 12...judgment unit, 13...instruction unit, 14...memory unit, 101...bus, 102...processor, 103...main memory device, 104...communication interface, 105...auxiliary memory device, 106...input / output I / O, 107...display device, NW...network.

Claims

1. a learning unit configured to construct a multivariate probabilistic model using each of a plurality of processing resource usages used in the execution of processes corresponding to a plurality of process IDs observed in each time period within an observation period, and to generate a matrix representing conditional dependencies between the plurality of processing resource usages corresponding to the plurality of process IDs by estimating parameters of the constructed multivariate probabilistic model; a determination unit configured to determine an operational abnormality of processes corresponding to any two process IDs among the plurality of process IDs based on a value indicating the conditional dependency between processing resource usage amounts used in execution of processes corresponding to the two process IDs in the matrix generated by the learning unit; and An abnormality detection device comprising:

2. 2. The abnormality detection device according to claim 1, The learning unit estimates, by maximum likelihood estimation, the parameters of a multivariate normal distribution model in which a sample mean is set to 0 for multivariate data having variables each of the plurality of processing resource usages used in the execution of the processes corresponding to the plurality of process IDs, thereby generating a precision matrix of the multivariate normal distribution model as the matrix indicating the conditional dependency between the plurality of processing resource usages corresponding to the plurality of process IDs. An abnormality detection device characterized by:

3. 3. The abnormality detection device according to claim 2, the learning unit generates a first matrix that is the matrix in a first observation period and a second matrix that is the matrix in a second observation period after the first observation period; The determination unit further a change from a zero value to a non-zero value when comparing a value of an element in the first matrix indicating the conditional dependency between processing resource usage amounts used in the execution of processes corresponding to the two process IDs with a value of a corresponding element in the second matrix; a change from a non-zero value to a zero value when comparing a value of an element in the first matrix indicating the conditional dependency between processing resource usage amounts used in the execution of processes corresponding to the two process IDs with a value of the corresponding element in the second matrix; the value of the corresponding element of the second matrix exceeds a threshold value; and determining an operational anomaly of the process based on at least one of the conditions: An abnormality detection device characterized by:

4. 2. The abnormality detection device according to claim 1, The method further includes: acquiring the usage amounts of the plurality of processing resources used in the execution of the processes corresponding to the plurality of process IDs; An abnormality detection device characterized by:

5. 2. The abnormality detection device according to claim 1, Further, an instruction unit configured to issue an instruction to execute a predetermined measure to resolve the operational abnormality of the process when the determination unit determines that the process is operating abnormally. An abnormality detection device characterized by:

6. A computer-implemented anomaly detection method, comprising: a learning step of constructing a multivariate probabilistic model using each of a plurality of processing resource usages used in the execution of processes corresponding to a plurality of process IDs observed in each time period within an observation period, and estimating parameters of the constructed multivariate probabilistic model to generate a matrix representing conditional dependencies between the plurality of processing resource usages corresponding to the plurality of process IDs; a determining step of determining an operational abnormality of processes corresponding to any two process IDs among the plurality of process IDs based on a value indicating the conditional dependency between amounts of processing resources used in the execution of the processes corresponding to the two process IDs in the matrix generated in the learning step; An anomaly detection method comprising:

7. 7. The anomaly detection method according to claim 6, The learning step estimates the parameters of a multivariate normal distribution model using maximum likelihood estimation, with a sample mean set to 0 for multivariate data having variables each of the plurality of processing resource usages used in the execution of the processes corresponding to the plurality of process IDs, thereby generating a precision matrix of the multivariate normal distribution model as the matrix indicating the conditional dependency between the plurality of processing resource usages corresponding to the plurality of process IDs.

1. An anomaly detection method comprising:

8. The anomaly detection method according to claim 7, the learning step generates a first matrix that is the matrix in a first observation period and a second matrix that is the matrix in a second observation period after the first observation period; The determining step further comprises: a change from a zero value to a non-zero value when comparing a value of an element in the first matrix indicating the conditional dependency between processing resource usage amounts used in the execution of processes corresponding to the two process IDs with a value of a corresponding element in the second matrix; a change from a non-zero value to a zero value when comparing a value of an element in the first matrix indicating the conditional dependency between processing resource usage amounts used in the execution of processes corresponding to the two process IDs with a value of the corresponding element in the second matrix; the value of the corresponding element of the second matrix exceeds a threshold value; and determining whether the process is abnormal based on at least one of the conditions.

1. An anomaly detection method comprising:

9. 7. The anomaly detection method according to claim 6, The method further includes the step of acquiring the usage amounts of the plurality of processing resources used in the execution of the processes corresponding to the plurality of process IDs.

1. An anomaly detection method comprising:

10. 7. The anomaly detection method according to claim 6, Further, the method includes an instruction step of issuing an instruction to execute a predetermined measure to resolve the operational abnormality of the process when the operational abnormality of the process is determined in the determination step.

1. An anomaly detection method comprising:

Images