Log analysis device, log analysis method, and program

Abstract

A log analysis device for analyzing a log obtained from software, said device being equipped with: a log parsing unit which has a plurality of log parsers; an inference unit which has a plurality of models for calculating loss from an input log which is outputted from each log parser among the plurality of log parsers; and a log analysis method determination unit for determining a pair comprising a log parser and a model to be used in an inference phase from among the plurality of log parsers and the plurality of models, on the basis of the loss outputted from each model among the plurality of models.

Description

[Technical field]

[0001] The present invention relates to analysis of logs obtained from software in ICT systems, large-scale distributed systems, and the like. [Background technology]

[0002] Logs, which are semi-structured text output from systems, software, devices, etc., record information about their execution and are important data that can be used to check the status and behavior of the systems at runtime (He, S., He, P., Chen, Z., Yang, T., Su, Y., and Lyu, MR, "A Survey on Automated Log Analysis for Reliability Engineering", arXiv:2009.07237, 2020.).

[0003] Logs obtained from large-scale, complex systems are often large in volume and complicated, and research into automated log analysis is underway to make log analysis more efficient. Log analysis after acquisition from a system involves the use of a log parser and the process of log mining. The results of log analysis are used to manage, monitor, and troubleshoot systems. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] https: / / github.com / logpai / loghub / tree / master / HDFS [Non-Patent Document 2] M. Nagappan and MA Vouk, "Abstracting log lines to log event types for mining software system logs," 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010), 2010, pp. 114-117. [Non-Patent Document 3] Tang, L., Li, T., and Perng, C., "LogSig: Generating system events from raw textual logs", Proceedings of the 20th ACM international conference on Information and knowledge management, 2011, pp. 785-794. [Non-Patent Document 4] A. Makanju, AN Zincir-Heywood and EE Milios, "A Lightweight Algorithm for Message Type Extraction in System Application Logs," in IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 11, pp. 1921-1936, Nov. 2012 Summary of the Invention [Problem to be solved by the invention]

[0005] There are various log parser methods in the log analysis process, such as frequent pattern mining (Non-Patent Document 2), which searches for frequently occurring words, clustering (Non-Patent Document 3), which performs grouping, and a method of iteratively dividing logs (Non-Patent Document 4).

[0006] In log mining, data trends are identified in advance through learning, etc., and a model to be used for log analysis is created. When creating a model in log mining, the learning log is passed through a log parser used for log analysis, and the result is used as the input log.

[0007] Therefore, it is believed that the trends extracted by log mining are also influenced by the parsing method. However, since the log text is created in a format specified by the developer of the system that is the log output source, the log text differs depending on the log output source, and the appropriate parsing method differs depending on the log.

[0008] If the log parser is not able to parse correctly, a highly accurate model cannot be created, which may result in poor log mining results and affect tasks such as system administration.

[0009] The present invention has been made in consideration of the above-mentioned points, and has an object to provide a technique that enables highly accurate log analysis by applying a parsing method suitable for logs. [Means for solving the problem]

[0010] According to the disclosed technology, there is provided a log analysis device that analyzes a log obtained from software, comprising: a log parsing unit having a plurality of log parsers; an inference unit having a plurality of models for calculating a loss from an input log output from each of the plurality of log parsers; a log analysis method determination unit that determines a pair of a log parser and a model to be used in an inference phase from among the plurality of log parsers and the plurality of models based on a loss output from each of the plurality of models; A log analysis device is provided. Effect of the Invention

[0011] According to the disclosed technology, a technology is provided that enables highly accurate log analysis by applying a parsing method suited to the log. [Brief description of the drawings]

[0012] [Figure 1] FIG. 11 is a diagram for explaining a flow of log analysis. [Diagram 2] FIG. 13 is a diagram illustrating an example of a log and parsing. [Diagram 3] 1 is a configuration diagram of a log analysis device 100. FIG. [Figure 4] 2 is a diagram for explaining the flow of processing executed by the log analysis device 100. FIG. [Diagram 5] FIG. 11 is a diagram for explaining detailed processing of a learning phase and a verification phase. [Figure 6] FIG. 11 is a diagram for explaining detailed processing in the inference phase. [Figure 7] FIG. 2 is a diagram showing an example of the structure of a model 200. [Figure 8] FIG. 11 is a diagram for explaining an example of the operation of the log analysis method decision engine 150. [Figure 9] FIG. 2 is a diagram illustrating input and output of a function f(X). [Figure 10] 1 is a diagram illustrating an example of a hardware configuration of a log analysis device 100. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0014] (Log analysis overview) First, an overview of log analysis will be explained with reference to Figures 1 and 2. As mentioned above, log analysis after acquiring logs from a system or the like consists of the processes of log parser and log mining.

[0015] Figure 1 shows the flow of log analysis. A log is acquired from the system, and in S1 (step 1), the log parser parses it. In parsing, the semi-structured text of each line of the log is decomposed into structured and unstructured parts, and the unstructured part is decomposed into a template part, which is fixed text, and a parameter part to which variables, etc. are assigned. Figure 2 shows an example of parsing a line of the HDFS log (Non-Patent Document 1).

[0016] The parsing process transforms the log into a form that can be used as input for the next log mining (such as the form of table data). Hereafter, the parsed log that has been transformed into a form that can be used as input for log mining is referred to as the input log.

[0017] Log mining is performed in S2 in Figure 1. In log mining, the input logs obtained from the log parser are used as input data, and meaningful patterns and trends are extracted by searching using machine learning, data mining, etc., and the status and operation of the system, etc. that is the log output source, are output. The results obtained from log analysis are used for managing, monitoring, and troubleshooting the system, etc.

[0018] As mentioned above, the text of a log varies depending on the log output source, and the parsing method suitable for each log varies. Therefore, in the conventional technology, it is not possible to create a highly accurate model, which may result in poor log mining results and affect tasks such as system management. The following describes the configuration and operation of a device for solving such problems.

[0019] (Device configuration example) 3 shows a configuration diagram of a log analysis device 100 according to the present embodiment. As shown in FIG. 3, the log analysis device 100 includes a log database 110, a log parser database 120, a trained model database 130, a log parsing engine 140, a log analysis method determination engine 150, a learning engine 160, an inference engine 170, and an integration engine 180.

[0020] A log database 110 stores logs obtained from a system or the like, and a log parser database 120 stores a log parser (program) used for log parsing.

[0021] Further, the trained model database 130 stores trained models. As described later, the model in this embodiment is configured from a function with weight parameters and the like. The model may be a neural network. When the model is stored in the database 130, it is stored as data such as weight parameters and function programs.

[0022] In addition, the log parsing engine 140, the log analysis method determination engine 150, the learning engine 160, the inference engine 170, and the integration engine 180 may be referred to as the log parsing unit 140, the log analysis method determination unit 150, the learning unit 160, the inference unit 170, and the integration unit 180, respectively.

[0023] In addition, the log parsing engine 140, the log analysis method decision engine 150, the learning engine 160, the inference engine 170, and the integration engine 180 may each be a functional unit realized by executing a program on a computer, or may be a hardware circuit (dedicated processing processor), or may be a combination of a functional unit realized by executing a program on a computer and a hardware circuit.

[0024] Furthermore, each database may be provided outside the log analysis device 100. In this case, the log analysis device 100 accesses the database via a network to store and read data.

[0025] (Overview of device operation) A flow of processing executed by log analysis device 100 in this embodiment is shown in Fig. 4. As shown in Fig. 4, for logs acquired from log database 110, a log used for learning is referred to as a learning log, a log used for verification is referred to as a verification log, and a log used for inference is referred to as an inference log. Hereinafter, the learning process is referred to as the learning phase, the verification process is referred to as the verification phase, and the inference process is referred to as the inference phase. Log analysis device 100 executes log analysis consisting of a learning phase, a verification phase, and an inference phase.

[0026] 4, in the learning phase, a model is trained, and the trained model is stored in the trained model database 130. In the verification phase, the inference engine 170 executes inference on the input log using the trained model, and the log analysis method determination engine 150 determines a log analysis method using a loss for the inference result. In the inference phase, inference is executed using the log analysis method (a combination of a log parser and a model) determined in the verification phase.

[0027] Note that one log analysis device 100A may execute the learning phase and the verification phase, and another log analysis device 100B may execute the inference phase using the log analysis method determined thereby.

[0028] Also, one log analysis device 100A may execute the learning phase, and another log analysis device 100B may execute the verification phase and the inference phase using the model thus learned.

[0029] Furthermore, the learning phase, the verification phase, and the inference phase may be executed by different log analysis devices 100 (for example, the log analysis device 100-1, the log analysis device 100-2, and the log analysis device 100-3).

[0030] (Details of the learning and validation phases) The processing in each of the learning phase and the verification phase will be described in detail with reference to FIG.

[0031] <Learning Phase> In the learning phase, the log parsing engine 140 first extracts N train Next, the log parsing engine 140 retrieves the learning log of the line from the log parser database 120. train Obtain and keep N log parsers train The learning log of a row is p train Parse with each of the log parsers.

[0032] The log parsing engine 140 processes the log train Type N train The input log of the row is input to the learning engine 160. The learning engine 160 train Type N train For each row of input log, p is used in the validation phase. train Create a model of the type train The types of models are stored in a trained model database 130.

[0033] <Verification phase> In the verification phase, first, N valid A verification log consisting of lines is input to the log parsing engine 140. The log parsing engine 140 used in this phase is configured with the same log parser as in the learning phase. The inference engine 170 also uses the log parser learned in the learning phase. train The types of models are read from the trained model database 130 and stored.

[0034] The log parsing engine 140 processes the log train Type N valid The input log of rows is calculated by p train The input to the inference engine 170 is composed of various models. The inference engine 170 calculates the loss (described later) calculated by the model in N valid Calculate the row and average it as the loss p train The number of types is calculated and input to the log analysis method decision engine 150 .

[0035] Log analysis method decision engine 150 train Among the set of log parsers and models, p is used in the inference phase. test Determine the type and the p test The identification number of the model (=log parser identification number) is output as an inference number. test The type inference number corresponds to the number of the log parser and model pair that was determined to be used during the inference phase.

[0036] (Details of the inference phase) The log parsing engine 140 test p identified by type inference number test The inference engine 170 reads and holds the types of log parsers from the log parser database 120. test p identified by type inference number test The types of models are read from the trained model database 130 and stored.

[0037] Alternatively, the log parsing engine 140 and the inference engine 160 used in the verification phase may be used as they are. In this case, the log parsing engine 140 test p identified by type inference number test Only one kind of log parser is used, and the inference engine 160 uses p test p identified by type inference number test Use only one model of the same kind.

[0038] The detailed process in the inference phase will be described with reference to FIG. test The inference log for the row is obtained from the log database 110 and input to the log parsing engine 140 .

[0039] In the log parsing engine 140, parsing is performed by each log parser. test Type N test Create an input log of lines and test Type Ntest The input log of rows is input to the inference engine 170 .

[0040] In the inference engine 170, each model is test Inference is made from the input log of rows, and the p obtained from the model test The output (inference result) of each type is input to the integration engine 180. In the integration engine 180, p test Input the various inference results to the function f(X) and extract N test Get the output of the row.

[0041] This N test The row output is the log analysis result by the log analysis device 100, and it is assumed that this result will then be used for, for example, a task of checking the state or operation of a system or the like that is the log output source, or a task of detecting anomalies.

[0042] (Example model) The models in the learning engine 160 and the inference engine 170 will now be described. An example of the structure of the model 200 is shown in FIG. 7. N lines of an input log obtained by parsing are input to the model 200. A data extraction unit 210 in the model extracts necessary data from the input log and inputs it to a function 220 in the model 200. Several examples of methods for extracting necessary data from the input log will be given below with reference to the input log in FIG. 2. All of the following extraction methods may be implemented, or some of the following extraction methods may be implemented, or extraction methods other than the following extraction methods may be implemented.

[0043] -Convert the template part into a numeric ID and extract the template part ID as input data.

[0044] The template part is converted into a numeric ID, and the parameter part's numeric value is extracted as labeled input data with the template part's ID attached.

[0045] The LEVEL, COMPONENT, and template parts are converted into numerical IDs, and labeled input data is extracted by attaching the LEVEL or COMPOMENT ID as a label to the template part ID.

[0046] -Convert the unstructured character strings into numeric data in character code and extract it as input data.

[0047] ·The numerical data given to TIME is appended to the input data given in the example above, and extracted as time series data.

[0048] The user of log analysis device 100 can freely set methods other than those described above for extracting data from the input log. Note that the data extracted here is desirably numeric data or numeric string data consisting of M columns, since it is to be input to function 220. In the following description, the extracted data will be described as numeric string data with N rows and M columns.

[0049] This numeric column data is X={x1,…,x N} Note that x n (n=1,…,N) is the nth row of numbers, and each column is x n =[x n,1 ,…,x n,M In addition, in the case of labeled data, it is preferable that the labels are also numeric data or numeric string data. In this paper, the labels of each row are assumed to consist of Q columns, and the label data is expressed as P={p1,…,p N}, pn=[p n,1 ,…,p n,Q ] (n=1,…,N).

[0050] Functions 220 in the model 200 include a weighting parameter function 221 , an output calculation function 222 , and a loss calculation function 223 .

[0051] The weighting parameter function 221 may be, for example, a linear function such as θX obtained by multiplying a matrix θ consisting of weighting parameters by numeric string data, or a nonlinear function constituting a neural network. During learning in the learning engine 160, the weighting parameters of this function 221 are learned.

[0052] The output from the weighting parameter function 221 is made to be numeric string data with N rows and R columns. Here, R is made to match the number of columns of the numeric string that serves as an index for comparison when calculating the loss (described later) in the loss calculation function 223. For example, when assuming a classification problem in which comparison is made with label data P, R=Q is set, and when assuming a regression problem in which an input numeric string is restored as in an autoencoder, R=M is ​​set accordingly in order to compare with the numeric string data X.

[0053] The output calculation function 222 calculates the calculation result of the weight parameterized function 221 into a shape suitable for output from the model 200. The output calculation function 221 uses an identity function that is often used when reducing to a regression problem, or a softmax function that is often used when reducing to a classification problem. The inference engine 170 in the inference phase outputs the calculation result of the output calculation function 222 from the model 200. In this embodiment, the calculation result of the output calculation function 222 is each row R columns, and Y={y1, ..., y N}, y n =[y n,1 ,…,y n,R ] (n=1,…,N).

[0054] In the loss calculation function 223, a function such as RMSE or cross entropy error is used to compare the calculation result of the output calculation function 222 with a numerical sequence serving as an index for each row, and the sum of the comparison results is calculated for each row. In this embodiment, this numerical value is called loss, and the loss of n rows (n=1,...,N) is expressed as l nThe numerical sequence serving as an index may be, for example, the label data P or the numerical sequence data X as described above. In learning by the learning engine 160, the average of the losses of each row is calculated as the loss L, and the weight parameters of the weight parameterized function 221 are learned so as to minimize the loss. In the inference engine 170 in the verification phase, the average of the losses calculated from the trained model is output from the model as the loss L.

[0055] (Example of operation of the log analysis method decision engine 150) An example of the operation of the log analysis method determination engine 150 in the verification phase will be described with reference to FIG. train ) loss from L n This is expressed as:

[0056] The log analysis method decision engine 150 first sorts the losses in ascending order. After that, the sorted losses are sorted in ascending order. T_1 ,L T_2 In the text of this specification, the subscripts of subscripts are expressed as, for example, L T_1 It is expressed using "_" as follows.

[0057] Among the sorted losses, L T_m (m=1,…,p test ) is used in the inference phase, the log analysis method decision engine 150 m (m=1,…,p test )p test The number of types is output as the inference number. test In advance, train You can specify any integer below L T_n ≦L th (L th is given by determining the maximum n that satisfies the threshold. test is assumed to be a number greater than 1, but may be 1.

[0058] (Example of function f(X)) Next, the function f(X) provided in the integration engine 180 in the inference phase will be described. The input and output of the function f(X) are shown in FIG. test Types of output

[0059]

number

[0060]

number

[0061]

number

[0062]

number

[0063] (Hardware configuration example) The log analysis device 100 can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0064] That is, the log analysis device 100 can be realized by executing a program corresponding to the processing performed by the log analysis device 100 using hardware resources such as a CPU and memory built into a computer. The program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The program can also be provided via a network such as the Internet or email.

[0065] Fig. 10 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 10 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected via a bus BS.

[0066] The program for realizing the processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0067] The memory device 1003 reads out and stores the program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes functions related to the log analysis device 100 in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) or the like according to a program. The input device 1007 is composed of a keyboard and mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the results of calculations.

[0068] (Effects of the embodiment) As described above, in this embodiment, parsing methods using multiple log parsers are evaluated when creating a log mining model, and a parsing method with minimal loss is selected. This enables parsing tailored to the log. In addition, by integrating the results of passing through multiple log parsers and models to obtain the final output, it is expected that the accuracy of the inference result will be improved.

[0069] (Additional Note) The following supplementary items are further disclosed regarding the above-described embodiment. (Additional note 1) Memory, at least one processor coupled to the memory; Including, The processor, Parsing the logs is performed by each of the multiple log parsers. Calculating a loss from an input log output from each of the plurality of log parsers by each of the plurality of models; A pair of a log parser and a model to be used in an inference phase is determined from among the plurality of log parsers and the plurality of models based on a loss output from each of the plurality of models. Log analysis device. (Additional note 2) The processor determines a predetermined number of pairs of log parsers and models that are ranked in order of smallest loss as pairs of log parsers and models to be used in an inference phase. Item 1. A log analysis device according to claim 1. (Additional note 3) The processor outputs an inference result obtained by integrating the predetermined number of inference results obtained by using the predetermined number of sets of log parsers and models. 3. A log analysis device according to claim 2. (Additional note 4) The processor outputs an average of the predetermined number of inference results or a weighted average of the predetermined number of inference results as the integrated inference result. 4. A log analysis device according to claim 3. (Additional note 5) 1. A log analysis method executed by a computer for analyzing a log obtained from software, comprising: each log parser in the plurality of log parsers performing parsing of the log; Each model in the plurality of models calculates a loss from an input log output from each log parser in the plurality of log parsers; determining a pair of a log parser and a model to be used in an inference phase from among the plurality of log parsers and the plurality of models based on a loss output from each model in the plurality of models; A log analysis method comprising: (Additional note 6) A non-transitory storage medium storing a program executable by a computer to perform a log analysis process, The log analysis process includes: Parsing the logs is performed by each of the multiple log parsers. Calculating a loss from an input log output from each of the plurality of log parsers by each of the plurality of models; A pair of a log parser and a model to be used in an inference phase is determined from among the plurality of log parsers and the plurality of models based on a loss output from each of the plurality of models. Non-transitory storage medium.

[0070] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0071] 100 Log analysis device 110 Log Database 120 Log Parser Database 130 trained model database 140 Log Parsing Engine 150 Log analysis method decision engine 160 Learning Engine 170 Inference Engine 180 Integrated Engine 1000 Drive Device 1001 Recording media 1002 Auxiliary storage device 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device 1008 Output device

Claims

1. A log analysis device that analyzes a log obtained from software, a log parsing unit having a plurality of log parsers; an inference unit having a plurality of models for calculating a loss from an input log output from each of the plurality of log parsers; a log analysis method determination unit that determines a pair of a log parser and a model to be used in an inference phase from among the plurality of log parsers and the plurality of models based on a loss output from each of the plurality of models; A log analysis device comprising:

2. The log analysis method determination unit determines a predetermined number of pairs of log parsers and models that are ranked in order of smallest loss as pairs of log parsers and models to be used in an inference phase. The log analysis device according to claim 1 .

3. an integration unit that receives as input the predetermined number of inference results obtained by the log parsing unit and the inference unit using the predetermined number of sets of log parsers and models, and outputs an inference result obtained by integrating the predetermined number of inference results; The log analysis device according to claim 2 , further comprising:

4. The integration unit outputs an average of the predetermined number of inference results or a weighted average of the predetermined number of inference results as the integrated inference result. The log analysis device according to claim 3 .

5. 1. A log analysis method executed by a computer for analyzing a log obtained from software, comprising: each log parser in the plurality of log parsers performing parsing of the log; Each model in the plurality of models calculates a loss from an input log output from each log parser in the plurality of log parsers; determining a pair of a log parser and a model to be used in an inference phase from among the plurality of log parsers and the plurality of models based on a loss output from each model in the plurality of models; A log analysis method comprising:

6. A program for causing a computer to function as each unit in the log analysis device according to any one of claims 1 to 4.

Images