Log data analysis device, log data analysis method, and program

Abstract

A log data analysis device of one embodiment of the present invention is equipped with: an acquisition unit that acquires a text log generated by a system to be analyzed; a feature amount calculation unit; and an abnormal log detection unit. The feature amount calculation unit calculates a feature amount of each text log from a sequence of acquired text logs. The abnormal log detection unit uses a first model, which was obtained by training a first neural network using a single text log as a target, to calculate an approximate value of an abnormality score from the text log sequence feature amounts of a portion of the text log sequence. If the approximate value is not greater than a prescribed value, the abnormal log detection unit classifies the text log sequence as either normal or abnormal on the basis of the approximate value. If the approximate value is greater than the prescribed value, the abnormal log detection unit uses a second model, which was obtained by training a second neural network using a log sequence including a plurality of text logs as a target, to calculate an abnormality score correction value from all the acquired text log sequence feature amounts, and classifies the text log sequence as either normal or abnormal on the basis of the sum of the approximate value of the abnormality score and the corrected value of the abnormality score.

Description

[Technical field]

[0001] The present invention relates to a log data analysis device, a log data analysis method, and a program. [Background technology]

[0002] When a malfunction occurs in a computer system, it affects all areas of modern society. To minimize the impact, there is a need to quickly detect abnormalities in service maintenance work. One method for detecting system abnormalities is to analyze the large amount of log data output by the devices and software that make up the system. Since this method is also used to analyze the cause after a failure occurs, there is a need to increase the speed of analysis and obtain results quickly.

[0003] Log data is often output in the form of text. Therefore, the application of text analysis technology using a deep learning model is being considered. For example, Non-Patent Documents 1 and 2 disclose anomaly detection using LSTM (Long short-term memory), which is one of the deep learning models. These documents mainly propose matters related to the design of features to be input to LSTM. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Du, Min et al. “DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning.” Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (2017). [Non-Patent Document 2] Meng, Weibin et al. “LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs.” IJCAI (2019). [Non-Patent Document 3] Adam J. Oliner, Jon Stearley. What Supercomputers Say: A Study of Five System Logs, in Proc. of IEEE / IFIP International Conference on Dependable Systems and Networks (DSN), 2007. Summary of the Invention [Problem to be solved by the invention]

[0005] It is difficult to scale deep learning model-based techniques to detect log anomalies on a large-scale text log sequence. For this reason, depending on the volume of text logs, it can take a long time to obtain analysis results. Accuracy is sacrificed when trying to reduce the processing time, and text log data is distributed biased toward either positive or negative examples, so the greater the bias, the lower the accuracy. The present invention provides a technique capable of reducing the processing time while suppressing the deterioration of accuracy. [Means for solving the problem]

[0006] In order to solve the above problem, a log data analysis device according to an embodiment of the present invention includes a storage unit that stores a first model obtained by training a first neural network for a single text log and a second model obtained by training a second neural network for a log sequence including a plurality of text logs, an acquisition unit, a feature calculation unit, and an abnormality log detection unit. The acquisition unit acquires a text log sequence generated by a system to be analyzed. The feature calculation unit calculates a feature of each text log from the acquired text log sequence. The abnormality log detection unit classifies the acquired text log sequence into either normal or abnormal based on the feature. The abnormality log detection unit calculates an approximate value of an anomaly score from the feature values ​​of a part of the acquired text log sequence using the first model. If the approximate value of the anomaly score is equal to or less than a specified value, the abnormality log detection unit classifies the text log sequence into either normal or abnormal based on the approximate value of the anomaly score. If the estimated anomaly score is greater than a specified value, the anomaly log detection unit calculates a corrected anomaly score value using a second model from the features of all the acquired text log sequences, and classifies the text log sequence as either normal or abnormal based on the sum of the estimated anomaly score value and the corrected anomaly score value. Effect of the Invention

[0007] According to one aspect of the present invention, it is possible to provide a technique capable of reducing processing time while suppressing deterioration in accuracy. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a system including a log data analysis device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a flowchart showing an example of a processing procedure of the log data analysis device 10 shown in FIG. [Diagram 3] FIG. 3 is a diagram showing a 2×2 cross-tabulation table for a classification problem where positive examples are abnormal and negative examples are normal. [Figure 4]FIG. 4 is a diagram showing an example of a result of the accuracy evaluation. [Diagram 5] FIG. 5 is a diagram showing an example of the results of efficiency evaluation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. (composition) Fig. 1 is a diagram showing an example of a system including a log data analysis device according to an embodiment of the present invention. In Fig. 1, a log data analysis device 10 includes a processor 11, a database 12, a storage 13, a memory 14, and an interface unit 15. In other words, the log data analysis device 10 is a computer, and is realized as, for example, a personal computer, a server computer, or the like.

[0010] The interface unit 15 is connected to the network 100 and, for example, accesses the maintenance target system 2 to transmit and receive various data. In addition, the interface unit 15 outputs the analysis result 3 generated by the log data analysis device 10 in response to a request from, for example, an operator.

[0011] The processor 11 is an arithmetic unit such as a Central Processing Unit (CPU) or a Micro Processing Unit (MPU), and realizes its functions by a program 14 a loaded into a memory 14 . The database 12 stores and accumulates the text log data 12a acquired from the maintenance target system 2.

[0012] The storage 13 is a non-volatile storage medium (block device) such as a hard disk drive (HDD) or a solid state drive (SSD). The database 12 stores an FFN model 13a and an LSTM model 13b in addition to basic programs such as an operating system (OS) and device drivers, and programs for implementing the functions of the log data analysis device 10.

[0013] The FFN model 13a is a model obtained by training an FFN (Feed Forward Network) targeting a single text log, and is used for classification by the FFN. The LSTM model 13b is a model obtained by training an LSTM targeting a log sequence including multiple text logs, and is used for classification by the LSTM. The memory 14 stores a program 14a and an analysis result 14b generated by the processor 11.

[0014] The processor 11 includes a log data acquisition unit 11a, a feature calculation unit 11b, an abnormality log detection unit 11c, and an output unit 11d as functional blocks (program modules) according to the embodiment. These functional blocks are processing functions that are realized by the processor 11 executing instructions included in the program 14a. That is, the log data analysis device 10 of the present invention can also be realized by a computer and a program. The program can be recorded on a recording medium such as an optical medium and distributed. Alternatively, the program can be provided through a network.

[0015] The log data acquisition unit 11a acquires text log sequences generated by the maintenance target system 2, which is the analysis target, via the network 100 and the interface unit 15, and accumulates them in the database 12 as text log data 12a.

[0016] The feature amount calculation unit 11b calculates the feature amount of each text log from the text log sequence of the text log data 12a. The abnormal log detection unit 11c classifies the text log sequence of the text log data 12a into either normal or abnormal based on the above-mentioned feature amount.

[0017] In addition, the abnormality log detection unit 11c calculates an approximate value of an anomaly score from the features of a part of the text log sequence of the text log data 12a using the FFN model 13a. If the approximate value of the anomaly score is equal to or smaller than a specified value, the abnormality log detection unit 11c classifies the text log sequence as either normal or abnormal based on the approximate value of the anomaly score. On the other hand, if the estimated anomaly score is greater than the specified value, the anomaly log detection unit 11c calculates a corrected anomaly score value from the features of all the acquired text log sequences using the LSTM model 13b, and classifies the text log sequence as either normal or abnormal based on the sum of the estimated anomaly score value and the corrected anomaly score value.

[0018] The output unit 11d outputs the analysis result calculated by the processor 11 (data indicating whether the text log is normal or abnormal).

[0019] (action) Fig. 2 is a flowchart showing an example of a processing procedure of the log data analysis device 10 shown in Fig. 1. In Fig. 2, the log data analysis device 10 reads only the immediately preceding log (for example, the log in the i-th line) from the text log data 12a of the database 12 and calculates a feature quantity t i Next, the log data analysis device 10 calculates the feature quantity t i The FFN model 13a is applied to the first feature value S coarse is calculated (step S3).

[0020] Next, the log data analysis device 10 calculates a predetermined threshold (pruning threshold α: hyperparameter) α and S coarse Compare the magnitude of α>S coarse Otherwise, α≦S coarse If so, the final value of the anomaly score S all Scoarse Substitute (S all =S coarse ) (step S5). Next, the log data analysis device 10 all A comparison is made between S and 0 (step S6). all If it is >0 (Yes), it is concluded that the log data is abnormal, and the abnormality is output as the analysis result (step S7). all If ≦0 (No), the log data analysis device 10 determines that the log data is normal, and outputs "normal" as the analysis result (step S8).

[0021] On the other hand, in step S4, α>S coarse If so (Yes), the log data analysis device 10 reads out the logs of the i-1th line, . . . , i-w+1th line from the text log data 12a of the database 12 in addition to the i-th line log that has already been read out of the log sequence (step S9), and calculates the feature quantity t i-1 ,…,t i-w+1 Calculate (step S9).

[0022] Next, the log data analysis device 10 calculates the feature quantity t i , and the feature quantity t obtained in step S9 i-1 ,…,t i-w+1 (Step S10), and obtain a detailed value S fine is calculated (step S11).

[0023] Next, the log data analysis device 10 calculates the final value S of the anomaly score. all S coarse +S fine Substitute (S all =S coarse +S fine ) (step S12), the procedure proceeds to step S6, all A normal / abnormal judgment is performed based on the value of

[0024] A pseudo-code algorithm for implementing the procedure of FIG. 2 is given below.

[0025]

number

[0026] In the embodiment, for example, a text log of a supercomputer system (Non-Patent Document 3) can be used as the dataset. A comparison is made between a case where an analysis is performed on this dataset using only an FFN model and a case where an analysis is performed using only an LSTM model without pruning. Evaluation methods include, for example, a method of evaluating the F1 value for positive examples (accuracy evaluation) and a method of evaluating the inference time (efficiency evaluation). Here, the F1 value is one of the methods for evaluating the accuracy of the prediction result, and can be expressed by the following formula.

[0027]

number

[0028] Figure 3 shows a 2x2 cross-tabulation table for a classification problem where positive examples are abnormal and negative examples are normal. TP, FN, and FP represent the number of data when the predicted results and true results are classified as shown in the table in Figure 3.

[0029] As in (Non-Patent Document 2), the feature can be the average of word embeddings (embeddings are initialized randomly) or a count vector (the number of occurrences of each template ID in the window).

[0030] (effect) FIG. 4 is a diagram showing an example of the result of the accuracy evaluation. When the F1 value is calculated using the above data set, it is shown that the F1 value when only the LSTM model is used is 0.536, the F1 value when only the FFN model is used is 0.626, and the F1 value in this embodiment is 0.779. In other words, it is shown that the accuracy is improved compared to the existing method. According to the configuration of the embodiment, most normal data that becomes noise in anomaly detection can be filtered in advance using only the FFN result. This makes it possible to reduce the ratio of normal data included in the input data of LSTM and suppress the deterioration of the accuracy of LSTM, and as a result, the accuracy of classification is improved.

[0031] FIG. 5 is a diagram showing an example of the results of efficiency evaluation. FIG. 5 shows that the efficiency is highest when only FFN is used (dash-dotted line), lowest when only LSTM is used (dotted line), and intermediate for the technology disclosed in the embodiment (solid line). Compared to the case of only LSTM, the execution time was about 3.1 times faster (batch size 1). Here, the batch size is an amount indicating the size of data input to the model at one time. It can be seen that the efficiency in the embodiment deteriorates as the batch size increases. This is thought to be mainly due to the fact that, when the batch size is increased, the number of data that cannot be pruned in the batch increases, and the processing of the data that cannot be pruned limits the processing speed of the entire batch.

[0032] As described above, in the embodiment, an anomaly detection model using a coarse-dense structure is constructed by focusing on the fact that some abnormal text log sequences can be determined from a small number of text logs, while others can be determined from multiple text logs. That is, when detecting simple anomalies, an efficient model is used, and when detecting difficult anomalies, a highly accurate model is used. This method can reduce the processing time caused by the deep learning sequence model. In addition, accuracy can be improved by filtering normal data, which accounts for the majority of anomaly detection, in advance to reduce the proportion of normal data.

[0033] In existing techniques, the computational cost of sequence models such as LSTM used to encode log sequences within a window is high and increases in proportion to the sequence length. In addition, when there is a bias in the ratio of positive and negative examples in text log data, conventional methods such as LSTM have the characteristic that the accuracy decreases as the bias increases.

[0034] In contrast, in the embodiment, a model (FFN model) that calculates an anomaly score for each log is used to calculate an anomaly score from the feature amount of the immediately preceding log through an efficient model, and only for those logs whose anomaly scores exceed a threshold, a sequence model (LSTM model) is calculated. That is, the inference time caused by the deep learning sequence model can be reduced by detecting an anomaly using a coarse-dense structure. In addition, the accuracy of anomaly detection is improved by allocating detection tasks with different levels of difficulty to different models. That is, by using a coarse-dense structure to partially bypass the processing of the sequence model, the speed of text log anomaly detection can be increased, and the accuracy of text log anomaly detection can be improved.

[0035] From these factors, according to the embodiment, it is possible to provide a technique that can reduce the processing time while suppressing a decrease in accuracy. It should be noted that the present invention is not limited to the above-described embodiment as it is, and that the components can be modified and embodied in the implementation stage without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0036] 2. Systems subject to maintenance 3…Analysis results 10...Log data analysis device 11…Processor 11a…Log data acquisition section 11b…Feature amount calculation unit 11c…Abnormal log detection section 11d…Output section 12. Database 12a…Text log data 13. Storage 13a…FFN model 13b…LSTM model 14…Memory 14a…Program 14b…Analysis results 15...Interface section 100…Network.

Claims

1. a storage unit that stores a first model obtained by training a first neural network targeting a single text log and a second model obtained by training a second neural network targeting a log sequence including a plurality of text logs; An acquisition unit that acquires a text log generated by a system to be analyzed; a feature calculation unit that calculates a feature of each text log from the acquired text log sequence; an anomaly log detection unit that classifies the acquired text log sequence into either normal or abnormal based on the feature amount; The abnormality log detection unit Calculating an approximate anomaly score from features of a portion of the acquired text log sequences using the first model; If the estimated anomaly score is equal to or smaller than a specified value, classifying the text log sequence as either normal or abnormal based on the estimated anomaly score; if the estimated anomaly score is greater than a specified value, a corrected anomaly score is calculated from feature quantities of all of the acquired text log sequences using the second model, and the text log sequence is classified as either normal or abnormal based on the sum of the estimated anomaly score and the corrected anomaly score.

2. The log data analysis device according to claim 1 , wherein the first neural network is a Feed Forward Network (FFN).

3. 3. The log data analysis device according to claim 1, wherein the second neural network is a Long Short Term Memory (LSTM).

4. A method for analyzing log data in a log data analysis device having a processor, comprising: The processor stores in a storage unit a first model obtained by training a first neural network targeting a single text log and a second model obtained by training a second neural network targeting a log sequence including a plurality of text logs; The processor acquiring a text log generated by a system to be analyzed; The processor calculates a feature amount of each text log from the acquired sequence of text logs; The processor calculates an approximate anomaly score from feature quantities of a portion of the acquired text log sequences using the first model; and if the estimated anomaly score is equal to or smaller than a predetermined value, classifying the text log sequence as either normal or anomalous based on the estimated anomaly score; and if the estimated anomaly score is greater than a specified value, the processor calculates a corrected anomaly score from feature quantities of all of the acquired text log sequences using the second model, and classifies the text log sequence as either normal or abnormal based on the sum of the estimated anomaly score and the corrected anomaly score.

5. A program causing a processor to function as each of the units of the log data analysis device according to any one of claims 1 to 3.

Images