Log sequence anomaly detection method based on hierarchical semantics

By introducing a hierarchical semantic structure and LayerLog framework in log sequence exception detection, the semantic features of log data are extracted, and the problems of inappropriate feature selection and unconsidered semantics in the existing methods are solved, achieving higher detection accuracy and resilience Great.

CN119004327APending Publication Date: 2024-11-22CHINA TOBACCO YUNNAN IND
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411090488.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing log-based anomaly detection methods have problems such as inappropriate feature selection, inefficient and poor adaptability, and have failed to fully consider the semantics of log data, resulting in high missing detection rates and detection accuracy being affected.

Method used

A log sequence anomaly detection method based on hierarchical semantics is proposed. Through the 'word-log-log-log sequence' hierarchy and LayerLog framework, two-way long and short-term memory using word embedding, part-of-speech annotation, word frequency-inverse file frequency algorithm and attention mechanism Network model extracts semantic features from log data and performs abnormal detection.

Benefits of technology

It achieves higher detection accuracy and robustness, can effectively extract semantic information from the original log, reduce dependence on the log parser, improve the accuracy and recall of exception detection, and perform better than other methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119004327A_ABST
    Figure CN119004327A_ABST
Patent Text Reader

Abstract

The invention provides a log sequence anomaly detection framework based on hierarchical semantics. The log sequence anomaly detection framework is a log sequence anomaly detection framework based on semantics of different modules of log data. Under the condition that a log parser is not used, the framework can effectively extract semantic features from each module and is a framework considering word, log and log sequence semantics. In addition, the framework can detect execution sequence anomalies, operation anomalies and incomplete anomalies of the log sequence in an end-to-end manner at the same time. According to the invention, sufficient experiments are carried out on two standard common data sets, and experiment results show that the framework has high detection accuracy and low omission ratio, and confirm the effectiveness of the framework.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Technical Language

[0002] The invention relates to a log sequence anomaly detection method based on hierarchical semantics, and belongs to the technical field of natural language processing. Background Art

[0003] With the rapid development of artificial intelligence and cloud computing, software systems are becoming more and more complex and large, which makes it more difficult to maintain them. At the same time, high availability and reliability are essential for large-scale software systems. Since these systems provide various services to a large number of users, small problems in the system may lead to user dissatisfaction or even cause significant financial losses. Therefore, anomaly detection is very important for the quality assurance of complex software systems. Existing studies have shown that logs have become important data for maintaining large-scale software (such as online service systems), which are generated during system operation to record events and status. By checking the recorded logs, developers can check the system status, detect anomalies, and diagnose the root cause. However, due to the large scale of the system and the huge amount of log data, it is very difficult to manually check the logs. Therefore, a large amount of work in the literature focuses on automated log analysis.

[0004] In automated log analysis, log-based anomaly detection is one of the most important aspects. Almost all existing log-based anomaly detection methods contain the same detection steps: 1) extracting log events (i.e., templates of log messages) and log sequences (i.e., a series of recorded log events) from log messages. 2) using machine learning or data mining techniques to build an anomaly detection model based on log sequences. These machine learning or data mining techniques include supervised methods and semi-supervised methods, which have been proven to be effective in corresponding studies.

[0005] In the field of log-based anomaly detection, He et al. evaluated the performance of six machine learning (ML) algorithms. These machine learning-based methods have some limitations, such as inappropriate feature selection, low efficiency, and poor adaptability. To address these problems, researchers have begun to adopt deep learning techniques and have achieved good results. Existing deep learning-based log anomaly detection methods have shown good results on common datasets, outperforming traditional machine learning-based methods. However, methods based on log count vectors and log key sequences do not take into account the semantics of log data, resulting in a high missing detection rate. Existing methods based on log semantics cannot fully consider the semantic information between words, logs, and log sequences. In addition, current methods require the use of log parsers in preprocessing, which may have a negative impact on detection accuracy and cause semantic loss. Summary of the invention

[0006] The present invention provides a "word-log-log sequence" hierarchy, i.e., a three-layer structure of log data, and proposes LayerLog, which can effectively extract semantic features from each layer. LayerLog is the first framework that considers the semantics of words, logs, and log sequences. The present invention evaluates the effectiveness of LayerLog by conducting experiments on two commonly used public datasets.

[0007] The technical solution of the present invention is: a log sequence anomaly detection method based on hierarchical semantics, the specific steps of the method are as follows:

[0008] A log sequence anomaly detection framework based on hierarchical semantics, characterized in that the method comprises the following steps:

[0009] Step 1: In the word module, the framework combines word embedding, part-of-speech tagging, and word frequency-inverse document frequency algorithm to generate the semantic vector corresponding to each word;

[0010] Step 2: Represent the words in the log as vectors and pass them to the log module to form a word vector sequence. Generate the semantic vector corresponding to each log through the bidirectional long short-term memory network model based on the attention mechanism.

[0011] Step 3: The log vector is transferred to the log sequence module to form a log vector sequence, and the bidirectional long short-term memory network model based on the attention mechanism is used again to generate the log sequence vector;

[0012] Step 4: The framework generates anomaly detection results through the anomaly detection module.

[0013] As a further solution of the present invention, step 1 includes the following specific steps:

[0014] Step 1.1, log data preprocessing, including two parts: word segmentation and data cleaning. After the log file is segmented, meaningless symbols (such as punctuation marks and separators) are deleted;

[0015] Step 1.2, word embedding: Use Google's open source tool Word2vec for word vector calculation to map words into word vectors;

[0016] Step 1.3, word importance calculation, this framework combines part-of-speech tagging with word frequency-inverse document frequency algorithm to calculate the importance of words;

[0017] Step 1.4: Combine word embedding with importance to generate word vectors Where i represents the i-th log sequence, j represents the j-th log in the i-th log sequence, and k represents the k-th word in the j-th log.

[0018] For the part-of-speech tagging in step 1.3, the present invention uses the natural language processing library NLTK (Natural Language Toolkit) designed by the University of Pennsylvania to mark the part-of-speech of the words. The word frequency-inverse file frequency algorithm method used in the present invention is based on the following assumption: if a word appears frequently in one log sequence and rarely appears in other log sequences, the word is more discriminative, so its importance is higher.

[0019] As a further solution of the present invention, step 2 includes the following specific steps:

[0020] Step 2.1: Log layer transforms word vector sequence As the input of Bi-LSTM, the hidden layer state value in LSTM at time t is recorded as The reverse LSTM is denoted as and The calculation and simplification of and Finally, they are recorded as

[0021] Step 2.2: Use single-layer MLP with word information Get Hidden Expression

[0022] Step 2.3, by calculating the context vector u in the word layer ij The similarity of words is measured The importance of is obtained through Softmax

[0023] Step 2.4: The final semantic representation vector is recorded as

[0024] As a further solution of the present invention, step 3 includes the following specific steps:

[0025] Step 3.1: Record the hidden layer state value in LSTM at time t as The reverse LSTM is denoted as and The calculation and simplification of and The final expression of these two vectors is

[0026] Step 3.2: Use single-layer MLP with word information Get Hidden Expression

[0027] Step 3.3, by calculating the context vector u in the word layer ij The similarity of words is measured The importance of is obtained through the SoftMax function

[0028] Step 3.4: Record the log representation vector as Through the above steps, the LogSeqVec of each log sequence is obtained as the output of the LogSeq layer, which is the semantic representation of the log sequence composed of logs.

[0029] As a further solution of the present invention, step 4 includes the following specific steps:

[0030] 1. Supervised Anomaly Detection Methods

[0031] In the supervised anomaly detection method, the anomaly detection of the log sequence is regarded as a binary classification problem (normal or abnormal), the probability distribution of the log sequence is calculated using the SoftMax function, and then the detection result corresponding to the maximum conditional probability is selected as the predicted output category.

[0032] The calculation process is as follows:

[0033] During the training phase, the cross entropy between the predicted output and the true label provided by the dataset is used as the loss and / or

[0034] 2. Semi-supervised anomaly detection methods

[0035] Only positive sample data is used to train the model, and a class of support vector machines is used to detect the novelty of high-dimensional data.

[0036] The convention follows the formula:

[0037] Where parameter o is the center of the hypersphere and r is the radius of the hypersphere. Then, minimize the volume V of the hypersphere (r) , to obtain the distribution space of normal data,

[0038] In the testing phase, the anomaly score is defined as follows: Score(X i )=‖X i -o‖ 2

[0039] By calculating the sample X i The distance from the center point o, determine X i Is it in the category and determine X i Is it abnormal?

[0040] The beneficial effects of the present invention are:

[0041] 1. This paper proposes a three-layer structure of log data, "word-log-log sequence hierarchy", and proposes a new log sequence anomaly detection framework LayerLog based on hierarchical semantics. LayerLog does not rely on any log parser in the preprocessing stage and can effectively extract semantic features from each layer.

[0042] 2. LayerLog has better robustness because we extract semantic information directly from raw logs instead of converting logs into log templates. We conducted extensive experiments on two commonly used public datasets, and the experimental results confirmed the effectiveness of LayerLog. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of the present invention; DETAILED DESCRIPTION

[0044] Example 1: Figure 1 As shown, a log sequence anomaly detection method based on hierarchical semantics, the specific steps of the method are as follows:

[0045] a1. Process the log file through the word module and generate a word vector at the end of the module, including:

[0046] 1) Find the log files in the software system and perform word segmentation and data cleaning preprocessing;

[0047] 2) Get word embedding representation through Word2vec;

[0048] 3) Combining part-of-speech tagging with the word frequency-inverse document frequency algorithm to calculate the importance of words;

[0049] 4) Finally, the word vector is generated in the last part of the word module

[0050] a2. In the log module, we aim to generate log vectors, but we cannot simply add word vectors to obtain log vectors, because this operation ignores the order and syntactic relationship between words. Therefore, we use the Bi-LSTM model with attention mechanism to extract features from the WordVec sequence in the previous step, capture the contextual information between words, and automatically adjust the importance of words, including:

[0051] 1) The log layer converts the word vector sequence As the input of Bi-LSTM, the hidden layer state value in LSTM at time t is recorded as The reverse LSTM is denoted as and The calculation and simplification of and Finally, they are recorded as

[0052] 2) Use single-layer MLP with word information Get Hidden Expression

[0053] 3) By calculating the context vector u in the word layer ij The similarity of words is measured The importance of is obtained through Softmax

[0054] 4) The final semantic representation vector is recorded as

[0055] a3. In the log sequence module, the implementation method is similar to that of the log module, and the purpose is to generate a semantic vector representation of the log sequence, including:

[0056] 1) The hidden layer state value in LSTM at time t is recorded as The reverse LSTM is denoted as and The calculation and simplification of and The final expression of these two vectors is

[0057] 2) Use single-layer MLP with word information Get Hidden Expression

[0058] 3) By calculating the context vector u in the word layer ij The similarity of words is measured The importance of is obtained through the SoftMax function

[0059] 4) The log representation vector is recorded as Through the above steps, the LogSeqVec of each log sequence is obtained as the output of the LogSeq layer, which is the semantic representation of the log sequence composed of logs;

[0060] a4. Treat the anomaly detection of log sequences as a binary classification problem (normal or abnormal), use the SoftMax function to calculate the probability distribution of the log sequence, and then select the detection result corresponding to the maximum conditional probability as the predicted output category. In the training stage, the present invention uses the cross entropy between the predicted output and the true label provided by the data set as the loss.

[0061] The experiments of the present invention are conducted on the log file HDFS and BGL datasets from the public Hadoop distributed file system and the supercomputer system of the Lawrence Livermore National Laboratory in Livermore, California. In order to measure the effectiveness of the log sequence anomaly detection method based on hierarchical semantics in log sequence anomaly detection, the present invention uses precision, recall, specificity and F1 value evaluation indicators for evaluation.

[0062] Precision (P): The percentage of log sequences that are correctly identified as abnormal among all log sequences identified as abnormal by the model. The calculation formula is as follows:

[0063]

[0064] Precision (P): The percentage of log sequences that are correctly identified as abnormal among all log sequences identified as abnormal by the model. The calculation formula is as follows:

[0065]

[0066] Specificity (S): The percentage of log sequences that are correctly identified as normal among all correct log sequences. The calculation formula is as follows:

[0067]

[0068] F1 value (F1): The harmonic mean of precision and recall. Its calculation formula is as follows:

[0069]

[0070] In this section, the performance of the hierarchical semantics-based log sequence anomaly detection framework is verified on HDFS and BGL datasets. The framework is compared with LR (a statistical method for binary classification problems), SVM (a machine learning algorithm for classification and regression analysis), PCA (a statistical method for dimensionality reduction), IM (a machine learning method for anomaly detection), LogClustering, LSTM-AE (a method based on log event counting), DeepLog (a method based on log key sequences), and LogAnomaly (a method based on log templates). Table 1 shows the comparison results of HDFS and BGL datasets.

[0071] From the experimental results, we can see that the hierarchical semantics-based log sequence anomaly detection framework performs better than other methods. The framework achieves an F1 score of 0.988 on the HDFS dataset and an F1 score of 0.985 on the BGL dataset. Methods based on log event counting cannot achieve both high precision and high recall. For example, LR and SVM achieve a precision of even 1 on the HDFS and BGL datasets, while their recall is relatively low, resulting in a low F1 score. At the same time, the robustness of methods based on log event counting is poor because the evaluation criteria of the two datasets are quite different. For example, SVM can achieve an F1 score of 0.926 on the HDFS dataset, but only 0.71 on the BGL set. Methods based on log key sequences generally perform better than those based on log event counting. The F1 scores of DeepLog on both datasets exceed 0.9, which emphasizes the importance of considering the execution order of log sequences for log sequence anomaly detection and verifies better robustness performance. The log template-based method achieves almost perfect experimental results, which shows that understanding the execution of log sequences from a semantic perspective can improve the ability to detect three different types of anomalies in log sequences. The performance of the log sequence anomaly detection framework based on hierarchical semantics is better than that of LogAnomaly, and its F1 score is nearly 3 percentage points higher than that of LogAnomaly. This confirms that the log sequence anomaly detection framework based on hierarchical semantics can extract richer semantic features from words, logs, and log sequences instead of just from log templates, so that more accurate hierarchical semantics can be learned and the best accuracy can be achieved.

[0072] Table 1 Results on HDFS and BGL datasets

[0073]

[0074] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A log sequence anomaly detection method based on hierarchical semantics, characterized by: The method comprises the following steps: Step 1: In the word module, the framework combines word embedding, part-of-speech tagging, and word frequency-inverse document frequency algorithm to generate the semantic vector corresponding to each word; Step 2: Represent the words in the log as vectors and pass them to the log module to form a word vector sequence. Generate the semantic vector corresponding to each log through the bidirectional long short-term memory network model based on the attention mechanism. Step 3: The log vector is transferred to the log sequence module to form a log vector sequence, and the bidirectional long short-term memory network model based on the attention mechanism is used again to generate the log sequence vector; Step 4: The framework generates anomaly detection results through the anomaly detection module.

2. The log sequence anomaly detection method based on hierarchical semantics according to claim 1, characterized in that: The step 1 comprises the following specific steps: Step 1.1: Log data preprocessing, which includes two parts: word segmentation and data cleaning. After the log file is segmented, meaningless symbols are deleted; Step 1.2, word embedding: Use Google's open source tool Word2vec for word vector calculation to map words into word vectors; Step 1.3, word importance calculation, this framework combines part-of-speech tagging with word frequency-inverse document frequency algorithm to calculate the importance of words; Step 1.4: Combine word embedding with importance to generate word vectors Where i represents the i-th log sequence, j represents the j-th log in the i-th log sequence, and k represents the k-th word in the j-th log.

3. The log sequence anomaly detection method based on hierarchical semantics according to claim 1, characterized in that: The step 2 comprises the following specific steps: Step 2.1: Log layer transforms word vector sequence As the input of Bi-LSTM, the hidden layer state value in LSTM at time t is recorded as The reverse LSTM is denoted as and The calculation and simplification of and Finally, they are recorded as Step 2.2: Use single-layer MLP with word information Get hidden expression Step 2.3, by calculating the context vector u in the word layer ij The similarity of words is measured The importance of is obtained through Softmax Step 2.4: The final semantic representation vector is recorded as 4. The log sequence anomaly detection method based on hierarchical semantics according to claim 1, characterized in that: The step 3 comprises the following specific steps: Step 3.1: Record the hidden layer state value in LSTM at time t as The reverse LSTM is denoted as and The calculation and simplification of and The final expression of these two vectors is Step 3.2: Use single-layer MLP with word information Get hidden expression Step 3.3, by calculating the context vector u in the word layer ij The similarity of words is measured The importance of is obtained through the SoftMax function Step 3.4: Record the log representation vector as Through the above steps, the LogSeqVec of each log sequence is obtained as the output of the LogSeq layer, which is the semantic representation of the log sequence composed of logs.

5. The log sequence anomaly detection method based on hierarchical semantics according to claim 1, characterized in that: The step 4 comprises the following specific steps:

1. Supervised Anomaly Detection Methods In the supervised anomaly detection method, the anomaly detection of log sequences is regarded as a binary classification problem of normal or abnormal. The probability distribution of the log sequence is calculated using the SoftMax function, and then the detection result corresponding to the maximum conditional probability is selected as the predicted output category. The calculation process is as follows: During the training phase, the cross entropy between the predicted output and the true label provided by the dataset is used as the loss and / or 2. Semi-supervised anomaly detection methods Only positive sample data is used to train the model, and a class of support vector machines is used to detect the novelty of high-dimensional data. The convention follows the formula: Where parameter o is the center of the hypersphere and r is the radius of the hypersphere. Then, minimize the volume V of the hypersphere (r) , to obtain the distribution space of normal data, In the testing phase, the anomaly score is defined as follows: Score(X i )=‖X i -o‖ 2 By calculating the sample X i The distance from the center point o, determine X i Is it in the category and determine X i Is it abnormal?

Citation Information

Patent Citations

  • Method and device for detecting log sequence abnormity and computer storage medium

    CN113407721A

  • Log anomaly detection method and system based on attention and Bi-LSTM

    CN118211592A