Log parameter anomaly detection method based on word embedding

Abstract

The invention discloses a log parameter anomaly detection method based on word embedding, and the method comprises the following steps: 1, analyzing all parameters in a log, and independently dividingdiscrete parameters in all the parameters; 2, converting the discrete parameters into continuous parameter word vectors; 3, training a parameter word vector by using a long-term and short-term memoryneural network model, and predicting the parameter word vector at a subsequent target moment by using the trained parameter word vector; 4, determining the association degree of the prediction parameter word vector and the target parameter word vector by using cosine similarity, calculating a loss value through the association degree, feeding back the loss value to the network, and updating and optimizing the model until convergence; and 5, acquiring a log to perform parameter anomaly detection, calculating cosine similarity between the prediction parameter and the target parameter, and if the cosine similarity is lower than a threshold value, determining that the log parameter is abnormal. The detection bottleneck caused by parameter dynamics and difference can be effectively solved, andthe overall accuracy of log detection is improved.

Description

technical field [0001] The invention belongs to the technical field of the Internet, and relates to a log parameter abnormality detection method based on word embedding. Background technique [0002] Logs are an important part of various computer systems. System logs can reflect the execution process and status of computer programs, and can also identify errors and exceptions that occur during the execution of systems and programs. Due to the importance of logs, log anomaly detection has always been a hot topic of research. Most of the current anomaly detection methods classify logs into fixed-format log templates. Through the timing information of log templates, data statistics, machine learning, Methods such as deep learning complete log anomaly detection. However, in various types of computer logs, in addition to fixed format and category log templates, there are also a large number of dynamic log parameters. Abnormalities in the program execution sequence can be identi...

AI Technical Summary

Problems solved by technology

This method is only for continuous type parameter variables, and the model is redundant and does not have the ability to be deployed in practice

Images