Method and device for detecting log sequence abnormity and computer storage medium

Abstract

The invention provides a method and a device for detecting log sequence abnormity and a computer storage medium. The method comprises the following steps: collecting original log sequence data from a data source; sequentially extracting semantic information of each layer according to a hierarchical structure of a log sequence in the log sequence data, and generating a semantic vector of a fixed dimension of each layer; and calculating probability distribution of the log sequence by utilizing a SoftMax function according to the semantic vector of the log sequence, and selecting a result corresponding to the maximum conditional probability as an output category. According to the invention, richer semantic features can be extracted from words, logs and log sequences, so that the device can learn more accurate hierarchical semantic expressions, a better detection effect is achieved, and the capability of detecting three log sequence anomalies is further improved.

Description

technical field [0001] The invention relates to the technical field of data mining, and in particular, to a method, a device and a computer storage medium for detecting log sequence anomalies. Background technique [0002] A large number of system logs are usually generated during the operation of modern systems, which record the operating information of the system in the form of text, and map the important activity states of the system at different key points. Log anomaly detection is helpful for anomaly location and cause analysis, thereby reducing error time and ensuring the normal operation of the system. The logs are arranged in order of execution time, forming a log sequence. Exception log sequences may not contain exception logs, as there may be cases where all individual logs are normal, but abnormal execution order or incomplete execution patterns of log sequences cause exceptions. Therefore, logs have context, and we need to detect log anomalies from the perspect...

AI Technical Summary

Problems solved by technology

However, due to the diversity of log formats in different systems, log parsers are not suitable for all log types

Furthermore, the robustness and accuracy of the log parser affects the performance of anomaly detection

Worse, usage of log parsers leads to loss of textual semantic information

[0004] (2) The method based on the event counting vector does not consider the execution order between logs, while the method based on the log key sequence only considers whether the next log is suitable for occurrence, ignoring the integrity of the log sequence

Existing methods based on log semantics do not consider the impact of word order and importance on log semantics

Images