Log variable semantic annotation method

Abstract

The invention discloses a log variable semantic annotation method, which comprises the following steps of: firstly, analyzing a log based on a log analysis algorithm to obtain a log mode of a log set; according to the position of the variable in the log mode, judging the part-of-speech of words near the position of the variable, and obtaining a log variable list with known semantics and a log variable list with unknown semantics; secondly, according to a variable value set in the log mode, describing the similarity between log variables from six dimensions including the overlapping performance of variable values, the distribution characteristic of the variable values, the diversity similarity of the variable values, the statistical characteristic similarity of the variable values, the variable position similarity and the neighbor variable similarity; and finally, based on an inference algorithm, judging whether the two log variables subjected to similarity description are matched or not, further identifying the log variables with unknown semantics as log variables with known semantics, and completing semantic annotation of the log variables. According to the method, the accuracy of the log variable labeling result can be effectively improved.

Description

technical field [0001] The invention relates to the technical field of log automatic parsing, in particular to a log variable semantic labeling method. Background technique [0002] With the rapid development and widespread popularization of Internet applications, log messages have exploded. At present, many studies analyze log messages to mine potential valuable information. However, log messages are mostly unstructured or semi-structured text data. Before analysis, structured fields need to be extracted from log messages. This process is called log parsing; after extracting fields, in order to help analysts understand the fields In order to facilitate the structural analysis of log messages using the log analysis platform, it is also necessary to assign appropriate semantics to the structured fields. This process is called log variable annotation. [0003] The problem that log variable labeling actually solves is how to assign reasonable semantics to log variables after ...

AI Technical Summary

Problems solved by technology

There are big defects in the description of entity similarity in this method. Only the similarity of variable values ​​is considered. In addition, it is not appropriate to use exact matching to measure the similarity between variable values ​​in the log.

Images