Online log analysis method and system and electronic terminal equipment thereof

Abstract

The invention provides an online log analysis method and system and electronic terminal equipment thereof. The method comprises the following steps: S1, conducting log preprocessing on each unanalyzedlog to obtain a plurality of unanalyzed log sequences with different log lengths, and classifying the unanalyzed log sequences into corresponding first log groups; S2, acquiring a log character string of each log sequence in the first log group, calculating the similarity of the log character strings, and performing online clustering based on the similarity of the log character strings; and S3, taking the unanalyzed log sequence as a query item, matching the common node with the template in the template spanning tree of the second log group to obtain the template. The method has the advantages that the logs are classified according to the length, the logs are subjected to secondary clustering based on the log character string similarity, finally, the template spanning tree is used for extracting the log template, the method can efficiently and accurately extract the log template from the unstructured logs, and a data analyst can conveniently carry out higher-level analysis and processing on the logs.

Description

technical field [0001] The invention relates to the technical field of log parsing, in particular to an online log parsing method, system and electronic terminal equipment based on character string clustering and template spanning tree. Background technique [0002] The system log records the operating information of the system. As important data for system online monitoring and abnormal detection, it can help managers debug system faults and analyze abnormalities, and further ensure system security. Usually logs are massive and unstructured. Log parsing can convert a large number of unstructured logs into structured data, that is, extract templates from logs. As the first step of system anomaly detection, log parsing improves the efficiency of anomaly detection. [0003] Generally, log parsing methods can be divided into offline methods and online methods. The offline method is a method of analyzing logs offline. It is necessary to collect the logs of the system within a ...

AI Technical Summary

Problems solved by technology

However, there are two shortcomings in the current online log analysis methods: first, the existing online methods have room for improvement in terms of accuracy and efficiency, and second, these online methods are designed for specific log systems, and the analysis parameters of each system are different. Universal, low parsing efficiency, poor accuracy

Images