Online log analysis method and system and electronic terminal equipment thereof

A parsing method and log technology, applied in the field of log parsing, can solve problems such as poor accuracy, low parsing efficiency, and uncommon parsing parameters, and achieve the effects of improving accuracy, improving log parsing efficiency, and reducing invalid comparison steps

Active Publication Date: 2020-03-17
SHANGHAI MUNICIPAL ELECTRIC POWER CO +2
View PDF3 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, there are two shortcomings in the current online log analysis methods: first, the existing online methods have room for improvement in terms of accuracy and efficiency, and second, these online methods are designed for specific log systems, and the analysis parameters of each system are different. Universal, low parsing efficiency, poor accuracy

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Online log analysis method and system and electronic terminal equipment thereof
  • Online log analysis method and system and electronic terminal equipment thereof
  • Online log analysis method and system and electronic terminal equipment thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0056] The present invention will be further elaborated below by describing a preferred specific embodiment in detail in conjunction with the accompanying drawings.

[0057] Log parsing is an efficient and accurate method to convert unstructured logs into structured forms, and it obtains hidden information from massive system logs to guide subsequent operations.

[0058] The invention discloses an online log parsing method, which comprises the following steps:

[0059] S1. Perform log preprocessing on each unanalyzed log to obtain multiple unanalyzed log sequences with different log lengths, and classify each unanalyzed log sequence into the corresponding first log group, and the first log group has A collection of log sequences with the same log length, the first log group includes parsed log sequences and unparsed log sequences.

[0060] Such as figure 1 As shown, in the log parsing process, each log m can be expressed as:

[0061] m{m h ,m c}

[0062] Among them, m h...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an online log analysis method and system and electronic terminal equipment thereof. The method comprises the following steps: S1, conducting log preprocessing on each unanalyzedlog to obtain a plurality of unanalyzed log sequences with different log lengths, and classifying the unanalyzed log sequences into corresponding first log groups; S2, acquiring a log character string of each log sequence in the first log group, calculating the similarity of the log character strings, and performing online clustering based on the similarity of the log character strings; and S3, taking the unanalyzed log sequence as a query item, matching the common node with the template in the template spanning tree of the second log group to obtain the template. The method has the advantages that the logs are classified according to the length, the logs are subjected to secondary clustering based on the log character string similarity, finally, the template spanning tree is used for extracting the log template, the method can efficiently and accurately extract the log template from the unstructured logs, and a data analyst can conveniently carry out higher-level analysis and processing on the logs.

Description

technical field [0001] The invention relates to the technical field of log parsing, in particular to an online log parsing method, system and electronic terminal equipment based on character string clustering and template spanning tree. Background technique [0002] The system log records the operating information of the system. As important data for system online monitoring and abnormal detection, it can help managers debug system faults and analyze abnormalities, and further ensure system security. Usually logs are massive and unstructured. Log parsing can convert a large number of unstructured logs into structured data, that is, extract templates from logs. As the first step of system anomaly detection, log parsing improves the efficiency of anomaly detection. [0003] Generally, log parsing methods can be divided into offline methods and online methods. The offline method is a method of analyzing logs offline. It is necessary to collect the logs of the system within a ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/17G06F16/18G06F16/903
CPCG06F16/1734G06F16/1815G06F16/90344G06F16/322G06F16/345
Inventor 吴金龙王秀兰何旭东张露维胡钧毅陈晓璐谢丽燕方晓蓉祝蓓李静
Owner SHANGHAI MUNICIPAL ELECTRIC POWER CO
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap