An online log parsing method, system and electronic terminal device thereof
An analysis method and log technology, which is applied in the field of log analysis, can solve the problems of poor accuracy, uncommon analysis parameters, and low analysis efficiency, and achieve the effects of reducing comparison steps, improving log analysis efficiency, and improving accuracy
Active Publication Date: 2022-07-22
SHANGHAI MUNICIPAL ELECTRIC POWER CO +2
View PDF3 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
However, there are two shortcomings in the current online log analysis methods: first, the existing online methods have room for improvement in terms of accuracy and efficiency, and second, these online methods are designed for specific log systems, and the analysis parameters of each system are different. Universal, low parsing efficiency, poor accuracy
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0056] The present invention will be further described below by describing a preferred specific embodiment in detail with reference to the accompanying drawings.
[0057] Log parsing is an efficient and accurate method to convert unstructured logs into structured forms, which obtain hidden information from massive system logs to guide subsequent operations.
[0058] The invention discloses an online log parsing method, which comprises the following steps:
[0059] S1. Perform log preprocessing on each unparsed log to obtain multiple unparsed log sequences with different log lengths, and classify each unparsed log sequence into a corresponding first log group, where the first log group has A set of log sequences with the same log length, the first log group includes parsed log sequences and unparsed log sequences.
[0060] like figure 1 As shown, in the log parsing process, each log m can be expressed as:
[0061] m{m h ,m c }
[0062] where m h Indicates the log header,...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The present invention provides an online log parsing method, system and electronic terminal equipment thereof. The method comprises the following steps: S1. Perform log preprocessing on each unparsed log to obtain a plurality of unparsed log sequences with different log lengths, be classified into the corresponding first log group; S2, obtain the log strings of each log sequence in the first log group, calculate the log string similarity, and perform online clustering based on the log string similarity; S3, classify the unparsed log sequences The template is acquired as a node in which the query item matches the template in the template spanning tree of the second log group in common. Its advantages are: classify logs according to length, perform secondary clustering on logs based on log string similarity, and finally use template spanning tree to extract log templates. This method can efficiently and accurately extract log templates from unstructured logs. , which is convenient for data analysts to perform more advanced analysis and processing of logs.
Description
technical field [0001] The invention relates to the technical field of log parsing, in particular to an online log parsing method, system and electronic terminal equipment based on string clustering and template spanning tree. Background technique [0002] The system log records the operating information of the system, which is an important data for online monitoring and anomaly detection of the system, which can help administrators to debug system faults and analyze anomalies, and further ensure the security of the system. Usually logs are massive and unstructured, and log parsing can convert a large number of unstructured logs into structured data, that is, extract templates from logs. As the first step of system anomaly detection, log parsing improves the efficiency of anomaly detection. [0003] Usually log parsing methods can be divided into offline methods and online methods. The offline method is a method of offline parsing logs. It needs to collect the logs of the ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/17G06F16/18G06F16/903
CPCG06F16/1734G06F16/1815G06F16/90344G06F16/322G06F16/345
Inventor 吴金龙王秀兰何旭东张露维胡钧毅陈晓露谢丽燕方晓蓉祝蓓李静
Owner SHANGHAI MUNICIPAL ELECTRIC POWER CO