Automatic log analysis system

Abstract

The invention discloses an automatic log analysis system, and aims to provide a quick, convenient and automatic log analysis system. According to the technical scheme, the method comprises the steps that a log preprocessing module sends a cleaned work log file to a text analysis slicing module for semantic analysis, a log adjacent text Simhash fingerprint is obtained through calculation, similarity calculation is carried out, and judgment is carried out to form a minimum non-repeated text block; a parameter name and numerical value extraction module constructs a regular expression matching mode according to the minimum non-repeated text block of the text or according to a pre-written log template, and selects a read-in log to perform parameter name and numerical value extraction processing; a data IO module stores the extracted text key information, parameter names and numerical values in a hard disk for subsequent reading and writing; and a statistical drawing and report generation module performs data statistics on the numerical values extracted from the hard disk according to classes, draws a visual graph according to a data statistical value, and generates an analysis report.

Description

technical field [0001] The invention belongs to the technical field of computers, and in particular relates to an automatic log analysis system for specific fields and cross-field texts. Background technique [0002] Text is one of the main ways of expressing information. On the Internet, there are a lot of text data in our work and life, and the processing of text is inseparable from daily data processing. The rapid development of the Internet has brought an increasing number of Internet users. In order to improve the service quality of enterprises, Internet service providers need to record user access information, network operation status, etc. The huge number of users makes the process of recording information generate a large number of logs. The analysis system for massive network logs provides Internet companies with effective log collection methods and log processing and analysis capabilities. The log is an integral part of a mature system, and it is the information...

AI Technical Summary

Problems solved by technology

But its disadvantages are: the information completeness of entries differs greatly from entry to entry, and the calculation accuracy cannot be guaranteed. The production method of network knowledge is public participation, which leads to a lack of professionalism in the text.

With the rapid growth of log data, there will be problems of inefficient storage and acquisition. Large-scale log data cannot be retrieved and analyzed in a centralized manner, and it is difficult to obtain valuable information from it. When problems occur during system operation, it is necessary to locate the problem in a short time.

Images