Real-time log clustering analysis method based on reverse table

Abstract

The invention provides a real-time log clustering analysis method based on a reverse table. The method comprises the following specific steps: 1, initialization: defining a packaging structure of eachword in a log; 2, preprocessing the original log, including regularization replacement, sensitive word bank extraction, word segmentation by a word segmentation device, part-of-speech tagging and public variable extraction; and 3, obtaining a template, including log grouping, inverted table scoring, template obtaining, template display layer content updating and inverted table updating. The method has a real-time log clustering function, the universality of the template is improved, the logs can be processed in parallel, and the analysis and processing speed is increased.

Description

technical field

[0001] The present invention relates to a technology in the computer field, in particular to a real-time log cluster analysis method based on an inverted list. Background technique

[0002] Log analysis is particularly important for an enterprise. If the operation and maintenance personnel of the enterprise cannot understand the security status of the server in real time, it will cause inestimable losses to the enterprise. Analyzing the logs can not only understand the operating status of the software and hardware equipment, but also understand the source of the error log, determine whether the error is caused by the application or the system itself, etc., so as to make timely remedies and better improve the enterprise High availability of hardware and software equipment. In short, there are two most direct and obvious purposes of log analysis, one is website security self-inspection to understand the security incidents happening on the server, and the other...

Images