Network safety log template extraction method and device

A network security and log technology, applied in the field of information security, can solve the problems that analysts are difficult to extract log templates, log format changes, etc., achieve high accuracy, reduce system load, and improve computing efficiency

CN105049247AActive Publication Date: 2015-11-11INST OF INFORMATION ENG CAS
7 Cites 23 Cited by

Patent Information

Authority / Receiving Office
CN · China
Current Assignee / Owner
Publication Date
2015-11-11

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention relates to a network safety log template extraction method and device. The method comprises steps: firstly, original safety logs are subjected to data cleaning, and log information after filtering of time and IP addresses is obtained; secondly, the log information which does not containing time and IP addresses are subjected to clustering, and logs with similar formats are classified to the same class; thirdly, for logs in the each class, template words describing formats in the logs are extracted, and the template of the log format is obtained. Concretely, a DBSCAN algorithm or an OPTICS algorithm is employed to cluster logs with similar formats, and template words describing formats in logs are extracted by utilization of an LDA Gibbs sampling algorithm. The device comprises a data cleaning unit, an information clustering unit and a template extraction unit. Priori knowledge is not needed, a template of a network safety log format can be obtained automatically, the system load can be reduced, and operational efficiency and accuracy are raised.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the field of information security, in particular to a method and device for extracting a network security log template. The solution does not depend on any prior knowledge and can automatically extract a template describing an unknown network security log format. Background technique

[0002] Network security logs, including system logs generated by the operating system and alarm logs generated by network security devices, record various security events in the network environment and provide important clues for network abnormal diagnosis and network attack threat discovery. In the network security log analysis system, log format parsing is an essential step. Therefore, extracting network security log templates is of great significance for log analysis. At present, in some relatively mature network security log analysis products, such as OSSIM, Snort, OSSEC, etc., log parsing methods based on regular expressions are usually col...

Examples

Embodiment Construction

[0032] In order to make the above objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described below through specific embodiments and accompanying drawings.

[0033] The network security log template extraction method provided by the present invention, its main steps include data cleaning, information clustering and template extraction, such as figure 1 shown. Among them, data cleaning is to first filter the fields with specific formats in the log data, including date, IP address, etc.; information clustering is to use clustering algorithm to divide the logs with similar formats into one category after cleaning. Medium; template extraction is to extract the template words describing the format in the log for each log in the class, and only keep the template words in the original log to obtain the log format.

[0034] 1. Data cleaning

[0035] The data cleaning in this method is mainly to filter out...