NLP-based weblog processing system and method

A processing system and processing method technology, applied in the field of NLP-based network log processing system, can solve problems such as correlation analysis obstacles, poor applicability, and poor readability of syslog messages, so as to simplify the learning process, reduce the learning cycle, and improve usability Effect

Active Publication Date: 2020-05-08
INFORMATION & COMM BRANCH OF STATE GRID JIANGSU ELECTRIC POWER
View PDF9 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] 1. Since syslog is written based on the cognition of device manufacturers and developers, the expressions of the content with the same meaning in different manufacturers / models of devices are also quite different
[0007] 2. The readability of the syslog message itself is poor, and too many technical terms require managers to have a lot of professional background knowledge to understand the meaning of the message
[0008] 3. There is no unified standard for the log event itself, which leads to the inability to effectively classify the alarm level classification and event classification, which will cause certain obstacles to the correlation analysis
[0009] 4. The existing template-based traditional translation technology has poor flexibility and applicability

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • NLP-based weblog processing system and method
  • NLP-based weblog processing system and method
  • NLP-based weblog processing system and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0039] Such as Figure 1-2 As shown, the present embodiment provides a network log processing system based on NLP, including a natural language processing component and a database; a thesaurus, a preset word meaning library and a language processing model are constructed in the database, and the thesaurus is set Key words based on multiple literal translation words specific to the device type or high-frequency words derived after word segmentation;

[0040] Specifically:

[0041] The first case: multiple classifiers are set, and the classifiers cover keywords corresponding to different characteristics of different types of equipment.

[0042] For example: routers extend BGP, OSPF, CPU; servers extend CPU, memory.

[0043] The second case: the processing of classifiers is performed by matching and segmenting the syslog based on the preset word meaning database. If there is a word matched by the preset word in the preset word meaning database, it is extracted as the type under t...

Embodiment 2

[0080] In order to more clearly explain the syslog logs related to the device model and manufacturer extraction in an NLP-based network log processing method, it will be explained with examples:

[0081] Network devices can be monitored through the syslog protocol, and the log information is transmitted to the remote server module in the form of User Datagram Protocol (UDP). The remote receiving log server module must monitor UDP port 514 through syslog, and configure The configuration of the machine processes the local machine, receives the log information of the access system, and writes the specified event into a specific file for background database management and response.

[0082] The specific implementation steps are as follows:

[0083] 1) Collect raw log information.

[0084] Data source: a device or system that provides log data in syslog format; the device may be a firewall, switch, router, server, or other host with a linux-like operating system installed.

[008...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an NLP-based weblog processing system. The weblog processing system comprises a natural language processing assembly and a database. A classification lexicon, a preset word meaning library and a language processing model are constructed in the database; the classification lexicon sets a plurality of literal translation words specifically corresponding to equipment types orkeywords taking high-frequency words derived after word segmentation processing as standards; the classification word bank is in a mapping relation with a preset word meaning bank, and the preset wordmeaning bank is associated with a language processing model; the natural processing component is used for carrying out classification on syslog source data and log files of the equipment, and analyzing and determining meanings contained in natural language statements. According to the method, the defect that an undefined log cannot be analyzed by a conventional template-based method is overcome,availability of the system is improved, and usability of a user is improved.

Description

technical field [0001] The invention relates to the technical field of network security, in particular to an NLP-based network log processing system and method. Background technique [0002] In a modern society where the popularity of the Internet is getting higher and higher, network monitoring and management is an important guarantee for the rational use of network resources and information. In order for network managers to quickly and conveniently understand and control the operation status of the entire network, and respond to problems and threats in the network in a timely manner, the current common practice is to centrally collect and monitor network logs through the log management component. Analyze and provide managers with the implementation and operation status of equipment in the network to achieve effective control of risks. [0003] In recent years, with the development of artificial intelligence technology, natural language processing (NLP) technology stands o...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/24
CPCH04L41/069
Inventor 冒佳明赵俊峰曹晶夏飞夏元轶
Owner INFORMATION & COMM BRANCH OF STATE GRID JIANGSU ELECTRIC POWER
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap