Distributed event collection probe, distributed event high-speed collection system and method

A technology for event collection and high-speed collection, which is applied in the field of network data and can solve the problems of data security hazards, log data collection and analysis solutions unable to meet requirements, etc.

Inactive Publication Date: 2018-03-30
THE THIRD RES INST OF MIN OF PUBLIC SECURITY
View PDF5 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In the actual operation process, many users use the same user name and password in different applications, so that if the hacker collects the FTP password, he may also obtain the password of the user's online account or other c

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed event collection probe, distributed event high-speed collection system and method
  • Distributed event collection probe, distributed event high-speed collection system and method
  • Distributed event collection probe, distributed event high-speed collection system and method

Examples

Experimental program
Comparison scheme
Effect test

example 1

[0059] This example implements the collection of received Syslog and Netflow logs.

[0060] For most devices and Linux and Unix systems, you can configure to record logs of a specified type and level, and specify a server to receive the logs, so that the logs are sent to the server that receives the logs through the Syslog protocol. When deploying this example, the address of the server that receives the logs is set to the address of the event collection probe of the auditing and tracing platform, thereby realizing forwarding the logs to the event collecting probe of the auditing and tracing platform.

[0061] Accordingly, the types of device forwarding logs that this instance can receive include Syslog logs and Netflow logs. The log receiver is written in C language, and it is mainly responsible for receiving and forwarding logs.

[0062] see figure 1 , which shows the flow chart of receiving Syslog and Netflow logs in this example. As can be seen from the figure, the proc...

example 2

[0068] This example implements obtaining Windows host logs through the WMI service.

[0069] The Windows system itself does not support log forwarding. To obtain Windows log information, you usually use a plug-in installed on the Windows system to forward or open the Windows system WMI service, and obtain it by calling the corresponding interface of the WMI service. The event collection probe supports obtaining Windows logs through the WMI service of the Windows system. When configuring, you need to enable the Windows WMI service and provide the necessary access rights for the collection probe.

[0070] To collect Windows system logs, you need to install the Sensor plug-in; the plug-in obtains logs through the WMI service interface and forwards the logs to the log collector.

[0071] see figure 2 , which shows the flow chart of Windows system log collection in this example. As can be seen from the figure, the process of collecting Windows system logs in this example is as f...

example 3

[0079]This example realizes reading the local log file of the event source.

[0080] In this example, some devices or systems do not support log forwarding, but save logs in local log files, and may set a policy to use these files in rotation. The plug-in needs to be installed in the device where the event source is located. The plug-in polls the log file regularly, and judges the new log by timestamp or event ID (depending on the event source device or system implementation method), and reads the new log through Syslog The protocol is forwarded to the event collection probe.

[0081] In this case, when this instance reads the local log files of the event source, the Sensor plug-in needs to be installed; the plug-in forwards the log files of the event source to the log collector by configuring the event source information.

[0082] The event source monitoring here supports event source directory monitoring and event source log file monitoring; in directory monitoring, you nee...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed event collection probe, a distributed event high-speed collections system and method. Through the scheme provided by the invention, the event collection is performed through the distributed probe, and a distributed log collection analysis system is proposed by using the Redis, ZeroMQ, Elasticsearch and like technical means in combination. The collection analysis system is formed through the cooperation of the distributed event collection probe and an analysis platform, can collect, analyze and display an access log of the Nginx in real time, thereby fundamentally solving the challenge on the log collection and analysis system, wherein the challenge is caused by the scale explosion of log data; and meanwhile, the effective information can be timely extracted from mass logs for the safety attack activity of the internet, thereby safely providing information support for the enterprise. The distributed event collection scheme provided by the inventionsupports the data collection and processing under the conditions of complex environment and huge data size.

Description

technical field [0001] The invention relates to network data technology, in particular to data collection and processing. Background technique [0002] Existing log collection methods include at least Syslog, SNMP Trap, Log File, JDBC, WMI, Ftp, etc., and support traffic data collection including NetFlow, sFlow, and NetStream. [0003] Among them, Syslog stores data in the form of plain text. If it is attacked by a hacker, the information can be stolen. When the intruder obtains root privileges, the file can be tampered with, and these operating system administrators cannot discover these intrusions at all. It is difficult to distinguish the authenticity of the data packaged by Syslog using UDP. [0004] The problem with the FTP file transfer protocol is that it sends usernames and passwords in clear text. Anyone can see the user name and password as long as a protocol analyzer is placed in a suitable place in the network; moreover, the data sent by FTP is also transmitted...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L12/24H04L29/06
CPCH04L41/069H04L63/1425
Inventor 李明陶源
Owner THE THIRD RES INST OF MIN OF PUBLIC SECURITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap