Log acquisition device and log acquisition method

Abstract

The invention relates to a log acquisition device and a log acquisition method. The device comprises at least one client for collecting logs and at least one server for recording the logs, wherein log information is transmitted between the client and the server through a network in real time. The method comprises that: the client generates the log information, writes the log information into a disk file of the client, reads the log information from the disk file of the client, and sends a connection request to the server; after receiving the request, the server establishes network connection with the client and receives the log information; the server sends the received log information to a server log writing module and a server log writing database module; the server log writing module writes the log information into the disk file of the server; and the server log writing database module writes the log information into the database. When the device and the method of the invention are used, the data analysis is more real-time, and the distributed acquisition of the logs is realized.

Description

technical field [0001] The invention relates to the technical field of computer Internet, in particular to a real-time collection device and method of distributed logs. Background technique [0002] In the current Internet field, the analysis of user behavior is becoming more and more important, and the basis of user behavior analysis is the user's usage log. For many distributed high-traffic products on the Internet, with the distribution of servers, log analysis becomes more and more troublesome. It is transferred to a server through FTP (File Transfer Protocol) for unified processing. Before the calculation and processing of some logs, the sorting of logs also needs to be considered. [0003] For example, a common log backup method at present: multiple business servers will record the log in the local log file in the form of text according to the designed log format, and truncate the log file by day or hour. When the business load is low in the early morning, the log fi...

AI Technical Summary

Problems solved by technology

[0006] 1. The period of log collection is too long. In order to ensure the normal operation of the business, it usually takes one day

[0007] 2. The scalability is poor, and the collection and processing of larger system logs will be more difficult

[0008] 3. The log file is too large

Due to the common use of text files to record files, the log files are not well compressed and require a lot of disk space

[0009] 4. The timeliness of log analysis is poor

Since some systems need to complete log analysis at a specific time, the existing method of uploading and merging logs will take up a lot of time for log analysis. The larger the log volume, the longer the time for log uploading and merging will be, resulting in log analysis that cannot be completed in time.

[0010] 5. Poor maintainability

The log format of the business changes, and the analysis method of the log changes accordingly. When the system is upgraded, it will face uncertainty

Images