Method, device and system for processing massive log information

Abstract

The invention discloses a processing method, device and system of mass log information. A client end is used for filtering collected newly-increased log information, the transmission bandwidth occupied by invalid log information can be reduced when the log information is transmitted, the log information with high real-time performance is transmitted to a server in real time, the log information with low real-time performance is delayed to be transmitted to the server, and the collecting cycle of the log information can be effectively shortened. The logic server transmits warning log information to a warning processing device timely, the demand for timely outputting the warning information in the log information can be met, non-warning log information is stored to different storage positions, the types of the log information correspond to the operation types of operation to the log information, so that in the process that a log operation platform conducts operation on the log information, the storage positions of the log information can be judged according to the operation types, the processing time of log data operation can be quickly and effectively shortened, and the processing efficiency is improved.

Description

technical field [0001] The present invention relates to data processing technology, in particular to a processing method, device and system for massive log information. Background technique [0002] For the business support network operation management system, as the complexity and diversity of the support network and business gradually increase, the granularity of business monitoring becomes finer and finer, resulting in more and more service log information, especially The data volume of business service logs is larger. The daily business service logs of a communication operator may exceed 200G, and these business service logs are distributed on dozens of servers. At the same time, the business monitoring center has higher and higher requirements for real-time alarm information. How to quickly complete the collection, extraction, processing, storage, and efficient query of logs has become the focus of the business support network operation management system. [0003] Cu...

AI Technical Summary

Problems solved by technology

[0016] Embodiments of the present invention provide a processing method, device, and system for massive log information, which are used to solve the problem of collecting and processing and storing and querying processing methods for massive log information in the prior art. The collection period is too long and invalid log information occupies too much The problem of transmission bandwidth, the operation and processing time of massive log information is long, the efficiency is low, and the problem of timely output of alarm information in log information cannot be met

Images