Mass information processing device

Abstract

The invention discloses a mass information processing device. A client filters acquired new log information so as to reduce transmission bandwidth occupied by invalid log information during transmission of the log information, log information with high real time is sent to a server in real time, and transmission of log information with low real time to the server is delayed, and then the cycle of log information acquisition can be effectively shortened; the log server timely sends alarm log information to an alarm processing device so as to meet the demand for instant output of the alarm information in the log information, and non-alarm log information is stored in different storage positions; and the class of the log information is corresponding to an operation class for operating the log information, and then a log operation platform can determines a storage position of the log information according to the operation class during operation of the log information. The processing time of log data operation can be quickly and effectively shortened, and the processing efficiency can be improved.

Description

technical field [0001] The invention relates to data processing technology, in particular to a massive information processing device. Background technique [0002] For the business support network operation management system, as the complexity and diversity of the support network and business gradually increase, the granularity of business monitoring becomes finer and finer, resulting in more and more service log information, especially The data volume of business service logs is larger. The daily business service logs of a communication operator may exceed 200G, and these business service logs are distributed on dozens of servers. At the same time, the business monitoring center has higher and higher requirements for real-time alarm information. How to quickly complete the collection, extraction, processing, storage, and efficient query of logs has become the focus of the business support network operation management system. [0003] Currently, there are two main ways to...

AI Technical Summary

Problems solved by technology

[0017] According to the technical solution of the embodiment of the present invention, in the processing method for massive information collection of log information provided by the embodiment of the present invention, the log client can filter the newly added log information collected, and can filter out the collected Invalid log information in the log information reduces the transmission bandwidth occupied by invalid log information when transmitting log information, and according to the corresponding relationship between predetermined log information and real-time performance level, carries real-time identification in the filtered log information. In the process of sending the filtered log information to the server, the log information with high real-time performance is sent to the server in real time, and the log information with low real-time performance is sent to the server later, which can effectively shorten the cycle of log information collection; in the present invention In the method for processing massive amounts of information stored in log information provided in the embodiment, the log server sends the alarm log information to the alarm processing device in time, can quickly send the alarm information to the alarm processing device, and can meet the requirements of the log information. According to the requirement of timely output of alarm information, according to the category of the non-alarm log information and the time of receiving the log information, the non-alarm log information is stored in the relational database, the memory database of the server or the file system respectively, and the category of the log information is related to The operation category corresponding to the operation on the log information enables the log operation platform to determine the storage location of the log information according to the operation category in the process of operating the log information, and the reading of data in the relational database and the memory database The acquisition speed is fast, which can quickly and effectively shorten the processing time of the log data operation and improve the processing efficiency, so as to solve the problems of the collection processing and storage query processing methods for massive information in the prior art, and the collection period is too long and the invalid log The problem of information occupying too much transmission bandwidth, the operation and processing time of massive information is long, the efficiency is low, and the problem of timely output of alarm information in log information cannot be met

Images