Method and system for processing journals

Abstract

Disclosed are a method and a system for processing journals. The method includes the steps of: collection, namely collecting isomeric journals generated by servers aiming at user operation, wherein the isomeric journals are generated on the basis of action items caused on each server, and the isomeric journals are enabled to include item identity fields indicating the action items; format conversion, namely converting the collected isomeric journals into journals with a uniform format, wherein the uniform format includes the item identity fields; combination, generating an intermediate journal file by combining the journals with the uniform format and storing the intermediate journal file on an intermediate server different from each server; statistical analysis, namely configuring a journal specification document on the basis of the item identity fields of the action items indicating expectation statistical analysis to automatically make statistical analysis on the action items of expectation statistical analysis so as to process massive journals in a standardized manner.

Description

technical field [0001] The present application relates to the field of network log service systems, and more specifically, relates to a method and system for processing logs in a standardized manner. Background technique [0002] With the development of Internet technology and information processing technology, more and more end users have access to the network, and spend a lot of time on the Internet in their daily work, study and life. For each operation of each user, many links on the network will generate corresponding network logs. Various network logs generated on the server side that provide various services reflect the operation status of each service, and also help service providers understand users' online behaviors, so as to provide technical guarantees or improvements. [0003] However, the amount of network logs is huge, and it is often necessary to process tens of billions of logs. Also, these logs are likely to have different formats depending on the source ...

AI Technical Summary

Problems solved by technology

It can be seen that the number of network logs is huge, the content is complicated, and the difference in the source of generation may also lead to the heterogeneity of the obtained logs. Therefore, it is difficult to effectively deal with the massive logs generated

[0004] In addition, processing network logs is an important part of network maintenance for service providers. In the prior art, in order to count these logs, it is often necessary to store network logs in the form of a table. After that, set the corresponding There are certain disadvantages in this method. Specifically, using tables to store massive log data will greatly reduce the processing speed of data, and the statistics of logs are also limited to the form of tables, and in order to count specific For statistical projects, it is necessary to repeatedly screen the massive logs in the table. Due to the huge network log data, this statistical method will consume a lot of time and lack flexibility

[0005] Therefore, when the existing log service system processes massive logs, its huge data volume brings great inconvenience to the storage and statistics of log data

In addition, when the logs have different formats due to different sources, it is difficult to perform statistical processing on these heterogeneous logs

Images