Log data modeling method and system

Abstract

The invention discloses a log data modeling method and system, and the system comprises a client and a log generalization unit, and the client is in communication connection with the log generalization unit. And the log generalization unit is set to be used for acquiring the log uploaded by the client and performing internal processing on the log uploaded by the client. According to the method and the system provided by the invention, modeling processing is performed on a large amount of log data through log generalization, so that later data retrieval and analysis are facilitated. And meanwhile, the log generalization process only depends on strategy configuration, and the strategy configuration can be managed on an interface, so that personalized customization of various types of log generalization processes can be conveniently supported. In the log generalization process, secondary processing on the generalization result is supported, and the scene that secondary processing needs to be carried out on the original log content can also be met. According to the method, big data architecture deployment is adopted, high performance and high throughput are guaranteed, and the requirements of large data volume and high performance requirements in actual application scenes are met.

Description

technical field [0001] The embodiments of the present application relate to the technical field of data analysis, and in particular to a log data modeling method and system. Background technique [0002] With the development of Internet technology, the software architecture of enterprises is gradually expanding in scale and becoming more and more complex. Usually, software applications provide convenience for later maintenance by recording logs. In this way, there are a lot of problems in an enterprise software architecture system. Continuous log data, which often contain information such as application processing key data, system status, and system configuration. In order to understand the user's access characteristics and needs, log analysis is required to discover the user's access behavior and rules, and optimize the business structure in order to bring a better experience to users. At the same time, with the development of log data analysis and processing technology, p...

AI Technical Summary

Problems solved by technology

[0004] This application provides a log data modeling method and system to solve the lack of a modeling management method for log data management in the prior art, resulting in the time-consuming and laborious problem that managers need to find key data and other information when managing data

Images