Log monitoring method and device and server

Abstract

The invention discloses a log monitoring method and device and a server, and the method comprises the steps: obtaining log data through a buried point according to a preset format of the log data, andpushing the log data to a distributed publishing and subscribing system through a data processing pipeline; performing operation processing on log data in the distributed publishing and subscribing system to generate a plurality of items of monitoring data; and setting a data calling interface for monitoring data and front-end communication, and realizing log monitoring through the front end. According to the technical scheme, the distributed publishing and subscribing system serves as a cache and is suitable for data collection services of internet services generating a large amount of data,so that operation analysis of log data is more sufficient, and the log monitoring process is optimized; moreover, the monitoring items are wider and more comprehensive, and the intelligent level of log monitoring is improved.

Description

technical field [0001] The invention relates to the field of big data processing, in particular to a log monitoring method, device and server. Background technique [0002] In the process of business development, software products are continuously developed and iterated, new functions are continuously added, and various services are continuously deployed. In order to detect software and hardware problems and user usage in each service module earlier, a complete set of service log monitoring and alarm system is required. [0003] The current mainstream log monitoring system and some open source solutions only perform simple monitoring on the logs generated by the server, and can only provide simple alarms on the service log ERROR. Complex monitoring requirements such as user behavior and links generated by each call cannot be well resolved, and the current log monitoring system cannot analyze and count the usage, frequency of use, number of active users, and user operations ...

AI Technical Summary

Problems solved by technology

Complex monitoring requirements such as user behavior and links generated by each call cannot be well resolved, and the current log monitoring system cannot analyze and count the usage, frequency of use, number of active users, and user operations of each functional module every day

[0004] In addition, for some internal services of the company, such as the daily read and write traffic of databases such as Oracle, the timing scheduling operation of the XXL-JOB distributed task scheduling platform, and the alarm monitoring of the TDH big data processing platform, there is no unified processing

Images