Multi-data source log data centralized collection and storage method and device

Abstract

The invention provides a multi-data source log data centralized collection and storage method, which comprises the following steps: installing a log collection system client on each application server, collecting log data of each application server through the log collection system client, and sending the log data to a log processing system; after receiving the log data, copying the original log data into a log storage system for storage by the log processing system; screening, cutting, classifying and merging the original log data, and storing the processed new log data in the log storage system; and establishing a log index in a log display system, and obtaining the log data from the log storage system according to the log index for display. The invention further provides a device corresponding to the method. Through the multi-data source log data centralized collection and storage method, the multi-data-source log collection, storage and retrieval are achieved; the fault processingefficiency can be improved; the data analysis capability is improved; and a good foundation is laid for later troubleshooting.

Description

Technical field [0001] The invention relates to the field of log processing, in particular to a method and device for centralized collection and storage of log data from multiple data sources. Background technique [0002] Distributed applications are applications composed of different components running in separate operating environments, usually interconnected on different platforms through a network. Typical distributed applications are two-terminal (Client / Server), three-terminal (client / middleware / server) and n-terminal (client / multiple middleware / multiple server). [0003] At present, distributed application clusters have been widely used. When an abnormality or failure occurs in a distributed application cluster, in order to reduce the impact caused by the abnormality or failure, technicians need to perform abnormal troubleshooting or fault location as soon as possible. Traditionally, steps such as hardware resource detection, operating system status check, and application...

AI Technical Summary

Problems solved by technology

However, due to factors such as geographical dispersion of server clusters and numerous troubleshooting steps, it usually takes a lot of time to perform specific troubleshooting or positioning, and it is impossible to fix faults in the first place, which also leads to a decline in enterprise SLA indicators , and seriously affected the user experience of the system, and also caused economic losses to the enterprise

Images