Log maintenance method and system based on full-text retrieval

Abstract

The invention relates to a log maintenance method and system based on full-text retrieval. The method comprises the following steps: 1) collecting logs, and storing collected log acquisition files into storage nodes; 2) generating indexes for all log acquisition files stored into the storage nodes; 3) establishing a retrieval agent for each node; 4) receiving an external retrieval request by a server, and sending the retrieval request to the retrieval agent to generate a retrieval port; 5) obtaining retrieval result data; and 6) feeding the retrieval result data back to the server through the retrieval port. The logs of all business systems are subjected to centralized management, which is favorable for the maintenance of the log files. A quick means for analyzing business faults is provided for business maintenance personnel, and maintenance efficiency is improved. The log storage nodes which are deployed in a cluster mode make a preparation for business increase. Retrieval and index generation are processed in parallel on each storage node to provide a quick processing speed.

Description

Technical field [0001] The invention relates to a log maintenance method and system based on full-text retrieval. Background technique [0002] In the business support system of telecom operators, due to the huge daily business transaction volume, massive log data will be generated. However, it is inevitable to analyze the log data in the process of user report and troubleshooting during daily maintenance. The method is that the background maintenance personnel analyze the content of the failure report, first guess the subsystem that may cause the failure, and then log in to the operating host of the subsystem (at this stage, a considerable proportion of the operators' systems are running in the x86 cluster environment, the same subsystem It may be composed of dozens of hosts), and then manually search the relevant log files at the time of the failure on the host. This process may consume a lot of time and labor costs. [0003] For the processing of log files in existing systems, ...

AI Technical Summary

Problems solved by technology

[0004] 1. Some subsystems will write the logs of different businesses into the same file in the way of centralized output, such as java applications, which flip the log files according to time or file size, and the flipped log files are based on business needs or In the case of storage space, delete or back up the older ones to other hosts regularly. The log files generated in this case are usually large in size, and it will be slow to find data in these large files;

[0005] 2. In addition, some systems will output logs to different files according to different businesses or processes. These log files will not be processed by flipping, but will be cleared directly by external shell scripts regularly, which will lead to such Situation: When you want to check the relevant logs after a period of time, they have been cleared and cannot be found

[0006] It is always a laborious and time-consuming task for maintenance personnel to frequently search for clues in these log files of various forms during the maintenance process. ten seconds

Images