Method and device for managing file objects in memory in log collection process

Abstract

The invention provides a method and a device for managing a file object in a memory in a log acquisition process. The method comprises the following steps: acquiring a log reading position record file; determining the latest sequence mark of each rolling log instance according to the log reading position record file, and storing the latest sequence mark into a log file latest sequence mark mapping; reading a to-be-collected log file, and obtaining a latest sequence mark of the to-be-collected log file; performing matching query on the latest sequence mark of the log file to be acquired and thelatest sequence mark mapping of the log file, opening a handle of the log file to be acquired through multiple judgment rules to generate a log file to be read, and putting the log file to be read into a read file list of a memory; traversing the read file list, obtaining the latest sequence mark of the to-be-read log file, comparing the latest sequence mark of the to-be-read log file with the updated latest sequence mark mapping of the log file, closing a handle of the to-be-read log file according with a set rule, and clearing the handle from the read file list of the memory.

Description

technical field [0001] The invention relates to the technical field of computer information processing, in particular to a method and device for managing file objects in memory during log collection. Background technique [0002] This section is intended to provide a background or context for implementations of the invention that are recited in the claims. The descriptions herein are not admitted to be prior art by inclusion in this section. [0003] With the continuous development of computer technology, logs can no longer be used only for engineers to manually troubleshoot errors, but can also become data sources in fields such as big data analysis and intelligent operation and maintenance. Therefore, an excellent log collection tool is essential. In the process of text log collection, in order to reduce data duplication and data loss, it is necessary to record the location where each log file is collected in the memory, and periodically persist it, so that the breakpoin...

AI Technical Summary

Problems solved by technology

[0006] The shortcomings of the existing technology mainly include: if the parameter close_inactive is set unreasonably, it will lead to performance degradation in some scenarios, and in some scenarios it will cause too many file handles to be occupied

If the parameter clean_inactive is set unreasonably, in some scenarios, the data will be inaccurate, and in some scenarios, the log collection program will take up a lot of memory, which will affect the performance of the program.

Images