A statistical method for process file page cache based on BCC tool

Abstract

The present invention relates to the field of computer technology, and specifically provides a statistical method for process file page cache based on a BCC tool, comprising starting to monitor the memory distribution of a Linux system; judging whether a process that needs to monitor the file page cache is mapped, and if not, continuing to monitor other memory items; if so, the add_to_page_cache_lru kernel function is called, using the BCC tool, monitoring the calling process of the add_to_page_cache_lru kernel function based on the kprobes plugging technology, obtaining the address of the address_space structure through parameters; parsing the address_space structure, finding the inode number and the nr_pages member; and calculating the file page cache size through nr_pages*PAGE_SIZE. Through the above scheme, the technical problem of lack of monitoring of non-fixed file page cache is solved.

Description

Technical Field

[0001] The invention relates to the technical field of computers, and in particular provides a statistical method for process file page cache based on a BCC tool. Background Art

[0002] The memory distribution of Linux system is: anonymous memory (mmap+shmem+malloc), file cache, slab memory and percpu memory. In many business scenarios, memory bottlenecks often occur, resulting in the inability of applications and system components to apply for memory smoothly. The high cache usage is particularly frequent. The open source tools provided by default in the Linux system can only count the total usage of cache (file page cache + directory entry cache + slab cache), but for the file page cache, it is impossible to count how many file page caches are mapped in the process address space when a process opens and reads a file.

[0003] In modern Linux systems, memory management is an important part of ensuring system stability and performance. As a part of memory, file page cache can significantly improve the efficiency of file access, but its specific impact on each process is not easy to monitor.

[0004] In HPC clusters and elastic cloud network environments, information technology personnel lack efficient and easy-to-use tools and means to perform memory performance analysis, troubleshooting, memory monitoring, etc. on business environments. The introduction of eBPF technology allows technicians to perform white-box detection on the Linux operating system. By analyzing the time and frequency of function calls, it helps find performance bottlenecks, monitors system calls and function execution in real time, and can quickly locate problems, monitor memory allocation events, help memory security analysis, system calls, and assist in memory auditing. BCC is a tool set for eBPF technology in the Linux kernel, providing high-level APIs and tools to simplify the creation and management of eBPF programs. eBPF is a bytecode that runs in the Linux kernel and can be used for a variety of tasks, including network monitoring, performance analysis, security monitoring, etc.

[0005] Therefore, how to use the existing BCC tool to solve the problem that the existing system operation and maintenance lacks monitoring of non-fixed file page cache is the key problem to be solved by the present invention. Accordingly, the art needs a statistical solution for process file page cache based on the BCC tool to solve the above problem. Summary of the invention

[0006] In order to overcome the above-mentioned defects, the present invention is proposed to provide a technical solution to the lack of monitoring of non-fixed file page cache.

[0007] The present invention provides a statistical method for process file page cache based on a BCC tool, comprising the following steps:

[0008] Start monitoring Linux system memory distribution;

[0009] Determine whether the process that needs to monitor the file page cache is mapped. If not, continue monitoring other memory items;

[0010] If yes, the add_to_page_cache_lru kernel function is called. Use the BCC tool to monitor the calling process of the add_to_page_cache_lru kernel function based on the kprobes instrumentation technology, and obtain the address of the address_space structure through the parameters.

[0011] Parse the address_space structure and find the inode number and nr_pages member;

[0012] The file page cache size is calculated by nr_pages*PAGE_SIZE.

[0013] Furthermore, the add_to_page_cache_lru kernel function is used to add a memory page to the file page cache.

[0014] Further, parse the address_space structure to find the inode number, including:

[0015] Get the file's inode number through an inode structure pointed to by the address_space structure, and then retrieve the corresponding file name through the file system interface in user mode.

[0016] Further, parse the address_space structure to find the nr_pages member, including:

[0017] Access the nr_pages member of the address_space structure to record changes in the file page cache.

[0018] Working principle and beneficial effects of the invention:

[0019] In the technical solution of the present invention, the add_to_page_cache_lru kernel function is tracked by the kprobe dynamic plugging technology based on the BCC development tool, so as to obtain the information of the dynamic change of the file page cache. It helps the development and operation personnel to deeply explore the technical principle of the kernel file page cache, and the highly free customized development feature also reduces the cost of system operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In addition, similar numbers in the figures are used to represent similar components, among which:

[0021] Figure 1 It is a flow chart of the main steps of a statistical method for process file page cache based on a BCC tool of the present invention;

[0022] Figure 2 It is a schematic diagram of the actual effect output after running a statistical method for process file page cache based on a BCC tool in the present invention. DETAILED DESCRIPTION

[0023] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0024] Figure 1 This is a flow chart of the main steps of a statistical method for process file page cache based on the BCC tool of the present invention. Figure 1 As shown, a method for counting process file page cache based on a BCC tool in this embodiment mainly includes the following steps S10 to S50.

[0025] Step S10: Start monitoring the Linux system memory distribution;

[0026] Step S20: determine whether the process that needs to monitor the file page cache is mapped, if not, continue monitoring other memory items;

[0027] Step S30: If yes, the add_to_page_cache_lru kernel function is called, and the BCC tool is used to monitor the calling process of the add_to_page_cache_lru kernel function based on the kprobes plugging technology, and the address of the address_space structure is obtained through the parameters.

[0028] In this embodiment, the process mapping of the file page cache needs to be monitored, and the add_to_page_cache_lru kernel function will be called only when the file data is read into the memory.

[0029] The kernel add_to_page_cache_lru function adds a memory page to the file page cache. There is no need to rely on the kernel process structure and the mapping of the file to the file page cache space. When the add_to_page_cache_lru triggers the call, it is injected into the corresponding kernel space through the function designed in the user layer, and the address of the address_space structure of the function parameter is obtained. When scanning the cardinality structure, multi-process parallel monitoring can be performed, and the changes of the file page cache page are counted while the cache is updated.

[0030] Step S40: Parse the address_space structure to find the inode number and nr_pages member.

[0031] Get the file's inode number through an inode structure pointed to by the address_space structure, and then retrieve the corresponding file name through the file system interface in user mode.

[0032] Access the nr_pages member of the address_space structure to record the changes in the file page cache. The size change of the file cache can be calculated through nr_pages*PAGE_SIZE. The member i_mapping of the Linux kernel inode index node points to address_space (file address structure space structure), and the member f_mapping of the file instance structure also points to address_space. It can be understood that the storage structure (radix tree) of each file page cache pagecache is stored in address_space. The member nr_pages is the sum of the file page cache pagecache corresponding to the file address space.

[0033] Step S50: Calculate the file page cache size by nr_pages*PAGE_SIZE.

[0034] The file page cache size corresponding to the process is calculated through nr_pages*PAGE_SIZE, and the statistical cache information is stored in the data structure defined by the injection function itself, and returned to the user state through kernel-level communication.

[0035] In the Linux kernel, the file page cache is used to store data read from the disk to reduce the number of disk accesses and improve system performance. Each file has a corresponding address_space structure, which manages the file page cache. The nr_pages member in the address_space structure indicates the total number of page caches corresponding to the file address space.

[0036] Based on the above steps S10 to S50, the add_to_page_cache_lru kernel function is tracked through the kprobe dynamic plugging technology based on the BCC development tool, so as to obtain the information of the dynamic changes of the file page cache. It helps the development and operation personnel to deeply explore the technical principles of the kernel file page cache, and the highly free customized development features also reduce the cost of system operation and maintenance.

[0037] In one implementation, the method of this embodiment can be used to monitor the changes in the file page cache and the corresponding read and write objects in real time. The actual effect is as follows: Figure 2 As shown, the first column is the timestamp (with nanosecond precision), the second column contains the process name and process number, the fourth column is the file cache size mapped by the process (the default size is Mb), and the fifth column is the file index number and the corresponding file name.

[0038] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0039] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A statistical method for process file page cache based on BCC tool, characterized in that: The following steps are involved: Start monitoring Linux system memory distribution; Determine whether the process that needs to monitor the file page cache is mapped. If not, continue monitoring other memory items; If yes, the add_to_page_cache_lru kernel function is called. Use the BCC tool to monitor the calling process of the add_to_page_cache_lru kernel function based on the kprobes instrumentation technology, and obtain the address of the address_space structure through the parameters. Parse the address_space structure and find the inode number and nr_pages member; The file page cache size is calculated by nr_pages*PAGE_SIZE.

2. According to the method for counting process file page cache based on BCC tool in claim 1, it is characterized in that: The add_to_page_cache_lru kernel function is used to add a memory page to the file page cache.

3. The method for counting process file page cache based on the BCC tool according to claim 1, characterized in that: Parse the address_space structure to find the inode number, including: Get the file's inode number through an inode structure pointed to by the address_space structure, and then retrieve the corresponding file name through the file system interface in user mode.

4. The method for counting process file page cache based on the BCC tool according to claim 1, characterized in that: Parse the address_space structure to find the nr_pages member, including: accessing the nr_pages member of the address_space structure to record changes in the file page cache.

Images