Method for detecting linux kernel memory leak

Abstract

The invention discloses a method for detecting linux kernel memory leak. The method comprises the following steps: acquiring a use index of a system kernel memory through a monitoring program; determining whether memory leak occurs or not according to the use index; and in response to determining that the memory leak occurs, starting an eBPF detection program to monitor memory allocation and release activities in the kernel, and analyzing the memory allocation and release activities to generate a memory leak report to locate the memory leak. According to the method, the system kernel memory is monitored by depending on the system information, the use index of the system kernel memory is acquired through the monitoring program, and whether memory leakage occurs or not is judged according to the use index, so that the use condition of the kernel memory can be monitored in real time with very small system overhead, and the system reliability is improved. And detecting whether the kernel memory leak occurs or not.

Description

Technical Field

[0001] The present invention relates to the technical field of memory detection, and particularly to a method for detecting memory leaks in the Linux kernel. Background Art

[0002] Memory leak refers to the situation where a program fails to release memory resources that are no longer in use during its operation, resulting in these resources being unable to be reused by other parts. For the system kernel, memory leaks may lead to serious performance degradation or even system crashes. Traditional memory leak detection tools usually rely on external libraries or modify the kernel, which not only increases complexity but also affects the stability and security of the system. eBPF provides a new non-invasive means to monitor and diagnose memory leak problems in the kernel. However, eBPF also wastes CPU resources and consumes memory during daily monitoring. For each event processed by eBPF, if the processing logic is complex, it will cause delays in normal activities. At the same time, eBPF programs may need to exchange data with user-space applications frequently, which involves I / O operations and may increase the I / O burden of the system. Using eBPF for monitoring in daily production and life will cause waste of system resources. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for detecting memory leaks in the Linux kernel to solve the above problems.

[0004] The present invention provides a method for detecting memory leaks in the Linux kernel, including:

[0005] Collecting usage metrics of the system kernel memory through a monitoring program;

[0006] Determining whether a memory leak has occurred based on the usage metrics;

[0007] In response to determining that a memory leak has occurred, enabling the eBPF detection program to monitor memory allocation and release activities in the kernel, and analyzing the memory allocation and release activities to generate a memory leak report for locating the memory leak.

[0008] Further, the usage metrics include at least one of the following: the VmallocUsed value representing the total amount of kernel virtual memory, the SUnreclaim value representing the total amount of non-reclaimable kernel memory, and the MemFree value representing the total amount of unallocated physical memory;

[0009] The determining whether a memory leak has occurred based on the usage metrics includes:

[0010] If the usage metrics obtained from consecutive multiple collections are all greater than the threshold values corresponding to the usage metrics, it is determined that a memory leak has occurred.

[0011] Further, the threshold value is the average value of the usage metrics collected multiple times when there is no memory leak, including: collecting the VmallocUsed value, SUnreclaim value, and MemFree value under / proc / meminfo multiple times, and respectively taking the average values of the collected VmallocUsed value, SUnreclaim value, and MemFree value as the first threshold value, the second threshold value, and the third threshold value.

[0012] Further, enable the eBPF detection program to monitor memory allocation and release activities in the kernel, including:

[0013] The eBPF detection program includes a vmalloc leak detection function, a slab leak detection function, and a page leak detection function;

[0014] If the vamllocused value exceeds the corresponding first threshold value, enable the vmalloc leak detection function for memory allocation and release;

[0015] If the SUnreclaim value exceeds the corresponding second threshold value, enable the slab leak detection function for memory allocation and release;

[0016] If the MemFree value exceeds the corresponding third threshold value, enable the page leak detection function for memory allocation and release.

[0017] Further, if the collected vamllocused values all exceed the corresponding first threshold value, enable the vmalloc leak detection function for memory allocation and release, including:

[0018] Instrument the allocation function vmalloc to obtain sampling information, where the sampling information includes: the process name, pid, application address, kernel stack, hash id value of the kernel stack, and the size of the allocated memory;

[0019] Using the application address as the key, store the sampling information in the slab_map table;

[0020] Using the hash id value of the kernel stack as the key, query whether there is such a value in the slab_total_map table. If there is, add the size of the current allocated memory to the original total value and increment the total number of allocation times by 1. If not, add the total size as the size of the current allocated memory, the total number of allocation times as 1, and record the kernel stack and the hash id value of the kernel stack;

[0021] Instrument the release function vmfree to sample the released memory address;

[0022] Query in the slab_map table using the memory address as the key to check if this value exists. If not, return. If it exists, obtain the stackid, query in the slab_total_map table using the stackid as the key, subtract the memory size value released this time from the total total value of the data, record the decrease in the total application count, and delete this data item in the slab_map table.

[0023] Furthermore, if all the collected SUnreclaim values exceed the corresponding second threshold value, enable the slab leak detection function for memory allocation and release, including:

[0024] Instrument the allocation functions kmem_cache_alloc, kmalloc, kmalloc_node to obtain sampling information, where the sampling information includes: the process name of the allocated memory, pid, application address, kernel stack, hash id value of the kernel stack, and the size of the allocated memory;

[0025] Use the application address as the key and store the sampling information in the slab_map table;

[0026] Use the hash id value of the kernel stack as the key to query in the slab_total_map table to check if this value exists. If it exists, add the size of the memory allocated this time to the original total value and record an increase of 1 in the total application count. If not, add a total size of the memory allocated this time, a total application count of 1, record the kernel stack and the hash id value of the kernel stack;

[0027] Instrument the release functions kfree, kmem_cache_free to sample the released memory address;

[0028] Query in the slab_map table using the memory address as the key to check if this value exists. If not, return. If it exists, obtain the stackid, query in the slab_total_map table using the stackid as the key, subtract the memory size value released this time from the total total value of the data, record the decrease in the total application count, and delete this data item in the slab_map table.

[0029] Furthermore, if all the collected MemFree values exceed the corresponding third threshold value, enable the page leak detection function for memory allocation and release, including:

[0030] Instrument the allocation function mm_page_alloc to obtain sampling information, where the sampling information includes: the process name of the allocated memory, pid, application address, kernel stack, hash id value of the kernel stack, and the size of the allocated memory;

[0031] Using the application address as the key, store the sampling information in the slab_map table;

[0032] Using the hash id value of the kernel stack as the key, query in the slab_total_map table to see if there is such a value. If there is, add the size of the memory applied for this time to the original total value and increment the total application count by 1. If not, add the total size as the size of the memory applied for this time, the total application count as 1, record the kernel stack and the hash id value of the kernel stack;

[0033] Instrument the release function mm_page_free to sample the released memory address;

[0034] Query in the slab_map table using the memory address as the key to see if there is such a value. If not, return. If there is, obtain the stackid, query in the slab_total_map table using the stackid as the key, subtract the size of the memory released this time from the total value of the data, decrement the total application count, and delete this data item from the slab_map table.

[0035] Furthermore, analyze the memory allocation and release activities to generate a memory leak report for locating memory leaks, including:

[0036] Traverse the map table with the stackid as the key, compare the total application size or count with the threshold value. If it exceeds the threshold value, print the kernel stack to a file, and traverse the map table with the address or page as the key, filter the map table based on whether the stackid is the same, and print the process name and process ID to the file to generate a memory leak report for locating memory leaks.

[0037] The present invention has at least the following beneficial effects:

[0038] The present invention monitors the system kernel memory by relying on system information, collects the usage metrics of the system kernel memory through a monitoring program, and determines whether a memory leak occurs based on the usage metrics, achieving real-time monitoring of the usage of kernel memory with very little system overhead and detecting whether there is a kernel memory leak. When a kernel memory leak is detected, enable the eBPF detection program to monitor the memory allocation and release activities in the kernel, and generate a memory leak report by analyzing the memory allocation and release activities to accurately locate the memory leak. If no memory leak occurs, the eBPF detection program does not work, thus avoiding the problem of resource waste caused by the eBPF detection program performing daily monitoring. Furthermore, the present invention can be used for long-term memory monitoring of devices and can also be used to find memory leaks during the development process.

[0039] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. Description of the Drawings

[0040] Figure 1 It is a flowchart of a method for detecting memory leaks in the Linux kernel provided by the present invention. Detailed Embodiments

[0041] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0042] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present invention.

[0043] Embodiment 1: In combination with Figure 1 Illustrate this embodiment

[0044] The present invention provides a method for detecting memory leaks in the Linux kernel, including:

[0045] S01: Collect the usage metrics of the system kernel memory through a monitoring program;

[0046] S02: Determine whether a memory leak has occurred according to the usage metrics;

[0047] S03: In response to determining that a memory leak has occurred, enable the eBPF detection program to monitor the memory allocation and release activities in the kernel, and analyze the memory allocation and release activities to generate a memory leak report for locating the memory leak.

[0048] The present invention monitors the system kernel memory by relying on system information, collects the usage metrics of the system kernel memory through a monitoring program, and determines whether a memory leak has occurred based on the usage metrics, so as to realize real-time monitoring of the usage of the kernel memory with very small system overhead and detect whether there is a kernel memory leak. When it is detected that there is a kernel memory leak, an eBPF detection program is enabled to monitor the memory allocation and release activities in the kernel, and a memory leak report is generated by analyzing the memory allocation and release activities to accurately locate the memory leak. If no memory leak occurs, the eBPF detection program does not work, thus avoiding the problem of resource waste caused by the eBPF detection program performing daily monitoring. Furthermore, the present invention can be used for long-term memory monitoring of devices and can also be used to find memory leaks during the development process.

[0049] In a possible implementation manner, the above usage metrics include at least one of the following: the VmallocUsed value representing the total amount of kernel virtual memory, the SUnreclaim value representing the total amount of non-reclaimable kernel memory, and the MemFree value representing the total amount of unallocated physical memory. Correspondingly, the above step S02 may include:

[0050] If the usage metrics obtained from continuous multiple collections are all greater than the threshold value corresponding to the usage metrics, it is determined that a memory leak has occurred.

[0051] For example, compare the collected usage metrics with the threshold value of the corresponding usage metrics. If the usage metrics exceed the threshold value, the record is incremented by 1. If the usage metrics do not exceed the threshold value, it is reset, so as to obtain the number of times that the collected usage metrics are greater than the corresponding threshold values continuously for multiple times. Furthermore, when this number is greater than or equal to a preset threshold value, it can be determined that a memory leak has occurred. The preset threshold value is, for example, 10, that is, if the collected usage metrics are greater than the corresponding threshold values continuously for 10 times, it is determined that a memory leak has occurred.

[0052] Optionally, for the threshold value of each different type of usage metric, the usage metric can be collected multiple times in the case where no memory leak has occurred, and the average value of the usage metrics collected multiple times can be calculated, and then the threshold value of this type of usage metric can be determined according to the average value. Specifically, it may include: collecting the VmallocUsed value, the SUnreclaim value, and the MemFree value under / proc / meminfo multiple times, and respectively taking the average values of the collected VmallocUsed value, SUnreclaim value, and MemFree value as the first threshold value, the second threshold value, and the third threshold value. The specific number of collections for each type of usage metric can be, for example, 10 times.

[0053] In a possible implementation, the eBPF detection program includes a vmalloc leakage detection function, a slab leakage detection function, and a page leakage detection function. Correspondingly, enabling the eBPF detection program to monitor memory allocation and release activities in the kernel may include enabling at least one of the vmalloc leakage detection function, the slab leakage detection function, and the page leakage detection function to monitor memory allocation and release activities in the kernel according to the size relationship between different usage metrics and corresponding threshold values.

[0054] For example, if the collected vamllocused values all exceed the corresponding first threshold value, the vmalloc leakage detection function is enabled to perform memory allocation and release;

[0055] If the collected SUnreclaim values all exceed the corresponding second threshold value, the slab leakage detection function is enabled to perform memory allocation and release;

[0056] If the collected MemFree values all exceed the corresponding third threshold value, the page leakage detection function is enabled to perform memory allocation and release.

[0057] Furthermore, if the collected vamllocused values all exceed the corresponding first threshold value, enabling the vmalloc leakage detection function to perform memory allocation and release may specifically include:

[0058] Instrument the allocation function vmalloc to obtain sampling information, where the sampling information includes: the process name, pid, application address, kernel stack, hash id value of the kernel stack, and the size of the allocated memory;

[0059] Using the application address as the key, store the sampling information in the slab_map table;

[0060] Using the hash id value of the kernel stack as the key, query whether there is such a value in the slab_total_map table. If there is, add the size of the currently allocated memory to the original total value and increment the total number of allocation times by 1. If not, add the total size as the size of the currently allocated memory, the total number of allocation times as 1, and record the kernel stack and its hash id value;

[0061] Instrument the release function vmfree to sample the released memory address;

[0062] Query whether there is such a value in the slab_map table with the memory address as the key. If not, return; if so, obtain the stackid, query in the slab_total_map table with the stackid as the key, subtract the memory size value released this time from the total total value of the data, record the decrease in the total application count, and delete this data item in the slab_map table.

[0063] Further, if the collected SUnreclaim values all exceed the corresponding second threshold value, enable the slab leak detection function for memory allocation and release, which specifically includes:

[0064] Instrument the allocation functions kmem_cache_alloc, kmalloc, and kmalloc_node to obtain sampling information, where the sampling information includes: the process name, pid, application address, kernel stack, hash id value of the kernel stack, and the memory size applied for;

[0065] Store the sampling information in the slab_map table with the application address as the key;

[0066] Query whether there is such a value in the slab_total_map table with the hash id value of the kernel stack as the key. If so, add the memory size applied for this time to the original total value and record an increase in the total application count by 1. If not, add a total size of the memory size applied for this time, a total application count of 1, and record the kernel stack and its hash id value;

[0067] Instrument the release functions kfree and kmem_cache_free to sample the memory address released;

[0068] Query whether there is such a value in the slab_map table with the memory address as the key. If not, return; if so, obtain the stackid, query in the slab_total_map table with the stackid as the key, subtract the memory size value released this time from the total total value of the data, record the decrease in the total application count, and delete this data item in the slab_map table.

[0069] Further, if the collected MemFree values all exceed the corresponding third threshold value, enable the page leak detection function for memory allocation and release, including:

[0070] Instrument the allocation function mm_page_alloc to obtain sampling information, where the sampling information includes: the process name, pid, application address, kernel stack, hash id value of the kernel stack, and the memory size applied for;

[0071] Using the application address as the key, store the sampling information in the slab_map table;

[0072] Using the hash id value of the kernel stack as the key, query in the slab_total_map table to see if there is such a value. If there is, add the memory size applied this time to the original total value and increment the total application count by 1. If not, add a new entry with the total size being the memory size applied this time, the total application count being 1, and record the kernel stack and its hash id value;

[0073] Instrument the release function mm_page_free to sample the released memory address;

[0074] Query in the slab_map table using the memory address as the key to see if there is such a value. If not, return. If there is, obtain the stackid, query in the slab_total_map table using the stackid as the key, subtract the memory size released this time from the total value of the data, decrement the total application count, and delete this entry in the slab_map table.

[0075] Furthermore, analyzing memory allocation and release activities to generate a memory leak report for memory leak localization can specifically include:

[0076] Traverse the map table with the stackid as the key, compare the total application size or count with the threshold value. If it exceeds the threshold value, print the kernel stack to a file, and traverse the map table with the address or page as the key, filter the map table based on whether the stackid is the same, and print system information such as the process name and process ID to the file to generate a memory leak report for memory leak localization.

[0077] During the memory leak localization process, the kernel memory leak point can be accurately located based on detailed call stack information, memory allocation paths, and information such as the processes used.

[0078] It should also be noted that the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent in such process, method, commodity or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, commodity or device including the element. Words such as first, second, etc. are used to denote names and do not denote any particular order. The present invention and its implementation manners have been schematically described above. The description is not restrictive. Without departing from the spirit or basic features of the present invention, the present invention can be implemented in other specific forms. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto. Any reference signs in the claims should not limit the claims involved. Therefore, if those of ordinary skill in the art are inspired thereby and design similar structural manners and embodiments without creative efforts without departing from the purpose of this creation, they shall fall within the protection scope of this patent.

Claims

1. A method for detecting memory leaks in the Linux kernel, characterized in that, Including: Collecting the usage metrics of the system kernel memory through a monitoring program; Determining whether a memory leak has occurred according to the usage metrics; In response to determining that a memory leak has occurred, enabling the eBPF detection program to monitor the memory allocation and release activities in the kernel, and analyzing the memory allocation and release activities to generate a memory leak report for locating the memory leak.

2. The method for detecting Linux kernel memory leakage according to claim 1, wherein The usage metrics include at least one of the following: the VmallocUsed value representing the total amount of kernel virtual memory, the SUnreclaim value representing the total amount of non-reclaimable kernel memory, and the MemFree value representing the total amount of unallocated physical memory; The determining whether a memory leak has occurred according to the usage metrics includes: If the usage metrics obtained from consecutive multiple collections are all greater than the threshold values corresponding to the usage metrics, it is determined that a memory leak has occurred.

3. The method for detecting memory leakage in the Linux kernel according to claim 2, wherein The threshold values are the average values of the usage metrics collected multiple times in the case of no memory leak, including: collecting the VmallocUsed value, SUnreclaim value, and MemFree value under / proc / meminfo multiple times, and respectively taking the average values of the collected VmallocUsed value, SUnreclaim value, and MemFree value as the first threshold value, the second threshold value, and the third threshold value.

4. A method for detecting memory leaks in the Linux kernel according to claim 3, characterized in that, Enabling the eBPF detection program to monitor the memory allocation and release activities in the kernel includes: The eBPF detection program includes a vmalloc leak detection function, a slab leak detection function, and a page leak detection function; If the collected vamllocused values all exceed the corresponding first threshold value, enable the vmalloc leak detection function for memory allocation and release; If the collected SUnreclaim values all exceed the corresponding second threshold value, enable the slab leak detection function for memory allocation and release; If the collected MemFree values all exceed the corresponding third threshold value, enable the page leak detection function for memory allocation and release.

5. A method for detecting memory leaks in the Linux kernel according to claim 4, characterized in that, If the collected vamllocused values all exceed the corresponding first threshold value, enabling the vmalloc leak detection function for memory allocation and release includes: Instrumenting the allocation function vmalloc to obtain sampling information, where the sampling information includes: the process name, pid, application address, kernel stack, hashid value of the kernel stack, and the size of the allocated memory; Storing the sampling information into the slab_map table with the application address as the key; Using the hash id value of the kernel stack as the key, querying in the slab_total_map table to see if there is such a value. If there is, add the size of the currently allocated memory to the original total value and increment the total number of allocation times by 1. If not, add the total size as the size of the currently allocated memory, the total number of allocation times as 1, and record the kernel stack and its hashid value; Instrumenting the release function vmfree to sample the released memory address; Query whether there is such a value in the slab_map table with the memory address as the key. If not, return. If so, obtain the stackid, query in the slab_total_map table with the stackid as the key, subtract the memory size value released this time from the total total value of the data, record the total number of applications decreased, and delete this data item in the slab_map table.

6. A method for detecting memory leaks in the Linux kernel according to claim 5, characterized in that, If the collected SUnreclaim values all exceed the corresponding second threshold value, enable the slab leak detection function for memory allocation and release, including: Instrument the allocation functions kmem_cache_alloc, kmalloc, kmalloc_node to obtain sampling information, where the sampling information includes: the process name of the allocated memory, pid, application address, kernel stack, hashid value of the kernel stack, and the size of the allocated memory; Store the sampling information in the slab_map table with the application address as the key; Query whether there is such a value in the slab_total_map table with the hash id value of the kernel stack as the key. If so, add the size of the memory applied this time to the original total value and record the total number of applications increased by 1. If not, add the total size as the size of the memory applied this time, the total number of applications as 1, record the kernel stack and the hashid value of the kernel stack; Instrument the release functions kfree, kmem_cache_free to sample the released memory address; Query whether there is such a value in the slab_map table with the memory address as the key. If not, return. If so, obtain the stackid, query in the slab_total_map table with the stackid as the key, subtract the memory size value released this time from the total total value of the data, record the total number of applications decreased, and delete this data item in the slab_map table.

7. A method for detecting memory leaks in the Linux kernel according to claim 6, characterized in that, If the collected MemFree values all exceed the corresponding third threshold value, enable the page leak detection function for memory allocation and release, including: Instrument the allocation function mm_page_alloc to obtain sampling information, where the sampling information includes: the process name of the allocated memory, pid, application address, kernel stack, hashid value of the kernel stack, and the size of the allocated memory; Store the sampling information in the slab_map table with the application address as the key; Query whether there is such a value in the slab_total_map table with the hash id value of the kernel stack as the key. If so, add the size of the memory applied this time to the original total value and record the total number of applications increased by 1. If not, add the total size as the size of the memory applied this time, the total number of applications as 1, record the kernel stack and the hashid value of the kernel stack; Instrument the release function mm_page_free to sample the released memory address; Query whether there is such a value in the slab_map table with the memory address as the key. If not, return; if so, obtain the stackid, query in the slab_total_map table with the stackid as the key, subtract the memory size value released this time from the total total value of the data, record the reduction of the total application times, and delete this data item in the slab_map table.

8. A method for detecting memory leaks in the Linux kernel according to claim 7, characterized in that, Analyze memory allocation and release activities to generate a memory leak report for locating memory leaks, including: Traverse the map table with the stackid as the key, compare the total application size or times with the threshold value. If it exceeds the threshold value, print the kernel stack to a file, and traverse the map table with the address or page as the key, filter the map table according to whether the stackid is the same, and print the process name and process ID to the file to generate a memory leak report for locating memory leaks.

Images