DPDK large page memory leak detection method, device and system
The eBPF user space probe receives and processes memory change information of DPDK large page memory in the kernel, and solves the problem that the existing technology cannot detect DPDK large page memory leaks, realizes efficient memory leak detection without intrusion, and improves detection Reliability and convenience.
Patent Information
- Application Number
- CN202311404599.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-26
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-10-26
AI Technical Summary
Existing memory detection tools cannot effectively detect memory leaks on DPDK large pages, and they need to intrude into the user-side applications and occupy a large amount of resources.
The eBPF user space probe is used to receive memory change information of DPDK large page memory in the kernel, update the memory record table, and send the identification information to the user side when a suspected leakage is detected, realizing non-invasive DPDK large page memory leak detection.
It realizes effective leakage detection of DPDK large page memory, without intrusion into user-side applications, reduces resource occupancy, and improves the reliability, convenience and timeliness of detection.
Smart Images

Figure CN117574354B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of memory detection technology, and in particular to a DPDK large page memory leak detection method, device and system. Background Art
[0002] DPDK (Data Plane Development Kit) is a data plane development tool set provided by Intel, which provides library functions and driver support for efficient data packet processing in user space. One of the key technologies of DPDK is to reduce memory access overhead (cache miss) by using huge pages of memory (Hugepages). Therefore, DPDK does not use conventional memory allocation functions, such as "malloc()". DPDK manages pre-allocated huge pages of memory through its own API. DPDK can greatly improve data processing performance and throughput by using multi-core CPUs, and improve the work efficiency of data plane applications. However, it also makes it impossible to apply the original debugging methods based on ordinary applications to DPDK applications.
[0003] At present, most memory detection tools, such as ASAN (Address Sanitizer), usually apply for a shadow memory to record the availability of the main memory used by the user program, reflecting whether the access to each byte in the real memory is safe. Shadow memory cannot be directly read and written by users, and the compiler needs to generate relevant code to access it. The allocation and release of memory will be written to the shadow memory, and the legality of the access will be judged by the shadow memory before each reading and writing; then, by establishing a red zone marked as poisoned around the protected heap, stack, and global variables, the memory is not released directly when it is released, but marked and isolated, so that when it is accessed again, the abnormality can be found according to the status.
[0004] However, the existing memory detection tools based on the above principles cannot be effectively applied to DPDK large page memory leak detection. The specific reason is: DPDK detects memory by inserting memory application release functions during the compilation phase, setting red zones before and after the memory when the program applies for memory, and adding it to the shadow area. Even with a certain compression ratio, the shadow memory requires a large continuous virtual address space. However, it can only detect stack overflows, illegal pointer references and other problems, but cannot detect whether there is a memory leak on the large page heap. In addition, these solutions often require adding some debugging code or additional compilation options in the code or at compile time, and cannot be non-invasive to the application.
[0005] Therefore, there is an urgent need to design a memory detection method that can realize DPDK large page memory leak detection, does not need to invade the user-side application, and can reduce the application resource occupancy rate required for DPDK large page memory leak detection. Summary of the invention
[0006] In view of this, embodiments of the present application provide a DPDK large page memory leak detection method, device, and system to eliminate or improve one or more defects in the prior art.
[0007] One aspect of the present application provides a DPDK large page memory leak detection method, including:
[0008] Receive memory change information for DPDK hugepage memory in the kernel based on the eBPF userspace probe preset on the DPDK hugepage memory stack on the user side;
[0009] A memory record table preset in the kernel is updated according to the memory change information, and when suspected leaked memory is detected in the memory record table, identification information corresponding to the suspected leaked memory is sent to the user side for output.
[0010] In some embodiments of the present application, the memory change information includes: memory allocation information;
[0011] Correspondingly, the eBPF user space probe based on the DPDK large page memory stack preset on the user side receives memory change information for the DPDK large page memory in the kernel, including:
[0012] When the DPDK large page memory is called, based on the eBPF user space probe preset on the DPDK large page memory stack, the memory allocation information for the DPDK large page memory is obtained from the interface of the DPDK large page memory stack, wherein the memory allocation information includes: the stack ID, memory address and memory size to which the called memory belongs.
[0013] In some embodiments of the present application, the memory change information includes: memory release information;
[0014] Correspondingly, the eBPF user space probe based on the DPDK large page memory stack preset on the user side receives memory change information for the DPDK large page memory in the kernel, including:
[0015] When the DPDK large page memory is released, based on the eBPF user space probe preset on the DPDK large page memory stack, the memory release information for the DPDK large page memory is obtained from the interface of the DPDK large page memory stack, wherein the memory release information includes: a memory address to which the released memory belongs.
[0016] In some embodiments of the present application, the memory record table includes: an allocation information hash table and a stack trace table;
[0017] Correspondingly, updating a memory record table preset in the kernel according to the memory change information includes:
[0018] If the memory allocation information is received, a corresponding relationship between the stack ID, memory address and memory size to which the called memory belongs is added to the allocation information hash table to form a memory allocation record in the allocation information hash table, and the stack tracking table is updated accordingly;
[0019] If the memory release information is received, the memory allocation record corresponding to the memory address to which the released memory belongs in the memory release information is searched in the allocation information hash table, the memory allocation record is deleted in the allocation information hash table, and the stack tracking table is updated accordingly.
[0020] In some embodiments of the present application, before sending the identification information corresponding to the suspected leaked memory to the user side for output, the method further includes:
[0021] Receive a memory leak detection request sent by the user to the allocation information hash table, and search the allocation information hash table for whether it contains continuous memory allocation records for the same stack ID based on the memory leak detection request. If so, determine that the DPDK large page memory stack corresponding to the stack ID is currently suspected of having a memory leak.
[0022] In some embodiments of the present application, when the suspected leaked memory is detected in the memory record table, sending identification information corresponding to the suspected leaked memory to the user side for output includes:
[0023] Extracting identification information corresponding to the DPDK large page memory stack currently suspected of having a memory leak from the allocation information hash table and the stack tracking table, the identification information including all memory allocation records of the DPDK large page memory stack currently suspected of having a memory leak and context information of the DPDK large page memory stack;
[0024] The identification information corresponding to the DPDK large page memory stack currently suspected of having a memory leak is sent to the user side to update the mapping table on the user side, so that the user side outputs the mapping table for display.
[0025] In some embodiments of the present application, before the eBPF user space probe based on the DPDK large page memory stack preset on the user side receives the memory change information for the DPDK large page memory in the kernel, it also includes:
[0026] Set eBPF user space probes on the allocation memory interface and release content interface corresponding to each DPDK huge page memory stack on the user side;
[0027] And, a memory record table for storing memory change information for the DPDK large page memory is constructed in the kernel.
[0028] Another aspect of the present application provides a DPDK large page memory leak detection device, including:
[0029] The probe monitoring module is used to receive memory change information for DPDK huge page memory in the kernel based on the eBPF user space probe preset on the DPDK huge page memory stack on the user side;
[0030] The leakage detection module is used to update the memory record table preset in the kernel according to the memory change information, and when suspected leaked memory is detected in the memory record table, send the identification information corresponding to the suspected leaked memory to the user side for output.
[0031] A third aspect of the present application provides a DPDK large page memory leak detection system, including: an eBPF backend program unit set in the kernel and an eBPF frontend program unit set on the user side;
[0032] The eBPF backend program unit is used to execute the DPDK large page memory leak detection method;
[0033] The eBPF front-end program unit is used to obtain identification information corresponding to the suspected leaked memory and output it on the user side when detecting that there is suspected leaked memory in the memory record table.
[0034] In some embodiments of the present application, the eBPF front-end program unit is specifically used to perform the following:
[0035] Sending a memory leak detection request to the memory record table of the kernel in real time to search in the memory record table whether there is a continuous memory allocation record for the same stack ID based on the memory leak detection request, and if so, determining the DPDK large page memory stack corresponding to the stack ID as a suspected memory leak;
[0036] Obtain identification information corresponding to the DPDK large page memory stack that is currently suspected of having a memory leak from the memory record table in the kernel, update the mapping table on the user side accordingly, and output the mapping table for display, wherein the identification information includes all memory allocation records of the DPDK large page memory stack that is currently suspected of having a memory leak and context information of the DPDK large page memory stack.
[0037] A fourth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the DPDK large page memory leak detection method when executing the computer program.
[0038] A fifth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the DPDK large page memory leak detection method when executed by a processor.
[0039] The DPDK large page memory leak detection method provided by the present application is based on an eBPF user space probe preset on the DPDK large page memory stack on the user side, and receives memory change information for the DPDK large page memory in the kernel; updates the memory record table preset in the kernel according to the memory change information, and when suspected leaked memory is detected in the memory record table, sends the identification information corresponding to the suspected leaked memory to the user side for output, which can effectively realize the leakage detection of the DPDK large page memory without intruding the application on the user side, and can reduce the application resource occupancy rate required for the DPDK large page memory leak detection, thereby effectively improving the reliability, convenience and timeliness of the DPDK large page memory leak detection, and can effectively improve the operation security of the application.
[0040] Additional advantages, purposes, and features of the present application will be partially described in the following description, and will become partially apparent to those skilled in the art after studying the following, or may be learned from the practice of the present application. The purposes and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification and the drawings.
[0041] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description, and the above and other purposes that can be achieved by the present application will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The drawings described herein are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. The components in the drawings are not drawn to scale, but are only for the purpose of illustrating the principles of the present application. In order to facilitate the illustration and description of some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, they may become larger relative to other components in the exemplary device actually manufactured according to the present application. In the drawings:
[0043] Figure 1 This is a first flow chart of a DPDK large page memory leak detection method in an embodiment of the present application.
[0044] Figure 2 This is a second flow chart of a DPDK large page memory leak detection method in an embodiment of the present application.
[0045] Figure 3 Schematic diagram of the structure of a DPDK large page memory leak detection device in one embodiment of the present application.
[0046] Figure 4 Schematic diagram of the structure of a DPDK large page memory leak detection system in an embodiment of the present application.
[0047] Figure 5 This is a schematic diagram of the contents executed by the eBPF front-end program unit in the DPDK large page memory leak detection system in one embodiment of the present application.
[0048] Figure 6 An example diagram of the DPDK large page memory leak detection process during the allocation and release of DPDK large page memory provided for the application example of this application. DETAILED DESCRIPTION
[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the implementation modes and the accompanying drawings. Here, the illustrative implementation modes and descriptions of the present application are used to explain the present application, but are not intended to limit the present application.
[0050] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the scheme according to the present application are shown in the accompanying drawings, while other details that are not very relevant to the present application are omitted.
[0051] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.
[0052] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.
[0053] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.
[0054] In order to solve the problems existing in the existing memory detection method, such as the inability to detect DPDK large page memory leaks, the need to invade the application on the user side and the need to occupy the virtual address space of the application, the embodiments of the present application respectively provide a DPDK large page memory leak detection method, a DPDK large page memory leak detection device for executing the DPDK large page memory leak detection method, a physical device and a computer-readable storage medium, and also provide a DPDK large page memory leak detection system including an eBPF back-end program unit set in the kernel and an eBPF front-end program unit set on the user side, which can realize DPDK large page memory leak detection without invading the application on the user side and can reduce the application resource occupancy rate required for DPDK large page memory leak detection.
[0055] The details are described in detail through the following examples.
[0056] Based on this, the embodiment of the present application provides a DPDK large page memory leak detection method that can be implemented by a DPDK large page memory leak detection device, see Figure 1 The DPDK large page memory leak detection method specifically includes the following contents:
[0057] Step 100: Based on the eBPF user space probe preset on the DPDK large page memory stack on the user side, memory change information for the DPDK large page memory is received in the kernel.
[0058] It should be noted that in one or more embodiments of the present application, eBPF (Extended Berkeley Packet Filter) is a powerful network and performance analysis tool on the Linux kernel, which allows developers to dynamically load, update and run user-defined code while the kernel is running. The use of eBPF greatly improves the observability of the kernel and applications.
[0059] Among them, eBPF (Extended Berkeley Packet Filter) user space probe refers to the user space probe of eBPF, namely Uprobe (User-space probes), which is a tracing technology application based on eBPF. Uprobe probes allow dynamic insertion in user space programs. The insertion locations include: function entry, specific offset, and function return. When an Uprobe is defined, the kernel will create a fast breakpoint instruction on the attached instruction. When the program executes the instruction, the kernel will trigger an event, the program will fall into the kernel state, and call the probe function as a callback function. After executing the probe function, it will return to the user state to continue to execute the subsequent instructions. It should be noted that the combination of DPDK large pages and eBPF features can solve the problem of difficult memory positioning, but the existing technology cannot realize the detection of DPDK large page memory leaks, and it is also impossible to solve the security problems caused by attacking applications using memory vulnerabilities.
[0060] In one or more embodiments of the present application, the user side specifically refers to the application side in user mode. The kernel refers to the CPU kernel state. The application scenario corresponding to the DPDK large page memory leak detection method mentioned in the present application is a general scenario using DPDK, such as DPU, etc.
[0061] Step 200: updating a memory record table preset in the kernel according to the memory change information, and when detecting that there is suspected leaked memory in the memory record table, sending identification information corresponding to the suspected leaked memory to the user side for output.
[0062] In step 200, the operation of updating the memory record table may include adding a new record or deleting a record in the memory record table, which needs to be processed according to the specific content of the memory change information. The specific processing method will be described in detail in the subsequent embodiments.
[0063] Among them, the process of detecting whether there is suspected leaked memory in the memory record table can be executed by the DPDK large page memory leak detection device of the kernel, or other program modules on the user side can access the memory record table in the kernel to detect whether there is suspected leaked memory in the memory record table.
[0064] In one or more embodiments of the present application, the suspected leaked memory may refer to memory that has been allocated but not released. Specifically, if continuous memory allocation records appear for the same stack ID, it can be considered that the DPDK large page memory stack corresponding to the stack ID is suspected of having a memory leak.
[0065] It can be understood that the DPDK large page memory stack mentioned in the embodiments of the present application specifically refers to the DPDK large page memory heap and / or the DPDK large page memory stack, which can be set according to the specific application scenario.
[0066] From the above description, it can be seen that the DPDK large page memory leak detection method provided in the embodiment of the present application can effectively realize the leakage detection of DPDK large page memory without invading the application on the user side, and can reduce the application resource occupancy rate required for DPDK large page memory leak detection, thereby effectively improving the reliability, convenience and timeliness of DPDK large page memory leak detection.
[0067] In order to further improve the timeliness and reliability of memory allocation monitoring during the DPDK large page memory leak detection process, in a DPDK large page memory leak detection method provided in an embodiment of the present application, the memory change information includes: memory allocation information;
[0068] For the corresponding Figure 2 , step 100 in the DPDK large page memory leak detection method specifically includes the following contents:
[0069] Step 110: When the DPDK large page memory is called, based on the eBPF user space probe preset on the DPDK large page memory stack, the memory allocation information for the DPDK large page memory is obtained from the interface of the DPDK large page memory stack, wherein the memory allocation information includes: the stack ID, memory address and memory size to which the called memory belongs.
[0070] Specifically, the obtaining of the memory allocation information for the DPDK large page memory from the interface of the DPDK large page memory stack based on the eBPF user space probe preset on the DPDK large page memory stack can specifically refer to: when a user calls the DPDK large page memory, the memory size of the DPDK large page memory stack requested by the user is obtained based on the eBPF user space probe preset at the entry of the interface API of the DPDK large page memory stack with a known stack ID; if the application is successful, the address of the memory allocated to the user by the DPDK large page memory stack is obtained based on the eBPF user space probe preset at the exit of the interface API of the DPDK large page memory stack.
[0071] In order to further improve the timeliness and reliability of memory release monitoring during the DPDK large page memory leak detection process, in a DPDK large page memory leak detection method provided in an embodiment of the present application, the memory change information includes: memory release information;
[0072] For the corresponding Figure 2, step 100 in the DPDK large page memory leak detection method further specifically includes the following contents:
[0073] Step 120: When the DPDK large page memory is released, based on the eBPF user space probe preset on the DPDK large page memory stack, the memory release information for the DPDK large page memory is obtained from the interface of the DPDK large page memory stack, wherein the memory release information includes: the memory address to which the released memory belongs.
[0074] Specifically, for large page memory release, at the entry of the release API, the eBPF kernel program obtains the parameters of the API, that is, the address of the memory to be released.
[0075] In order to further improve the comprehensiveness and effectiveness of memory records in the process of DPDK large page memory leak detection, in a DPDK large page memory leak detection method provided in an embodiment of the present application, the memory record table includes: an allocation information hash table and a stack trace table;
[0076] For the corresponding Figure 2 In the DPDK large page memory leak detection method, step 200 further specifically includes the following contents executed after step 110:
[0077] Step 210: If the memory allocation information is received, the corresponding relationship between the stack ID, memory address and memory size of the called memory is added to the allocation information hash table to form a memory allocation record in the allocation information hash table, and the stack tracking table is updated accordingly.
[0078] The stack trace table is used to store stack calls of various DPDK large page memory applications and is referenced by a keyword ID (such as a stack ID of each DPDK large page memory stack).
[0079] For the corresponding Figure 2 In the DPDK large page memory leak detection method, step 200 further specifically includes the following contents executed after step 120:
[0080] Step 220: If the memory release information is received, search the memory allocation record corresponding to the memory address to which the released memory in the memory release information belongs in the allocation information hash table, delete the memory allocation record in the allocation information hash table, and update the stack trace table accordingly.
[0081] At the same time, in order to further improve the reliability of detecting whether there is suspected leaked memory in the memory record table, in a DPDK large page memory leak detection method provided in an embodiment of the present application, see Figure 2In the DPDK large page memory leak detection method, step 200 further specifically includes the following contents executed after step 210 and step 220:
[0082] Step 230: Receive a memory leak detection request sent by the user to the allocation information hash table, and search the allocation information hash table based on the memory leak detection request to see whether it contains continuous memory allocation records for the same stack ID. If so, the DPDK large page memory stack corresponding to the stack ID is determined to be suspected of having a memory leak.
[0083] In order to further improve the timeliness and convenience of users in learning suspected DPDK large page memory leaks, in a DPDK large page memory leak detection method provided in an embodiment of the present application, see Figure 2 In the DPDK large page memory leak detection method, step 200 also specifically includes the following contents executed after step 230:
[0084] Step 240: Extract identification information corresponding to the DPDK large page memory stack that is currently suspected of having a memory leak from the allocation information hash table and the stack tracking table, where the identification information includes all memory allocation records of the DPDK large page memory stack that is currently suspected of having a memory leak and context information of the DPDK large page memory stack.
[0085] Specifically, the embodiment of the present application combines the ability of eBPF to trace the function call stack and records the context information of the leak point when a large page memory leak occurs in the application, thereby making it easier to discover the problem.
[0086] Step 250: Send identification information corresponding to the DPDK large page memory stack currently suspected of having a memory leak to the user side to update the mapping table of the user side, so that the user side outputs the mapping table for display.
[0087] In order to further improve the timeliness and reliability of DPDK large page memory leak detection, in a DPDK large page memory leak detection method provided in an embodiment of the present application, see Figure 2 The DPDK large page memory leak detection method further includes the following contents before step 100:
[0088] Step 010: Set eBPF user space probes on the allocation memory interface and release content interface corresponding to each DPDK large page memory stack on the user side.
[0089] Step 020: Construct a memory record table in the kernel for storing memory change information for the DPDK large page memory.
[0090] From the software level, the present application also provides a DPDK large page memory leak detection device for executing all or part of the DPDK large page memory leak detection method, see Figure 3 The DPDK large page memory leak detection device specifically includes the following contents:
[0091] The probe monitoring module 10 is used to receive memory change information for the DPDK huge page memory in the kernel based on the eBPF user space probe preset on the DPDK huge page memory stack on the user side;
[0092] The leakage detection module 20 is used to update the memory record table preset in the kernel according to the memory change information, and when suspected leaked memory is detected in the memory record table, send the identification information corresponding to the suspected leaked memory to the user side for output.
[0093] The embodiment of the DPDK large page memory leak detection device provided in the present application may specifically be a backend program set in the kernel, such as an eBPF backend program unit.
[0094] The DPDK large page memory leak detection device provided in the present application is used to execute the processing flow of the embodiment of the DPDK large page memory leak detection method in the above embodiment. Its functions are not repeated here, and reference can be made to the detailed description of the above DPDK large page memory leak detection method embodiment.
[0095] From the above description, it can be seen that the DPDK large page memory leak detection device provided in the embodiment of the present application can effectively realize the leakage detection of DPDK large page memory without invading the application on the user side, and can reduce the application resource occupancy rate required for DPDK large page memory leak detection, thereby effectively improving the reliability, convenience and timeliness of DPDK large page memory leak detection.
[0096] Based on the aforementioned DPDK large page memory leak detection method and / or DPDK large page memory leak detection device embodiment, the present application also provides a DPDK large page memory leak detection system specific application example, see Figure 4 The DPDK large page memory leak detection system specifically includes the following contents:
[0097] An eBPF backend program unit 1 provided in the kernel and an eBPF frontend program unit 2 provided on the user side;
[0098] The eBPF backend program unit 1 is used to execute the DPDK large page memory leak detection method in the aforementioned embodiment;
[0099] The eBPF front-end program unit 2 is used to obtain identification information corresponding to the suspected leaked memory and output it on the user side when detecting that there is suspected leaked memory in the memory record table.
[0100] In order to further improve the reliability and effectiveness of DPDK large page memory leak detection, in the DPDK large page memory leak detection system provided in the embodiment of the present application, see Figure 5 The eBPF front-end program unit 2 in the DPDK large page memory leak detection system is specifically used to execute the following contents:
[0101] Step 300: Send a memory leak detection request to the memory record table of the kernel in real time to search the memory record table for whether there is a continuous memory allocation record for the same stack ID based on the memory leak detection request. If so, the DPDK large page memory stack corresponding to the stack ID is determined to be suspected of having a memory leak.
[0102] Step 400: Obtain identification information corresponding to the DPDK large page memory stack that is currently suspected of having a memory leak from the memory record table in the kernel and update the mapping table on the user side accordingly, and output the mapping table for display, wherein the identification information includes all memory allocation records of the DPDK large page memory stack that is currently suspected of having a memory leak and context information of the DPDK large page memory stack.
[0103] Based on the above embodiments, in order to further illustrate the DPDK large page memory leak detection method provided by the present application, the present application also provides a specific application example of using the above-mentioned DPDK large page memory leak detection system to perform the DPDK large page memory leak detection method. Specifically, the application example of the present application is aimed at the DPDK application program being unable to use effective means to debug memory-related problems during the use of large page memory. A system and method for detecting large page memory leaks in DPDK based on eBPF technology is proposed, that is, using the Uprobe dynamic tracking technology of eBPF to find out the leak points when the application program uses large page memory. In addition, combined with the ability of eBPF to track the function call stack, when a large page memory leak occurs in the application program, the context information of the leak point is recorded, so that the problem can be found more simply and conveniently.
[0104] First, before executing the DPDK large page memory leak detection process, the preprocessing steps provided by the application example of this application are as follows:
[0105] S0.1: Define the eBPF allocation information hash table, which can be abbreviated as the allocation hash table, and the stack trace table in the eBPF backend program (i.e., the program running in the kernel).
[0106] S0.2: Add Uprobe observation points to the memory allocation APIs (such as rte_malloc_socket and rte_realloc_socket) and memory release APIs (such as rte_free) on the DPDK large page memory heap, and insert eBPF backend tracking programs at the API entry and return.
[0107] Then, see Figure 6 The application example of this application is based on the DPDK large page memory allocation and release provided by the eBPF front-end program and the eBPF back-end program. The DPDK large page memory leak detection process is as follows:
[0108] S1.1: The application allocates large pages of memory.
[0109] S1.2: The memory request operation is captured by Uprobe.
[0110] S1.3: Record the allocation information and add it to the allocation hash table.
[0111] Among them, an example of allocating a hash table is shown in Table 1:
[0112] Table 1
[0113]
[0114] In Table 1, each row is used to store a different memory allocation record, where N is a positive integer greater than 3.
[0115] That is to say, in S1.1 to S1.3, for large page memory allocation, at the allocation API entry, the size of the memory requested for allocation is recorded. At the return point, if the application is successful, the requested address is recorded, and the address is used as the keyword of the allocation hash table. The size, time, and call stack ID of this allocation are added to the allocation hash table as values.
[0116] S2.1: The application releases huge page memory.
[0117] S2.2: The memory release operation is captured by Uprobe.
[0118] S2.3: Search in the allocation hash table based on the released information.
[0119] S2.4: Delete the entry in the allocation Hash table. An example of the allocation Hash table after deletion is shown in Table 2:
[0120] Table 2
[0121]
[0122] That is to say, in S2.1 to S2.4, for the release of large page memory, at the release API entry, the eBPF kernel program obtains the parameters of the API, that is, the address of the memory to be released, and searches the allocation hash table based on the address, and deletes the corresponding item in the hash table.
[0123] S0.3: For the large page memory reallocation release interface (such as rte_realloc_socket), the eBPF program obtains the allocated address as the keyword, searches the allocation hash table, deletes the corresponding item in the hash table, and then uses the newly allocated address as the keyword of the hash table to update the hash table.
[0124] S3.1: The eBPF (also written as ebdf) front-end program traverses the kernel allocation hash table.
[0125] Specifically, in the eBPF front-end program (the program running in user space), the allocation hash table and stack tracking table of the eBPF backend program are obtained, the allocation hash table is traversed, and a new mapping is created with the stack ID in the stack tracking table as the keyword. The mapped value is the number of times the stack allocates memory and the total size of the allocated memory.
[0126] The mapped value is embodied in a mapping table, and an example of the mapping table is shown in Table 3:
[0127] Table 3
[0128]
[0129] S3.2: eBPF frontend program displays the results.
[0130] Specifically, the mapping generated by the eBPF front-end program is printed out. During the running of the DPDK program, if there is memory that has been allocated but not released, the allocated stack will be displayed.
[0131] It can be seen that the application example of this application performs uprobe tracking on the API interface for large page memory allocation and release, and finds the call stack that has been applying for memory but not releasing memory during the program running, so as to find the point of memory leak; after the DPDK application has been running for a period of time, because the allocation and release API interfaces are tracked, if the application has a memory leak, the eBPF program will record the size of the memory leak and the call stack where the leak occurred, and print these out, so that the memory leak problem can be clearly observed.
[0132] In summary, according to the DPDK large page memory leak detection method provided by the application example of this application, the allocation of large page memory of the DPDK application is tracked by eBPF, and the large page memory usage in the DPDK application is obtained, which can effectively monitor whether there is a memory leak, thereby solving the current problem of being unable to effectively locate the large page memory leak in the DPDK application. It has the following advantages:
[0133] 1. No additional operations are required on the application code or during compilation, so there is no modification to the application.
[0134] 2. When a memory leak occurs, the program's call stack can be recorded to facilitate problem analysis.
[0135] The embodiment of the present application also provides an electronic device, which may include a processor, a memory, a receiver and a transmitter, wherein the processor is used to execute the DPDK large page memory leak detection method mentioned in the above embodiment, wherein the processor and the memory may be connected via a bus or other means, such as by bus connection. The receiver may be connected to the processor and the memory via wired or wireless means.
[0136] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.
[0137] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the DPDK large page memory leak detection method in the embodiment of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the DPDK large page memory leak detection method in the above method embodiment is implemented.
[0138] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0139] The one or more modules are stored in the memory, and when executed by the processor, the DPDK large page memory leak detection method in the embodiment is executed.
[0140] In some embodiments of the present application, the user equipment may include a processor, a memory, and a transceiver unit, which may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.
[0141] As an implementation method, the functions of the receiver and the transmitter in the present application can be considered to be implemented through a transceiver circuit or a dedicated chip for transceiver, and the processor can be considered to be implemented through a dedicated processing chip, a processing circuit or a general chip.
[0142] As another implementation method, it is possible to use a general-purpose computer to implement the server provided in the embodiment of the present application, that is, to store the program code for implementing the functions of the processor, receiver, and transmitter in a memory, and the general-purpose processor implements the functions of the processor, receiver, and transmitter by executing the code in the memory.
[0143] The embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned DPDK large page memory leak detection method. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.
[0144] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.
[0145] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0146] In this application, the features described and / or illustrated for one embodiment may be used in the same manner or in a similar manner in one or more other embodiments, and / or combined with or replace the features of other embodiments.
[0147] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A DPDK large page memory leak detection method, characterized in that: include: Set eBPF user space probes on the allocation memory interface and release content interface corresponding to each DPDK huge page memory stack on the user side; Receive memory change information for DPDK hugepage memory in the kernel based on the eBPF userspace probe preset on the DPDK hugepage memory stack on the user side; According to the memory change information, a memory record table preset in the kernel is updated, and when suspected leaked memory is detected in the memory record table, identification information corresponding to the suspected leaked memory is sent to the user side for output; The memory change information includes: memory allocation information; Correspondingly, the eBPF user space probe based on the DPDK large page memory stack preset on the user side receives memory change information for the DPDK large page memory in the kernel, including: When the DPDK large page memory is called, based on the eBPF user space probe preset on the DPDK large page memory stack, the memory allocation information for the DPDK large page memory is obtained from the interface of the DPDK large page memory stack, wherein the memory allocation information includes: the stack ID, memory address and memory size to which the called memory belongs; The memory change information includes: memory release information; Correspondingly, the eBPF user space probe based on the DPDK large page memory stack preset on the user side receives memory change information for the DPDK large page memory in the kernel, including: When the DPDK large page memory is released, based on the eBPF user space probe preset on the DPDK large page memory stack, the memory release information for the DPDK large page memory is obtained from the interface of the DPDK large page memory stack, wherein the memory release information includes: a memory address to which the released memory belongs; The memory record table includes: an allocation information hash table and a stack tracking table; Correspondingly, updating a memory record table preset in the kernel according to the memory change information includes: If the memory allocation information is received, a corresponding relationship between the stack ID, memory address and memory size to which the called memory belongs is added to the allocation information hash table to form a memory allocation record in the allocation information hash table, and the stack tracking table is updated accordingly; If the memory release information is received, searching the memory allocation record corresponding to the memory address to which the released memory belongs in the memory release information in the allocation information hash table, deleting the memory allocation record in the allocation information hash table, and updating the stack tracking table accordingly; Before sending the identification information corresponding to the suspected leaked memory to the user side for output, the method further includes: Receive a memory leak detection request sent by the user to the allocation information hash table, and search the allocation information hash table for whether it contains continuous memory allocation records for the same stack ID based on the memory leak detection request. If so, determine that the DPDK large page memory stack corresponding to the stack ID is currently suspected of having a memory leak.
2. The DPDK large page memory leak detection method according to claim 1, characterized in that: When the suspected leaked memory is detected in the memory record table, the identification information corresponding to the suspected leaked memory is sent to the user side for output, including: Extracting identification information corresponding to the DPDK large page memory stack currently suspected of having a memory leak from the allocation information hash table and the stack tracking table, the identification information including all memory allocation records of the DPDK large page memory stack currently suspected of having a memory leak and context information of the DPDK large page memory stack; The identification information corresponding to the DPDK large page memory stack currently suspected of having a memory leak is sent to the user side to update the mapping table on the user side, so that the user side outputs the mapping table for display.
3. The DPDK large page memory leak detection method according to claim 1 or 2, characterized in that: Before the eBPF user space probe based on the DPDK large page memory stack preset on the user side receives the memory change information for the DPDK large page memory in the kernel, it also includes: And, a memory record table for storing memory change information for the DPDK large page memory is constructed in the kernel.
4. A DPDK large page memory leak detection device, characterized in that: include: The probe monitoring module is used to receive memory change information for DPDK huge page memory in the kernel based on the eBPF user space probe preset on the DPDK huge page memory stack on the user side; A leakage detection module, configured to update a memory record table preset in the kernel according to the memory change information, and when a suspected leaked memory is detected in the memory record table, send identification information corresponding to the suspected leaked memory to the user side for output; The DPDK large page memory leak detection device is also used to perform the following contents: Before receiving memory change information for DPDK huge page memory in the kernel, set up eBPF user space probes on the allocation memory interface and release content interface corresponding to each DPDK huge page memory stack on the user side; The memory change information includes: memory allocation information; Correspondingly, the eBPF user space probe based on the DPDK large page memory stack preset on the user side receives memory change information for the DPDK large page memory in the kernel, including: When the DPDK large page memory is called, based on the eBPF user space probe preset on the DPDK large page memory stack, the memory allocation information for the DPDK large page memory is obtained from the interface of the DPDK large page memory stack, wherein the memory allocation information includes: the stack ID, memory address and memory size to which the called memory belongs; The memory change information includes: memory release information; Correspondingly, the eBPF user space probe based on the DPDK large page memory stack preset on the user side receives memory change information for the DPDK large page memory in the kernel, including: When the DPDK large page memory is released, based on the eBPF user space probe preset on the DPDK large page memory stack, the memory release information for the DPDK large page memory is obtained from the interface of the DPDK large page memory stack, wherein the memory release information includes: a memory address to which the released memory belongs; The memory record table includes: an allocation information hash table and a stack tracking table; Correspondingly, updating a memory record table preset in the kernel according to the memory change information includes: If the memory allocation information is received, a corresponding relationship between the stack ID, memory address and memory size to which the called memory belongs is added to the allocation information hash table to form a memory allocation record in the allocation information hash table, and the stack tracking table is updated accordingly; If the memory release information is received, searching the memory allocation record corresponding to the memory address to which the released memory belongs in the memory release information in the allocation information hash table, deleting the memory allocation record in the allocation information hash table, and updating the stack tracking table accordingly; Before sending the identification information corresponding to the suspected leaked memory to the user side for output, the method further includes: Receive a memory leak detection request sent by the user to the allocation information hash table, and search the allocation information hash table for whether it contains continuous memory allocation records for the same stack ID based on the memory leak detection request. If so, determine that the DPDK large page memory stack corresponding to the stack ID is currently suspected of having a memory leak.
5. A DPDK large page memory leak detection system, characterized in that: include: An eBPF backend program unit set in the kernel and an eBPF frontend program unit set on the user side; The eBPF backend program unit is used to execute the DPDK large page memory leak detection method according to any one of claims 1 to 3; The eBPF front-end program unit is used to obtain identification information corresponding to the suspected leaked memory and output it on the user side when detecting that there is suspected leaked memory in the memory record table.
6. The DPDK large page memory leak detection system according to claim 5, characterized in that: The eBPF front-end program unit is specifically used to execute the following contents: Sending a memory leak detection request to the memory record table of the kernel in real time to search in the memory record table whether there is a continuous memory allocation record for the same stack ID based on the memory leak detection request, and if so, determining the DPDK large page memory stack corresponding to the stack ID as a suspected memory leak; Obtain identification information corresponding to the DPDK large page memory stack that is currently suspected of having a memory leak from the memory record table in the kernel, update the mapping table on the user side accordingly, and output the mapping table for display, wherein the identification information includes all memory allocation records of the DPDK large page memory stack that is currently suspected of having a memory leak and context information of the DPDK large page memory stack.
