Shared memory leak detection method based on multi-process coroutine model

By combining invasive and non-invasive detection modes under the multi-process coroutine model, tracking memory allocation and release information, the problem of the existing technology being unable to detect cross-process memory leaks is solved, and effective detection of multi-process shared memory is achieved. Improves detection flexibility and accuracy.

CN117971626BActive Publication Date: 2024-11-01TIANJIN NANKAI UNIV GENERAL DATA TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410362240.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-28
Publication Date
2024-11-01
Estimated Expiration
2044-03-28

AI Technical Summary

Technical Problem

The existing memory leak detection technology cannot detect memory leaks across processes, and cannot start and stop the entire system for observation at will in large systems, and cannot effectively detect memory leaks in shared memory under multiple processes.

Method used

Using a shared memory leak detection method based on a multi-process coroutine model, combining intrusive and non-invasive detection modes, the memory is judged and located by obtaining the initial and final state of the memory pool, tracking the address and stack information of the memory allocation and release functions. Leakage, realizes memory leak detection under multi-process coroutine model.

Benefits of technology

It realizes that memory leaks can be detected without modifying program codes under a multi-process and shared memory architecture, and can observe memory leaks without starting and stopping the program, improving the flexibility and accuracy of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117971626B_ABST
    Figure CN117971626B_ABST
Patent Text Reader

Abstract

The present invention provides a shared memory leak detection method based on a multi-process coroutine model, including an intrusive detection mode and a non-intrusive detection mode. The process of using the intrusive detection mode to perform memory leak detection is as follows: before executing the operation instruction of the database, obtain the usage of the memory pool where the current instruction is located to obtain the initial state of the memory pool; after the operation instruction of the database is executed, obtain the usage of the memory pool where the current instruction is located again to obtain the final state of the memory pool; compare the initial state and the final state of the memory pool, if the memory usage increases monotonically, it indicates that there is a memory leak; if the memory usage remains unchanged, it indicates that there is no memory leak. Beneficial effects of the present invention: the idea of ​​the commonly used memory leak detection tool is applied to the program of this architecture, and the memory leak of some instructions of the program can be observed without starting and stopping the program.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of computer technology, and in particular relates to a shared memory leak detection method based on a multi-process coroutine model. Background Art

[0002] The concept of "coroutine" was first proposed by Melvin Conway in 1958. A coroutine can be understood as a pure user-mode thread that switches through collaboration rather than preemption. Compared with processes or threads, all operations of a coroutine can be completed in user mode, and the creation and switching costs are lower. In general, coroutines provide a runtime abstraction for collaborative tasks, which is very suitable for collaborative multitasking scheduling and data stream processing. In modern operating systems and programming languages, because the switching cost of user-mode threads is lower than that of kernel-mode threads, coroutines have become a lightweight multitasking model.

[0003] However, the existing technology has the following shortcomings in the process of memory leak in coroutine mode:

[0004] Existing memory leak detection technology can only detect memory within a process, but cannot detect memory leaks across processes. In a multi-process shared memory architecture, existing technology cannot detect memory leaks.

[0005] Moreover, existing memory leak detection technologies can only check the scenario where memory is requested but not released from before the process is started to after it is stopped. However, for large systems, the entire system cannot be started and stopped at will, and only local observations can be performed. Summary of the invention

[0006] In view of this, the present invention aims to propose a shared memory leak detection method based on a multi-process coroutine model, in order to solve at least one of the above-mentioned technical problems.

[0007] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0008] The first aspect of the present invention provides a shared memory leak detection method based on a multi-process coroutine model, including an intrusive detection mode and a non-intrusive detection mode:

[0009] The process of performing memory leak detection using intrusive detection mode is as follows:

[0010] Before executing the database operation instruction, obtain the usage of the memory pool where the current instruction is located and obtain the initial state of the memory pool;

[0011] After the database operation instruction is executed, the usage of the memory pool where the current instruction is located is obtained again to obtain the final state of the memory pool;

[0012] Compare the initial and final states of the memory pool. If the memory usage increases monotonically, it indicates a memory leak.

[0013] If the memory usage remains the same, then there is no memory leak.

[0014] Furthermore, the process of performing memory leak detection using non-intrusive detection mode is as follows:

[0015] Use kernel detection tools to track the address, size, and stack of memory allocated by memory allocation and release functions;

[0016] Based on the results of the kernel detection tool, determine whether the memory allocation function has allocated memory but has not been released by the memory release function;

[0017] If it exists, use the kernel detection tool to trace the stack information and find the specific location of the memory leak.

[0018] Furthermore, the intrusive detection mode and the non-intrusive detection mode are combined to perform the memory leak detection process as follows:

[0019] After the operation instructions of the database obtained in the intrusive detection mode are executed, the usage of the memory pool where the current instruction is located is decomposed into memory allocation and release information, and passed into the non-intrusive detection mode as input data;

[0020] The non-intrusive detection mode directly determines whether there is a situation where the memory allocation function has allocated memory but has not been released through the memory release function, and finds the specific location of the memory leak.

[0021] Furthermore, both the intrusive detection mode and the non-intrusive detection mode are shared memory leak detection modes designed based on a multi-process coroutine model, where the structure of the multi-process coroutine model is specifically as follows:

[0022] In the multi-process coroutine model, the process is only a container for the execution body, and the thread is the execution body of the program. Multiple threads can be migrated between different processes.

[0023] In addition, the multi-process coroutine model uses shared memory independent of the process, and different processes use the same memory area to interact.

[0024] Furthermore, the workflow of the memory allocation function is as follows:

[0025] After calling the memory allocation function, the memory allocation function allocates memory from the current memory pool. If the space in the current memory pool is sufficient, the memory in the memory pool is used directly; if the space in the current memory pool is insufficient, memory is allocated from the segment.

[0026] If there is enough space in the segment, memory is allocated from the segment; if there is insufficient space in the segment, it is determined whether the memory to be allocated exceeds the upper limit of the total memory capacity;

[0027] If the total memory capacity is exceeded, an error message is returned and allocation is stopped; if the total memory capacity is not exceeded, a new segment is created and memory is requested in the new segment.

[0028] Furthermore, the following operations are performed during the working process of the memory allocation function:

[0029] When the thread is started, the kernel detection tool records the number of memory pools used by each thread;

[0030] Each time a memory allocation or release operation is performed, the usage of the current memory pool is recorded and the size of the memory used by each memory pool is updated;

[0031] Each time a memory allocation operation is performed, the specific information of each memory application is recorded, including the application point, the address and size of the memory applied for.

[0032] The second aspect of the present invention provides an electronic device, comprising a processor and a memory that is communicatively connected to the processor and is used to store executable instructions of the processor, characterized in that the processor is used to execute any shared memory leak detection method based on a multi-process coroutine model as described in the first aspect above.

[0033] The third aspect of the present invention provides a server, characterized in that it includes at least one processor and a memory communicatively connected to the processor, the memory storing instructions executable by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes the shared memory leak detection method based on the multi-process coroutine model as described in any one of the first aspects.

[0034] The fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the shared memory leak detection method based on a multi-process coroutine model as described in any one of the first aspects is implemented.

[0035] Compared with the prior art, the shared memory leak detection method based on the multi-process coroutine model described in the present invention has the following beneficial effects:

[0036] 1) Apply the idea of ​​commonly used memory leak detection tools to programs of this architecture without intruding into the program (that is, without modifying the program code). At the same time, the memory leak of some instructions of the program can be observed without starting or stopping the program.

[0037] 2) It can detect memory leaks non-invasively in multi-process programs based on coroutine shared memory architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0039] Figure 1 A schematic diagram of the execution flow of the intrusive detection mode according to an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of the execution flow of the non-intrusive detection mode according to an embodiment of the present invention;

[0041] Figure 3 The figure is a schematic diagram of the structure of the multi-process coroutine model described in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0043] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0044] Explanation of some technical terms described in the present invention:

[0045] Coroutine: For the operating system, there are only processes and threads. The control of coroutines is explicitly scheduled by the application and is non-preemptive. The execution of coroutines ultimately depends on threads. The application schedules the coroutine to select the appropriate thread to obtain execution rights. The switching is very fast and low-cost. The stack size usually occupies is much smaller than that of threads (coroutines are at the KB level, threads are at the MB level), so more coroutines can be opened. Coroutines are lighter than threads.

[0046] Intrusive: The product source code needs to have the corresponding function. In the intrusive detection mode, the program obtains the memory allocation of a session (or connection) (getMemoryInfo()), including the memory address and memory size of each allocation. The interface for program memory allocation is shm_malloc, and the interface for program memory release is shm_Free. The memory allocation here means: in the coroutine model, shared memory is used between multiple processes to share information.

[0047] The first process will allocate a large block of shared memory from the operating system in advance, and the subsequent processes will share this memory with the first process. Then, they will manage the memory themselves, and the program's memory allocation interface and the program's memory release interface will allocate memory from the shared memory that has been requested from the operating system.

[0048] Non-intrusive: No changes are required to the product source code. Observation is performed through external means. In non-intrusive detection mode, operating system-level observation tools are used to record the address, size, and function call stack of the memory allocated by shm_malloc.

[0049] Multi-process model: The program will fork out many sub-processes.

[0050] Coroutine model: managed by the thread itself and can run in different processes.

[0051] Shared memory: The memory of multiple threads is applied for by the first process to the operating system, and then it is self-managed. The so-called memory allocation and memory release refer to the management of this part of memory.

[0052] Embodiment 1:

[0053] A shared memory leak detection method based on a multi-process coroutine model, including an intrusive detection mode and a non-intrusive detection mode:

[0054] The process of performing memory leak detection using intrusive detection mode is as follows:

[0055] A1. Before executing the database operation instruction, obtain the usage of the memory pool where the current instruction is located and obtain the initial state of the memory pool;

[0056] A2. After the database operation instruction is executed, the usage of the memory pool where the current instruction is located is obtained again to obtain the final state of the memory pool;

[0057] A3. Compare the initial and final states of the memory pool. If the memory usage increases monotonically, it indicates a memory leak.

[0058] A4. If the memory usage remains unchanged, there is no memory leak.

[0059] The process of performing memory leak detection using non-intrusive detection mode is as follows:

[0060] B1. Use kernel detection tools to track the address, size and stack of memory allocated by memory allocation and release functions;

[0061] B2. Based on the results obtained by the kernel detection tool, determine whether the memory allocation function has allocated memory but has not been released by the memory release function;

[0062] B3. If it exists, use the kernel detection tool to track the stack information and find the specific location of the memory leak.

[0063] Combining the intrusive detection mode and the non-intrusive detection mode, the process of performing memory leak detection is as follows:

[0064] After the operation instructions of the database obtained in the intrusive detection mode are executed, the usage of the memory pool where the current instruction is located is decomposed into memory allocation and release information, and passed into the non-intrusive detection mode as input data;

[0065] The non-intrusive detection mode directly determines whether there is a situation where the memory allocation function has allocated memory but has not been released through the memory release function, and finds the specific location of the memory leak.

[0066] like Figure 3 As shown: Both the intrusive detection mode and the non-intrusive detection mode are based on the shared memory leak detection mode designed under the multi-process coroutine model, where the structure of the multi-process coroutine model is specifically as follows:

[0067] In the multi-process coroutine model, the process is only a container for the execution body, and the thread is the execution body of the program. Multiple threads can be migrated between different processes.

[0068] In addition, the multi-process coroutine model uses shared memory independent of the process, and different processes use the same memory area to interact.

[0069] The workflow of the memory allocation function is as follows:

[0070] After calling the memory allocation function, the memory allocation function allocates memory from the current memory pool. If the space in the current memory pool is sufficient, the memory in the memory pool is used directly; if the space in the current memory pool is insufficient, memory is allocated from the segment.

[0071] If there is enough space in the segment, memory is allocated from the segment; if there is insufficient space in the segment, it is determined whether the memory to be allocated exceeds the upper limit of the total memory capacity;

[0072] If the total memory capacity is exceeded, an error message is returned and allocation is stopped; if the total memory capacity is not exceeded, a new segment is created and memory is requested in the new segment.

[0073] The following operations are performed during the working process of the memory allocation function:

[0074] When the thread is started, the kernel detection tool records the number of memory pools used by each thread;

[0075] Each time a memory allocation or release operation is performed, the usage of the current memory pool is recorded and the size of the memory used by each memory pool is updated;

[0076] Each time a memory allocation operation is performed, the specific information of each memory application is recorded, including the application point, the address and size of the memory applied for.

[0077] Working process:

[0078] A session (or connection) to a database;

[0079] Before executing a SQL statement, use getMemoryInfo() to record the total memory used by the current session in memInfo_begin.txt;

[0080] Start the non-intrusive observation tool and record the address, size and function stack of each memory allocation by shm_Malloc to memMallocInfo.txt;

[0081] Execute SQL statements;

[0082] discontinue non-intrusive observation tools;

[0083] Use getMemoryInfo() again to record the total memory used by the current session into memInfo_end.txt;

[0084] Compare the memory differences between step 5 and step 1:

[0085] If there is no difference, then there is no memory leak;

[0086] If there is a difference, there is a memory leak. According to the memory address of the leaked part, find the corresponding stack from memMallocInfo.txt, and you can quickly locate the memory leak point of the product.

[0087] Embodiment 2:

[0088] An electronic device comprises a processor and a memory which is in communication with the processor and is used to store instructions executable by the processor, wherein the processor is used to execute the shared memory leak detection method based on a multi-process coroutine model as described in any one of the above-mentioned embodiments.

[0089] Embodiment three:

[0090] A server, characterized in that it includes at least one processor and a memory communicatively connected to the processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the processor so that the at least one processor executes a shared memory leak detection method based on a multi-process coroutine model as described in any one of Embodiment 1.

[0091] Embodiment 4:

[0092] A computer-readable storage medium stores a computer program, characterized in that: when the computer program is executed by a processor, the shared memory leak detection method based on a multi-process coroutine model described in any one of Embodiment 1 is implemented.

[0093] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0094] In the several embodiments provided in the present application, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the division of the units described above is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The above-mentioned units may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A shared memory leak detection method based on a multi-process coroutine model, including an intrusive detection mode, characterized in that: The process of performing memory leak detection using intrusive detection mode is as follows: Before executing the database operation instruction, obtain the usage of the memory pool where the current instruction is located and obtain the initial state of the memory pool; After the database operation instruction is executed, the usage of the memory pool where the current instruction is located is obtained again to obtain the final state of the memory pool; Compare the initial and final states of the memory pool. If the memory usage increases monotonically, it indicates a memory leak. If the memory usage remains the same, then there is no memory leak; Combining the intrusive detection mode and the non-intrusive detection mode, the process of performing memory leak detection is as follows: After the operation instructions of the database obtained in the intrusive detection mode are executed, the usage of the memory pool where the current instruction is located is decomposed into memory allocation and release information, and passed into the non-intrusive detection mode as input data; The non-intrusive detection mode directly determines whether the memory allocation function has allocated memory but has not been released through the memory release function, and finds the specific location of the memory leak; Both the intrusive detection mode and the non-intrusive detection mode are shared memory leak detection modes designed based on the multi-process coroutine model, where the structure of the multi-process coroutine model is as follows: In the multi-process coroutine model, the process is only a container for the execution body, and the thread is the execution body of the program. Multiple threads can be migrated between different processes. In addition, the multi-process coroutine model uses shared memory independent of the process, and different processes use the same memory area for interaction; The process of performing memory leak detection using non-intrusive detection mode is as follows: Use kernel detection tools to track the address, size, and stack of memory allocated by memory allocation and release functions; Based on the results traced by the kernel detection tool, determine whether there is a memory allocation function that has allocated memory; If it exists and is not released through the memory release function, use the kernel detection tool to trace the stack information to find the specific location of the memory leak; The workflow of the memory allocation function is as follows: After calling the memory allocation function, the memory allocation function allocates memory from the current memory pool. If the space in the current memory pool is sufficient, the memory in the memory pool is used directly; if the space in the current memory pool is insufficient, memory is allocated from the segment. If there is enough space in the segment, memory is allocated from the segment; If there is insufficient space in the segment, determine whether the memory to be allocated exceeds the upper limit of the total memory capacity; If the total memory capacity exceeds the upper limit, an error message is returned and allocation is stopped; if the total memory capacity does not exceed the upper limit, a new segment is created and memory is requested in the new segment; The following operations are performed during the working process of the memory allocation function: When the thread is started, the kernel detection tool records the number of memory pools used by each thread; Each time a memory allocation or release operation is performed, the usage of the current memory pool is recorded and the size of the memory used by each memory pool is updated; Each time a memory allocation operation is performed, the specific information of each memory application is recorded, including the application point, the address and size of the memory applied for; Can observe memory leaks of some instructions of the program.

2. An electronic device, comprising a processor and a memory connected to the processor for storing instructions executable by the processor, characterized in that: The processor is used to execute the shared memory leak detection method based on the multi-process coroutine model described in claim 1 above.

3. A server, characterized in that: It includes at least one processor and a memory that is communicatively connected to the processor, wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes the shared memory leak detection method based on the multi-process coroutine model as described in claim 1.

4. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the shared memory leak detection method based on a multi-process coroutine model described in claim 1 is implemented.