Memory compression method for Shadow Memory memory leak detection mechanism

Abstract

The present invention provides a memory compression method for Shadow Memory memory leak detection mechanism, belonging to the field of computer applications. In order to solve the problem of large memory overhead caused by small object size, the method of the present invention uses a mechanism of fuzzy representation of object information - object fusion, to perform Shadow Memory memory compression. For objects with a minimum size of k bytes, if the pointer difference between two adjacent objects does not exceed 2k bytes, the two objects share an object identification unit; at the same time, in order to eliminate the object omissions caused by this, two solutions are provided: outputting supplementary information, and randomly overwriting and combining the output results of multiple leak detection runs to solve the object omissions. The method of the present invention makes an appropriate trade-off between the accuracy of leak detection and memory load, so that the memory consumption of the leak detection module is reduced by half, and the workload of checking leaking objects is also reduced, thereby accelerating the efficiency of leak detection.

Description

Technical Field

[0001] The invention belongs to the field of computers, and in particular relates to a memory compression method for a Shadow Memory memory leak detection mechanism. Background Art

[0002] Most interpreted languages ​​rely on the runtime's garbage collection mechanism for object allocation and automatic deallocation. In real-world development, poor code design often leads to "cold" objects that remain unaccessed and occupy memory for extended periods, resulting in wasted memory space and a memory leak.

[0003] Shadow Memory is an effective solution to the problem of memory leaks. The virtual machine allocates an additional memory space outside the stack to record object access, allocation point, and other information. It then inserts code into the logic of creating, accessing, modifying, and deleting objects, storing the access information in Shadow Memory. The object access information stored therein is then scanned and maintained at set time intervals, generating object allocation point information that may leak in the process. This solution can effectively detect objects that are potentially prone to memory leaks, and the workload for customized development for different runtime systems is relatively small, making it highly versatile.

[0004] However, the above solution cannot solve the problem of high memory load. The solution uses the pointer of the object in the memory to represent the object, and relies on this pointer to retrieve the access and allocation point information corresponding to the object in the Shadow Memory, such as Figure 1 As shown in the figure, the space occupied by the additional Shadow Memory is directly related to the compression ratio (runtime memory size / Shadow Memory size), which is in turn limited by object size. When the object size is small, the compression limit is low, resulting in more memory space being consumed to store the object's access and allocation point information. For some devices with limited memory resources, excessive memory overhead is unacceptable. Summary of the Invention

[0005] To address the potential problem of large memory overhead caused by the small object size mentioned above, the present invention proposes a mechanism for fuzzy representation of object information - object fusion, and implements a memory compression method for the Shadow Memory memory leak detection mechanism, making an appropriate trade-off between the accuracy of leak detection and memory load to solve the above problem.

[0006] The present invention provides a memory compression method for the Shadow Memory memory leak detection mechanism, and the implementation means include the following:

[0007] (1) Assuming that the minimum size of a storage object in an application scenario is k bytes, then when the pointers of two adjacent objects differ by no more than 2k bytes, the two objects share the same object identification unit;

[0008] (2) Use the following two solutions to eliminate the problem of object omission:

[0009] Solution 1: When an object overlay occurs, logs are output for both objects involved in the overlay as supplementary information output by the leak detection module.

[0010] Solution 2: In different leakage detection cycles, the leakage detection module initializes different coverage strategies, namely first-come-first-come or last-come-last-come priority; when the first-come-first-come strategy is adopted, the object identification unit will not be updated when an object is covered; when the last-come-last-come strategy is adopted, the object identification unit will be updated to the content of the last-come-last-come object when an object is covered; the outputs of the leakage detection modules of leakage detection cycles with multiple different coverage strategies are integrated to eliminate the problem of object omission.

[0011] The advantages and positive effects of the present invention are: (1) the method of the present invention improves the compression efficiency of the shadow memory with an original compression ratio of 1:k to 1:2k, thereby reducing the memory consumption of the leakage detection module by half; (2) the method of the present invention can accurately detect the leakage object on the basis of reducing the memory consumption of the leakage detection module, thereby reducing the workload of checking the leakage object and accelerating the leakage detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is the principle diagram of Shadow Memory memory leak detection object identification;

[0013] Figure 2 This is a schematic diagram of the object fusion identification scheme adopted by the method of the present invention;

[0014] Figure 3 This is a state update flow chart of the object fusion scheme using random coverage in the present invention. DETAILED DESCRIPTION

[0015] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0016] This embodiment of the present invention considers a scenario where the minimum object size is k = 8 bytes to illustrate the proposed memory compression method for the ShadowMemory memory leak detection mechanism. k is the size of the data structure (or handler) representing the object within the virtual machine. Taking the JavaScript virtual machine as an example, the minimum object size in V8 is 8 bytes; in JerryScript, it is 16 bytes; and in QuickJS, it is 72 bytes. The smaller the object size of the virtual machine, the greater the benefits of applying the present method.

[0017] In scenarios where the minimum object size is 8 bytes, the compression ratio according to the original object representation scheme is a maximum of 8:1, that is, each object identification unit in Shadow Memory can represent a maximum of 8 bytes of heap memory, so the minimum memory card overhead is 12.5%. The present invention allows two objects to share the same object identification unit. In this scenario, the compression ratio is set to 16:1. When the pointers of two objects differ by no more than 16 bytes, they may be mapped to the same object identification unit in Shadow Memory, such as Figure 2 When the object identification unit retrieved from Shadow Memory according to the pointer of the second object is already occupied, the object will ignore the information of the first object and overwrite the identification unit.

[0018] The drawback of two objects sharing the same object identification unit is that the allocation point information of a later object may overwrite the allocation point information of an earlier object, resulting in information mismatch when the object access information is sent to the object identification unit. For example, if the allocation point information of object B overwrites the allocation point information of object A, the access of object A will be recorded by ShadowMemory on object B. If object B is not actually accessed, then this will result in a potential leak of object B.

[0019] To address the problem of underreporting, the present invention proposes two solutions.

[0020] Solution 1: Output supplementary information. When an object overlap occurs, the allocation points of both parties involved in the overlap are output as logs as supplementary information when outputting the leaked information.

[0021] Solution 2: Random Overwrite. Before running the leak detection program, the overwrite strategy is randomly initialized to either first-come-first or last-come-last. When an object is overwritten, if the overwrite strategy is first-come-first, the object identifier is not updated; if the overwrite strategy is last-come-last, the object identifier is updated with the content of the last-come-last object. By comparing the output of multiple runs of the leak detection module, the problem of missed objects can be resolved.

[0022] Both of the above solutions can solve the potential underreporting problem caused by object fusion to a certain extent.

[0023] The above two solutions can be used simultaneously and complement each other.

[0024] The logging mechanism in Solution 1 is as follows: if two objects A and B overlap, a log message is output: A overlaps B (or B overlaps A). This log message indicates a potential missed detection. This log message is then taken into consideration when analyzing missed detections.

[0025] In the second scheme, random coverage is implemented by initializing the coverage strategy during each leak detection cycle, determining whether to prioritize first-arrived or last-arrived objects. Therefore, if only the results of a single leak detection cycle are considered, missed detections may occur. Therefore, it is necessary to initialize the leak detection module to adopt different coverage strategies during different leak detection cycles. Leak detection cycles running with different coverage strategies will produce different outputs. The results of leak detection cycles with different coverage strategies can then be combined to eliminate missed detections. For example, unioning the results of two different leak detection cycles can eliminate missed detections.

[0026] like Figure 3 As shown in Figure 1, the mechanism for updating Shadow Memory using the object fusion scheme with random overwriting strategy includes the following:

[0027] Step 1) When the stub code in the runtime system detects that a new object is created at a certain allocation point p, the stub code forwards the allocation point information and the pointer addr of the object to Shadow Memory;

[0028] Step 2) Shadow Memory retrieves the object identification unit according to addr. The retrieval position is addr / R, where R is the compression unit. Then check whether the identification unit is occupied by other objects. If not, jump to step 4; otherwise, continue to step 3.

[0029] Step 3) If the cell is already occupied, check whether the object overwrite policy initialized before the leak detection program runs is "later objects first," allowing later objects to overwrite the allocation points and access information of earlier objects in Shadow Memory. If so, proceed to Step 4; otherwise, go to Step 5.

[0030] Step 4) Write the allocation point and access information of the currently captured object into the object identification unit.

[0031] Step 5) The access point information of the current object is stored in the access point table so that the actual access point information can be retrieved according to the hash(p) stored in ShadowMemory. Hash(p) refers to the hash value of the allocation point p.

[0032] Shadow memory consumes additional memory, and its size is a crucial metric for such memory detection tools; the smaller the better. As can be seen from the above examples, the method of the present invention achieves the goal of significantly reducing the size of shadow memory at the expense of a small amount of detection accuracy, effectively reducing the size of shadow memory exponentially.

[0033] Except for the technical features described in the specification, all other technical features are known to those skilled in the art. The present invention omits descriptions of well-known components and well-known technologies to avoid redundancy and unnecessary limitation of the present invention. The implementation methods described in the above embodiments do not represent all implementation methods consistent with the present application. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the present invention.

Claims

1. A memory compression method for Shadow Memory memory leak detection mechanism, characterized in that: The steps include: (1) Assuming that the minimum size of an object stored in the application scenario is k bytes, then when the pointers of two adjacent objects differ by no more than 2k bytes, the two objects share the same object identification unit; k is also the size of the data structure representing the object within the virtual machine; the original compression ratio k in the shadow memory is changed to 2k; (2) Use one or a combination of the following two solutions to eliminate the problem of underreporting of objects: Solution 1: When an object overlay occurs, logs are output for both objects involved in the overlay as supplementary information output by the leak detection module. Solution 2: In different leakage detection cycles, the leakage detection module initializes different coverage strategies, namely first-come-first-come or last-come-last-come priority; when the first-come-first-come strategy is adopted, the object identification unit will not be updated when an object is covered; when the last-come-last-come strategy is adopted, the object identification unit will be updated to the content of the last-come-last-come object when an object is covered; the outputs of the leakage detection modules of leakage detection cycles with multiple different coverage strategies are integrated to eliminate the problem of object omission.

2. The method according to claim 1, characterized in that In step (2), the mechanism for updating ShadowMemory using solution 2 is as follows: Step 1: When the instrumentation code in the runtime system detects that a new object is created at an allocation point p, the instrumentation code forwards the allocation point information and the object pointer addr to the Shadow Memory; Step 2: Shadow Memory searches for the object identification unit based on addr. The search location is addr / R, where R is the compression unit. Then, Shadow Memory checks whether the object identification unit is occupied by other objects. If not, it jumps to step 4; otherwise, it continues to step 3. Step 3: If the object identification unit is already occupied, check whether the overwriting strategy initialized by the leak detection module is "later objects first". If so, proceed to step 4; otherwise, go to step 5. Step 4, writing the allocation point and access information of the currently captured object into the object identification unit; Step 5: Store the access point information of the current object into the access point table, and retrieve the actual access point information based on the hash (p) stored in Shadow Memory; hash(p) is the hash value of the assigned point p.

Images