Memory segment merging method and device, equipment, storage medium and program product
By merging memory segments in the Android system, the problem of too many memory segments caused by the Secondary allocator is solved, achieving a reduction in the number of memory segments while maintaining security and good compatibility.
Patent Information
- Application Number
- CN202511212411.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-27
- Publication Date
- 2025-11-21
AI Technical Summary
In the Android system, the frequent allocation of memory segments by the Secondary allocator causes the number of memory segments to increase rapidly, exceeding the system limit and triggering a memory overflow crash.
When the memory operation interface is called, the memory segments to be merged are obtained and merged after meeting preset conditions, including the merging of data segments and protection pages, thereby reducing the number of memory segments.
It effectively reduces the number of memory segments, avoids memory overflow issues, maintains memory safety, has good compatibility, and does not require modification of the heap memory allocator source code or the Android system kernel.
Smart Images
Figure CN120994398A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of mobile operating systems, specifically to methods, apparatus, devices, storage media, and program products for merging memory segments. Background Technology
[0002] The Scudo memory allocator (heap allocator) used in the Android system is a highly secure memory allocator. Its architecture mainly consists of two core components: the Primary allocator and the Secondary allocator.
[0003] The Primary allocator is mainly used to allocate small amounts of memory (typically no more than 64KB), while the Secondary allocator is mainly used to allocate large amounts of memory (typically more than 64KB). The memory allocated by the Secondary allocator has a special structure: guard pages + data segments + guard pages. The guard pages are non-readable and non-writable, used to detect buffer overflows and enhance memory security.
[0004] In large Android applications, when the Primary allocator runs out of memory, smaller memory allocations are forced to use the Secondary allocator path. Since the Secondary allocator creates three memory segments (one data segment and two guard pages) with each allocation, frequent memory allocations in large applications can cause the number of memory segments to increase rapidly, eventually exceeding the system's default maximum number of memory segments and triggering a memory overflow. Summary of the Invention
[0005] In view of this, this application provides a method, apparatus, device, storage medium, and program product for merging memory segments to solve the problem of memory overflow caused when small memory is allocated to a secondary allocator for allocation.
[0006] Firstly, this application provides a method for merging memory segments, the method comprising:
[0007] When the memory operation interface is called, the first memory segment to be merged and the second memory segment to be merged are obtained and entered into the secondary allocator for memory allocation. The first memory segment and the second memory segment include data segments or protection pages. When the first memory segment is a data segment, the second memory segment is a protection page. When the first memory segment is a protection page, the second memory segment is a data segment.
[0008] Determine whether the first memory segment and the second memory segment meet the preset conditions for merging, wherein the preset conditions are used to perform a merging consistency judgment on the first memory segment and the second memory segment using multiple parameters;
[0009] If the first memory segment and the second memory segment meet the preset conditions, the first memory segment and the second memory segment will be merged.
[0010] Secondly, this application provides a memory segment merging apparatus, the apparatus comprising:
[0011] The first acquisition module is used to acquire, when the memory operation interface is called, the first memory segment to be merged and the second memory segment to be merged that enter the secondary allocator to perform memory allocation. The first memory segment and the second memory segment include data segments or protection pages. When the first memory segment is a data segment, the second memory segment is a protection page. When the first memory segment is a protection page, the second memory segment is a data segment.
[0012] The determination module is used to determine whether the first memory segment and the second memory segment meet the preset conditions for merging. The preset conditions are used to perform a merging consistency judgment on the first memory segment and the second memory segment using multiple parameters.
[0013] The merging module is used to merge the first memory segment and the second memory segment when the first memory segment and the second memory segment meet preset conditions.
[0014] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the memory segment merging method described in the first aspect or any corresponding embodiment.
[0015] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the memory segment merging method described in the first aspect or any corresponding embodiment thereof.
[0016] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to execute the memory segment merging method described in the first aspect or any corresponding embodiment thereof.
[0017] The method, apparatus, device, storage medium, and program product for merging memory segments provided in this application embodiment, when the memory operation interface is called, obtains the first memory segment to be merged and the second memory segment to be merged that enter the secondary allocator to perform memory allocation, and merges the first memory segment and the second memory segment after the first memory segment and the second memory segment meet the preset conditions for merging. In this way, this application embodiment merges the first memory segment and the second memory segment that enter the secondary allocator allocation path to perform memory allocation, realizes the aggregation of data segments and protection pages, reduces the number of memory segments, and avoids memory overflow problems. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this application, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0019] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this application;
[0020] Figure 2 This is a flowchart illustrating a method for merging memory segments according to an embodiment of this application;
[0021] Figure 3 This is a flowchart illustrating another method for merging memory segments according to an embodiment of this application;
[0022] Figure 4 This is a structural block diagram of a memory segment merging apparatus according to an embodiment of this application;
[0023] Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0025] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.
[0026] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.
[0027] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0028] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.
[0029] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.
[0030] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.
[0031] The heap allocator used in the Android system mainly consists of two core components: the primary allocator and the secondary allocator, to implement memory allocation.
[0032] The primary allocator is mainly used for allocating small amounts of memory (typically no more than 64KB). It consists of 33 partitions, each with a fixed size (256MB). Each partition is further divided into blocks of equal size, with varying block sizes across different partitions, increasing from partition 0 to partition 32. When allocating memory, the heap allocator first attempts to allocate from the partition containing the best-matching block size. If that fails, it then tries the subsequent partitions in sequence.
[0033] The secondary allocator is primarily used for allocating large amounts of memory (typically exceeding 64KB). It directly uses the mmap system call to allocate memory. The memory allocated by the secondary allocator has a special structure: safe pages + data segments + safe pages.
[0034] Therefore, when the heap allocator is running, if the primary allocator runs out of memory, small memory allocations are forced to go through the secondary allocator path. However, since the secondary allocator creates three memory segments (one data segment and two safety pages) with each allocation, frequent memory allocations in large applications can cause the number of memory segments to increase rapidly. Under high load scenarios, this can easily trigger system limits, and when it eventually exceeds the system limit (65530, which is obtained by subtracting 5 reserved segments from the maximum value of an unsigned short integer), it can cause a memory overflow crash.
[0035] Based on this, this application proposes a method for merging memory segments as an optional application scenario for this application embodiment, such as... Figure 1 As shown, the application scenario includes: Android terminal device 101 and optimization server 102. Android terminal device 101 and optimization server 102 establish a communication connection, and form a full-link optimization architecture for the Scudo memory segment over-limit problem based on Android terminal device 101 and optimization server 102.
[0036] Specifically, Android terminal device 101 (running a large application with frequent small memory allocation scenarios) sends an optimization request to optimization server 102 to address the Scudo memory segment over-limit issue. Optimization server 102 then deploys a targeted solution based on this request (including triggering memory segment merging logic in prctl after the system hook prctl is called, modifying the memory permission attributes of security pages, and unifying the naming of security pages and data segments), and pushes the solution to Android terminal device 101.
[0037] During the application execution process, if the memory operation interfaces such as mremap, mmap, madvise, prctl, mprotect, and ioctl in the kernel of Android terminal device 101 are called, the kernel will obtain the first and second memory segments currently being allocated by the secondary allocator and determine the preset conditions for merging the first and second memory segments. These preset conditions are set according to the solution pushed by optimization server 102. If the preset conditions are met, the first and second memory segments will be merged.
[0038] Android terminal device 101 can be a smartphone, tablet, Android smart TV, or other device running the Android system. It must support kernel memory management mechanisms and system call hooks (such as implementing prctl hooks through dynamic link library injection). Optimization server 102 must have the ability to deploy and adapt solutions, and may include cloud servers, edge servers, etc. It can adjust solution details according to the system version of the Android terminal device (such as Android 12 and above).
[0039] It should be noted that, Figure 1 This is merely an illustrative example of an application scenario and does not limit the scope of protection of this application.
[0040] According to an embodiment of this application, a method embodiment for merging memory segments is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0041] This embodiment provides a method for merging memory segments, which can be used in the aforementioned Android terminal device 101. Figure 2 This is a flowchart of a method for merging memory segments according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:
[0042] Step S201: When the memory operation interface is called, obtain the first memory segment to be merged and the second memory segment to be merged for memory allocation in the secondary allocator. The first memory segment and the second memory segment include data segments or protection pages. When the first memory segment is a data segment, the second memory segment is a protection page. When the first memory segment is a protection page, the second memory segment is a data segment.
[0043] Optionally, this embodiment of the application is an application of the memory segment merging mechanism in the kernel. Specifically, when memory operation interfaces such as mremap, mmap, madvise, prctl, mprotect, and ioctl are called, the first and second memory segments are obtained and entered into the secondary allocator for memory allocation. The current first and second memory segments are objects to be merged, and the merging operation is only performed after the merging conditions are met.
[0044] In addition, in this embodiment, the objects to be merged are data segments and protection pages. Therefore, when the first memory segment is a data segment, the second memory segment is a protection page, and when the first memory segment is a protection page, the second memory segment is a data segment, ensuring that the first memory segment and the second memory segment are not the same object.
[0045] It should be noted that the embodiments of this application are based on the special structure of memory allocation by the secondary allocator, which usually involves two protection pages, namely left and right protection pages. Therefore, when the first memory segment or the second memory segment is a protection page, the corresponding number is two by default.
[0046] Step S202: Determine whether the first memory segment and the second memory segment meet the preset conditions for merging, wherein the preset conditions are used to determine the consistency of merging the first memory segment and the second memory segment.
[0047] Optionally, a preset condition for merging the first memory segment and the second memory segment is currently set. This preset condition is mainly used to determine the consistency of the addresses, memory inclusion identifiers, and memory segment names of the first memory segment and the second memory segment, and to determine whether the first memory segment and the second memory segment meet the preset condition.
[0048] Here, the address can be the end address of the previous memory segment (e.g., the first memory segment) and the start address of the next memory segment (e.g., the second memory segment); the identifiers of the memory segment can be whether it is readable, writable, locked, measurable, or a large page, etc. Memory segment naming here refers to the process of calling the system interface to name the first and second memory segments during memory allocation.
[0049] Step S203: If the first memory segment and the second memory segment meet the preset conditions, merge the first memory segment and the second memory segment.
[0050] Optionally, if the first memory segment and the second memory segment meet the preset conditions for merging, the first memory segment and the second memory segment can be merged. If the preset conditions are not met, a new first memory segment and a new second memory segment are obtained, and the above memory segment merging mechanism is executed until the preset conditions are met, thus achieving the merging of memory segments.
[0051] The method, apparatus, device, storage medium, and program product for merging memory segments provided in this application embodiment, when the memory operation interface is called, obtains the first memory segment to be merged and the second memory segment to be merged that enter the secondary allocator to perform memory allocation, and merges the first memory segment and the second memory segment after the first memory segment and the second memory segment meet the preset conditions for merging. In this way, this application embodiment merges the first memory segment and the second memory segment that enter the secondary allocator allocation path to perform memory allocation, realizes the aggregation of data segments and protection pages, reduces the number of memory segments, and avoids memory overflow problems.
[0052] This embodiment provides a method for merging memory segments, which can be used in the aforementioned Android terminal device 101. Figure 3This is a flowchart of another memory segment merging method according to an embodiment of this application, such as... Figure 3 As shown, the process includes the following steps:
[0053] Step S301: When the memory operation interface is invoked, obtain the first memory segment to be merged and the second memory segment to be merged for memory allocation in the secondary allocator. The first and second memory segments include data segments or protection pages. When the first memory segment is a data segment, the second memory segment is a protection page; when the first memory segment is a protection page, the second memory segment is a data segment. For details, please refer to [link to relevant documentation]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.
[0054] Step S302: Determine whether the first memory segment and the second memory segment meet the preset conditions for merging, wherein the preset conditions are used to determine the consistency of merging the first memory segment and the second memory segment.
[0055] Specifically, step S302 includes:
[0056] Step S3021: Obtain the end address of the first memory segment and the start address of the second memory segment.
[0057] Step S3022: When the end address of the first memory segment is the start address of the second memory segment, obtain the attribute identifier contained in the first memory segment and the attribute identifier contained in the second memory segment.
[0058] Step S3023: When the attribute identifier contained in the first memory segment is the same as the attribute identifier contained in the second memory segment, obtain the name of the first memory segment and the name of the second memory segment.
[0059] Step S3024: When the name of the first memory segment is the same as the name of the second memory segment, determine that the first memory segment and the second memory segment meet the preset conditions.
[0060] Optionally, the kernel may automatically merge adjacent memory segments under certain conditions, the main conditions of which include:
[0061] 1. Adjacent addresses: The end address of the previous memory segment is equal to the start address of the next memory segment;
[0062] 2. All identifiers in the memory segment are consistent: the identifiers include whether the memory is readable, writable, locked, measurable, and whether it is a large page memory, etc.
[0063] 3. Consistent naming of anonymous memory segments: (Scudo memory segments are all anonymous memory segments, and the first and second memory segments must have the same name).
[0064] Memory segment merging will only occur if all three conditions mentioned above are met simultaneously.
[0065] Specifically, the kernel can obtain the end address of the first memory segment and the start address of the second memory segment through the process's memory management data structures (such as struct mm_struct, struct vm_area_struct) or by calling the kernel interface.
[0066] The identification information for each memory segment is mainly stored in the key fields of `struct vm_area_struct`, which together constitute the "identity" of the memory segment. When the end address of the previous memory segment (such as the first memory segment) is equal to the start address of the next memory segment (such as the second memory segment), the kernel directly reads `vm_flags` and parses the bit flags to obtain the attribute flags contained in the first memory segment and the attribute flags contained in the second memory segment.
[0067] If the attribute identifiers contained in the first memory segment and the attribute identifiers contained in the second memory segment are the same, the names of the first and second memory segments are obtained. Specifically, when the memory segment is a file mapping (mapped via mmap), the kernel associates it with the `struct file` through the `vm_file` field of `struct vm_area_struct`, and then obtains the file path as the "name" of that memory segment. For example, the code segment of the ` / bin / ls` program will use the file path ` / bin / ls` as its identifier.
[0068] If the first memory segment and the second memory segment have the same name, such as both being named "scudo:secondary_unguard", then the first memory segment and the second memory segment meet the preset conditions, and then the merge operation can be performed.
[0069] Step S303: If the first memory segment and the second memory segment meet preset conditions, merge the first memory segment and the second memory segment. For details, please refer to [link to relevant documentation]. Figure 2 Step S203 of the illustrated embodiment will not be described again here.
[0070] As an optional embodiment, before step S202 above, the method further includes:
[0071] Step a1: If it is determined that the first interface of the system is called and the corresponding execution behavior is to trigger memory segment merging, obtain the memory address corresponding to the data segment.
[0072] Step a2: Determine the address of the protected page based on the memory address.
[0073] Step a3: Invoke the system's second interface based on the address of the protected page, and change the function parameters of the second interface based on the attribute identifier contained in the data segment.
[0074] Step a4: Based on the function parameters of the first interface, modify the naming of the protected page and data segment.
[0075] Optionally, in this embodiment of the application, by monitoring whether small memory (less than 40KB) is allocated through the secondary allocator, when small memory is detected to be allocated through the secondary allocator, the memory permissions of the protection page are modified to be readable and writable, consistent with the data segment. At the same time, the naming scope is expanded from the original data segment to include the protection page, and the data segment and the protection page are named uniformly to achieve naming consistency. This makes the protection page and the data segment meet the conditions for kernel memory segment merging (adjacent address, consistent attribute identifier (including consistent permissions), and consistent naming). Thus, when the memory operation interface is called in the application phase, the kernel's memory segment merging mechanism is triggered to reduce the number of memory segments.
[0076] Specifically, by hooking the prctl system call, the code related to the specific scheme for triggering memory segment merging is injected into prctl, and memory segment merging is triggered in prctl.
[0077] Here, `prctl` is a system call interface, referred to here as the first interface. Therefore, when it's determined that `prctl` has been called and the current execution triggers memory segment merging, the kernel obtains the memory address range of the data segment through the memory management data structure `struct vm_area_struct`, and then calculates the address ranges of the left and right guard pages based on this memory address range.
[0078] The left protected page is located one page before the start address of the data segment, and the right protected page is located one page after the end address of the data segment. Therefore, the kernel obtains the page size (e.g., 4096 bytes) through the PAGE_SIZE macro (or the getpagesize() function), and the page mask is PAGE_MASK = ~(PAGE_SIZE-1) (used for address alignment).
[0079] Calculate the left protected page address:
[0080] The starting address of the left protected page immediately adjacent to the data segment is:
[0081] left_guard_start=data_start-PAGE_SIZE
[0082] Because memory is page aligned, data_start is an integer multiple of PAGE_SIZE, therefore left_guard_start must also be page aligned, and its address range is [left_guard_start, data_start).
[0083] Calculate the right protected page address
[0084] The right protected page is adjacent to the end address of the data segment, and its starting address is:
[0085] right_guard_start = data_end (because data_end is page-aligned, it happens to be the start of the next page)
[0086] Its address range is [right_guard_start, right_guard_start + PAGE_SIZE).
[0087] Subsequently, in this embodiment of the application, the protection page is remapped using mmap (i.e., the second interface). The function parameters of the second interface are changed based on the attribute identifier contained in the data segment. Then, the left and right protection pages call the mmap interface once each, so that the attribute identifier of the protection page can be changed based on the attribute identifier in the data segment, thereby achieving consistency between the attribute identifiers of the data segment and the protection page.
[0088] The process of changing the function parameters of the second interface based on the attribute identifier contained in the data segment is as follows:
[0089] First, the function prototype of mmap is: void* mmap(void* addr, size_t length, int prot, int flags, int fd, off_t offset).
[0090] Change the prot parameter of mmap to be readable and writable (PROT_READ|PROT_WRITE).
[0091] Changing the `flags` parameter of `mmap` to `MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED` will set the memory attribute flags of the protected pages to be consistent with the attribute flags of the data segments.
[0092] Change the address information (addr) and length information (length) of mmap to the address range of the protected page.
[0093] Then, by modifying the parameters of the prctl call, the naming scope was expanded to unify the naming of protected pages and data segments, for example, both could be named "scudo:secondary_unguard".
[0094] In this embodiment, non-intrusive optimization of the heap memory allocator is achieved by hooking the prctl system call. This method does not require modification of the heap memory allocator's source code or the Android system kernel, and can be implemented directly at the application level, offering good compatibility and portability.
[0095] As an optional embodiment, step a1 above includes:
[0096] Step b1: Determine whether the operation type parameter in the function of the first interface is the first target value after the first interface is called.
[0097] Step b2: If the operation type parameter is the first target value, determine whether the first parameter in the function of the first interface is the second target value.
[0098] Step b3: If the first parameter is the second target value, determine whether the value of the second parameter in the function of the first interface is less than the threshold.
[0099] Step b4: If the value of the second parameter is less than the threshold, determine whether the character of the third parameter in the function of the first interface is the target character.
[0100] Step b5: If the character in the third parameter is the target character, determine that the execution behavior is to trigger memory segment merging.
[0101] Optionally, the criteria for determining whether memory segment merging needs to be triggered in prctl are as follows:
[0102] The prototype of prctl is as follows:
[0103] int prctl(int option,unsigned long arg2,unsigned long arg3,unsigned long arg4,unsigned long arg5);
[0104] Here, option represents the specific operation instruction, arg2 represents whether the current operation is naming an anonymous memory segment, arg3 represents the starting address of the memory segment, arg4 represents the address range of the memory segment, and arg5 represents the name of the memory segment.
[0105] In the specific implementation, a threshold SCUDO_SECONDARY_DANGER_LINE (40KB) is first defined to filter out normal large memory allocations and only process small memory allocations that go through the allocator's allocation path.
[0106] First, determine whether the first interface, i.e., the prctl call, is naming a memory segment: simply check if the operation type parameter option is the first target value: PR_SET_VMA. If option is PR_SET_VMA, check if the first parameter arg2 in the prctl function is the second target value: PR_SET_VMA_ANON_NAME. If arg2 is PR_SET_VMA_ANON_NAME, then it is considered that this prctl call is naming a memory segment.
[0107] Next, determine if the memory segment size is less than the set threshold: This only requires checking if the value of the second parameter arg4 is less than the set threshold SCUDO_SECONDARY_DANGER_LINE. If it is, finally check if the memory name is scudo:secondary. To determine if "secondary" should be named, simply check if the third parameter arg5 is equal to the target string: scudo:secondary. If arg5 is scudo:secondary, then the current execution behavior is determined to trigger memory segment merging.
[0108] In this embodiment, the allocation path for small memory allocations is optimized. By setting a threshold, only memory allocations smaller than the threshold are processed, avoiding impact on normal large memory allocations. This also solves the problem of a rapid increase in the number of memory segments caused by small memory allocations when the main allocator runs out of memory.
[0109] As an optional embodiment, step a3 above includes:
[0110] Step c1: Set the address information and length information in the function parameters of the second interface, where the address information and length information are used to characterize the address range of the protected page.
[0111] Step c2: Modify the attribute identifier in the function parameters of the second interface to the attribute identifier contained in the data segment.
[0112] Step c3: Call the second interface and remap to the protection page based on the address information and length information, so that the attribute identifier corresponding to the protection page is consistent with the attribute identifier contained in the data segment.
[0113] Optionally, this application provides a detailed description of how the attribute identifiers (including memory permissions) of the protected page are modified.
[0114] Specifically, as can be seen from the above embodiments, this application remaps the protected page by calling the mmap interface, giving it the same attribute identifier as the data segment. Therefore, based on the mmap function prototype: void* mmap(void* addr, size_t length, int prot, int flags, int fd, off_t offset), the address information (parameter addr) and length information (parameter length) in the function parameters of the second interface are first set to correspond to the address range of the protected page. Then, the attribute identifiers in the function parameters of the second interface are modified to the attribute identifiers contained within the data segment. For example, the prot parameter is set to readable and writable (PROT_READ|PROT_WRITE), and the flags parameter is set to MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED.
[0115] Next, the mmap interface is called to remap the data to the protection page based on addr and length. Then, the attribute identifiers from the function parameters of the second interface are applied to the protection page, changing the attribute identifiers of the protection page to match those of the data segment. Since the data segment has read and write permissions, the modified protection page also has read and write permissions.
[0116] This application embodiment reduces the number of memory segments by modifying the attribute identifier of the protected page to meet the conditions for kernel memory segment merging.
[0117] As an optional embodiment, step a4 above includes:
[0118] Step d1: Based on the function parameters called from the first interface, change the second and fourth parameters.
[0119] Step d2: Obtain the naming range based on the modified second and fourth parameters.
[0120] Step d3: Change the names of the protected pages and data segments uniformly based on the naming range.
[0121] Optionally, this application provides a detailed description of how to uniformly name protected pages and data segments.
[0122] Specifically, by modifying the parameters of the prctl call, the naming range is expanded to unify the naming of protected pages and data segments, such as naming them scudo:secondary_unguard.
[0123] Furthermore, when a small memory allocation path is detected that uses the secondary allocator, the kernel modifies the parameters of the prctl call:
[0124] The naming scope is expanded from the original data segment to include the left and right security pages (by modifying the second parameter arg3 and the fourth parameter arg4 of prctl, for example, by adding the address of the previous page to the starting address arg3 and adding the address range of two pages to the address range size arg4, thus expanding the naming scope to cover both the security pages and the data segment).
[0125] Then, the security page and data segment were uniformly named "scudo:secondary_unguard" (by modifying the third parameter arg5).
[0126] Based on the above, the protected page and the data segment have the same name.
[0127] This application embodiment reduces the number of memory segments by using the same name for protection pages and data segments to meet the conditions for kernel memory segment merging.
[0128] As a specific application embodiment of this application, taking a large Android social application (such as an "instant messaging + short video" app) as an example, this application has frequent small memory allocations (such as message item caching, temporary video frame data, etc., mostly 1KB-30KB in size) in high-concurrency scenarios (such as homepage information stream refresh, short video list scrolling). When the heap memory allocator allocates small memory, it is easy to cause memory segment over-limit and crash. Based on this, this application integrates memory segment optimization logic in the application development stage to avoid the problem of memory overflow crash triggered by small memory allocation after the application is launched.
[0129] The memory segment optimization logic involves merging protected pages and data segments. Specifically, when a small memory segment (less than the threshold SCUDO_SECONDARY_DANGER_LINE(40KB)) is detected traversing the allocator path, the code related to the specific scheme for triggering memory segment merging is injected into prctl by hooking the prctl system call. This triggers the memory segment merging within prctl.
[0130] 1. Determine whether memory segment merging needs to be triggered;
[0131] 2. If not needed, execute the logic in the original prctl of the system normally;
[0132] 3. If necessary, execute the following logic;
[0133] a. First, modify the memory permissions of the security page to make them consistent with those of the data segment;
[0134] b. Use a consistent naming convention for security pages and data segments.
[0135] The kernel performs the following checks when memory operation interfaces such as mremap, mmap, madvise, prctl, mprotect, and ioctl are called:
[0136] 1. Adjacent addresses: The end address of the previous memory segment is equal to the start address of the next memory segment;
[0137] 2. All flags in the memory segment are consistent: including whether the memory is readable, writable, locked, measurable, and whether it is a large page memory, etc.
[0138] 3. Anonymous named memory segments: consistent naming
[0139] The merging of memory segments (i.e., data segments and protected pages) will only occur if all three conditions are met.
[0140] The embodiments of this application have the following advantages:
[0141] 1. Effectively reduce the number of memory segments: By modifying the protected page attribute flag, the kernel memory segment merging mechanism is triggered, which greatly reduces the number of memory segments and avoids application crashes caused by memory segment overruns.
[0142] 2. Maintaining memory safety: While optimizing the use of memory segments, this application embodiment maintains the memory safety characteristics of the heap memory allocator. Although the permissions of the safe page are modified, the safe page still exists. It is just merged with the data segment into a memory segment, which can still detect buffer overflows to a certain extent.
[0143] 3. Non-intrusive implementation with good compatibility: Implemented by hooking system calls, it does not require modification of the source code of the heap memory allocator or the Android system kernel. It can be implemented directly at the application level, with good compatibility and portability. This implementation method allows the technical solution to be applied on different versions of the Android system without the need for adaptation for each system version.
[0144] 4. Targeted solution to small memory allocation problem: This application embodiment is specifically optimized for the allocation path of small memory using the secondary allocator, effectively solving the problem of the number of memory segments increasing rapidly when the main allocator runs out of memory.
[0145] This embodiment also provides a memory segment merging apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0146] This embodiment provides a device for merging memory segments, such as... Figure 4 As shown, it includes:
[0147] The first acquisition module 401 is used to acquire, when the memory operation interface is called, the first memory segment to be merged and the second memory segment to be merged that enter the secondary allocator to perform memory allocation. The first memory segment and the second memory segment include data segments or protection pages. When the first memory segment is a data segment, the second memory segment is a protection page. When the first memory segment is a protection page, the second memory segment is a data segment.
[0148] The determination module 402 is used to determine whether the first memory segment and the second memory segment meet the preset conditions for merging, wherein the preset conditions are used to perform a merging consistency judgment on the first memory segment and the second memory segment.
[0149] The merging module 403 is used to merge the first memory segment and the second memory segment when the first memory segment and the second memory segment meet preset conditions.
[0150] In some optional implementations, the determination module 402 includes:
[0151] The first acquisition unit is used to acquire the end address of the first memory segment and the start address of the second memory segment;
[0152] The second acquisition unit is used to acquire the attribute identifier contained in the first memory segment and the attribute identifier contained in the second memory segment when the end address of the first memory segment is the start address of the second memory segment.
[0153] The third acquisition unit is used to acquire the name of the first memory segment and the name of the second memory segment when the attribute identifier contained in the first memory segment and the attribute identifier contained in the second memory segment are the same.
[0154] The first determining unit is used to determine that the first memory segment and the second memory segment meet preset conditions when the naming of the first memory segment is the same as that of the second memory segment.
[0155] In some alternative embodiments, the device further includes:
[0156] The second acquisition module is used to acquire the memory address corresponding to the data segment before determining whether the first memory segment and the second memory segment meet the preset conditions for merging, and when it is determined that the first interface of the system is called and the corresponding execution behavior is to trigger the merging of memory segments.
[0157] The determination module is used to determine the address of the protected page based on the memory address;
[0158] The module is modified to call the system's second interface based on the address of the protected page, and to modify the function parameters of the second interface based on the attribute identifier contained in the data segment;
[0159] The modification module is used to modify the naming of protected pages and data segments based on the function parameters of the first interface.
[0160] In some alternative implementations, the second acquisition module includes:
[0161] The first judgment unit is used to determine whether the operation type parameter in the function of the first interface is the first target value after the first interface is called.
[0162] The second judgment unit is used to determine whether the first parameter in the function of the first interface is the second target value when the operation type parameter is the first target value.
[0163] The third judgment unit is used to determine whether the value of the second parameter in the function of the first interface is less than the threshold when the first parameter is the second target value.
[0164] The fourth judgment unit is used to determine whether the character of the third parameter in the function of the first interface is the target character when the value of the second parameter is less than the threshold.
[0165] The second determining unit is used to determine whether to trigger memory segment merging if the character in the third parameter is the target character.
[0166] In some alternative implementations, the module is modified, including:
[0167] The setting unit is used to set the address information and length information in the function parameters of the second interface, wherein the address information and length information are used to characterize the address range of the protected page;
[0168] The modification unit is used to modify the attribute identifier in the function parameter of the second interface to the attribute identifier contained in the data segment;
[0169] The mapping unit is used to call the second interface and remap to the protection page based on the address information and length information, so that the attribute identifier corresponding to the protection page is consistent with the attribute identifier contained in the data segment.
[0170] In some alternative implementations, the modified module includes:
[0171] The first modification unit is used to modify the second and fourth parameters based on the function parameters called through the first interface;
[0172] The unit is obtained to determine the naming range based on the modified second and fourth parameters;
[0173] The second modification unit is used to uniformly change the names of protected pages and data segments based on the naming range.
[0174] The memory segment merging apparatus provided in this disclosure can execute the memory segment merging method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.
[0175] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.
[0176] The following is a detailed reference. Figure 5 This diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present disclosure. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 502 or a program loaded from memory 505 into access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0177] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 505 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.
[0178] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 505, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the memory segment merging method of embodiments of this disclosure.
[0179] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0180] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, storage memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the memory segment merging method shown in the above embodiments is implemented.
[0181] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.
[0182] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.
Claims
1. A method for merging memory segments, characterized in that, The method includes: When the memory operation interface is called, the first memory segment to be merged and the second memory segment to be merged are obtained and entered into the secondary allocator for memory allocation. The first memory segment and the second memory segment include data segments or protection pages. When the first memory segment is a data segment, the second memory segment is a protection page. When the first memory segment is a protection page, the second memory segment is a data segment. Determine whether the first memory segment and the second memory segment meet the preset conditions for merging, wherein the preset conditions are used to perform a merging consistency judgment on the first memory segment and the second memory segment; If the first memory segment and the second memory segment meet the preset conditions, the first memory segment and the second memory segment are merged.
2. The method according to claim 1, characterized in that, The determination of whether the first memory segment and the second memory segment meet the preset conditions for merging includes: Obtain the end address of the first memory segment and the start address of the second memory segment; When the end address of the first memory segment is the start address of the second memory segment, obtain the attribute identifier contained in the first memory segment and the attribute identifier contained in the second memory segment; If the attribute identifier contained in the first memory segment is the same as the attribute identifier contained in the second memory segment, obtain the name of the first memory segment and the name of the second memory segment. When the name of the first memory segment is the same as the name of the second memory segment, it is determined that the first memory segment and the second memory segment satisfy the preset condition.
3. The method according to claim 1, characterized in that, Before determining whether the first memory segment and the second memory segment meet the preset conditions for merging, the method further includes: If it is determined that the system's first interface is called and the corresponding execution behavior is to trigger memory segment merging, the memory address corresponding to the data segment is obtained; The address of the protected page is determined based on the memory address; The system's second interface is invoked based on the address of the protected page, and the function parameters of the second interface are changed based on the attribute identifier contained in the data segment. Based on the function parameters of the first interface, modify the naming of the protected page and the data segment.
4. The method according to claim 3, characterized in that, Determining whether the corresponding execution behavior triggers memory segment merging includes: After the first interface is called, determine whether the operation type parameter in the function of the first interface is the first target value; If the operation type parameter is the first target value, determine whether the first parameter in the function of the first interface is the second target value; If the first parameter is the second target value, determine whether the value of the second parameter in the function of the first interface is less than the threshold. If the value of the second parameter is less than the threshold, determine whether the character of the third parameter in the function of the first interface is the target character; If the character in the third parameter is the target character, it is determined that the execution behavior is to trigger memory segment merging.
5. The method according to claim 3, characterized in that, The second interface of the system, which is invoked based on the address of the protected page, and the function parameters of the second interface are changed based on the attribute identifier contained in the data segment, includes: The address information and length information in the function parameters of the second interface are set, wherein the address information and length information are used to characterize the address range of the protected page; Modify the attribute identifier in the function parameters of the second interface to the attribute identifier contained in the data segment; The second interface is invoked, and based on the address information and the length information, the data is remapped to the protection page so that the attribute identifier corresponding to the protection page is consistent with the attribute identifier contained in the data segment.
6. The method according to claim 3, characterized in that, Modifying the naming of the protected page and the data segment includes: Based on the function parameters called through the first interface, change the second and fourth parameters; The naming range is obtained based on the modified second and fourth parameters; The names of the protected pages and the data segments will be uniformly changed based on the naming range.
7. A device for merging memory segments, characterized in that, The device includes: The first acquisition module is used to acquire, when the memory operation interface is called, a first memory segment to be merged and a second memory segment to be merged that are entering the secondary allocator to perform memory allocation. The first memory segment and the second memory segment include data segments or protection pages. When the first memory segment is a data segment, the second memory segment is a protection page. When the first memory segment is a protection page, the second memory segment is a data segment. The determination module is used to determine whether the first memory segment and the second memory segment meet the preset conditions for merging, wherein the preset conditions are used to perform a merging consistency judgment on the first memory segment and the second memory segment; The merging module is used to merge the first memory segment and the second memory segment when the first memory segment and the second memory segment meet the preset conditions.
8. An electronic device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the memory segment merging method of any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of merging memory segments according to any one of claims 1 to 6.
10. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the method of merging memory segments according to any one of claims 1 to 6.