Memory management method, electronic device, and storage medium
By determining the physical page type according to the backend type in memory management, allocating or loading physical pages, the compatibility of too many page missing and large page management techniques in traditional page management technology is solved, improving memory performance and reducing memory overhead .
Patent Information
- Application Number
- PCT/CN2024/072509
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-17
- Filing Date
- 2024-01-16
- Publication Date
- 2024-11-21
AI Technical Summary
Traditional page management technology leads to excessive page missing processing in large-scale data processing, reduces the hit rate of page table cache, limits system performance, and the coupling between large page management technology and hardware memory management units is high, limiting the selection of large pages. Granularity leads to large memory overhead and compatibility problems also exist.
By determining different types of physical pages based on different backend types in memory management, allocating or loading physical pages, establishing a mapping relationship between virtual memory intervals and physical pages, improving the flexibility and efficiency of memory allocation, and reducing memory overhead .
Improve memory performance, reduce memory overhead, solve the problem of performance bottlenecks in large-scale data processing in traditional page management technology, and avoid the compatibility issues between large page management technology and hardware.
Smart Images

Figure CN2024072509_21112024_PF_FP_ABST
Abstract
Description
Memory management method, electronic device and storage medium
[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on May 17, 2023, with application number 202310554422.2 and application name “Memory Management Method, Electronic Device and Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of computer technology, and in particular to a memory management method, electronic device, and storage medium. Background Art
[0003] With the continuous development of computer hardware technology, in order to meet the needs of complex computing and processing, supporting larger and larger memories has become an important trend in the development of computer memory management.
[0004] Due to the small page granularity, traditional page management technology will lead to excessive page fault processing in large-scale data processing services. Large-scale data processing services involve a large range of memory access. Traditional page management technology will lead to a decrease in the page table cache hit rate, thereby limiting the overall service performance of the system.
[0005] Although large page management technology enables memory management at a larger page granularity within the operating system and can reduce the frequency of page faults in large-scale data processing services, thereby improving overall system performance, current large page management technology is highly coupled with the hardware memory management unit, limiting the selectable granularity of large pages and resulting in high memory overhead. Furthermore, using large page management technology can cause compatibility issues with operating systems that only support traditional page management technology.
[0006] Summary of the Invention
[0007] The present application provides a memory management method, electronic device and storage medium, which help to improve memory performance and reduce memory overhead.
[0008] In a first aspect, the present application provides a memory management method, comprising: allocating virtual memory in response to a received memory allocation request; creating a virtual memory interval for the allocated virtual memory, and binding the virtual memory interval to a target backend; wherein the backend is used to characterize the type of data source; determining a physical page type based on the target backend, the physical page type including an ordinary physical page type and a large physical page type; allocating or loading a physical page based on the physical page type, and establishing a first mapping relationship between the virtual memory interval and the allocated physical page or the loaded physical page.
[0009] In this application, by determining different types of physical pages based on different backends when allocating memory, the allocated physical pages can better adapt to the currently required memory, which can help improve memory performance and reduce memory overhead.
[0010] In one possible implementation, after binding the virtual memory interval to the target backend, the method further includes: in response to detecting a page fault, searching for a corresponding target virtual memory interval according to the page fault address; and determining the target backend using the target virtual memory interval.
[0011] In one possible implementation, after binding the virtual memory interval to the target backend, the method further includes: in response to receiving a mapping establishment request, searching for the corresponding target virtual memory interval according to the address of the mapping establishment request; and determining the target backend according to the target virtual memory interval.
[0012] In one possible implementation, determining the physical page type according to the target backend includes: determining a target intersection of a large page window and the target virtual memory interval, wherein the large page window is used to represent the memory space that can be accommodated by a large physical page; if the target intersection is equal to the memory space corresponding to the large physical page, determining that the physical page type is a large physical page type; or, if the target intersection is smaller than the memory space corresponding to the large physical page, determining that the physical page type is an ordinary physical page type; or, if the target intersection includes allocated physical pages or loaded physical pages, determining that the physical page type is an ordinary physical page type.
[0013] In one possible implementation, after allocating or loading a physical page according to the physical page type and establishing a first mapping relationship between the target virtual memory interval and the allocated physical page or the loaded physical page, the method further includes: updating, in response to a received update request, relevant information of the target virtual memory interval, the first mapping relationship, and one or more of the physical page information.
[0014] In one possible implementation, the allocated physical page or the loaded physical page is a large physical page, and establishing a first mapping relationship between the target virtual memory interval and the allocated physical page or the loaded physical page includes: mapping the target virtual memory interval to a partial area of the allocated physical page or a partial area of the loaded physical page; or mapping the target virtual memory interval to the entire area of the allocated physical page or the entire area of the loaded physical page.
[0015] In one possible implementation, after allocating or loading a physical page according to the physical page type and establishing a first mapping relationship between the target virtual memory interval and the allocated physical page or the loaded physical page, the method further includes: in response to detecting a reclaim request, selecting a reclaimable physical page from the allocated or loaded physical pages, and releasing the first mapping relationship between the virtual memory interval and the reclaimable physical page; wherein the reclaimable physical page is a normal physical page or a large physical page.
[0016] In one possible implementation, the reclaimable physical page is a large physical page, and the virtual memory range is mapped to a partial area of the reclaimable physical page, or the virtual memory range is mapped to the entire area of the reclaimable physical page.
[0017] In one possible implementation, the reclaimable physical page is a large physical page, and releasing the first mapping relationship between the virtual memory interval and the reclaimable physical page includes: finding the virtual memory interval corresponding to the reclaimable physical page according to a second mapping relationship, wherein the second mapping relationship is used to represent the mapping relationship between a physical page and one or more virtual memory intervals; and releasing the first mapping relationship between the virtual memory interval corresponding to the reclaimable physical page and the reclaimable physical page.
[0018] In a second aspect, the present application provides a memory management device, comprising one or more functional modules, wherein the one or more functional modules are used to implement the memory management method as described in the first aspect.
[0019] In a third aspect, the present application provides an electronic device comprising: a processor and a memory, wherein the memory is used to store a computer program; and the processor is used to run the computer program to implement the memory management method as described in the first aspect.
[0020] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer implements the memory management method as described in the first aspect.
[0021] In a fifth aspect, the present application provides a computer program, which, when executed on a processor of an electronic device, enables the electronic device to execute the memory management method as described in the first aspect.
[0022] In one possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged with the processor, or may be stored in whole or in part on a memory not packaged with the processor. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] FIG1 is a diagram of a system architecture provided by an embodiment of the present application;
[0024] FIG2 is a flow chart of an embodiment of a memory management method provided by the present application;
[0025] FIG3 is a flow chart of another embodiment of the memory management method provided by the present application;
[0026] 4a-4c are schematic diagrams of page missing scenarios provided in an embodiment of the present application;
[0027] FIG5 is a flow chart of another embodiment of the memory management method provided by the present application;
[0028] FIG6 is a schematic diagram of a large physical page splitting effect provided by an embodiment of the present application;
[0029] FIG7 is a schematic diagram of the structure of a memory management device provided in an embodiment of the present application;
[0030] FIG8 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0031] In the embodiments of this application, unless otherwise specified, the character " / " indicates that the associated objects are in an "or" relationship. For example, A / B can represent A or B. "And / or" describes the relationship between the associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exists simultaneously, or B exists alone.
[0032] It should be pointed out that the words "first", "second", etc. involved in the embodiments of this application are only used for distinguishing description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor can they be understood as indicating or implying order.
[0033] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. In addition, "at least one of the following" or similar expressions refers to any combination of these items, which may include any combination of single items or plural items. For example, at least one of A, B, or C can represent: A, B, C, A and B, A and C, B and C, or A, B and C. Among them, each of A, B, and C can be an element itself, or a set containing one or more elements.
[0034] In the embodiments of this application, the terms "exemplary," "in some embodiments," and "in another embodiment" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" in this application should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner.
[0035] In the embodiments of this application, the terms "of," "corresponding," and "relevant" may sometimes be used interchangeably. It should be noted that, when the distinction between them is not emphasized, the meanings they convey are the same. In the embodiments of this application, the terms "communication" and "transmission" may sometimes be used interchangeably. It should be noted that, when the distinction between them is not emphasized, the meanings they convey are the same. For example, "transmission" may include "sending" and / or "receiving," and may be either a noun or a verb.
[0036] In the embodiments of this application, "equal to" can be used in conjunction with "greater than" and is applicable to the technical solution adopted when "greater than" is used, and can also be used in conjunction with "less than" and is applicable to the technical solution adopted when "less than" is used. It should be noted that when "equal to" is used in conjunction with "greater than", it cannot be used in conjunction with "less than"; and when "equal to" is used in conjunction with "less than", it cannot be used in conjunction with "greater than".
[0037] The following explains some of the terms involved in the embodiments of the present application to facilitate understanding by those skilled in the art.
[0038] 1. Virtual Address Space (Vspace). The virtual address space in this application refers to the virtual memory address space corresponding to user processes in a computer operating system. This virtual address space is used to manage user process memory resources. Virtual addresses allocated to user processes are mapped to physical addresses in a page table to enable memory access.
[0039] 2. Page table. In this application, the page table is a data structure table used to store the mapping relationship between user virtual memory and physical address. Each user process's virtual address space will correspond to a page table data.
[0040] 3. Memory Mapping: Memory mapping in this application refers to the mapping relationship between the virtual address and physical address of the application process.
[0041] 4. Defragmentation: Defragmentation in this application refers to the technology of merging scattered physical pages into continuous physical pages.
[0042] With the continuous development of computer hardware technology, in order to meet the needs of complex computing and processing, supporting larger and larger memories has become an important trend in the development of computer memory management.
[0043] Due to the small page granularity, traditional page management technology will lead to excessive page fault processing in large-scale data processing services. Large-scale data processing services involve a large range of memory access. Traditional page management technology will lead to a decrease in the page table cache hit rate, thereby limiting the overall service performance of the system.
[0044] Although large page management technology enables memory management at a larger page granularity within the operating system and can reduce the frequency of page faults in large-scale data processing services, thereby improving overall system performance, current large page management technology is highly coupled with the hardware memory management unit, limiting the selectable granularity of large pages and resulting in high memory overhead. Furthermore, using large page management technology with operating systems that only support traditional page management technology can lead to compatibility issues.
[0045] Based on the above problems, an embodiment of the present application proposes a memory management method.
[0046] The memory management method provided in the embodiment of the present application is now described with reference to Figures 1-3, 4a-4c, and 5.
[0047] Figure 1 is a diagram of the system architecture provided by an embodiment of the present application. As shown in Figure 1, the system architecture includes an application layer, an operating system layer, and a hardware layer.
[0048] Among them, the application layer may include an application, which may be a system application, for example, a system application may be a driver; the application may also be a third-party application, which is not specifically limited in the embodiments of the present application.
[0049] The operating system layer may include a memory management unit (MMU), which may include a virtual address allocation module, a memory map backend module (MMB), a page table, and a page allocator, wherein the virtual address allocation module is used to allocate a continuous virtual memory range (Vregion) from the virtual address space (Vspace), and the continuous virtual memory range (Vregion) may be represented by a virtual address. The MMB may include multiple backend instances, exemplarily, the backend instances may include anonymous instances, file instances, device instances, shared instances, etc., wherein the backend is used to represent the type of data source, exemplarily, the backend may include an anonymous class backend, a file class backend, a device class backend, a shared class backend, etc.
[0050] Page table, a data structure table used to manage the mapping relationship between user virtual memory and physical addresses.
[0051] The page allocator is used to allocate physical pages corresponding to virtual addresses.
[0052] It is understood that in conventional technology, the page allocator is coupled to the hardware, meaning that the page description is shared between the page table and the page allocator. For example, the page allocator allocates physical pages at a 4KB page granularity, and the mapping between virtual addresses and physical addresses in the page table is also based on the 4KB page table granularity, resulting in limited flexibility in allocating physical pages.
[0053] In an embodiment of the present application, the page description of the page table and the page description of the page allocator are independent, thereby allowing the granularity between the page table and the page allocator to remain independent and no longer coupled, thereby improving the flexibility of physical page allocation and the efficiency of memory allocation. For example, the page allocator can allocate physical pages according to the 64KB page granularity, and the mapping relationship between virtual addresses and physical addresses in the page table can be mapped based on the 4KB page table granularity.
[0054] The hardware layer may include hardware such as a central processing unit (CPU) and a random access memory (RAM).
[0055] FIG2 is a flow chart of an embodiment of a memory management method provided by the present application, which specifically includes the following steps:
[0056] Step 201: In response to a memory allocation request from an application, the operating system allocates virtual memory from a virtual address space.
[0057] Specifically, the application program may initiate a memory allocation request, which is used to request memory allocation. In response to the memory allocation request of the application program, the operating system may allocate virtual memory from a virtual address space (Vspace).
[0058] Step 202: The operating system creates a corresponding virtual memory region (Vregion) for the allocated memory.
[0059] Specifically, after the operating system allocates virtual memory from the virtual address space (Vspace), a corresponding virtual memory interval (Vregion) can be created for the allocated virtual memory. It is understandable that the virtual memory interval (Vregion) may include relevant information such as interval information, access rights and reclaimable attributes. Among them, the interval information may include information such as the starting address and end address of the virtual memory interval (Vregion). The access rights are used to characterize the authority to access the physical memory corresponding to the virtual memory interval (Vregion). Exemplarily, the access rights may include readable, writable and readable & writable permissions, and the embodiments of the present application do not specifically limit this. The reclaimable attribute is used to characterize whether the physical memory corresponding to the virtual memory interval (Vregion) is reclaimable. Exemplarily, when memory resources are tight, the allocated physical memory can be reclaimed. However, this does not constitute a limitation on the embodiments of the present application. In some embodiments, the virtual memory interval (Vregion) may also include other information.
[0060] In step 203, the operating system binds the allocated virtual memory region (Vregion) to the target backend instance.
[0061] Specifically, after the operating system creates a virtual memory region (Vregion), the created virtual memory region (Vregion) may be bound to a target backend instance based on a preset mapping relationship.
[0062] The target backend can be obtained by searching the preset mapping relationship, which can be a mapping relationship between a request type and a backend instance. It is understood that when the operating system receives a memory allocation request, it can obtain the request type of the memory allocation request and then search the preset mapping relationship based on the request type of the memory allocation request to obtain the corresponding backend instance.
[0063] It is understood that after the virtual memory range (Vregion) is bound to the target backend instance, physical memory can be allocated and a first mapping relationship between the virtual memory range (Vregion) and the required physical memory can be established, thereby enabling access to the physical memory through the virtual memory space (Vregion). The specific process of allocating physical memory and establishing the first mapping relationship between the virtual memory range (Vregion) and the required physical memory can be referred to in subsequent embodiments and will not be repeated here.
[0064] In an embodiment of the present application, a virtual memory interval (Vregion) is allocated according to different types of memory allocation requests, and the virtual memory interval (Vregion) is bound to a backend instance, and a first mapping relationship between the required physical memory and the virtual memory interval (Vregion) is established, so that the application can use the virtual memory interval (Vregion) to access the corresponding physical memory through the first mapping relationship.
[0065] Next, allocating physical memory and establishing a first mapping relationship between virtual memory and physical memory will be exemplarily described with reference to FIG. 3 .
[0066] It is understandable that after completing the allocation of physical memory and establishing the first mapping relationship between virtual memory and physical memory, it can be considered that the memory allocation for the application is completed. During the subsequent operation of the application, the physical memory can be accessed through the virtual address.
[0067] FIG3 is a flow chart of another embodiment of the memory management method provided by the present application, which specifically includes the following steps:
[0068] Step 301: The operating system determines a target virtual memory region (Vregion).
[0069] In some optional embodiments, after the allocated virtual memory region (Vregion) is bound to the target backend instance, physical page allocation can be proactively performed. For example, the operating system can use the allocated virtual memory region (Vregion) as the target virtual memory region (Vregion) and execute step 302.
[0070] In some optional embodiments, after the allocated virtual memory range (Vregion) is bound to the target backend instance, physical page allocation can also be performed after receiving a mapping establishment request. For example, after being allocated to the virtual memory range, the application can also proactively initiate a mapping establishment request to request allocation of the corresponding physical memory. In response to receiving the mapping establishment request, the operating system can search for the corresponding target virtual memory range based on the address of the mapping establishment request; determine the target backend based on the target virtual memory range; and determine the corresponding physical page type based on the target backend.
[0071] In some optional embodiments, after the allocated virtual memory region (Vregion) is bound to the target backend instance, physical page allocation can be performed passively. For example, when a page fault is detected, the operating system can search for the corresponding target virtual memory region (Vregion) based on the faulted page address and execute step 302.
[0072] In some optional embodiments, in a page fault scenario, the operating system can also perform a legality check on the access to the virtual memory address.
[0073] If it is determined that the access to the virtual memory address is illegal, an abnormal page fault process can be performed, wherein the specific method of abnormal page fault process can refer to the existing technology, and the embodiment of the present application will not be described in detail. Or,
[0074] If it is determined that the access to the virtual memory address is a legitimate access, normal page fault processing can be performed and step 302 is executed.
[0075] In step 302, the operating system determines whether the virtual memory region (Vregion) is mapped to a large physical page.
[0076] Specifically, after determining the target virtual memory region (Vregion), the operating system may obtain the target backend instance bound to the target virtual memory region (Vregion).
[0077] Next, it can be determined whether to map large physical pages based on the target backend instance.
[0078] In some optional embodiments, the backend instance may determine to map a common physical page according to the type of the backend instance and execute step 303 .
[0079] In some optional embodiments, the backend instance may determine the mapping large physical page according to the type of the backend instance and execute step 305.
[0080] It is understood that the page granularity of a large physical page is larger than that of an ordinary physical page. For example, the page granularity of an ordinary physical page can be 4KB, and the page granularity of a large physical page can be 16KB, 32KB, 64KB, 128KB, 2MB, 1GB, etc. It is understood that the above values are only exemplary and the specific settings can be based on actual conditions. The embodiments of this application do not impose any special limitations on this.
[0081] In some optional embodiments, whether to map a large physical page may be determined according to a preset algorithm. Specifically, the algorithm may include: a backend instance may obtain a large page window, which is used to represent the memory space that can be accommodated by the large physical page. For example, if the page granularity of the large physical page is 64KB, the size of the large page window is 64KB.
[0082] Next, the intersection of the huge page window and the target virtual memory range (Vregion) is calculated, and whether to map the huge page is determined based on the intersection.
[0083] If the operating system determines to map a normal physical page, step 303 is executed. Alternatively,
[0084] If the operating system determines to map a large physical page, step 305 is executed.
[0085] A page fault scenario in which the page granularity of a normal physical page is 4 KB and the page granularity of a large physical page is 64 KB is taken as an example for illustrative description.
[0086] Figure 4a shows one of the page fault scenarios. Referring to Figure 4a, V1 is the virtual memory range (Vregion) corresponding to the page fault address, and P is the page fault address. W is the large page window, that is, the size of W is 64k. Since V1 is larger than W, the intersection of V1 and W is W. Then, the operating system can determine the page granularity of the intersection as the page granularity of the current physical page to be allocated, that is, the page granularity of W is used as the page granularity of the current physical page to be allocated, for example, 64k. The operating system can determine the virtual memory range (Vregion) corresponding to W to map the large physical page according to the page granularity of the physical page to be allocated.
[0087] It is understandable that the huge page windows may be aligned before calculating the intersection of V1 and W. There are two ways to align the huge page windows:
[0088] Method 1: Based on the offset of the virtual memory range (Vregion).
[0089] For backends without data provenance, for example, anonymous class backends, the page fault address can be offset in the virtual memory region (Vregion), and then the huge page window can be aligned with the offset page fault address.
[0090] Method 2: based on the backend offset.
[0091] For a backend with a data source type, such as a file-type backend or a device-type backend, the page fault address can be offset in the backend. For example, the page fault address can be converted to an address in the backend (such as an offset in the file), and then the converted address can be offset so that the large page window is aligned with the offset address.
[0092] Among them, the method of determining whether the page fault address needs to be offset can refer to the existing technology, and the embodiments of the present application will not be described in detail.
[0093] Figure 4b shows another page fault scenario. Referring to Figure 4b, V1 is the virtual memory region (Vregion) corresponding to the page fault address, and P is the page fault address. W is the large page window, that is, the size of W is 64k. The intersection of V1 and W is S. Since the large page portion of W other than the size of S will cause waste, the operating system can use the page granularity of the ordinary page as the page granularity of the current physical page to be allocated, for example, 4k. The operating system can determine the virtual memory region (Vregion) corresponding to S to map the ordinary physical page based on the page granularity of the physical page to be allocated.
[0094] Figure 4c shows another page fault scenario. Referring to Figure 4c, V1 is the virtual memory range (Vregion) corresponding to the page fault address, R is the occupied virtual address, that is, R is the virtual address with a physical page mapping, and P is the page fault address. W is a large page window, that is, the size of W is 64k. S is the intersection of V1 and W. Since V1 is greater than W, S is W. However, W contains the occupied virtual address R, that is, S contains a physical page that cannot be allocated. Therefore, the operating system can use the page granularity of the ordinary page as the page granularity of the current physical page to be allocated, for example, 4k. The operating system can determine the virtual memory range (Vregion) corresponding to S to map the ordinary physical page according to the page granularity of the physical page to be allocated.
[0095] It is understandable that the page granularity decision algorithm is not limited to the type in the above example.
[0096] Different backend instances have different page granularity decision algorithms, which are not limited in the embodiments of the present application.
[0097] Step 303: The operating system allocates or loads a normal physical page.
[0098] Specifically, if the operating system determines to map a normal physical page, the operating system can allocate or load the normal physical page. For example, the normal physical page can be a 4k physical page, but this does not constitute a limitation on the embodiments of the present application. The normal physical page can also be a physical page of other granularities.
[0099] In some optional embodiments, if the operating system successfully allocates or loads a normal physical page, step 304 is executed. Alternatively,
[0100] If the operating system fails to allocate or load a normal physical page, the operating system can reclaim the allocated or loaded normal physical page, reallocate or load the normal physical page, or enter the exception handling process. The specific processing method of the exception handling process can refer to the existing technology and will not be repeated here.
[0101] In step 304 , the operating system establishes a first mapping relationship between the target virtual memory region (Vregion) and a common physical page.
[0102] Specifically, the operating system can establish a first mapping relationship between the target virtual memory range (Vregion) and the ordinary physical page, that is, a mapping relationship between the virtual address and the physical address, so as to complete the allocation or loading and mapping establishment of the ordinary physical page.
[0103] Step 305: The operating system allocates or loads a large physical page.
[0104] Specifically, if the operating system determines to map a large physical page, the operating system may allocate or load the large physical page. For example, the large physical page may be a 64k physical page, but this does not constitute a limitation on the embodiments of the present application.
[0105] In some optional embodiments, if the operating system successfully allocates or loads the large physical page, step 306 is executed. Alternatively,
[0106] If the operating system fails to allocate or load a large physical page, the operating system may reclaim the allocated or loaded large physical page and reallocate or load the large physical page, or the operating system may perform defragmentation and reallocate or load the large physical page after the defragmentation is successful, or the operating system may change to allocating or loading a normal physical page. The specific method for allocating or loading a normal physical page can be found in the description of the above embodiment and will not be repeated here.
[0107] Step 306 : The operating system establishes a first mapping relationship between the target virtual memory region (Vregion) and the large physical page.
[0108] Specifically, the operating system can establish a first mapping relationship between the target virtual memory range (Vregion) and the large physical page, that is, a mapping relationship between the virtual address and the physical address, so as to complete the allocation or loading and mapping establishment of the large physical page.
[0109] In some optional embodiments, large physical pages may also support hybrid mapping. For example, a large physical page may be fully mapped, that is, a virtual memory region (Vregion) maps the entire large physical page at a preset mapping granularity, where the preset mapping granularity may be the page granularity of a normal physical page, for example, 4k. For example, with a preset mapping granularity of 4k and a large physical page granularity of 64k, after 16 mappings are completed between the virtual memory region (Vregion) and the entire large physical page, the reference count of the physical page is incremented by 1. In other words, the accumulation of physical pages is based on the page granularity of the large physical page.
[0110] Alternatively, the large physical page can be partially mapped, that is, the virtual memory interval (Vregion) maps a partial area of the large physical page at a preset mapping granularity. Taking the page granularity of 4k as an example, a partial area of a 64k large page can be mapped at a 4k granularity. Assuming that the virtual memory interval (Vregion) is 8k, the partial mapping of the virtual memory interval (Vregion) and the large physical page can be completed through two mappings. The virtual memory interval (Vregion) only maps an 8k area in the large physical page. It can be understood that each time the mapping is completed at the preset mapping granularity, the reference count of the physical page is increased by 1, that is, the accumulation of physical pages is based on the preset mapping granularity.
[0111] The above description uses Figure 3 to exemplify the allocation or loading of physical memory and the establishment of a mapping relationship between virtual memory and physical memory. During the use of an application, the application may also call a system interface to update the relevant information of a virtual memory region (Vregion), the first mapping relationship, and physical page information. Next, the following description uses Figure 4 to exemplify the updating of the relevant information of a virtual memory region (Vregion), the first mapping relationship, and physical page information.
[0112] FIG5 is a flow chart of another embodiment of the memory management method provided by the present application, which specifically includes the following steps:
[0113] Step 501 : In response to receiving an update request from an application, the operating system searches for a target virtual memory region (Vregion) in a virtual address space (Vspace).
[0114] Specifically, the application may initiate an update request to update one or more of the relevant information of the virtual memory range (Vregion), the first mapping relationship, and the physical page information.
[0115] In response to receiving an update request from an application, the operating system may determine a corresponding target virtual address space (Vspace) according to the update request, and then search for a target virtual memory region (Vregion) in the target virtual address space (Vspace).
[0116] In step 502 , the operating system traverses all allocated virtual memory regions (Vregion) in the virtual address space (Vspace).
[0117] Specifically, during the traversal process, when traversing to any allocated virtual memory interval (Vregion), the relevant information of the allocated virtual memory interval (Vregion), the first mapping relationship and one or more information of the physical page information can be updated, and step 403 is executed until all allocated virtual memory intervals (Vregion) in the target virtual address space (Vspace) are traversed.
[0118] In some optional embodiments, when traversing to any allocated virtual memory interval (Vregion), the legitimacy of the update request of the allocated virtual memory interval (Vregion) can also be judged. For example,
[0119] If the update request for the allocated virtual memory interval (Vregion) is determined to be a legitimate update request, then one or more of the related information, the first mapping relationship, and the physical page information in the allocated virtual memory interval (Vregion) may be updated. Or,
[0120] If it is determined that the update request for the allocated virtual memory interval (Vregion) is an illegal update request, an abnormal update process can be performed. The specific method of the abnormal update process can refer to the existing technology and will not be described in detail in the embodiments of the present application.
[0121] In step 503 , the operating system updates one or more of the information related to the allocated virtual memory region (Vregion), the first mapping relationship, and the physical page information.
[0122] Specifically, when updating the relevant information of any allocated virtual memory interval (Vregion), the first mapping relationship can be updated accordingly, and all physical pages mapped by the allocated virtual memory interval (Vregion) can be traversed, and the information of all physical pages mapped by the allocated virtual memory interval (Vregion) can be updated.
[0123] It is understandable that when updating the information of all physical pages, after the current physical page is updated, the next physical page can be updated until all physical pages in the currently allocated virtual memory region (Vregion) are updated.
[0124] It can be understood that after traversing all physical pages mapped to the currently allocated virtual memory interval (Vregion) and updating the relevant information of the allocated virtual memory interval (Vregion), the first mapping relationship and one or more of the physical page information, it can be considered that the update of the currently allocated virtual memory interval (Vregion) has been completed, and the update of the next allocated virtual memory interval (Vregion) can be carried out.
[0125] When updating any physical page, the page type of the physical page can be determined. The page type includes a normal page type and a large page type. The normal page type is used to indicate that the current physical page is a normal physical page, and the large page type is used to indicate that the current physical page is a large physical page.
[0126] If the current physical page is an ordinary physical page, the operating system can update the page information in the ordinary physical page. Exemplarily, the page information may include information such as physical address information and a second mapping relationship, but this does not constitute a limitation on the embodiments of the present application. In some embodiments, other information may also be included. Among them, the second mapping relationship can be considered to be a reverse mapping relationship corresponding to the first mapping relationship, and the second mapping relationship is used to characterize the mapping relationship between a physical address and one or more virtual memories. It can be understood that, contrary to the first mapping relationship, through the first mapping relationship, the corresponding physical memory can be accessed by the virtual memory interval (Vregion), and through the second mapping relationship, the corresponding virtual memory interval (Vregion) can be found by the current physical address.
[0127] If the current physical page is a large physical page, the operating system may determine a valid mapping region of the large physical page. For example, if the virtual memory region (Vregion) maps the entire large physical page, the valid mapping region is the entire large physical page; or if the virtual memory region (Vregion) maps a partial region of the large physical page, the valid mapping region is the partial region of the large physical page.
[0128] Next, the operating system may update page information corresponding to the valid mapping area according to the determined valid mapping area.
[0129] In some optional embodiments, after the current large physical page is updated, it may be determined whether the virtual memory region (Vregion) corresponding to the current large physical page needs to be split. For example, if an update request updates a portion of a large physical page, and the updated information, such as access permissions, of the portion of the physical page is inconsistent with other regions, the large physical page virtual memory region (Vregion) may be split to obtain multiple virtual memory regions (Vregions), and mapping information between the multiple virtual memory regions (Vregions) and physical pages may be created.
[0130] Figure 6 is a schematic diagram of the effect of splitting a virtual memory interval (Vregion) into multiple virtual memory intervals (Vregions). Referring to Figure 6, V1 is the allocated virtual memory interval (Vregion), which maps to a large physical page, U1 is the area to be updated by the update request, and U2 and U3 are other areas in V1 except U1. It can be seen that the update request is to update a part of the area in the large physical page. After U1 is updated, the access rights and other information of U1 may be inconsistent with those of U2 and U3. In this case, V1 can be split into three virtual memory intervals (Vregions): U1, U2, and U3. Since these three virtual memory intervals (Vregions) are not large enough to map to a large physical page, these three virtual memory intervals (Vregions) can be mapped to ordinary physical pages respectively.
[0131] Now, an example of updating the relevant information of the virtual memory interval (Vregion) to reclaim the physical page is used for illustration. For example, when the operating system detects that the memory resources are tight, it can reclaim the allocated physical page or the loaded physical page to reclaim the physical memory. It is understandable that in addition to detecting that the memory resources are tight, the triggering of the reclaiming of the physical page can also include other scenarios. For example, the application can actively release the physical memory to reclaim the physical memory, or, as mentioned above, when the operating system fails to allocate or load the physical page (including the ordinary physical page or the large physical page), the allocated or loaded physical page can be reclaimed. The embodiment of the present application does not make any special limitation on this.
[0132] It is understood that when any of the above scenarios occurs, the operating system can initiate a reclaim request to reclaim the allocated or loaded physical page. In addition, after the operating system determines the physical page to be reclaimed, it can use the second mapping relationship to find the virtual memory range (Vregion) corresponding to the physical page to be reclaimed.
[0133] If the physical page to be reclaimed is a normal physical page, the mapping between the normal physical page to be reclaimed and the virtual memory region (Vregion) can be released, and the reference count and the second mapping relationship of the normal physical page to be reclaimed can be updated. Or,
[0134] If the current physical page to be reclaimed is a large physical page, the mapping area of the virtual memory interval (Vregion) in the large physical page to be reclaimed can be calculated, and the first mapping relationship between the mapping area and the virtual memory interval (Vregion) can be released based on the mapping area, and the reference count and second mapping relationship of the large physical page to be reclaimed can be updated.
[0135] For example, the virtual memory interval corresponding to the large physical page to be reclaimed may be found according to the second mapping relationship, and then the first mapping relationship between the virtual memory interval corresponding to the physical page to be reclaimed and the physical page to be reclaimed may be released.
[0136] The updating method of the reference count of the large physical page may be determined according to whether the updated large physical page supports hybrid mapping. The specific implementation method of the reference count of the large physical page can refer to the relevant description in the above embodiment and will not be repeated here.
[0137] FIG7 is a schematic diagram of the structure of an embodiment of the memory management device of the present application. As shown in FIG7 , the memory management device 70 may include: an allocation module 71, a creation module 72, a determination module 73 and a mapping module 74; wherein,
[0138] an allocation module 71 for allocating virtual memory in response to a received memory allocation request;
[0139] A creation module 72 is configured to create a virtual memory interval for the allocated virtual memory and bind the virtual memory interval to a target backend; wherein the backend is used to represent the type of data source;
[0140] A determination module 73 is configured to determine a physical page type according to the target backend, where the physical page type includes a common physical page type and a large physical page type;
[0141] The mapping module 74 is configured to allocate or load a physical page according to the physical page type, and establish a first mapping relationship between the virtual memory interval and the allocated physical page or the loaded physical page.
[0142] In one possible implementation, the determination module 73 is further configured to, in response to detecting a page fault, search for a corresponding target virtual memory interval according to the page fault address; and determine the target backend based on the target virtual memory interval.
[0143] In one possible implementation, the determination module 73 is further configured to, in response to receiving a mapping establishment request, search for a corresponding target virtual memory interval according to an address of the mapping establishment request; and determine the target backend according to the target virtual memory interval.
[0144] In one possible implementation, the determining module 73 is further configured to determine a target intersection of a large page window and the target virtual memory range, wherein the large page window is configured to represent a memory space that can be accommodated by a large physical page.
[0145] If the target intersection is equal to the memory space corresponding to the large physical page, determine that the physical page type is a large physical page type; or,
[0146] If the target intersection is smaller than the memory space corresponding to the large physical page, the physical page type is determined to be a common physical page type; or
[0147] If the target intersection includes an allocated physical page or a loaded physical page, it is determined that the physical page type is a common physical page type.
[0148] In one possible implementation, the memory management device 70 further includes:
[0149] An update module is configured to update, in response to a received update request, one or more of the relevant information of the target virtual memory range, the first mapping relationship, and the physical page information.
[0150] In one possible implementation, the allocated physical page or the loaded physical page is a large physical page.
[0151] In one possible implementation, the mapping module 74 is further configured to map the target virtual memory interval to a partial area of the allocated physical page or the loaded physical page; or
[0152] The target virtual memory range is mapped to the entire area of the allocated physical page or the loaded physical page.
[0153] In one possible implementation, the memory management device 70 further includes:
[0154] a release module, configured to, in response to detecting a reclaim request, select a reclaimable physical page from the allocated or loaded physical pages, and release the first mapping relationship between the virtual memory interval and the reclaimable physical page;
[0155] The reclaimable physical page is a common physical page or a large physical page.
[0156] In one possible implementation, the reclaimable physical page is a large physical page, and the virtual memory range is mapped to a partial area of the reclaimable physical page, or the virtual memory range is mapped to the entire area of the reclaimable physical page.
[0157] In one possible implementation, the reclaimable physical page is a large physical page, and the release module is further configured to find a virtual memory interval corresponding to the reclaimable physical page according to a second mapping relationship, wherein the second mapping relationship is used to represent a mapping relationship between a physical page and one or more virtual memory intervals;
[0158] The first mapping relationship between the virtual memory interval corresponding to the reclaimable physical page and the reclaimable physical page is released.
[0159] The memory management device 70 provided in the embodiment shown in FIG7 can be used to execute the technical solution of the method embodiment shown in this application. Its implementation principle and technical effects can be further referred to the relevant description in the method embodiment.
[0160] It should be understood that the division of the various modules of the memory management device 70 shown in Figure 7 above is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software calling through processing elements; they can also all be implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. During the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or the instructions in the form of software.
[0161] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more microprocessors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, these modules may be integrated together to implement a system-on-a-chip (SOC).
[0162] The following further describes an exemplary electronic device provided in the embodiments of the present application in conjunction with Figure 8. Figure 8 shows a schematic structural diagram of an electronic device 800.
[0163] The above-mentioned electronic device 800 may include: at least one processor; and at least one memory communicatively connected to the above-mentioned processor, wherein: the above-mentioned memory stores program instructions that can be executed by the above-mentioned processor, and the processor calls the above-mentioned program instructions to execute the method provided in the embodiment shown in this application.
[0164] Figure 8 shows a block diagram of an exemplary electronic device 800 suitable for implementing embodiments of the present application. The electronic device 800 shown in Figure 8 is merely an example and should not limit the functions and scope of use of the embodiments of the present application.
[0165] As shown in Figure 8, electronic device 800 is implemented as a general-purpose computing device. Components of electronic device 800 may include, but are not limited to, one or more processors 810, memory 820, a communication bus 840 connecting various system components (including memory 820 and processor 810), and a communication interface 830.
[0166] Communication bus 840 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of such architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnection (PCI) bus.
[0167] The electronic device 800 typically includes a variety of computer system readable media, which can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.
[0168] The memory 820 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (Random Access Memory; hereinafter referred to as: RAM) and / or a cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Although not shown in Figure 8, a disk drive for reading and writing a removable non-volatile disk (such as a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (such as a Compact Disc Read Only Memory (hereinafter referred to as: CD-ROM), a digital versatile disc read-only disk (hereinafter referred to as: DVD-ROM) or other optical media) may be provided. In these cases, each drive can be connected to the communication bus 840 via one or more data medium interfaces. The memory 820 may include at least one program product having a set (e.g., at least one) program modules that are configured to perform the functions of each embodiment of the present application.
[0169] A program / utility having a set (at least one) of program modules may be stored in memory 820. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. The program modules generally perform the functions and / or methods of the embodiments described herein.
[0170] The electronic device 800 can also communicate with one or more external devices (e.g., a keyboard, pointing device, display, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication can be performed via a communication interface 830. Furthermore, the electronic device 800 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter (not shown in FIG. 8 ). The network adapter can communicate with other modules of the electronic device via a communication bus 840. It should be understood that, although not shown in FIG. 8 , other hardware and / or software modules can be used in conjunction with the electronic device 800, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0171] The processor 810 executes various functional applications and data processing by running the programs stored in the memory 820, such as implementing the method provided in the embodiment of the present application.
[0172] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 800. In other embodiments of the present application, the electronic device 800 may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.
[0173] In the above embodiments, the processor involved may include, for example, a CPU, a DSP, a microcontroller, or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units; hereinafter referred to as: NPU) and an image signal processor (hereinafter referred to as: ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASICs, or one or more integrated circuits for controlling the execution of the program of the technical solution of this application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.
[0174] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer executes the method provided by the embodiment shown in the present application.
[0175] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program product is run on a computer, it enables the computer to execute the method provided by the embodiment shown in the present application.
[0176] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b, c can be single or multiple.
[0177] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented using a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0178] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0179] In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), magnetic disk or optical disk, and other media that can store program code.
[0180] The above description is merely a specific embodiment of the present application. Any person skilled in the art may easily conceive of variations or substitutions within the technical scope disclosed in this application, and such variations or substitutions shall be within the scope of protection of this application. The scope of protection of this application shall be subject to the scope of protection of the claims.
Claims
1. A memory management method, characterized in that: The method comprises: In response to the received memory allocation request, allocating virtual memory; Creating a virtual memory interval for the allocated virtual memory, and binding the virtual memory interval to a target backend; wherein the backend is used to characterize the type of data source; Determine a physical page type according to the target backend, where the physical page type includes a common physical page type and a large physical page type; A physical page is allocated or loaded according to the physical page type, and a first mapping relationship between the virtual memory interval and the allocated physical page or the loaded physical page is established.
2. The method according to claim 1, characterized in that After binding the virtual memory interval to the target backend, the method further includes: In response to detecting a page fault, searching a corresponding target virtual memory interval according to the page fault address; The target backend is determined according to the target virtual memory interval.
3. The method according to claim 1, characterized in that After binding the virtual memory interval to the target backend, the method further includes: In response to receiving a mapping establishment request, searching for a corresponding target virtual memory interval according to an address of the mapping establishment request; and determining the target backend according to the target virtual memory interval.
4. The method according to claim 1, characterized in that: Determining the physical page type according to the target backend includes: Determining a target intersection of a large page window and the target virtual memory interval, wherein the large page window is used to represent a memory space that can be accommodated by a large physical page; If the target intersection is equal to the memory space corresponding to the large physical page, determine that the physical page type is a large physical page type; or, If the target intersection is smaller than the memory space corresponding to the large physical page, the physical page type is determined to be a common physical page type; or, If the target intersection includes an allocated physical page or a loaded physical page, it is determined that the physical page type is a common physical page type.
5. The method according to any one of claims 1 to 4, characterized in that: After allocating or loading a physical page according to the physical page type and establishing a first mapping relationship between the target virtual memory interval and the allocated physical page or the loaded physical page, the method further includes: In response to the received update request, one or more of the relevant information of the target virtual memory range, the first mapping relationship and the physical page information is updated.
6. The method according to any one of claims 1 to 5, characterized in that: The allocated physical page or the loaded physical page is a large physical page, and the establishing of a first mapping relationship between the target virtual memory interval and the allocated physical page or the loaded physical page includes: Mapping the target virtual memory interval to a partial area of the allocated physical page or a partial area of the loaded physical page; or, The target virtual memory interval is mapped to the entire area of the allocated physical page or the entire area of the loaded physical page.
7. The method according to claim 1, characterized in that After allocating or loading a physical page according to the physical page type and establishing a first mapping relationship between the target virtual memory interval and the allocated physical page or the loaded physical page, the method further includes: In response to detecting a reclaim request, selecting a reclaimable physical page from the allocated or loaded physical pages, and releasing a first mapping relationship between the virtual memory interval and the reclaimable physical page; The recyclable physical page is a common physical page or a large physical page.
8. The method according to claim 7, characterized in that The reclaimable physical page is a large physical page, and the virtual memory interval is mapped to a partial area of the reclaimable physical page, or the virtual memory interval is mapped to the entire area of the reclaimable physical page.
9. The method according to claim 7, characterized in that: The reclaimable physical page is a large physical page, and the releasing of the first mapping relationship between the virtual memory interval and the reclaimable physical page includes: Finding a virtual memory interval corresponding to the reclaimable physical page according to a second mapping relationship, wherein the second mapping relationship is used to represent a mapping relationship between a physical page and one or more virtual memory intervals; The first mapping relationship between the virtual memory interval corresponding to the reclaimable physical page and the reclaimable physical page is released.
10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program; and the processor is used to run the computer program to implement the method according to any one of claims 1 to 9.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the method according to any one of claims 1 to 9 is implemented.
Citation Information
Patent Citations
Memory management method, electronic equipment and storage medium
CN119003142A
Virtual machine thermal migration method and device, electronic equipment and computer storage medium
CN112328354A
Memory mapping method and device of host machine, electronic device and computer readable medium
CN113326094A
Page missing exception processing method and device, electronic equipment and storage medium
CN115543532A
RDMA enabled I / O adapter performing efficient memory management
US20060236063A1
Cited By
Data processing method, electronic equipment, system, storage medium and program product
CN121166043A
