Page integration method and apparatus, electronic device, and storage medium

Abstract

The application provides a page integration method and device, electronic equipment and storage medium, and relates to the technical field of operating systems. The method comprises the following steps: dividing a virtual memory area into at least one sub-memory area with equal size; determining a target sub-memory area from the sub-memory areas according to the page access heat of the sub-memory areas; and integrating a first page in the target sub-memory area into a second page, wherein the size of the first page is smaller than the size of the second page. According to the technical scheme of the application, on the one hand, the page integration is constrained by using the page access heat, the use of TLB resources can be controlled, the occupation of TLB resources by large pages is reduced, and on the other hand, the sub-memory area with high page access heat and high addressing demand is preferentially used with TLB, the advantages of large pages are brought into play, and the TLB hit rate is ensured.

Description

Technical Field

[0001] This application relates to the field of operating system technology, and in particular to a page integration method, apparatus, electronic device, and storage medium. Background Technology

[0002] When operating systems manage memory, they typically use 4KB as the management unit, meaning a page is 4KB. As applications have grown larger, management units such as 16KB, 64KB, 2MB, and 1GB have emerged, meaning a page is 16KB, 64KB, 2MB, or 1GB.

[0003] The Translation Lookaside Buffer (TLB), also known as the page table cache, is primarily used to accelerate the translation from virtual to physical addresses, reducing addressing time and processor memory read operations. Since smaller pages result in larger page table entries, increasing lookup time and overhead during addressing, page consolidation is necessary during memory management, combining multiple small pages into larger pages as needed. However, too many large pages not only waste memory space, causing memory fragmentation and reducing memory utilization, but may also exceed the operating system's TLB resources, leading to a decline in system performance. Summary of the Invention

[0004] This application provides a page integration method, apparatus, electronic device, and storage medium to provide an adaptive page integration method.

[0005] In a first aspect, embodiments of this application provide a page consolidation method, comprising: dividing a virtual memory area into at least one sub-memory area of ​​equal size; determining a target sub-memory area from the sub-memory areas based on the page access frequency of the sub-memory areas; and consolidating a first page in the target sub-memory area into a second page, wherein the size of the first page is smaller than the size of the second page.

[0006] Secondly, embodiments of this application provide a page consolidation apparatus, comprising: a sub-memory area partitioning module, configured to partition a virtual memory area into at least one sub-memory area of ​​equal size; a target sub-memory area determination module, configured to determine a target sub-memory area from the sub-memory areas based on the page access frequency of the sub-memory areas; and a page consolidation module, configured to consolidate a first page in the target sub-memory area into a second page, wherein the size of the first page is smaller than the size of the second page.

[0007] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements the method provided in any embodiment of this application when executing the computer program.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method provided in any embodiment of this application.

[0009] According to the method in this application embodiment, the virtual memory area of ​​the application process is divided into sub-memory areas of equal size, and pages in the sub-memory areas with high page access frequency are merged. This involves selectively converting pages with high page access frequency in the virtual memory area into larger pages. On one hand, using page access frequency to constrain page merging can control the use of TLB resources and reduce the TLB resource occupation by large pages. On the other hand, TLB is prioritized for sub-memory areas with high page access frequency and high addressing requirements, leveraging the advantages of large pages to ensure TLB hit rate. Furthermore, before page access frequency statistics, adaptive virtual memory areas larger than a first threshold are identified based on the size of the virtual memory area. Therefore, only the larger virtual memory areas (adaptive virtual memory areas) of the process are merged based on page access frequency, ensuring full utilization of TLB resources and making the algorithm more reasonable. In addition, the upper limit of the number of large pages (i.e., second pages) will not exceed the number of TLB resources corresponding to them, thereby avoiding TLB shortages due to excessive use of large pages.

[0010] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of this application will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0011] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0012] Figure 1 This is an example diagram of a page integration method based on THP;

[0013] Figure 2 A flowchart of the page integration method provided in Embodiment 1 of this application;

[0014] Figure 3 A flowchart of the page integration method provided in Embodiment 2 of this application;

[0015] Figure 4 This is an application example diagram of the page integration method provided in Embodiment 2 of this application;

[0016] Figure 5 This is a schematic diagram of the page integration device provided in Embodiment 3 of this application;

[0017] Figure 6 This is a block diagram of the electronic device provided in Embodiment 3 of this application. Detailed Implementation

[0018] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0019] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and they all fall within the protection scope of the embodiments of this application.

[0020] The following terms are used in this document

[0021] Virtual memory area (VMA): The virtual address space of a process in the kernel is managed in virtual memory areas, which include the code segment virtual memory area and the data segment virtual memory area.

[0022] Page: In memory paging management, a process's virtual memory space is divided into several equal-sized segments called pages. Therefore, a page is a unit of memory management. Page sizes vary, such as 4KB, 16KB, 64KB, 2MB, and 1GB pages.

[0023] Normal page: mainly refers to a smaller page. In this application embodiment, it can be simply referred to as a small page, such as a 4KB page, a 16KB page, or a 64KB page.

[0024] Huge Pages: Also known as standard huge pages, these are larger pages compared to regular pages, such as 2MB pages or larger. The purpose is to use larger memory page sizes to accommodate the ever-increasing system memory, allowing the operating system to support the large page capacity of the hardware architecture.

[0025] Transparent Huge Pages (THPs): Huge pages are highly beneficial for databases using shared memory and those with a large number of processes. However, they are relatively cumbersome to configure manually and inconvenient to manage. When the shared memory exceeds the size of the huge pages, the configured huge pages become unusable, requiring reconfiguration and a system restart. THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. Transparent Huge Pages include transparent file pages and transparent anonymous pages.

[0026] Page consolidation: Consolidate multiple ordinary pages into one large page.

[0027] Large page consolidation (khugepaged) thread: The kernel can consolidate ordinary pages into large pages by starting the khugepaged thread, such as consolidating multiple 4KB pages into a 2M page.

[0028] Page access popularity: In this embodiment of the application, page access popularity refers to the access popularity (such as read operation or write operation) of a certain page in memory management.

[0029] Page tables divide physical memory space into several storage blocks of the same size as pages, called physical blocks or page frames. In memory management, the pages of a process are stored discretely in different physical blocks (page frames) of physical memory. Page tables establish a mapping between virtual addresses and physical addresses for each process; that is, the page table records the physical address corresponding to the virtual address of each page. Addressing is performed according to the page table, thus locating the physical block corresponding to each page in physical memory.

[0030] Page Table Entry (PTE): In a memory paging storage mechanism, the constituent elements of a page table, that is, each entry in the page table, is a page table entry.

[0031] Translation Lookaside Buffer (TLB): Also known as page table cache, it is a type of CPU cache primarily used to accelerate the translation from virtual addresses to physical addresses and reduce CPU memory read operations. The TLB distinguishes between code and data, called the data TLB and the code TLB, respectively. Furthermore, depending on the size of the pages cached, different TLBs are used, such as a 4KB TLB or a 2MB TLB, used to cache 4KB and 2MB pages, respectively.

[0032] TLB Miss / TLB Hit: During the CPU's virtual address to physical address translation process, if the corresponding physical address is not found in the TLB, it will read the corresponding physical address from memory. This process is called a TLB miss or hit. Conversely, if the corresponding physical address is directly found in the TLB during the same translation process, no memory access will occur. This process is called a TLB hit.

[0033] Code Cache: The Code Cache is a temporary storage area for frequently accessed code in a Java application process. Code compiled by the just-in-time compiler is placed here, and its main storage instruction is the code.

[0034] Application scenarios

[0035] Because regular pages are small, processes tend to occupy more pages, leading to excessively long page tables and increased lookup time and overhead during addressing. Using large pages reduces the operating system's maintenance of page state and improves TLB hit rate.

[0036] In related technologies, page consolidation is achieved through on / off control. For example, taking Transparent Huge Page (THP) page consolidation as an example, Linux systems typically mark different virtual memory regions by calling the "madvise" function. THP can be set to "never", "madvise", or "always". Specifically, when THP is set to "never", page consolidation is not performed on pages in the virtual memory region; when THP is set to "madvise", page consolidation is performed on file pages in the virtual memory region marked by the "madvise" function; when THP is set to "always", page consolidation is performed on anonymous pages in the virtual memory region, causing anonymous pages to use transparent huge pages. For example, as shown... Figure 1 As shown, the code cache for Java applications is used as an example. The code cache for Java applications is an anonymous page. When THP is set to "always", the code cache is added to the scan queue of the khugepaged thread, allowing the khugepaged thread to perform page khugepaged on the pages in the code cache, enabling the Java application's code cache to use transparent khugepaged.

[0037] However, this approach is not suitable for scenarios with large virtual memory areas. For example, in cloud computing scenarios, most Java applications use approximately 150MB to 400MB of virtual memory for their code segments. Using this approach will exceed the system's configured 2MB code TLB resource, leading to more TLB misses and consequently, system performance degradation. Another example is when the application's executable binary is loaded into memory. The operating system, based on memory management methods, uses file pages for the code segment and anonymous pages for the data segment. If the application's executable binary is too large, using the above memory management method for file page or anonymous page consolidation can also result in the 2MB code TLB resource being oversaturated, leading to more TLB misses and further system performance degradation.

[0038] Table 1 shows the configuration of the 2M code TLB resources for Platform 1 and Platform 2. L1 represents the Level 1 cache, and L2 represents the Level 2 cache. In Platform 1, the number of 2M L1 code TLB entries per processor core is 8, and the number of 2M L2 code TLB entries is 1536. In Platform 2, the number of 2M L1 code TLB entries per processor core is 64, and the number of 2M L2 code TLB entries is 1024.

[0039] Platform 1 8 1536 Platform 2 64 1024

[0040] Table 1

[0041] Tests revealed that for Java applications with large code caches or applications with large code segments, integrating instruction big pages through this page can easily lead to excessive use of instruction big pages. The 2M L1 code TLB resources it occupies exceed the L1 code TLB resources configured in Table 1 above, and the number of TLB missing cases increases instead of decreasing, thus affecting application performance.

[0042] Therefore, in related technologies, transparent big pages are usually disabled by default for scenarios with large virtual memory areas, and page consolidation is not performed. In other words, due to the shortcomings of using big pages in such scenarios, big pages cannot be widely applied on relevant platforms.

[0043] This application aims to provide a page consolidation method, apparatus, electronic device, and storage medium, applied to an operating system. Specifically, the kernel scans application processes, divides the virtual memory area of ​​the application processes into sub-memory areas of equal size, and then performs page consolidation on sub-memory areas with high page access frequency, while page consolidation can be abandoned for sub-memory areas with low page access frequency. This utilizes page access frequency to constrain page consolidation, controls the use of TLB resources, and ensures TLB hit rate.

[0044] Therefore, compared to page integration controlled by a switch, the technical solution of this application is an adaptive page integration scheme based on page access popularity. It can convert some pages with high page access popularity in the virtual memory area into large pages without the application's awareness and under the user's control. This reduces the TLB resource occupation of large pages, leverages the advantages of large pages, optimizes the TLB hit rate, and expands the compatibility of large pages on different platforms, increasing their application scenarios and value.

[0045] Example 1

[0046] This application provides a page integration method that can be used in an operating system and executed by the operating system. Figure 2 A flowchart illustrating a page integration method according to an embodiment of this application is shown. Figure 2 As shown, the page integration method includes:

[0047] Step S201: Divide the virtual memory area into at least one sub-memory area of ​​equal size.

[0048] In an operating system, a process's virtual address space is managed using virtual memory areas (VMAs), which include a code segment virtual memory area and a data segment virtual memory area. When an application starts, the operating system kernel scans the application process's virtual memory area and divides it into at least one sub-memory area of ​​equal size. "At least one" includes one or more sub-memory areas. For example, a 16MB virtual memory area can be divided into eight 2MB sub-memory areas.

[0049] Step S202: Determine the target sub-memory area from the sub-memory areas based on the page access popularity of the sub-memory areas.

[0050] In this embodiment, "access" can include read or write operations. The page access popularity of a certain page refers to the access frequency of that page in memory management. The page access popularity of a sub-memory area can be statistically analyzed using the access popularity of all pages in that sub-memory area. For example, the page access popularity of a sub-memory area can be represented by the number of pages accessed twice within a certain time interval. Furthermore, based on the page access popularity of each sub-memory area, the sub-memory area with high page access popularity can be selected as the target sub-memory area.

[0051] In other words, compared to other sub-memory areas, the target sub-memory area has a higher page access frequency, and therefore, the addressing requirements of the target sub-memory area and the usage requirements of the TLB are also higher.

[0052] For example, the operating system kernel determines the page access frequency of sub-memory areas by starting an access frequency statistics (kscand) thread. For instance, the virtual memory area is added to the scan queue of the kscand thread, and then the kscand thread performs page access frequency statistics on the sub-memory areas within the virtual memory area.

[0053] Step S203: Integrate the first page in the target sub-memory area into a second page, wherein the size of the first page is smaller than the size of the second page.

[0054] For example, the first page is a small page, such as a 4KB page, a 16KB page, or a 64KB page, and the second page is a large page, such as a 2MB page or a 1GB page. In one application example, by starting a khugepaged thread, scanning the virtual memory area of ​​the application process, dividing the virtual memory area into at least one sub-memory area of ​​equal size, and performing page consolidation, multiple first pages can be consolidated into a second page, such as consolidating multiple 4KB pages into a 2MB page.

[0055] Based on steps S201-203, the virtual memory area of ​​the application process is divided into sub-memory areas of equal size, and pages in the sub-memory areas with high page access frequency are consolidated, that is, the pages with high page access frequency in the virtual memory area are converted into large pages. On the one hand, using page access frequency to constrain page consolidation can control the use of TLB resources and reduce the occupation of TLB resources by large pages. On the other hand, TLB is preferentially used for sub-memory areas with high page access frequency and high addressing requirements, giving full play to the advantages of large pages and ensuring TLB hit rate. Furthermore, the technical solution of this application embodiment can be implemented without the application's awareness, thereby expanding the compatibility of large pages on different platforms and increasing its application scenarios and application value.

[0056] In particular, steps S202 and S203 can be executed asynchronously in parallel. That is, the page access popularity statistics of the kscand thread and the page integration of the khugepaged thread are executed asynchronously, which can reduce the load of the khugepaged thread. This ensures that the application can obtain the performance benefits brought by large pages, while preventing performance regression caused by excessive use of large pages.

[0057] In one implementation, the virtual memory area is obtained by scanning application processes, where the number of second pages in the application processes is less than or equal to a second threshold, which is the number of Translation Lookaside Buffers (TLBs) configured for a single processor core. The processor is deployed on the execution device of the method in this embodiment, i.e., the processor runs the aforementioned operating system. Further, the TLB corresponds to the second page; that is, the TLB is used to cache the second page. For example, if the second page is a 2MB page, then the TLB is a 2MB TLB. More specifically, if the second page is a 2MB code page, then the TLB is a 2MB code TLB.

[0058] In other words, within the same process, the number of second pages (i.e., large pages) after page consolidation must be less than or equal to the number of TLBs (secondary pages used for caching) owned (configured) by a single processor core. For example, if the second page is 2MB and the processor core is configured with 8 2MB TLBs, then the number of second pages in the same process must be less than or equal to 8. This ensures that the maximum number of large pages in the same process will not exceed the number of corresponding TLB resources, thus avoiding TLB shortages due to excessive use of large pages.

[0059] In one implementation, the size of the sub-memory region is equal to the size of the second page. For example, if the second page is a 2MB page, then the size of the sub-memory region obtained by partitioning the virtual memory region is 2MB. That is, in step 202, page access frequency statistics can be performed at the size of the second page, thereby ensuring that the size of the target sub-memory region matches the second page after page consolidation.

[0060] For example, the size of the sub-memory region is equal to the size of the second page; further, the number of second pages in the application process is equal to the second threshold. That is, the same number (if any) of target sub-memory regions can be selected for page consolidation based on the actual number of TLB resources. For example: if the second page is a 2MB page, and the operating system processor single-core is configured with 8 2MB TLBs, then in the same process, the number of target sub-memory regions is 8, and consequently, the number of second pages after consolidation is 8.

[0061] In one embodiment, the method of this application may further include: when the page access popularity of the target sub-memory region decreases, splitting the second page in the target sub-memory region into multiple first pages, and redetermining the target sub-memory region. Thus, the small pages for page consolidation in the virtual memory region can be adaptively adjusted according to changes in page access popularity.

[0062] For example, during the adaptive adjustment of the target sub-memory area, the number of second pages in the same application process is less than or equal to the second threshold mentioned above, thereby ensuring that the upper limit of the number of large pages in the same process remains unchanged, that is, it does not exceed the configured number of its corresponding TLB resources.

[0063] The following describes a specific implementation method for determining the page access popularity of a sub-memory area.

[0064] In one embodiment, the method of this application may further include: initializing the page access bits in the page table entries of the sub-memory area; after a preset time interval, determining the number of accessed pages in the sub-memory area based on the value of the page access bits; and determining the page access frequency of the sub-memory area based on the number of accessed pages.

[0065] A Page Table Entry (PTE) is an entry in the page table. A PTE includes a page access bit. For example, when a page is loaded into memory, the page access bit is initialized to 0. Then, if the page is accessed (e.g., a read or write operation), the page access bit is set to 1. The page access bit can be obtained by scanning the PTE.

[0066] For example, the page access bits of each page in the sub-memory area can be initialized first, such as clearing all PTEs corresponding to the sub-memory area; then, at a preset time interval, such as 2 seconds, the PTEs of each page in the sub-memory area can be scanned to obtain the value of the page access bits. For example, the timer function provided by the Linux kernel can be used to asynchronously count the number of PTEs that were cleared and accessed by the application again after 2 seconds, thereby counting the number of pages in the sub-memory area that were accessed again, and thus obtaining the page access popularity of the sub-memory area.

[0067] Based on this, page access popularity can be determined by scanning page table entries to perform hotspot statistics. However, this application's embodiments are not limited to this; for example, the execution frequency of specific instructions on a page can be obtained through hardware sampling and counting. However, if the sampling frequency is set high, this method will have a significant impact on system performance. The aforementioned method of determining page access popularity based on page table entries can be implemented without hardware processing, thereby reducing the impact on system performance.

[0068] As an example, the determination of page access popularity, that is, the statistics of page access popularity in each sub-memory area, can be performed by the kscand thread, which can reduce the load on the khugepaged thread. This ensures that the application can obtain the performance benefits brought by large pages, while preventing performance regression caused by excessive use of large pages.

[0069] Example 2

[0070] This application provides a page integration method that can be used in an operating system and executed by the operating system. The technical solutions in Embodiment 1 can be partially or wholly incorporated into this application. Figure 3 A flowchart illustrating a page integration method according to an embodiment of this application is shown. Figure 3 As shown, the page integration method includes:

[0071] Step S301: Determine the size of the virtual memory area. If the size of the virtual memory area is greater than the first threshold, proceed to step S302.

[0072] Step S302: Divide the virtual memory area into at least one sub-memory area of ​​equal size;

[0073] Step S303: Determine the target sub-memory region from the sub-memory regions based on the page access frequency of the sub-memory regions;

[0074] Step S304: Integrate the first page in the target sub-memory area into a second page, wherein the size of the first page is smaller than the size of the second page.

[0075] Steps S302, S303, and S304 can be performed in the same or similar manner as steps S201, S202, and S203, respectively, and will not be described again in this embodiment.

[0076] During the scanning process of the virtual memory area, virtual memory areas larger than a first threshold are identified as those requiring page access popularity statistics (hereinafter referred to as adaptive virtual memory areas). Steps S303 to S304 are then executed on these areas to perform page access popularity statistics and consolidate some smaller pages based on the statistical results. For example, the starting address of the adaptive virtual memory area can be saved as marker information in a separate array variable for further processing.

[0077] Taking the khugepaged thread as an example, the khugepaged thread will not consolidate all pages in the adaptive virtual memory area, but only consolidate small pages in the target sub-memory area (pages with high access frequency) in the adaptive virtual memory area, that is, only consolidate a portion of small pages in the adaptive virtual memory area.

[0078] According to the method in the embodiments of this application, by scanning the adaptive virtual memory region with a large process, and further selecting the target sub-memory region with high page access frequency from the adaptive virtual memory region for page integration, that is, only performing partial page integration based on page access frequency on the virtual memory region with a large process, ensuring full utilization of TLB resources and making the algorithm more reasonable.

[0079] In one implementation, before step S302, the method may further include: obtaining the number configuration information of the TLBs corresponding to the second page; and determining a first threshold. The number configuration information includes the number N of TLBs configured for a single processor core; and the first threshold is a cache size greater than or equal to N TLBs.

[0080] That is, the first threshold can be configured accordingly based on TLB resources. For example, if the second page is a 2M page, the TLB corresponding to the second page is a 2M TLB. If a single processor core has 8 2M TLB entries, then an executable virtual memory area with a size exceeding (length greater than) 16M can be used as an adaptive virtual memory area.

[0081] Based on this, it can be ensured that the maximum number of large pages will not exceed the number of their corresponding TLB resources, thereby avoiding TLB shortages due to excessive use of large pages.

[0082] For example, the device's TLB support information can be configured into the program in advance. For instance, the number of various TLBs owned by each processor core can be configured into the program. During the operating system startup process, the corresponding TLB number information (configured number) can be automatically matched according to the current processor model, that is, the number configuration information can be automatically generated, thereby saving the step of manual setting.

[0083] In one implementation, such as Figure 3 As shown, the method in this embodiment may further include step S305. In step S301, if the size of the virtual memory region is less than or equal to a first threshold, proceed to step S305: consolidate the first page in the virtual memory region into a second page. That is, for virtual memory regions with smaller processes, consolidating their pages using larger pages will not result in excessive TLB resource usage, so page consolidation can be performed according to the original page consolidation logic.

[0084] In one embodiment, the method of this application may further include: determining marking information based on the starting address of the virtual memory region when the size of the virtual memory region is greater than a first threshold; and determining the page access popularity of the sub-memory region when the marking information is identified.

[0085] For example, after determining that an adaptive virtual memory region exceeds a first threshold based on its size, the starting address of the adaptive virtual memory region can be used as a marker and stored in a separate array variable. That is, each time the `kscand` thread is awakened, if the marker information is detected, the array variable containing that marker information is processed first, and then the adaptive virtual memory region is added to the `kscand` thread's scan queue. Page access frequency statistics are then performed on the sub-memory regions within the adaptive virtual memory region, including but not limited to: initializing the page access bits in the page table entries of the sub-memory regions; determining the number of accessed pages in the sub-memory region based on the value of the page access bits after a preset time interval; and determining the page access frequency of the sub-memory region based on the number of accessed pages. This utilizes a separate asynchronous `kscand` thread to reduce the load on the `khugepaged` thread, ensuring that the application can obtain the performance benefits of large pages while preventing performance regression caused by excessive use of large pages.

[0086] In one implementation, the virtual memory area is a code segment virtual memory area, such as the code cache for Java class applications, where the first page and the second page are both code pages. In one application example, the code cache for Java class applications can be executed asynchronously based on the Linux kernel's khugepaged thread and kscand thread for access heat statistics. The first page, i.e., the small page, is an example of a 16KB page, and the second page, i.e., the large page, is an example of a 2MB page.

[0087] Specifically, such as Figure 4 As shown, the first step is to determine the adaptive virtual memory area (Adaptive VMA) by the size of the virtual memory area during the scanning process of the virtual memory area by the kernel's big page consolidation thread. For example, it can be executed in accordance with the methods of steps S301 and S302, and the adaptive VMA starting address is selected and saved to a separate array variable. That is, the big page consolidation logic of the big page consolidation thread skips all adaptive virtual memory areas.

[0088] Step 2: Each time the access popularity statistics thread is awakened, it prioritizes processing the array variable containing the starting address of the adaptive VMA (Virtual Memory Area) from Step 1, and performs page access popularity statistics on the sub-memory areas within the adaptive VMA. For example: clearing all page table entries (PTEs) in the adaptive VMA, such as initializing the page access bits in the PTEs of each sub-memory area in the adaptive VMA; using the timer function provided by the Linux kernel at preset time intervals, such as 2 seconds, asynchronously counting the number of PTEs accessed again by the application in the cleared PTEs, such as determining the number of accessed pages in the sub-memory area based on the value of each page access bit; determining the page access popularity of the sub-memory area based on the number of re-accessed PTEs. Here, the second page is a 2MB page; therefore, the size of the sub-memory area is 2MB, meaning page access popularity statistics are performed at a 2MB granularity.

[0089] Step 3: When the large page consolidation thread scans the application process again, it determines whether the access heat statistics thread is ready for the page access heat (i.e. hot spot) statistics of adaptive VMA. If it is ready, a fixed number (such as the second threshold) of 2M regions (sub-memory regions) with high hot spots are selected as target sub-memory regions and added to the page consolidation queue of the large page consolidation thread, as in step S304.

[0090] Step 4: For adaptive VMA, since the page access frequency of regions that have been integrated into large pages (such as the target sub-memory region) may decrease, it is necessary to determine this situation: split the large pages with decreased page access frequency, and at the same time integrate the new 2M region with increased page access frequency (i.e., the redefined target sub-memory region). In addition, this step needs to ensure that the upper limit (second threshold) of the number of adaptive instruction large pages (i.e., second pages) of the same process remains unchanged, which is the number of 2M code TLB entries owned by a single processor core;

[0091] By employing the four steps above and fully leveraging the large page consolidation mechanism of the Linux kernel's large page consolidation thread, the performance benefits of large pages for applications are ensured while performance regression caused by excessive large page usage is prevented. This addresses the shortcomings of the original Linux system's support for large page caching in Java applications. Furthermore, by incorporating adaptive VMAs (Virtual Memory Areas) size into the virtual memory area and dividing it into 2MB granularities (i.e., each sub-memory area of ​​the adaptive VMAs is 2MB), and by statistically analyzing page access frequency within a preset time interval (i.e., the number of pages accessed again within a 2MB area – a maximum of 512 page accesses per 2MB area), the use of instruction large pages is guaranteed not to exceed the platform's 2MB code TLB resource limit. Finally, based on the actual 2MB code TLB resource quantity, an equal number of 2MB areas (target sub-memory areas) are selected and consolidated into large pages.

[0092] In this application example, an adaptive VMA selection module, an access popularity statistics thread, and a TLB quantity configuration module can be added to the existing large page integration thread. The adaptive VMA selection module can be used to execute the method in step S301; the access popularity statistics thread can be used to execute steps S302, S303, and the method for determining page access popularity; and the TLB quantity configuration module is used to automatically configure the corresponding number of TLBs based on the current processor model, i.e., automatically generate the quantity configuration information.

[0093] The test results for the above application examples are presented below. The test scenarios included read-only and point-select scenarios in the database, with concurrency ranging from 64, 128, to 1500 concurrent users. Table 2 shows the physical machine test data. It can be seen that the above application examples achieve an average performance improvement of 2% to 3.5% in both read-only and point-select scenarios.

[0094] Concurrency: 64 27820.07 28482.51 2.4％ Concurrency: 128 32947.65 33649.61 2.1％ Concurrency: 512 35382.07 35960.02 1.6％ Concurrency: 1024 33496.60 34479.92 2.9％ Concurrency: 1500 33277.70 34111.18 2.5％ point select Figure 1 TPS solution The above application examples Performance improvement Concurrency: 64 671170.0533 691669.1033 3.1％ Concurrency: 128 844220.75 871115.6567 3.2％ Concurrency: 512 1021145.333 1058160.743 3.6％ Concurrency: 1024 898287.8467 937607.2433 4.3％ Concurrency: 1500 896833.5267 929651.03 3.7％

[0095] Table 2

[0096] Example 3

[0097] Corresponding to the method provided in the embodiments of this application, the embodiments of this application also provide a page integration apparatus. For example... Figure 5 As shown, the page consolidation device may include: a sub-memory area partitioning module 501, used to divide the virtual memory area into at least one sub-memory area of ​​equal size; a target sub-memory area determination module 502, used to determine a target sub-memory area from the sub-memory areas based on the page access frequency of the sub-memory areas; and a page consolidation module 503, used to consolidate a first page in the target sub-memory area into a second page, wherein the size of the first page is smaller than the size of the second page.

[0098] In one embodiment, the page integration apparatus of this application may further include a virtual memory area size determination module, which is used to determine the size of the virtual memory area before dividing the virtual memory area into at least one sub-memory area of ​​equal size, and to trigger the sub-memory area division module 501 to divide the virtual memory area into at least one sub-memory area of ​​equal size if the size of the virtual memory area is greater than a first threshold.

[0099] In one embodiment, the page integration apparatus of this application embodiment may further include a TLB quantity configuration module, used to obtain translation back buffer (TLB) quantity configuration information; wherein, the quantity configuration information includes the number N of TLBs configured for a single processor core, and the TLBs are used to cache second pages; and a first threshold is determined; wherein, the first threshold is greater than or equal to the cache size of N TLBs.

[0100] In one implementation, the page consolidation module 503 is further configured to consolidate the first page in the virtual memory area into a second page when the size of the virtual memory area is less than or equal to a first threshold.

[0101] In one embodiment, the page integration apparatus of this application may further include: a tag information determination module, configured to determine tag information based on the starting address of the first virtual memory region when the size of the virtual memory region is greater than a first threshold; and a page access heat determination module, configured to determine the page access heat of the sub-memory region when the tag information is identified.

[0102] In one embodiment, the page integration apparatus of this application may further include a page access popularity determination module, configured to: initialize the page access bits in the page table entries of the sub-memory area; after a preset time interval, determine the number of accessed pages in the sub-memory area based on the value of the page access bits; and determine the page access popularity of the sub-memory area based on the number of accessed pages.

[0103] In one embodiment, the target sub-memory area determination module 502 is further configured to split the second page in the target sub-memory area into multiple first pages and redetermine the target sub-memory area when the page access popularity in the target sub-memory area decreases.

[0104] In one implementation, the virtual memory area is obtained by scanning application processes, where the number of second pages in the application processes is less than or equal to a second threshold, which is the number of TLBs configured for a single processor core, and the TLBs are used to cache second pages.

[0105] In one implementation, the size of the sub-memory area is equal to the size of the second page.

[0106] In one implementation, the virtual memory area is a code segment virtual memory area, and both the first page and the second page are code pages.

[0107] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, user operation information, etc.), data (including but not limited to data used for processing, data analyzed, data stored, data displayed, etc.) and signals involved in this application are all information, data and signals authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of such information, data and signals must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0109] Figure 6 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 6 As shown, the electronic device includes a memory 601 and a processor 602. The memory 601 stores a computer program that can run on the processor 602. When the processor 602 executes the computer program, it implements the methods described in the above embodiments. The number of memories 601 and processors 602 can be one or more.

[0110] The electronic device also includes a communication interface 603 for communicating with external devices and exchanging and transmitting data.

[0111] If the memory 601, processor 602, and communication interface 603 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0112] Optionally, in a specific implementation, if the memory 601, processor 602 and communication interface 603 are integrated on a single chip, the memory 601, processor 602 and communication interface 603 can communicate with each other through an internal interface.

[0113] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in any embodiment of this application.

[0114] This application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device on which the chip is installed to perform the method provided in any embodiment of this application.

[0115] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in any embodiment of the application.

[0116] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0117] Optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include: Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM).

[0118] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0119] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0120] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0121] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0122] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0123] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0124] Furthermore, the functional units in the various embodiments of this application can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0125] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A page integration method, comprising: Obtain the quantity configuration information of the translation backup buffer (TLB), wherein the quantity configuration information includes the number N of the TLB configured for a single processor core; The number of application processes that obtain the second page is less than or equal to a second threshold, where the second threshold is the configured number; Scan the virtual memory area of ​​the application process. If the size of the virtual memory area is greater than a first threshold, divide the virtual memory area into at least one sub-memory area of ​​equal size, wherein the first threshold is greater than or equal to the cache size of N TLBs. The target sub-memory region is determined from the sub-memory regions based on the page access popularity of the sub-memory regions; The first page in the target sub-memory area is integrated into the second page, wherein the size of the first page is smaller than the size of the second page.

2. The method according to claim 1, further comprising: If the size of the virtual memory area is less than or equal to the first threshold, the multiple first pages in the virtual memory area are integrated into the second page.

3. The method according to claim 1, further comprising: If the size of the virtual memory region is greater than the first threshold, the marking information is determined based on the starting address of the virtual memory region; Upon recognizing the tagging information, the page access frequency of the sub-memory area is determined.

4. The method according to claim 1, further comprising: Initialize the page access bits in the page table entries of the sub-memory area; After a preset time interval, the number of pages accessed in the sub-memory area is determined based on the value of the page access bit. The page access popularity of the sub-memory area is determined based on the number of pages accessed.

5. The method according to claim 1, further comprising: If the page access popularity in the target sub-memory area decreases, the second page in the target sub-memory area is split into multiple first pages, and the target sub-memory area is redefined.

6. The method according to any one of claims 1 to 5, wherein, The size of the sub-memory area is equal to the size of the second page.

7. The method according to any one of claims 1 to 5, wherein, The virtual memory area is a code segment virtual memory area, and both the first page and the second page are code pages.

8. A page integration device, comprising: The sub-memory area partitioning module is used to obtain the quantity configuration information of the Translation Lookahead Buffer (TLB), wherein the quantity configuration information includes the configuration quantity N of the TLB for a single processor core; obtain application processes whose number of second pages is less than or equal to a second threshold, wherein the second threshold is the configuration quantity; scan the virtual memory area of ​​the application process, and if the size of the virtual memory area is greater than a first threshold, divide the virtual memory area into at least one sub-memory area of ​​equal size, wherein the first threshold is greater than or equal to the cache size of N TLBs; The target sub-memory area determination module is used to determine the target sub-memory area from the sub-memory areas based on the page access popularity of the sub-memory areas; A page integration module is used to integrate the first page in the target sub-memory area into the second page, wherein the size of the first page is smaller than the size of the second page.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1-7.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1-7.

Images