Method and apparatus for offloading hierarchical memory management

Abstract

A method and apparatus offloads hierarchical memory management. The method includes obtaining a pointer to a stored memory management structure associated with a hierarchical memory, the memory management structure including a plurality of memory management entries, each memory management entry of the plurality of memory management entries including information about a memory section in one of the hierarchical memories. In some examples, the method includes scanning at least a portion of the plurality of memory management entries. In a particular example, the method includes generating a memory profile list, the memory profile list including a plurality of profile entries, each profile entry of the plurality of profile entries corresponding to a scanned memory management entry in the memory management structure.

Description

[Background technology]

[0001] Memory controllers, including direct memory access (DMA) controllers, are circuits that translate accesses generated by memory access agents, such as data processors, into one or more commands understood by computer memory. Memory controllers can be implemented as separate chips or integrated with other components, such as data processors, on a single integrated circuit chip. In the latter case, the memory controller is usually called an integrated memory controller. Integrated memory controllers support various bus protocols, such as dynamic random-access memory (DRAM) bus protocols and double data-rate dynamic random-access memory (DDR DRAM) bus protocols. The need for tight coupling between memory and computing resources such as processors, including central processing units (CPUs), graphics processing units (GPUs), and digital signal processors (DPSs), presents challenges for system designers related to memory capacity requirements, memory controller availability, memory lifetime limitations, and memory bandwidth availability to the CPU. The need for increased memory capacity is growing due to features such as in-memory workloads and server virtualization. Furthermore, improvements in CPU performance (e.g., one or more cores) necessitate more memory channels per socket. For example, memory capacity requirements are determined by the number of CPUs in order to maintain balanced computing resources for a heavy workload.

[0002] In an effort to address these evolving needs, designers have developed new types of memory and memory systems. For example, one new type of memory known as storage class memory (SCM) uses dual inline memory modules (DIMMs), which are similar to standard DDR DRAM but use NAND flash or other phase change memory as a backing store and use DRAM as a local cache for active data. Hierarchical memory refers to memory having various memory access speeds and / or memory spaces. For example, one tier of memory includes memory having a faster bus transfer speed than another tier. As another example, one tier has a smaller memory space than another tier, and such DIMM modules can be connected using the JEDEC (Joint Electron Device Engineering Council) NVDIMM-P (non-volatile dual in-line memory module-persistent) specification, which enables a new memory type structured differently from conventional DDR DRAM to be placed on this bus while leveraging the existing DDR DRAM bus.

[0003] In existing x86 systems, the "Accessed" and "Dirty" bits (A / D bits) in the operating system and / or hypervisor (OS / HV) page table are updated by either hardware, the x86 CPU in the case of I / O access, or the IOMMU (input-output memory management unit) when a particular page of memory is accessed or written. Some OS paging algorithms periodically reset these A / D bits and then examine which A / D bits are periodically set and cleared to evaluate whether a particular portion of memory is "hot" (frequently accessed) or "cold" (rarely accessed). However, such OS paging algorithms use the x86 CPU core to perform evaluations, and only very slowly and / or "on demand" (when the x86 CPU core needs to page something out due to "insufficient memory"). In this case, the OS paging algorithm uses the x86 CPU to scan (i.e., crawl) the page table, but such a scan only scans all of the DRAM every 30 to 120 seconds for the current DRAM size. Such a method would not scale performance-wise to much larger tiered memory systems, such as a second tier memory eight times larger than the DRAM.

[0004] In addition, CPU processor cores are used to periodically walk through page tables, etc., to find hot memory pages and / or cold memory pages, which divertes valuable resources from the CPU processor cores to launch software code (e.g., applications, software programs).

[0005] Furthermore, fabric memory systems are emerging that leverage these new memory interconnection standards and interfaces. One new type of memory system, known as GenZ, uses standard interface protocols between processors and media within a communication fabric to support both directly connected memory and different types of multi-level fabric-connected remote memory.

[0006] The embodiments will be more easily understood by considering the following description accompanied by the following figures, where similar reference numerals represent similar elements. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic block diagram illustrating a computing architecture for managing data in hierarchical memory according to a specific embodiment described in this disclosure. [Figure 2] This flowchart shows an example of a method for offloading hierarchical memory management according to a specific embodiment described in this disclosure. [Figure 3] This flowchart shows an example of a method for offloading hierarchical memory management according to a specific embodiment described in this disclosure. [Figure 4] These are exemplary examples of page tables and page profile lists according to specific embodiments described in this disclosure. [Figure 5] This flowchart shows an example of a method for offloading hierarchical memory management according to a specific embodiment described in this disclosure. [Modes for carrying out the invention]

[0008] This disclosure relates, in general and in particular, to methods and apparatus for offloading the management of at least a portion of hierarchical memory. In some examples, hierarchical memory refers to memory having varying memory access speeds and / or memory spaces. When a computing processor and / or input / output (I / O) device touches (e.g., accesses, reads, modifies, deletes) a particular type of memory and hierarchical memory, a memory management data structure (e.g., a page table) is automatically updated. As used herein, a computing processor refers to a processor that executes software code. In some embodiments, a memory controller offloads at least a portion of memory management from the computing processor. In some examples, the memory controller scans these memory management structures to compile a memory profile list and provides the memory profile list to the computing processor.

[0009] In some variations, the memory controller includes one or more integrated memory controllers, non-integrated memory controllers, or direct memory access (DMA) controllers, scans a page table containing memory allocation and usage information across memory pages of multiple hierarchical memories, and generates a page profile list containing page profile information for memory pages in the hierarchical memory. As used herein, an integrated memory controller refers to a memory controller integrated with a processor on an integrated circuit chip. In some examples, the integrated memory controller is integrated with a data processor that does not boot an operating system, hypervisor, and / or software application (OS / HV / application). In a particular example, the processing system has several central processing unit (CPU) cores that can boot the OS / HV / application, and many supporting processing units (e.g., data fabric, DMA controller, other microcontrollers) that may or may not be physically integrated on the same chip as the CPU cores (e.g., system-on-a-chip, SOC), but these supporting processing units cannot be used to boot the OS / HV / application. In some cases, the memory controller is a separate circuit chip that offloads at least some of the memory management from the computing processor, such as the main central processing unit (CPU). In certain cases, the memory controller identifies hot memory pages on slower tiered memory and moves or copies such hot memory pages to faster tiered memory, either automatically or under command. In one example, the memory controller is called a DMA controller or DMA engine.

[0010] There are various memory interface standards for connecting unconventional memory, such as SCMs and fabric memory systems, to computer systems. In some embodiments, these standards allow different types of memory to be directly addressed (e.g., load, store, etc.) by application code launched on the CPU, without requiring a block-based (e.g., disk-like) storage interface (e.g., PCIe (Peripheral Component Interconnect Express) or something like SATA (Serial Advanced Technology Attachment)). In fact, the memory is logically equivalent to conventional DRAM in the way that the CPU and computing processor access it. Memory access latency, bandwidth, and other performance-centric characteristics may differ substantially in some cases. Several memory interface standards, including the JEDEC NVDIMM-P standard and the Compute Express Link (CXL) standard, allow memory devices to be connected using differential serializer / deserializer (SerDes) links.

[0011] Hierarchical memory can be managed transparently or opaquely. In transparently managed systems, a single memory space is represented to the operating system and / or hypervisor (OS / HV). This requires additional hardware-level translation and / or caching / tracking mechanisms. In opaquely managed systems, hierarchical memory is managed by the OS / HV (e.g., via an existing non-uniform memory access (NUMA) style memory topology reporting mechanism) using memory management structures (e.g., page tables). In some embodiments, the OS / HV is responsible for optimizing application memory placement by leveraging the existing paging mechanism.

[0012] Some embodiments of this disclosure relate to methods and apparatus for memory management for non-transparent memory systems. In one example, a tiered memory includes a first-tier memory of DRAM and a second-tier memory connected using either a standard interface (e.g., NVDIMM-P or CXL). Some examples of the second-tier memory may include phase-change memory, phase-change-like memory, SCM, legacy DRAM technology behind a CXL memory buffer chip to enable the reuse of DRAM recovered from legacy systems having a CPU, etc. In one example, the second-tier memory may be a fabric memory that supports dynamic reallocation of memory capacity between multiple host systems based on the memory needs of target applications on those host systems.

[0013] In certain embodiments, a method performed by the memory controller includes obtaining a pointer to a stored memory management structure associated with a hierarchical memory, the memory management structure containing a plurality of memory management entries, each of which contains information about a memory section in any of the hierarchical memories. In some examples, the method includes scanning at least a portion of the plurality of memory management entries. In certain examples, the method includes generating a memory profile list, the memory profile list containing a plurality of profile entries, each of which corresponds to a scanned memory management entry in the memory management structure.

[0014] In some examples, the method includes receiving a profiling command to generate a memory profile list, and, after receiving the profiling command, scanning at least a portion of multiple memory management entries. In some examples, the profiling command includes at least one of a pointer to a stored memory management structure, a memory range of any hierarchical memory, and a matching criterion. In certain examples, the matching criterion includes a criterion mask that indicates a portion of each of the multiple memory management entries to be evaluated, and a criterion value that indicates a predetermined value for the portion of the memory management entry.

[0015] In some examples, a portion of each memory management entry includes at least one of the access bits and dirty bits. In some examples, the memory profile list includes a first list of one or more hot memory pages in the second tier of tiered memory, where the second tier has a slower memory access rate than the first tier of tiered memory. In certain examples, the memory profile list includes a second list of one or more cold memory pages in the first tier of memory.

[0016] In certain examples, the method includes the memory controller obtaining a memory location for a memory profile list, the memory controller storing the memory profile list in the memory location, and the memory controller sending an indicator that the memory profile list is being generated. In some examples, the method includes receiving a move command and moving at least one hot memory page in the second-tier memory to the first-tier memory. In some examples, each profile entry in a set of profile entries includes a memory management entry for the corresponding memory section.

[0017] In some embodiments, the integrated circuit includes a memory controller, which includes obtaining a pointer to a stored memory management structure associated with a hierarchical memory, the memory management structure includes a plurality of memory management entries, each of which includes information about a memory section in any of the hierarchical memories. In some examples, the memory controller scans at least a portion of the plurality of memory management entries to generate a memory profile list, the memory profile list includes a plurality of profile entries, each of which corresponds to a scanned memory management entry in the memory management structure.

[0018] In some examples, the memory controller receives a profiling command to generate a memory profile list and, after receiving the profiling command, scans at least a portion of several memory management entries. In some examples, the profiling command includes at least one of a pointer to a stored memory management structure, a memory range of any hierarchical memory, and a matching criterion. In some variations, the matching criterion includes a criterion mask that indicates a portion of each of the multiple memory management entries being evaluated, and a criterion value that indicates a predetermined value for the portion of the memory management entry. In some examples, each portion of the memory management entry includes at least one of an access bit and a dirty bit.

[0019] In certain examples, the memory profile list contains a first list of one or more hot memory pages in the second tier of tiered memory, where the second tier has slower memory access speeds than the first tier of tiered memory. In some examples, the memory profile list contains a second list of one or more cold memory pages in the first tier of memory.

[0020] In some examples, the memory controller obtains a memory location for a memory profile list, stores the memory profile list in the memory location, and sends an indicator that the memory profile list is being generated. In some examples, the memory controller receives a move command and moves at least one hot memory page in the second tier memory to the first tier memory. In some variations, each profile entry in a group of profile entries includes a memory management entry for the corresponding memory section. In some embodiments, the integrated circuit includes a first processor including a memory controller, a data fabric operably coupled to the memory controller, and a second processor configured to interface with the tiered memory.

[0021] In certain embodiments, the system includes a hierarchical memory configured to store application data, a processor configured to launch an application, and a memory controller operably coupled to the hierarchical memory. In some examples, the memory controller receives a pointer from the processor to a page table, the page table containing a plurality of page table entries, each page table entry containing page information about a page in the hierarchical memory. In some variations, the memory controller scans at least a portion of the plurality of page table entries in the page table to generate a page profile list, the page profile list containing a plurality of profile entries, each profile entry of the plurality of profile entries corresponding to the scanned page table entry in the page table.

[0022] In some examples, the memory controller receives a profiling command to generate a memory profile list, and after receiving the profiling command, scans at least a portion of a plurality of memory management entries. In some examples, the profiling command includes at least one of a pointer to a stored memory management structure, any memory range of a hierarchical memory, and a matching criterion. In some variations, the matching criterion includes a reference mask indicating a portion of each memory management entry of a plurality of memory management entries to be evaluated, and a reference value indicating a predetermined value of a portion of the memory management entry. In a particular variation, the portion of each memory management entry includes at least one of an access bit and a dirty bit.

[0023] In a particular example, the memory profile list includes a first list of one or more hot memory pages within a second-level memory of a hierarchical memory, and the second-level memory has a slower memory access speed than a first-level memory of the hierarchical memory. In some examples, the memory profile list includes a second list of one or more cold memory pages within the first-level memory.

[0024] In some examples, the memory controller obtains a memory location for the memory profile list, stores the memory profile list at the memory location, and transmits an indication that the memory profile list has been generated. In some examples, the memory controller receives a move command and moves at least one hot memory page within the second-level memory to the first-level memory. In some variations, each profile entry of a plurality of profile entries includes a memory management entry for a corresponding memory section.

[0025] In some embodiments, the integrated circuit includes a memory controller coupled to a hierarchical memory, obtains a pointer to a stored memory management structure associated with the hierarchical memory, the memory management structure includes a plurality of memory management entries, and each memory management entry of the plurality of memory management entries includes information about a memory section in any of the hierarchical memories. In some examples, the memory controller scans at least a portion of the plurality of memory management entries to generate a memory profile list, the memory profile list includes a plurality of profile entries, and each profile entry of the plurality of profile entries corresponds to a scanned memory management entry in the memory management structure.

[0026] In some examples, the memory controller receives a profiling command for generating a memory profile list, and after receiving the profiling command, scans at least a portion of the plurality of memory management entries. In some examples, the profiling command includes at least one of a pointer to the stored memory management structure, a memory range in any of the hierarchical memories, and a matching criterion.

[0027] In some variations, the matching criterion includes a criterion mask indicating a portion of each memory management entry of the plurality of memory management entries to be evaluated, and a criterion value indicating a predetermined value of a portion of the memory management entry.

[0028] In a specific example, a portion of each memory management entry includes at least one of an access bit and a dirty bit. In some examples, the memory profile list includes a first list of one or more hot memory pages in a second-level memory of the hierarchical memory, and the second-level memory has a slower memory access speed than a first-level memory of the hierarchical memory. In some variations, the memory profile list includes a second list of one or more cold memory pages in the first-level memory.

[0029] In some examples, the memory controller obtains a memory location for the memory profile list, stores the memory profile list in the memory location, and sends an indicator that the memory profile list is being generated. In some examples, the memory controller receives a move command and moves at least one hot memory page in the second-tier memory to the first-tier memory. In some variations, each profile entry in a multi-profile entry contains a memory management entry for the corresponding memory section.

[0030] Figure 1 is a schematic block diagram showing a computing system 100 for managing data in hierarchical memory according to a particular embodiment described herein. In some embodiments, the computing system 100 is implemented by any type of computing device suitable for implementing aspects of the embodiments of the disclosed invention. Examples of computing devices include, but are not limited to, laptops, desktops, tablet computers, handheld devices, display devices, media players, televisions, game consoles, printers, servers, cloud computing platforms, integrated circuits, and the like, all of which are intended within the scope of Figure 1 with respect to various components of the computing device 100.

[0031] In some examples, the computing system 100 includes a data processor 110, hierarchical memory 140, computing processors 150 (e.g., a central processing unit (CPU), a graphical processing unit (GPU), a digital signal processor (DSP), etc.), and one or more I / O devices 170. The computing system 100 also includes any number of additional components, different components, and / or combinations of components. One or more of the components are optional to the computing system 100. In some examples, each hierarchical memory is directly addressable memory.

[0032] In some embodiments, the computing system 100 includes one or more address buses and / or data buses (e.g., 162, 163, 164, 165, etc.) that directly and / or indirectly connect various components of the computing device 100. In some designs, any number of components of the computing device 100 or combinations thereof may be distributed and / or replicated across several computing devices. In some variations, the computing system 100 includes any number of processors (e.g., CPUs, GPUs, etc.). For example, in one variation, the computing system 100 includes a data processor 110 and a computing processor 150. In some variations, the computing system 100 includes multiple data processors and / or computing processors. As used herein, a processor refers to one or more CPUs, GPUs, GPGPUs, APUs, and / or other processing units. In some embodiments, processors 110 and / or processors 150 are part of a larger number of processors interconnected via ports and other communication links within a server system managed by a hypervisor to provide a virtualized processing system (e.g., a guest machine, a virtual machine).

[0033] In some embodiments, the processor includes a data fabric 120 and an input / output host controller (IOHC) 132. In some variations, the data fabric 120 facilitates access, reading, writing, assigning, moving, copying, and other data functions associated with hierarchical memory 140 via a communication link 162. In some examples, the data fabric 120 includes one or more data fabric components. In some variations, the IOHC 132 includes a DMA controller 130, also called a direct memory access (DMA) engine 130, coupled to the data fabric 120. In some examples, the DMA engine 130 includes a microcontroller 135 that performs memory management functions for I / O devices 170 that can access hierarchical memory, as opposed to, for example, the processor core of a processor 150. In some variations, one or more I / O devices 170 are coupled to the data fabric 120 via a communication link 163.

[0034] In some embodiments, the computing system 100 includes a computing processor 150. In one example, the computing processor 150 is an x86 CPU. In some variations, the computing processor 150 includes an operating system and / or hypervisor (OS / HV) 155. In some variations, the computing processor 150 sends and receives commands, data and / or signals to and from a DMA engine 130 via a communication link 165. In some examples, the computing processor 150 sends profiling commands 166 to the DMA engine 130. In some examples, the profiling commands 166 include addresses and / or pointers to memory management structures (e.g., page tables) 146. In one example, the profiling commands 166 include addresses and / or points to a memory profile list 148.

[0035] In some embodiments, the computing system 100 includes a tiered memory 140 comprising a first-tier memory 142 and a second-tier memory 144. In some examples, the first-tier memory 142 has a faster access speed than the second-tier memory 144. In some examples, the first-tier memory 142 has a smaller memory space than the second-tier memory 144. In one example, the first-tier memory 142 is DRAM. In one example, the second-tier memory 144 is storage-class memory (SCM).

[0036] In some embodiments, the hierarchical memory 140 includes a memory management structure (e.g., a page table) 146. In some variations, the memory management structure 146 is a page table containing a plurality of page table entries, each page table entry (PTE) containing page information for a memory page. In some examples, the page information includes one or more of the following: a physical address, a virtual address, an access bit (A bit), a dirty bit (D bit), and other memory page information. In some variations, the computing processor 150 provides an address and / or pointer to the memory management structure (e.g., a page table) 146 in a profiling command 166. In one example, the page information includes an access bit indicating whether the corresponding page was accessed within a given period. In another example, the page information includes a dirty bit (e.g., a modification bit) indicating whether the corresponding page was written within a given period.

[0037] In some examples, the processor and / or input / output (I / O) devices access a page table (e.g., a memory management structure (e.g., a page table) 146) as part of their normal operation to translate virtual addresses to physical addresses (actual memory locations), depending on the system architecture. Other address translations, such as translation from host physical addresses to system physical addresses, can be used. As used herein, virtual addresses include guest physical addresses or host physical addresses. These translation / paging mechanisms enable a computer system to host multiple virtual machines (VMs) and launch multiple programs simultaneously that have private memory (i.e., program-level views) but utilize different regions of physical memory. These translations are typically cached in a structure such as a translation lookaside buffer (TLB) to speed up access. In some examples, these translation tables are used to track when pages are accessed or written. This information can be used to inform hierarchical memory management to determine hot and cold memory pages.

[0038] In some embodiments, the hierarchical memory 140 includes a memory profile list 148 (e.g., a page profile list) generated by the DMA engine 130. In some embodiments, in response to a profiling command 166 from the hypervisor or OS, the DMA engine 130 begins scanning the memory management structure 146 and generating the memory profile list 148. In some examples, the DMA engine 130 scans the memory management structure 146 at a predetermined frequency, each scan also referred to as a profiling cycle. In some variations, the DMA engine 130 scans the memory management structure 146 at a frequency higher than once every 5 seconds, based on received profiling commands or a predetermined frequency. In some variations, the DMA engine 130 scans the memory management structure 146 at a frequency higher than once per second, based on received profiling commands or a predetermined frequency. In one example, the DMA engine 130 scans the memory management structure 146 once using a profiling command 166 received from the computing processor 150. In some embodiments, the DMA engine 130 scans some or all of the page table entries in a memory management structure (e.g., a page table) 146. In some variations, the DMA engine 130 reads the access bits and / or dirty bits in the page table entries to determine whether the corresponding memory page has been accessed.

[0039] In certain embodiments, the DMA engine 130 provides special code / interface with a processor 150 (e.g., CPU) that understands the x86 page table format. In some modifications, the DMA engine 130 is given a simple pointer (or otherwise) description of "which page table to walk" and returns a list of pages with, for example, an "A bit" set. The DMA engine 130 can be further informed of the memory ranges defining the first-tier memory 142 and / or the second-tier memory 144 so that it finds only hot pages in the second-tier memory 144. In some modifications, most pages in the second-tier memory 144 are not expected to be "hot," and therefore this scan process can see many PTEs (page table entries), most of which are of no interest, for example, the access bits and / or dirty bits are clear, indicating that they have not been used "recently." In certain modifications, the DMA engine 130 includes one or more small microcontrollers (e.g., microcontroller 135) so that it can run a simple program.

[0040] In some embodiments, the memory profile list 148 includes one or more memory management entries. In some examples, the memory management entries are from a memory management structure 146 that includes a page table. In one example, the memory management entries are from a memory management structure 146 that is different from the page table structure. In some variations, the memory profile list 148 includes a list of hot memory pages in the second-tier memory 144. As used herein, a hot memory page is also called a hot page and is a memory page that is accessed more frequently than a given threshold and / or within a given period of time. In some variations, the memory profile list 148 includes a list of cold memory pages in the first-tier memory 142. As used herein, a cold memory page is also called a cold page and is a memory page that is accessed less frequently than a given threshold and / or not within a given period of time. In some examples, the memory profile list 148 includes a plurality of memory profile entries, each memory profile entry corresponding to a scanned memory management entry in the memory management structure. In a specific example, a memory profiling entry is a page profiling entry that includes an indicator of whether the corresponding memory page is a hot memory page.

[0041] In certain embodiments, the profiling command 166 includes an address range for each level of memory in the hierarchical memory. In some cases, the profiling command 166 includes a first address range corresponding to the physical address range of the first level memory and a second address range corresponding to the physical address range of the second level memory. In some examples, the DMA engine 130 scans a memory management structure (e.g., a page table) 146 and, at least partially based on the address range, determines the specific hierarchical memory (e.g., first-level memory, second-level memory) associated with each scanned memory management entry. While some examples in this disclosure use two levels of hierarchical memory, the memory profiling technique can be used with three or more levels of hierarchical memory.

[0042] In some examples, the DMA engine 130 scans only the memory management entries (e.g., PTEs) associated with memory pages in a specific hierarchical memory (e.g., first-tier memory, second-tier memory, etc.). In some examples, the DMA engine 130 scans all memory management entries (e.g., PTEs). In certain examples, the DMA engine 130 generates a memory profile list 148 for a specific hierarchical memory (e.g., first-tier memory, second-tier memory, etc.). In some examples, the DMA engine 130 generates a memory profile list 148 for specific types of pages in a particular hierarchical memory (e.g., a hot page list in second-tier memory, a cold page list in first-tier memory, etc.). In certain examples, the DMA engine 130 generates a memory profile list 148 for all memory pages in a particular hierarchical memory, along with an indicator of whether the corresponding memory page is hot or cold. In some examples, the DMA engine 130 generates a memory profile list 148 for a specific type of page in one hierarchical memory and for all memory pages in another hierarchical memory. For example, memory profile list 148 includes a list of hot memory pages in second-tier memory and all memory pages in first-tier memory. In a specific example, page profiling command 166 includes information indicating what type of page profile list is generated and / or what information is included in the page profile list.

[0043] In some embodiments, the DMA engine 130 stores the memory profile list 148 (e.g., a page profile list) at the address and / or pointer of the memory profile list 148. After the memory profile list 148 is generated and / or updated, the DMA engine 130 sends an interrupt 167 to the computing processor 150 indicating the completion of memory profiling. In some embodiments, the computing processor 150 sends a move command 168 (e.g., a page move command) to the DMA engine 130. In one example, the DMA engine 130 moves some or all hot memory pages in the second tier memory 144 to the first tier memory 142 via the data fabric 120. In another example, the DMA engine 130 moves some or all cold memory pages in the first tier memory 142 to the second tier memory 144 via the data fabric 120.

[0044] In some variations, some or all of the components of the data processor 110 are combined into a single integrated circuit (IC). In certain variations, all of the components of the data processor 110 are combined into a single integrated circuit (IC). In some variations, one or more components of the data processor 110 (for example, part of the data fabric 120) are mounted on another IC. In some variations, some or all of the components of the data processor 110 and some or all of the components of the computing processor 150 are combined into a single IC. In certain variations, some or all of the components of the data processor 110 and some or all of the components of the hierarchical memory 140 are combined into a single IC.

[0045] Figure 2 is a flowchart illustrating an example of Method 200 for offloading hierarchical memory management according to a particular embodiment described herein. Embodiments of Method 200 are carried out, for example, by a computing device (e.g., a device implementing the computing system 100 in Figure 1) or a memory controller (e.g., the DMA engine 130 in Figure 1). In some embodiments, one or more steps of Method 200 are optional and / or modified by one or more steps of other embodiments described herein. In some embodiments, one or more steps of other embodiments described herein are added to Method 200. In this example, the memory controller receives a profiling command (e.g., a page profiling command) (210) to perform memory profiling, for example, on hierarchical memory. In some variations, the profiling command is sent from the CPU's operating system and / or hypervisor (OS / HV) to offload memory profiling from the CPU.

[0046] In some variations, the profiling command includes a pointer, memory location, and / or address of the page table associated with the hierarchical memory (e.g., memory management structure 146 in Figure 1). In certain variations, the profiling command includes the address range of each hierarchical memory. For example, the profiling command includes a first address range for the first hierarchical memory (e.g., addresses 0000 to 0100, etc.) and a second address range for the second hierarchical memory (e.g., addresses 0101 to 0999, etc.). In some examples, the first hierarchical memory has a faster access rate than the second hierarchical memory. In some examples, the first hierarchical memory has a smaller memory space than the second hierarchical memory. In one example, the first hierarchical memory is DRAM. In one example, the second hierarchical memory is storage class memory (SCM).

[0047] In some examples, the memory management structure is a page table containing multiple page table entries, each page table entry containing page information about a memory page or memory section in hierarchical memory. In some examples, the page information includes one or more of the following: physical address, virtual address, access bit, dirty bit (e.g., modification bit), and other memory page information. In certain modifications, the profiling command includes a pointer to a memory profile list (e.g., memory profile list 148 in Figure 1), a memory location, and / or address.

[0048] In some embodiments, the memory controller scans the memory management structure associated with the hierarchical memory (215). In some examples, the memory controller scans the memory management structure (e.g., page tables) at a predetermined frequency. In one example, the memory controller (e.g., integrated memory controller, DMA engine / controller, etc.) scans the memory management structure once for each profiling command received from the CPU's OS / HV. In some variations, the memory controller (e.g., DMA engine / controller, etc.) scans the memory management structure at a frequency greater than once every 5 seconds. In some variations, the memory controller (e.g., DMA engine / controller, etc.) scans the memory management structure at a frequency greater than once every second. In some embodiments, the memory controller (e.g., DMA engine / controller, etc.) scans some or all of the memory management structure. In some variations, the memory controller (e.g., DMA engine / controller, etc.) reads the access bits and / or dirty bits in the page table entries to determine whether the corresponding memory page has been accessed.

[0049] In some embodiments, the memory controller generates a memory profile list (e.g., memory profile list 148 in Figure 1) associated with a memory management structure (e.g., a page table) and corresponding memory pages in hierarchical memory. In some variations, the memory profile list is a page profile list containing a list of hot memory pages in second-tier memory (e.g., memory with slower memory access speeds, such as SCM). In certain variations, the memory profile list is a page profile list containing a list of cold memory pages in first-tier memory (e.g., memory with faster memory access speeds, such as DRAM). In some examples, the memory profile list contains multiple memory profile entries, each memory profile entry corresponding to a scanned memory management structure entry in the memory management structure. In certain examples, each memory profile entry is a page profile entry containing an indicator of whether the corresponding memory page is a hot memory page.

[0050] In certain embodiments, a profiling command received by a memory controller includes an address range for the corresponding tiered memory. For example, the profiling command includes a first address range corresponding to the physical address range of the first tier memory and a second address range corresponding to the physical address range of the second tier memory. In some examples, a memory controller (e.g., an integrated memory controller, a DMA engine / controller, etc.) scans a page table and determines the specific tiered memory (e.g., first tier memory, second tier memory) associated with each scanned page table entry. While some examples in this disclosure use two tiered memories, the page profiling technique can be used with three or more tiered memories.

[0051] In some examples, a memory controller (e.g., an integrated memory controller, DMA engine / controller, etc.) scans only the memory management entries in the memory management structure associated with memory pages in a specific tier of memory (e.g., first-tier memory, second-tier memory, etc.). In some examples, the memory controller scans all page table entries. In some examples, the memory controller generates a page profile list for a specific tier of memory (e.g., first-tier memory, second-tier memory, etc.). In some examples, the memory controller generates a page profile list for specific types of pages in a particular tier of memory (e.g., a hot page list in second-tier memory, a cold page list in first-tier memory, etc.).

[0052] In certain examples, the memory controller generates a memory profile list for all memory pages in a particular hierarchical memory, along with an indicator of whether the corresponding memory page is hot or cold. In some examples, the memory controller generates a memory profile list (e.g., a page profile list) for a specific type of page in one hierarchical memory and for all memory pages in another hierarchical memory. For example, the memory profile list includes a list of hot memory pages in the second hierarchical memory and a list of all memory pages in the first hierarchical memory. In some embodiments, the profiling command includes specific matching criteria that specify the types of pages to be scanned and / or included in the memory profile list. In some examples, the memory controller scans only the memory management entries specified in the profiling command. In certain examples, the profiling command includes information indicating which type of page profile list is generated and / or what information is included in the page profile list.

[0053] In some embodiments, a memory controller (e.g., a DMA engine / controller) stores a memory profile list at the address and / or pointer of the page profile list, which is either provided in a profiling command or received from the CPU. After the memory profile list is generated and / or updated, the memory controller sends an interrupt to the CPU indicating the completion of page profiling. In some modifications, the memory controller receives a page move command from the CPU.

[0054] In some embodiments, the memory controller migrates a specific memory section in the memory profile list (225). In some examples, the memory controller migrates a specific memory page in response to a page move command from the CPU. In some examples, the memory controller migrates a specific memory page in accordance with a page move command. In some examples, the memory controller migrates some or all of the hot memory pages in the second tier memory to the first tier memory. In one example, the memory controller migrates some or all of the cold memory pages in the first tier memory to the second tier memory.

[0055] Figure 3 is a flowchart illustrating an example of Method 300 for offloading hierarchical memory management according to a particular embodiment described herein. Embodiments of Method 300 are carried out, for example, by a computing device (e.g., a device implementing the computing system 100 in Figure 1), a memory controller (e.g., an integrated memory controller, the DMA engine 130 in Figure 1), and a CPU (e.g., the CPU 150 in Figure 1). In some embodiments, one or more steps of Method 300 are optional and / or modified by one or more steps of other examples described herein. In some embodiments, one or more steps of other examples described herein are added to Method 300. In some examples, the CPU sends a page profiling command (e.g., a page profiling command 166 in Figure 1) to the memory controller (305). In some variations, the page profiling command is sent from the CPU's operating system and / or hypervisor (OS / HV) to offload page profiling from the CPU. In this example, the page profiling command includes a page table pointer and the requested page profiling information.

[0056] In some variations, the page profiling command includes a pointer, memory location, and / or address of the page table associated with the hierarchical memory (e.g., the memory management structure (e.g., page table) 146 in Figure 1). In certain variations, the page profiling command includes the address range of each hierarchical memory. For example, the profiling command includes a first address range for the first hierarchical memory (e.g., addresses 0000 to 0100, etc.) and a second address range for the second hierarchical memory (e.g., addresses 0101 to 0999, etc.). In some examples, the first hierarchical memory has a faster access rate than the second hierarchical memory. In some examples, the first hierarchical memory has a smaller memory space than the second hierarchical memory. In one example, the first hierarchical memory is DRAM. In one example, the second hierarchical memory is storage class memory (SCM).

[0057] In some examples, a page table contains multiple page table entries, each page table entry containing page information about a memory page or memory section in hierarchical memory. In some examples, the page information includes one or more of the following: physical address, virtual address, access bit, dirty bit (e.g., modification bit), and other memory page information. In certain modifications, a page profile command contains a pointer, a memory location, and / or the address of a page profile list (e.g., memory profile list 148 in Figure 1).

[0058] In certain embodiments, a page profile command includes requested page profile information. For example, the requested page profile information includes the type of page list (e.g., cold memory page list, hot memory page list, or both), one or more specified hierarchical memory levels, access information, and / or similar. In some examples, the requested page profile information is a page profile list of hot memory pages in second-tier memory. In certain examples, the requested page profile information is a page profile list of cold memory pages in first-tier memory.

[0059] In some embodiments, a memory controller (e.g., an integrated memory controller, a DMA engine / controller, etc.) receives a page profiling command (310) to perform, for example, page profiling to hierarchical memory. In some variations, the memory controller accesses the page table via a data fabric (e.g., the data fabric 120 in Figure 1). In some examples, the memory controller scans the page table associated with the hierarchical memory (315). In some examples, the memory controller scans the page table at a predetermined frequency. In one example, the memory controller scans the page table once for each profiling command received from the CPU's OS / HV. In some variations, the memory controller scans the page table more frequently than once every 5 seconds. In some variations, the memory controller scans the page table more frequently than once every second. In some embodiments, the memory controller scans some or all of the page table entries in the page table. In some variations, the memory controller reads the access bits and / or dirty bits in the page table entries to determine whether the corresponding memory page has been accessed.

[0060] In some embodiments, the memory controller generates a page profile list (e.g., memory profile list 148 in Figure 1) associated with a page table and corresponding memory pages in hierarchical memory (320). In some variations, the page profile list includes a list of hot memory pages in second-tier memory (e.g., memory with slower memory access speeds, such as SCM). In certain variations, the page profile list includes a list of cold memory pages in first-tier memory (e.g., memory with faster memory access speeds, such as DRAM). In some examples, the page profile list includes multiple page profiles, each of which corresponds to a scanned page table entry in the page table. In certain examples, each of which includes an indicator of whether the corresponding memory page is a hot memory page.

[0061] In certain embodiments, a page profiling command received by a memory controller includes the address ranges of the corresponding tiered memory. For example, a page profiling command includes a first address range corresponding to the physical address range of the first tier memory and a second address range corresponding to the physical address range of the second tier memory. In some examples, a memory controller (e.g., a DMA engine / controller) scans a page table and determines the specific tiered memory (e.g., first tier memory, second tier memory) associated with each scanned page table entry. While some examples in this disclosure use two tiered memories, the page profiling technique can be used with three or more tiered memories.

[0062] In some embodiments, the memory controller generates a page profile list according to the requested page profile information received from the CPU. In some examples, according to the requested page profile information, the memory controller (e.g., DMA engine / controller, integrated memory controller, etc.) scans only the page table entries associated with memory pages in a particular tier memory (e.g., first tier memory, second tier memory, etc.). In some examples, according to the requested page profile information, the memory controller scans all page table entries. In certain examples, the memory controller generates a page profile list for a particular tier memory (e.g., first tier memory, second tier memory, etc.). In some examples, according to the requested page profile information, the memory controller generates a page profile list for a specific type of page in a particular tier memory (e.g., a hot page list in second tier memory, a cold page list in first tier memory, etc.).

[0063] In certain cases, the memory controller generates a page profile list for all memory pages in a particular hierarchical memory, along with an indicator of whether the corresponding memory page is hot or cold. In some cases, the memory controller generates page profile lists for a specific type of page in one hierarchical memory and for all memory pages in another hierarchical memory. For example, the page profile list might include a list of hot memory pages in the second hierarchical memory and a list of all memory pages in the first hierarchical memory. In certain cases, the page profile command includes information indicating which type of page profile list will be generated and / or what information will be included in the page profile list.

[0064] In some embodiments, a memory controller (e.g., an integrated memory controller, a DMA engine / controller, etc.) stores the page profile list in an address and / or pointer to the page profile list, which is either provided in a page profile command or received from the CPU (325). After the page profile list has been generated and / or updated, in some modifications, the memory controller sends a completion indicator to the CPU indicating that page profiling is complete (330). In some examples, the completion indicator is an indicator that the page profile list has been generated. In some examples, the completion indicator is an interrupt to the CPU.

[0065] In certain embodiments, after receiving a completion indicator, the CPU retrieves and interprets the page profile list (335). In some examples, the CPU evaluates the page profile list and selects a specific memory page for the transition. In some examples, the CPU selects a specific hot memory page in second-tier memory from the page profile list. In certain examples, the CPU selects a specific cold memory page in first-tier memory from the page profile list. In some examples, memory page selection is optional.

[0066] In some embodiments, the CPU sends a page move command to the memory controller (340). In some examples, the page move command includes specific parameters such as page number, source address, target address, and / or other information. In some examples, the page move command includes a request to move all hot memory pages in second-tier memory. In some examples, the memory controller (or main processor core) performs various functions to block and disable the current page translation, copy data from the source page to the destination page, and set up a new page translation when the data translation is complete. In some examples, the memory controller (or main processor core) performs various functions to copy data from the source page to the destination page and provide an intermediate page translation when the data translation is complete.

[0067] In some variations, the memory controller receives a page move command and migrates a specific memory page (345). In some examples, the memory controller migrates a specific memory page in accordance with the page move command. In some examples, the memory controller migrates some or all of the hot memory pages in the page profile list in the second-tier memory to the first-tier memory. In some specific examples, the memory controller migrates a specific hot memory page in the second-tier memory to the first-tier memory in accordance with the page move command. In some examples, the memory controller migrates some or all of the cold memory pages in the first-tier memory to the second-tier memory. In some examples, the memory controller migrates a specific cold memory page in the first-tier memory to the second-tier memory in accordance with the page move command.

[0068] Figure 4 is an exemplary example of a page table structure 410 and a page profile list 450 for address translation according to a particular embodiment described herein. In this example, the page table structure 410 includes a set of hierarchical address translation tables. The page table structure 410 translates a virtual address 430 to a physical address 420. The virtual address 430 has six fields, four of which are mapped to the page table structure 410, the physical page offset field provides the address offset in the physical page 420, and the sign extension field is an additional field. The page table structure 410 includes a page map level 4 table 412 that provides mappings to page map level 4 offset fields in the virtual address 430. The page table structure 410 includes a page directory pointer table 414 that provides mappings to page directory pointer offset fields in the virtual address 430. The page table structure 410 includes a page directory table 416 that provides mappings to page directory offset fields in the virtual address 430. The page table structure 410 includes a page table 418 that provides mappings to page table fields in the virtual address 430. The page table structure 410 further includes a CR3 register (Control Register) 413 for pointing to a page map level 4 base address.

[0069] The page table structure 410 includes a number of page table entries (PTEs) 412. In this example, each page table entry 412 includes a physical page base address, an "A" or access bit indicating whether the physical page has been accessed, and a "D" or dirty bit indicating whether the physical page has been modified.

[0070] The memory controller (e.g., the DMA engine 130 in Figure 1) receives a profiling command 440 and generates a page profile list 450. In some embodiments, the profiling command 440 is an example of the profiling command 166 shown in Figure 1. In some examples, the page profile list 450 includes two or more lists 452. Table 1 shows an example of a profiling command 440 that includes multiple command fields. In this example, the profiling command includes a “page table base” field, which is the starting memory location of a page table structure (e.g., the memory management structure 146 in Figure 1, the page table structure 410 in Figure 4). The profiling command also includes a “page profile list base” that provides the starting physical address for storing the page profile list 450, and a “page profile list limit” that provides the ending physical address of the page profile list 450. The profiling command 440 further includes a “page table minimum” that provides the starting virtual address (including a virtual address or guest physical address) to be scanned, and a “page table maximum” that provides the ending virtual address to be scanned. The two fields, "Page Table Minimum" and "Page Table Maximum," are optional. In some variations, the two fields "Page Table Minimum" and "Page Table Maximum" can be used to limit the number of scans and completion time per profiling command for very large address spaces.

[0071] [Table 1]

[0072] In some embodiments, the command field of the profiling command 440 includes a "system physical minimum[I]" which provides the lowest system physical address that identifies the hierarchy "I" memory (e.g., hierarchy 1 memory, hierarchy 2 memory), and a "system physical maximum[I]" which provides the highest system physical address that identifies the hierarchy "I" memory. The command field further includes a "match criterion mask[J]" which provides a description of the criterion J in the format of a masked field (e.g., a field to be scanned), and a "match criterion value[J]" which provides the required values ​​for the masked field in the criterion J. For example, if the criterion J is to make the access bit and dirty bit part of the match (e.g., bits 6, 5), the match criterion mask is set to 0x0000_0000_0000_0060, and the match criterion value is set to 0x0000_0000_0000_0020 to match only pages where the "A" bit is set (e.g., has a value of 1) and the "D" bit is not set (e.g., has a value of 0).

[0073] Other matching criteria can be added to the page profiling command 440. For example, a matching criterion called a page profiling criterion is to request a specific page that is accessed in multiple profiling commands or iterations of profiling. Another example is a page profiling criterion requesting a memory page that has not been accessed in multiple profiling commands or iterations of profiling. Yet another example is a page profiling criterion requesting a memory page associated with a specific value of one or more bits in a PTE entry that follows a particular pattern and / or assignment.

[0074] In some embodiments, the memory controller scans the page table structure 410 to generate a page profile list 450. In some examples, the page profile list 450 includes multiple page profile entries 452. In the example shown in Figure 4, the page profile entry 452 includes a reserved or "R" field, an address (e.g., physical address) field, a page table entry content or PTE content field, and a status or "S" field. In one example, the "R" field indicates whether the entry is valid. In some examples, the "R" field indicates whether the entry and the remaining entries in the page profile list 450 are valid. In some examples, the "R" field indicates the end of the page profile list 450. In some examples, the address field includes a virtual address or guest physical address that satisfies the criteria identified in the page profile command 440 (e.g., matching criterion mask and matching criterion value). The PTE content field includes the system physical address and the associated page metadata in the page table structure 410 corresponding to the virtual address. The "S" field includes status information associated with the corresponding page (e.g., hot, cold, etc.).

[0075] In certain embodiments, according to matching criteria, the profiling command 440 generates a list of entries that satisfy the matching criteria. Page profile lists 455 and 456 show lists of such entries. In one example, page profile list 455 includes a list of hot memory pages in second-tier memory that have a slower memory access rate than first-tier memory. In some variations, page profile list 455 includes one or more profile entries 147. In some examples, page profile entries 457 include a reserved or "R" field, an address (e.g., physical address) field, and a page table entry content or PTE content field. In one example, the "R" field indicates whether the entry is valid. In some examples, the "R" field indicates whether the entry and the remaining entries in page profile list 455 are valid. In some examples, the address field includes a virtual address or guest physical address that satisfies the criteria identified in page profile command 440 (e.g., matching criterion mask and matching criterion value). In certain examples, the "R" field indicates the end of page profile list 455. In some examples, the PTE content field includes the system physical address and the associated page metadata in the page table structure 410 corresponding to the virtual address.

[0076] In one example, page profile list 456 contains a list of cold memory pages in first-tier memory that have a faster memory access rate than second-tier memory. In some variations, page profile list 456 contains one or more profile entries 148, and in some examples, page profile entry 458 contains a reserved or "R" field, an address (e.g., physical address) field, and a page table entry content or PTE content field, these fields being the same as those in the profile entries described above. In some examples, page profile entry 458 contains a reserved or "R" field, an address (e.g., physical address) field, and a page table entry content or PTE content field. In one example, the "R" field indicates whether the entry is valid. In some examples, the "R" field indicates whether the entry and the remaining entries in page profile list 456 are valid. In some examples, the address field contains a virtual address or guest physical address that satisfies the criteria (e.g., matching criterion mask and matching criterion value) identified in page profile command 440. In certain examples, the "R" field indicates the end of page profile list 456. In some examples, the PTE content field includes the system physical address and the associated page metadata in the page table structure 410 corresponding to the virtual address.

[0077] Figure 5 is a flowchart illustrating an example of Method 500 for offloading hierarchical memory management according to a particular embodiment described herein. Embodiments of Method 500 are implemented, for example, by a computing device (e.g., a device implementing the computing system 100 in Figure 1), a memory controller (e.g., an integrated memory controller, the DMA engine 130 in Figure 1), and a CPU (e.g., the CPU 150 in Figure 1). In some embodiments, one or more steps of Method 500 are optional and / or modified by one or more steps of other examples described herein. In some embodiments, one or more steps of other examples described herein are added to Method 500. In some examples, the CPU or computing processor sends a page profiling command (e.g., profiling command 440 in Figure 4) to the memory controller, which includes a virtual address range, matching criteria, and other command information (505). In some examples, the page profiling command includes a page table base address and a page profile list base address.

[0078] In certain embodiments, a page profiling command includes requested page profile information, including matching criteria. For example, the matching criteria specify the type of page list, such as a cold memory page list, a hot memory page list, a page with an access bit set, a page with an access bit set for a given profiling cycle, a page with both an access bit and a dirty bit set, a page with an access bit set but no dirty bit set, and so on. In some examples, the requested page profile information includes the page table range to scan, a physical address range, and other information.

[0079] In some embodiments, a memory controller (e.g., an integrated memory controller, a DMA engine / controller, etc.) receives a page profiling command (510) and performs page profiling to, for example, hierarchical memory. In some examples, the memory controller traverses the page table (515) to identify entries that meet matching criteria. In certain examples, the memory controller scans only valid page table entries, thereby significantly reducing the number of entries scanned. In some examples, the memory controller scans the page table at a predetermined frequency. In one example, the memory controller scans the page table once for each profiling command received from the CPU's OS / HV. In some variations, the memory controller scans the page table more frequently than once every 5 seconds based on received profiling commands or a predetermined frequency. In some variations, the memory controller scans the page table more frequently than once per second based on received profiling commands or a predetermined frequency. In some embodiments, the memory controller scans some or all of the page table entries in the page table. In some variations, the memory controller reads the access bits and / or dirty bits in the page table entry to determine whether the corresponding memory page has been accessed and / or modified.

[0080] In some embodiments, during a page table scan, for each page table entry, the memory controller determines whether the page table entry meets the matching criteria (520). If the matching criteria are met, the memory controller writes the PTE content of the page table entry to the page profile list (e.g., page profile list 450 in Figure 4) (525). The memory controller determines, for example, whether the scan is complete for the page table range and physical address range specified in the page profiling command (530). If the scan is not complete, the memory controller continues traversing the page table (515). If the scan is complete, the memory controller stores the page profile list in the memory location specified in the page profiling command (535). In some embodiments, the CPU or computing processor is notified, for example, via interrupts, memory location polling, or other notification methods (540).

[0081] While features and elements are described above in specific combinations, each feature or element can be used alone without other features and elements, or in various combinations with or without other features and elements. In some embodiments, the devices described herein may be implemented in computer programs, software, or firmware embedded in a non-temporary computer-readable storage medium for implementation by a general-purpose computer or processor. Examples of computer-readable storage media include read-only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (e.g., internal hard disks and removable disks), magneto-optical media, and optical media (e.g., CD-ROM disks and digital versatile disks (DVDs)).

[0082] In the detailed descriptions of various embodiments described above, references have been made to the accompanying drawings illustrating specific preferred embodiments that form part thereof and can be used to carry out the present invention. These embodiments are described in sufficient detail to enable those skilled in the art to carry out the present invention, and it should be understood that other embodiments may be used and logical, mechanical, and electrical modifications may be made without departing from the scope of the present invention. In order to avoid details that are not necessary to enable those skilled in the art to carry out the present invention, the descriptions may omit certain information known to those skilled in the art. Furthermore, many other various embodiments incorporating the teachings of this disclosure can be readily constructed by those skilled in the art. Accordingly, the present invention is not intended to be limited to the specific forms described herein, but rather to encompass such alternative forms, modifications, and equivalents that can reasonably be included within the scope of the present invention. Accordingly, the detailed descriptions described above should not be construed as restrictive, and the scope of the present invention is defined solely by the appended claims. The detailed descriptions of the embodiments and examples described herein are presented for illustrative and explanatory purposes only, and not limiting. For example, the operations described may be performed in any suitable order or method. Accordingly, the present invention is intended to encompass any modifications, variations, or equivalents that fall within the scope of the basic principles disclosed above and claimed herein.

[0083] The above detailed description and the examples described herein are provided for illustrative and explanatory purposes only, and not for limitation.

Claims

1. A method implemented by the memory controller, In response to the memory controller receiving a profiling command from the first processor for generating a memory profile list, the memory controller obtains a pointer to a stored memory management structure associated with a hierarchical memory, wherein the stored memory management structure includes a plurality of memory management entries, and each of the plurality of memory management entries includes information about a memory section in any of the hierarchical memories. The memory controller, after receiving the profiling command, scans at least a portion of the plurality of memory management entries, The memory controller generates the memory profile list for the first processor, wherein the memory profile list includes a plurality of profile entries, and each of the plurality of profile entries corresponds to a scanned memory management entry in the stored memory management structure. method.

2. The profiling command includes at least one of the following: a pointer to the stored memory management structure, a memory range of any hierarchical memory, and a matching criterion. The method according to claim 1.

3. The matching criterion includes a criterion mask indicating the portion of each of the plurality of memory management entries to be evaluated, and a criterion value indicating a predetermined value of the portion of the memory management entry. The method according to claim 2.

4. Each of the hierarchical memory has a different memory access speed, and the portion of each memory management entry includes at least one of an access bit and a dirty bit. The method according to claim 3.

5. The memory profile list includes a first list of one or more hot memory pages in the second tier memory of the hierarchical memory, wherein the second tier memory has a slower memory access rate than the first tier memory of the hierarchical memory. The method according to claim 1.

6. The memory profile list includes a second list of one or more cold memory pages in the first tier memory. The method of claim 5.

7. The memory controller obtains the memory locations for the memory profile list from the profiling command, The memory controller stores the memory profile list in the memory location, The memory controller further includes transmitting an indicator that the memory profile list for the first processor has been generated. The method according to claim 6.

8. A memory controller comprising a circuit, The aforementioned circuit is In response to receiving a profiling command from a first processor for generating a memory profile list, the process involves obtaining a pointer to a stored memory management structure associated with a hierarchical memory, wherein the stored memory management structure includes a plurality of memory management entries, and each of the plurality of memory management entries includes information about a memory section in any of the hierarchical memories. After receiving the profiling command, scan at least a portion of the multiple memory management entries, The process involves generating the memory profile list for the first processor, wherein the memory profile list includes a plurality of profile entries, and each of the plurality of profile entries corresponds to a scanned memory management entry in the stored memory management structure. It is configured to do, Memory controller.

9. The profiling command includes at least one of the following: a pointer to the stored memory management structure, a memory range of any hierarchical memory, and a matching criterion. The memory controller according to claim 8.

10. The matching criterion includes a criterion mask indicating the portion of each of the plurality of memory management entries to be evaluated, and a criterion value indicating a predetermined value of the portion of the memory management entry. The memory controller according to claim 9.

11. Each of the hierarchical memory has a different memory access speed, and the portion of each memory management entry includes at least one of an access bit and a dirty bit. A memory controller according to claim 10.

12. The memory profile list includes a first list of one or more hot memory pages in the second tier memory of the hierarchical memory, wherein the second tier memory has a slower memory access rate than the first tier memory of the hierarchical memory. The memory controller according to claim 8.

13. The memory profile list includes a second list of one or more cold memory pages in the first tier memory. A memory controller according to claim 12.

14. Obtaining the memory location for the aforementioned memory profile list, The memory profile list is stored in the memory location, Sending an indicator that the aforementioned memory profile list has been generated, It is configured to do, The memory controller according to claim 13.

15. An integrated circuit comprising a memory controller according to any one of claims 8 to 14, A first processor comprising the memory controller, A data fabric operably coupled to the memory controller, The system further comprises a second processor configured to interface with the aforementioned hierarchical memory. Integrated circuit.

16. A system comprising a memory controller according to any one of claims 8 to 14, A hierarchical memory configured to store application data, A processor configured to launch an application, The memory controller is operably coupled to the hierarchical memory, The aforementioned memory controller Receiving a pointer to a page table from the processor, wherein the page table includes a plurality of page table entries, and each page table entry includes page information relating to a page in the hierarchical memory, Scanning at least a portion of the multiple page table entries in the page table, The process involves generating a page profile list, wherein the page profile list includes a plurality of profile entries, and each of the plurality of profile entries corresponds to a scanned page table entry in the page table. It is configured to do, system.

17. An integrated circuit comprising the system of claim 16.

Images