Extended Page Information Coprocessor

Abstract

The processing system includes a primary processor and a coprocessor. The primary processor is connectable to a memory subsystem having at least one memory and is operative to execute system software that uses memory address translation based on one or more page tables stored in the memory subsystem. The coprocessor is also connectable to the memory subsystem and is operative to perform page table walk iterations through one or more page tables maintained in the memory subsystem and to perform one or more page management operations on behalf of the system software based on the page table walk iterations. The page management operations performed by the coprocessor include analytical data aggregation, free list management and page allocation, page migration management, and page table error detection.

Description

[Background technology]

[0001] Many processing systems employ a paged memory allocation scheme in which data is moved between a secondary storage (e.g., mass storage) and memory in fixed-size blocks, typically called "pages." Each page represents a corresponding subset of memory address locations. A set of one or more page tables facilitates translating a virtual memory address of a referenced memory location to a physical memory address of that memory location. Typically, these page tables are stored in memory, except for a translation lookaside buffer (TLB), which acts as a cache that stores a copy of a small subset of the page table entries. Thus, unless an address translation involves one of the small number of page table entries present in the TLB, the address translation process involves accessing a page table entry of a page table in memory, and thus one or more memory access operations. Thus, an operating system or hypervisor dedicates a significant portion of the processing bandwidth of a central processing unit (CPU) or other primary processor to page table walking in memory and otherwise managing the page tables. While memory sizes grow, page sizes tend to remain fixed (e.g., 4 kilobytes) for legacy compatibility and other reasons, and thus the page tables managed by the primary processor become larger and larger, placing an ever-increasing burden on the page management process on the primary processor system.

[0002] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by reference to the following drawings, in which: The use of the same reference numbers in different drawings indicates similar or identical items. [Brief description of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram illustrating a processing system that uses a co-processor for memory page management, according to some embodiments. [Diagram 2] 2 is a block diagram illustrating the coprocessor of FIG. 1 in further detail, according to some embodiments. [Diagram 3] 2 is a block diagram illustrating a page table and corresponding page table entries (PTEs) according to some embodiments. [Figure 4] FIG. 3 is a flow diagram illustrating a method of evaluating and managing page tables provided by the coprocessor of FIGS. 1 and 2 according to some embodiments. [Diagram 5] FIG. 3 is a flow diagram illustrating a method of page transition provided by the co-processor of FIGS. 1 and 2 according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0004] In conventional processing systems, system software (i.e., either the operating system (OS) or hypervisor (HV) or both) periodically performs a page table walk, reading and selectively clearing the accessed (A), dirty (D) and other status bits. By sampling the pages during the page table walk, the system software can gather a limited amount of usage information that is provided to a memory manager component and used to determine which pages of data to evict in order to accept incoming pages that satisfy a memory request. However, while the size of system memory has increased, the page size has remained relatively fixed (e.g., 4 kilobyte pages) for a variety of reasons. As a result, the size of the page tables has grown, and thus the percentage of processor cycles that the system software spends collecting page statistics has increased, which impairs the system's ability to execute user tasks and other high priority tasks.

[0005] To ease the burden of memory management on a central processing unit (CPU), graphics processing unit (GPU), or other primary processor that executes system software, in at least one embodiment the processing system employs a separate, dedicated extended page information coprocessor (EPIC) that offloads some or all of the page management duties traditionally used by system software, and in some embodiments also operates to provide additional extended page management functionality. This coprocessor, referred to herein as a "memory processing unit" or "MPU," has limited general-purpose processing power compared to the primary processor, and is configured by the system software to perform page table walk iterations of one or more page tables, and to aggregate or generate various information during the page table walks without utilizing the resources of the primary processor. Information generated during the page walk iterations includes, for example, information about which pages were accessed, which pages were modified (i.e., "dirty"), frequency of access, and the completeness of the page table entries or the page tables themselves. The MPU uses this information to generate one or more ordered lists, such as an accessed page list or a dirty page list, which in some embodiments are provided to system software for use in selecting pages for eviction by the system software, or in other embodiments the MPU performs page selection and eviction based on this information and in accordance with policies specified by the system software, independent of the primary processor. Further, in some embodiments the processing system employs a non-uniform memory access (NUMA) architecture having multiple memories, and the MPU identifies candidate pages for migration between memories, and in some embodiments notifies system software of these candidate pages for system software controlled page migration, or in other embodiments operates to perform page migration independent of the system software.By offloading various page management operations to reduce the burden on system software, the MPU allows the primary processor to spend a greater percentage of its cycles performing other tasks.

[0006] 1 illustrates a processing system 100 that uses an Extended Page Information Coprocessor (EPIC) to offload memory page management from a primary processor and to provide supplemental memory page management operations, according to some embodiments. The processing system 100 may be utilized in any of a variety of electronic devices, including desktop computers, laptop computers, tablet computers, servers, computing-enabled mobile phones (smartphones), game consoles, computing-enabled televisions, and the like. The processing system 100 includes one or more primary processors 102, a memory subsystem 104 comprised of one or more memories 106 (e.g., memory 106-1 and memory 106-2), and an EPIC in the form of a memory processing unit (MPU) 108. The primary processor 102, memory 106, and MPU 108 are coupled for communication of signals and data via one or more buses, networks, or other interconnects, which are omitted from FIG. 1 for ease of illustration.

[0007] The primary processor 102 operates to execute one or more system software applications, having executable instructions stored in at least one memory 106, for overall management of the system 100 and to facilitate execution of user software applications. These system software applications may include, for example, an operating system (OS) or a hypervisor (HV), which, in combination or alternatively, are referred to as an OS / HV 110. Examples of primary processors 102 include central processing units (CPUs), graphics processing units (GPUs), artificial intelligence (AI) processors, application specific integrated circuits (ASICs), etc. For purposes of explanation, in an exemplary embodiment, the primary processor 102 is described below as a CPU, and more specifically as a CPU implementing the x86 instruction set architecture (ISA), although corresponding descriptions apply to other processor types and other ISAs as well, unless otherwise noted.

[0008] Primary processor 102 includes one or more processor cores 112, such as processor cores 112-1 and 112-2, that operate to execute OS / HV 110 and user software applications implemented in system 100. Additionally, primary processor 102 includes one or more interfaces to memory subsystem 104, such as one or more of memory management unit (MMU) 114, input / output MMU (IOMMU) 116, and direct memory access (DMA) engine 118. As is known in the art, the MMU 114 typically operates to transfer data between the processor core 112 (and one or more caches in an associated cache hierarchy (not shown)) and one or more memories 106, the IOMMU 116 typically operates to transfer data between the cache hierarchy, the one or more memories 106, and one or more peripheral devices (not shown), and the DMA engine 118 operates to transfer data to and from the one or more memories 106 without the involvement of the processor core 112 or the MMU 114.

[0009] In at least one embodiment, each memory 106 of the memory subsystem 104 is implemented as a "paged" memory, whereby a set (or a subset) of the physical storage locations of the memory 106 is logically divided into pages 120 of a set of pages 122. Typically, each page 120 of a given memory 106 is the same fixed size as all other pages of the same memory 106, with the page size often being set by policies defined by the OS / HV 110. For various reasons, including backward compatibility, for example, the physical storage locations of memory in an x86-based system are organized as pages 120 of 4 KB each, although different page sizes may be mixed in certain circumstances. With respect to primary processor 102 accesses to the memory 106, each page 120 is managed as a monolithic block. Thus, when a page management operation is performed to transfer identified data from memory 106 to a secondary storage component (e.g., a disk drive) of computer system 100, typically the entire page 120 containing the data is transferred to memory subsystem 104 as part of the page operation. Similarly, when a page-out operation is performed to transfer identified data from memory 106 to secondary storage, the entire page 120 containing the particular data is transferred. Although page sizes may vary, each page is transferred as a unit.

[0010] Because the addressable memory space of the processing system 100 is typically much larger than the size of the one or more memories 106, only a subset of the available pages of data may be stored in the memory 106 at any given time. Thus, in at least one embodiment, the OS / HV 110 employs a virtual memory management scheme having a virtual memory space that uses a set of virtual addresses (also commonly referred to as "linear addresses") that is larger than the set of physical addresses of the storage locations of the one or more memories 106. In this scheme, the OS / HV 110 uses a virtual-to-physical address translation process to translate each virtual address to a corresponding physical address, and then uses the physical address to access a corresponding page 120 in the memory 106 (assuming that the referenced page 120 has actually been loaded into the memory 106). To facilitate this virtual-to-physical address translation, the processing system 100 uses a page table structure 124 made up of one or more page tables 126. In at least one embodiment, page table structure 124 is a hierarchical page table structure, e.g., one subfield of a virtual address identifies an entry in a first page table, which points to a second page table, another subfield of the virtual address identifies an entry in the second page table, which points to a third page table, and so on.

[0011] To illustrate, one conventional long mode nesting paging scheme for 4KB pages and 64-bit virtual addresses employs four layers of page tables. At the first layer is a Page Map Level 4 (PML4) table that contains multiple entries, each of which points to a specific one of multiple Page Directory Pointer (PDP) tables. A 9-bit PML4 offset field in the virtual address specifies which PML4 table entry is to be used. Each entry in each PDP table points to a specific one of multiple Page Directory (PD) tables, and a 9-bit PDP offset subfield in the virtual address specifies a specific entry in the specific PDP table to be used. Each entry in each PD table points to a specific one of multiple last level page tables, and a 9-bit PD offset field in the virtual address specifies a specific entry in the specific PD table to be used. Each last level page table contains a collection of page table entries (PTEs), each of which stores a 52-bit physical address portion that is combined with bits of the 12-bit physical page offset subfield of the virtual address to generate the physical address that maps to the virtual address.

[0012] It will be appreciated that the processing system 100 typically accommodates multiple processes simultaneously through context switches or another time division multiplexing technique, with some or all of these processes often having their own respective virtual address space. Thus, each process or context typically uses its own separate page table structure 124. In the example long mode hierarchical paging scheme described above, the base address of the upper table (PML4 table) is specified using a specific control register 128 that stores an identifier of the currently active process / context, which serves to specify the base address. By way of example, in an x86 context, this control register 128 is implemented by a CR3 register that stores a process context identifier (PCID), which identifies the current process / context and is used to calculate the base address of the PML4 table of the page table structure 124 used for the identified process / context. For ease of explanation, the operation of the processing system 100 is described below with reference to the long mode paging scheme described above, and an example format of the PTE is described below with reference to FIG. 3. However, it will be understood that the techniques described herein are not limited to this exemplary embodiment, but rather may be applied to any of a variety of paging schemes using the guidelines provided herein.

[0013] Because the page tables 126 are stored in one or more memories 106, performing each translation from a virtual address to a physical address can involve a long memory access operation. Thus, to facilitate fast address translation, in at least one embodiment, the primary processor 102 utilizes a translation lookaside buffer (TLB) 130, which is a cache structure that operates to locally store copies of a subset of recently used PTEs for easy access by the MMU 114. Thus, each time a page 120 is allocated in the memory 106, the PTEs of the page tables 126 for that page 120 are also stored in the TLB 130 until the TLB 130 is full. One or more TLB management schemes are then used to identify which PTEs to store copies for in the TLB 130 and which PTEs to evict from the TLB 130 to make room for new incoming TLB entries. Such schemes are often based on the recency of access, the frequency of access, and the like. Traditionally, the MMU 114 managed the TLB 130. However, as described below, in some embodiments, MPU 108 operates to perform some or all of the management functions of TLB 130 on behalf of MMU 114 .

[0014] The MPU 108 is implemented as a co-processor for one or more primary processors 102 to offload traditional page management operations performed by the OS / HV 110 in conventional systems, thus consuming a portion of the execution bandwidth of the processor cores 112, and / or to leverage the capabilities of the MPU 108 to provide enhanced page management operations that dedicate most or all of the execution bandwidth of the MPU 108 to page management operations. That is, in at least one embodiment, except for page management, the MPU 108 has limited general-purpose processing capabilities compared to the primary processors 102 (i.e., cannot perform the general-purpose tasks of the primary processors 102). Thus, in some embodiments, the MPU 108 is implemented as a software or firmware instruction executing processor, such as a reduced instruction set computer (RISC) processor, or the like, with software / firmware instructions stored in memory 106, a basic input / output system (BIOS) or other read-only memory (ROM), or the like, that operate the MPU 108 to perform the functions described herein. In other embodiments, MPU 108 is implemented as hard-coded logic (e.g., an ASIC or other fixed function integrated circuit) or programmable logic (e.g., a programmable logic device) that performs the described functions. In yet other embodiments, MPU 108 is implemented as some combination of an instruction execution processor and hard-coded or programmable logic.

[0015] The MPU 108 operates to perform its various page management operations in response to commands from the OS / HV 110 or independent of any triggering commands from the OS / HV 110. To illustrate, the page allocation process of the processing system 100 relies on the use of at least one free list 132 that identifies which physical pages 120 of one or more memories 106 are "free" i.e., available for allocation to corresponding virtual pages. In some embodiments, the MPU 108 operates independently to periodically or otherwise recursively walk the page table 126 of each page table structure 124, identify pages 120 that are not in use (e.g., are not referenced and do not contain modified data), and mark such pages 120 as free in the free list 132. In other embodiments, the MPU 108 performs a page table walk in response to a command from the OS / HV 110 to perform a page table walk, which command may specify that only the identified page table structures 124 are to be "walked" for free list analysis. As another example, which will be described in more detail below, MPU 108 generates a list of pages that are candidates for page migration between respective memories 106 and provides this list to OS / HV 110, which then selects the pages to migrate and controls the migration of the selected pages, while in other embodiments MPU 108 identifies the pages to migrate and then performs the actual page migration independent of OS / HV 110, such that the page migration that occurs is transparent to OS / HV 110. As yet another example, in some embodiments, when MPU 108 walks page table 126, MPU 108 operates to collect and analyze or manipulate various page statistics or other analytical data (analysis page data 134) for pages 120, such as the number or ratio of modified pages to unchanged pages, etc.In some embodiments, MPU 108 operates to collect and process this data independent of instructions from OS / HV 110, while in other embodiments, MPU 108's data collection process is triggered via a command from OS / HV 110, which may also specify the type of data to collect and analyze.

[0016] Regardless of whether a particular page management operation performed by MPU 108 is triggered by system software or independent of commands from system software, in at least one embodiment such operation is performed according to one or more policies specified by OS / HV 110. By way of example, OS / HV 110 may set policies regarding management of a free list of pages available for allocation, the types of page information to collect, the selection and prioritization of page migration candidates, etc.

[0017] Communication and signaling of commands, data, policies, and other information between primary processor 102 and MPU 108 may be implemented in any of a variety of ways and combinations of ways. In some embodiments, processing system 100 uses a collection of one or more queues 136, such as queues 136-1 and 136-2, in memory 106 to facilitate communication. By way of example, in one embodiment, queue 136-1 is implemented as an input queue that receives commands, configuration information, and other data from primary processor 102 for access by MPU 108, while queue 136-2 is implemented as an output queue that receives analysis page data 134, updated free list 132, etc. from MPU 108 for access by OS / HV 110. In other embodiments, a single queue 136 is used for both commands and information from OS / HV 110 to MPU 108 and vice versa, where a "packet" of information includes an identifier of the destination (primary processor or MPU) and an identifier of the type of communication (e.g., command, list, statistics, configuration data, etc.). In still other embodiments, different types of communication utilize different queues. For example, commands from primary processor 102 to MPU 108 are inserted into one queue 136, free list 132 is inserted by MPU 108 into another queue 136, etc. In addition to or instead of using memory-based queues or other memory-based data structures for commands, data, and information passed between primary processor 102 and MPU 108, in some embodiments, communication between primary processor 102 and MPU 108 uses one or more buses or other interconnects extending between primary processor 102 and MPU 108. For example, primary processor 102 may issue a command to MPU 108 via a Peripheral Component Interconnect Express (PCIE) bus, an inter-chip Global Memory Interconnect (xGMI) bus, or other interconnect, and MPU 108 may signal the result of the operation by issuing an interrupt via the same or a different interconnect, while one or more queues 136 or other memory locations may be used to buffer data or information related to the command or interrupt.

[0018] In an effort to assist primary processor 102, there is a risk that MPU 108 and primary processor 102 may attempt to simultaneously access the same memory page, the same page table or page table entry, the same TLB entry, etc. Thus, in at least one embodiment, MPU 108 works in concert with primary processor 102 to prevent such simultaneous access attempts. In some embodiments, this involves the use of selectively asserted "flags" so that only one component operates on a corresponding memory component at any given time, such as by preventing conflicting scheduling decisions, to facilitate a safety or system integrity policy. These "flags" are typically memory locations programmed with a signal value (e.g., a mutex or semaphore) such that when the memory location is set to a specified value, other processors avoid one or more types of memory management operations.

[0019] FIG. 2 illustrates an embodiment of the MPU 108 in more detail. In some embodiments, the MPU 108 includes one or more of a page table walker 202, a data aggregation / reporting module 204, a PTE management module 206, and a page migration module 208. As noted above, these components may be implemented in the MPU 108 through hard-coded or programmable logic, or a combination thereof, via execution of corresponding instructions by a processor implementing the MPU 108. The page table walker 202 operates to walk the page tables 126 of the page table structures 124 currently resident in one or more memories 106. To this end, the page table walker 202 implements logic similar to the page table walk component found in a conventional MMU, which similarly performs page table walks using any of a variety of conventional or proprietary page table walk algorithms. However, whereas a conventional MMU actually attempts to perform a virtual to physical address translation while performing a page table walk, the page table walker 202 does not need to perform such a translation, but rather walks the entire page table to examine the contents of each PTE contained in the page table. Additionally, while conventional MMUs are typically limited to walking only the page tables of the current process / context, in some embodiments, the page table walker 202 is not so limited, but instead walks the page tables 126 of some or all of the process / context that have page table structures 124 resident in memory 106. Illustratively, the page table walker 202 implements an array that stores the CR3 values ​​of each process / context, selects one CR3 value, identifies the top-level page table of the page table structures 124 of the process / context associated with that CR3 value, walks the page tables of the page table structures 124, and upon completion, selects the next CR3 value in the array and begins the page table walk process again with the page table structures 124 indexed with the newly selected CR3 value.The page table walk process is described in more detail below with reference to FIG.

[0020] The data aggregation / reporting module 204 operates to collect data regarding the PTEs of the page tables 126 as the page table walker 202 walks the page tables 126 of the selected page table structure 124. This collected data may be aggregated or analyzed at one or more levels of granularity, such as for the entire memory subsystem 104, per memory, per region, per page, etc. By way of example, the data aggregation / reporting module 204 may determine the number of modified (i.e., "dirty") pages per region, or the average number of modified pages for multiple memory regions of one of the memories 106, etc. The data aggregation / reporting module 204 further operates to provide a representation of this captured / analyzed data to the primary processor 102 as analyzed page data 134. The PTE management module 206 operates to utilize the data collected by the data aggregation / reporting module 204 to manage the page tables 126 itself. This includes, for example, changing the status / attribute fields of PTEs in page table 126, marking pages as candidates for page migration, detecting and reporting errors in page table 126, identifying and addressing orphaned pages, populating and evicting pages from free list 132, updating TLB 130 to reflect changes in page locations and status, and identifying pages for power regulation purposes. Further details regarding the operation of modules 204 and 206 are described below with reference to FIG.

[0021] In some embodiments, the memory subsystem 104 has a non-uniform memory access (NUMA) configuration that allows the primary processor 102 to access different memories 106 at different speeds. In some embodiments, this difference in access time is based, at least in part, on the implementation of different types of memory. For example, one or more memories 106 are volatile random access memory (RAM), while one or more other memories 106 are flash memory or other non-volatile memory. This difference in architecture can also be the result of different memory technologies. For example, one memory implements dynamic RAM (DRAM) technology while another memory 106 implements static RAM (SRAM) technology, or one memory is a Dynamic Data Rate 2 (DDR2) based DRAM and another memory is a Dynamic Data Rate 4 (DDR4) based memory. Other factors that may cause differences in access speed include, for example, different clock speeds, different distances to the primary processor 102, different bus widths, etc. Furthermore, in addition to or instead of different access times, different memories 106 often provide different advantages and disadvantages. For example, some types of memory may have slower access times but offer better storage capacity or better power consumption, while other memories may have faster access times but be more susceptible to write wear and other degradation in proportion to the amount of memory usage.

[0022] Thus, in at least one embodiment, the processing system 100 uses page migration to opportunistically identify situations where moving an allocated page 120 from one memory 106 to another memory 106 would provide benefits, whether in terms of access speed, reduced power consumption, freeing up space for additional pages in faster memory, reduced write wear, etc. To this end, when the page table walker 202 walks the page table 126, the page migration module 208 operates to determine whether the page 120 associated with the PTE currently being accessed and evaluated is a candidate for page migration based on any of a variety of considerations, such as information from the PTE, instructions from the OS / HV 110, or data collected during a previous page table walk. In response to identifying page 120 as a page migration candidate, page migration module 208, in some embodiments, modifies the associated PTE to indicate that page 120 is a page migration candidate, or in other embodiments, adds an identifier for page 120 to a migration candidate list 138 ( FIG. 1 ) maintained in memory 106 of memory subsystem 104. In some embodiments, migration candidate list 138 is available to OS / HV 110, such that OS / HV 110 selects pages to migrate from migration candidate list 138 and then monitors page migration for each page so selected. In other embodiments, MPU 108 is authorized to handle page migration autonomously, without the involvement of OS / HV 110, such that page migration module 208 selects a page to migrate and then directs the migration of the page using a DMA engine (not shown) implemented in MPU 108, or alternatively, using DMA engine 118 of primary processor 102 (note that typically DMA engine 118 operates to perform memory operations including moving data without the direct involvement of OS / HV 110). The page migration process used by MPU 108 is described in more detail below with reference to FIG. 5.

[0023] FIG. 3 illustrates an exemplary format of a PTE of a page table 126, according to at least one embodiment. As discussed above, a hierarchical page table structure 124 or other last level of a page table structure 124 includes multiple page tables 126, each of which has multiple PTEs 302, such as the illustrated PTEs 302-1 through 302-N. Each valid PTE 302 corresponds to an allocated page 120 residing in memory 106 and includes various information regarding the corresponding page 120. Illustratively, PTE 302 includes a physical base address field 304, which includes a value representing a base value of a physical address associated with the corresponding page. PTE 302 further includes one or more status / attribute fields for the corresponding page 120. In at least one embodiment, some or all of the status / attribute fields of PTE 302 may be modified by MPU 108 based on policies specified by OS / HV 110, based on data aggregated by data aggregation / reporting module 204, or a combination thereof.

[0024] The status / attribute fields of the PTE 302 typically include one or more of a present (P) bit field 306 that stores a bit value indicating whether the corresponding page 120 actually exists in memory, an accessed (A) field 308 that stores a bit value indicating whether the page 120 has been accessed, a dirty (D) field 310 that stores a bit value indicating whether the page 120 has been modified since it was loaded into memory, and a global page (G) field 312 that indicates whether the corresponding page 120 is a global page (the TLB entry for the global page is not invalidated when there is a process / context switch that results in an update to the CR3 register 128).

[0025] As noted above, in some embodiments, memory subsystem 104 employs a NUMA architecture with multiple memory types ("type" in this context refers to any of the various differences between memories 106, including architecture, technology, protocol, clock speed, distance, etc.), and thus, in one embodiment, the status / attribute field of PTE 302 includes a type field 314 that stores a value representing an identifier of the type of memory in which the corresponding page 120 is located. The type identifier identifies the appropriate type based on, for example, volatile or non-volatile, architecture or memory architecture such as DRAM or SRAM, a particular technology such as Flash or DDR SDRAM, a class of memory access speed (e.g., Class I (fastest), Class II (medium), Class III (slowest)), storage size class, power consumption class, and vulnerability to write wear, etc. In other embodiments, the memory types are represented by separate data structures indexed, for example, using a physical base address or other page identifier in field 304. Similarly, in one embodiment, PTE 302 includes a location (LOC) field 316 that stores a value specifying the memory 106 currently storing the corresponding page 120, i.e., the "location" of the page 120 within the memory subsystem 104, with each memory 106 being assigned a corresponding location identifier.

[0026] Also, as described above, in some embodiments, the processing system 100 uses page migration to move pages from one memory 106 to another memory 106 for the purposes of increasing access speed, reducing power consumption, or reducing write wear. Thus, in some embodiments, the PTE 302 includes one or both of a ready to migrate (RTM) field 318 and a migration priority field 320. The RTM field 318 stores a bit value indicating whether the corresponding page 120 is ready to migrate, while the migration priority field 320 stores a priority value used to weigh the selection of the corresponding page 120 for page migration. For example, a seldom accessed page may be marked for migration to a slower accessed memory or a memory "further" away, in which case the RTM field 318 is set to an asserted bit field and the migration priority field 320 is set to a value indicating the priority of such migration based, for example, on whether the page is seldom accessed, the priority of the data, or the type of migration being attempted. The page migration module 208 sets this priority value based on policies or other instructions from the OS / HV 110, such as policies that designate certain types of migrations as higher priority than others, e.g., prioritizing page migrations that reduce the average number of accesses per page over page migrations that reduce power consumption, or vice versa, and based on an analysis of data collected by the data aggregation / reporting module 204, e.g., if a first page is accessed significantly more frequently than a second page, then a higher page migration priority is assigned to the first page and a lower page migration priority is assigned to the second page. In other embodiments, the RTM status and migration priority status of a set of one or more pages 120 are stored in one or more data structures in the memory 106 and indexed, e.g., using the physical base addresses or other identifiers of the corresponding pages.

[0027] 4, a method 400 of operating MPU 108 to provide memory page management operations that are offloaded from primary processor 102 or that augment conventional page management procedures provided by system software is described according to some embodiments. Method 400 includes a page table walk process performed by page table walker 202 of MPU 108, as well as a PTE data collection, analysis and reporting process and a PTE management process performed by data aggregation / reporting module 204 and PTE management module 206 of MPU 108, respectively.

[0028] In at least one embodiment, the aggregation, analysis, and reporting of page information, and management of PTEs, are performed as part of a page table walk process performed by page table walker 202, whereby each PTE is contacted during the page table walk and the PTE is analyzed and any associated analyzed page data 134 is updated. Thus, the description of method 400 begins with the page table walk process sequencing or "walking" through each PTE (e.g., PTE 302) of each page table 126 of an identified page table structure 124. As discussed above, in some implementations, the base address of a given page table structure 124 is identified based on the PCID of the process / context that the page table structure 124 represents. In one embodiment, page table walker 202 maintains a list of all relevant PCIDs in an array, selects each PCID in turn, performs a page table walk through the page table structure 124 identified by that PCID, and upon completion, selects the next PCID in the sequence and performs a page table walk of the page table structure 124 associated with that PCID, and so on. This approach provides MPU 108 with more comprehensive management of memory pages 120, but at the expense of less responsiveness or up-to-date information about the currently executing process / context. In another embodiment, page table walker 202 walks the page table structures 124 of the currently executing process / context, and upon a context switch to the next process / context, page table walker 202 switches to walk the page table structures 124 associated with that next process / context (and identified, for example, using the value loaded into CR3 register 128). This page table switch, in accordance with the context switch approach described below, facilitates retention of up-to-date page information for the current process / context, at the expense of less comprehensive evaluation of page information for all active processes / contexts.

[0029] In either approach, the page table walk process begins at block 402 with identifying the page table structure 124 to be traversed using either a PCID selected from the PCID array or a PCID accessed from the CR3 register 128, depending on the mode. The page table walk process may be initiated by the OS / HV 110 via a command or other signal, or the MPU 108 may independently manage the page table walk process, either cycling continuously through one or more page table structures 124 or initiating a page table walk in response to a timer or other trigger. Once the page table walk has been initiated, at block 404 the page table walker 202 selects the next (or first) PTE 302 (FIG. 3) of the page table structure 124 and accesses the selected PTE 302 from the corresponding page table 126.

[0030] For each accessed PTE 302, at block 406, the data aggregation / reporting module 204 collects and aggregates or analyzes the information represented by the PTE 302 based on policies set by the OS / HV 110 and incorporates the results into one or more instances of the analyzed page data 134. Illustratively, as represented by block 408, the data aggregation / reporting module 204 performs one or more statistical analyses of a particular group of PTEs 302, including, for example, all PTEs 302 associated with a process / context, all PTEs 302 associated with a particular memory 106, all PTEs 302 associated with a particular region of memory 106, all PTEs 302 associated with a particular type of memory 106, or any combination thereof. One example of a statistical analysis that may be performed includes maintaining a count of the number of analyzed PTEs that represent accessed pages 120 (indicated by the accessed field 308 (FIG. 3) being set), as well as a list of all identified pages that were identified as accessed, the ratio of accessed pages to unaccessed pages, or the ratio of accessed pages to the total number of analyzed pages, etc. Another example maintains a count of the number of analyzed PTEs that represent modified pages 120 (indicated by the dirty field 310 (FIG. 3) being set), as well as a list of all pages 120 that were identified as modified, the ratio of dirty pages to clean pages, or the ratio of dirty pages to the total number of pages, etc.

[0031] As represented by block 410, the analysis performed in block 406 includes the data aggregation / reporting module 204 determining one or both of the memory type (e.g., DRAM, SRAM, non-volatile RAM (NVRAM), etc.) and the memory location (e.g., an identifier for the particular memory 106) where the corresponding page 120 was found, and then setting the type field 314 and location field 316 (FIG. 3) of the PTE 302 to reflect this information and / or entering this information into corresponding fields in the instance of the analyzed page data 134. As represented by block 412, the analysis performed in block 406 also includes determining the least recently used (MRU) and least recently used (LRU) pages 120 in embodiments in which the PTE 302 includes a last access timestamp field or similar attribute, for example, by comparing the last access timestamp of the PTE 302 being accessed to the most recently and oldest access timestamps encountered so far during the progress of the page table walk.

[0032] As yet another example, block 414 represents an embodiment of the analysis performed in block 406 in which the data aggregation / reporting module 204 constructs and maintains a histogram or other statistical representation of memory page usage on a per page, per group of pages, per memory region, or per memory. To illustrate, as described below, in some embodiments, the PTE management module 206 operates to clear the accessed field 308 of a PTE 302 each time the PTE 302 is accessed during an iteration of a page table walk, or based on other policies specified by the OS / HV 110. Thus, if the accessed field 308 is set the next time the PTE 302 is accessed, this indicates that the page has been accessed, i.e., "used," and the data aggregation / reporting module 206 thus reflects this usage by incrementing the usage metric of the corresponding page or other granularity of the page. A histogram of page usage counts for a particular group of pages is then compiled and updated for each PTE access. Similarly, a histogram or other representation of the frequency of dirty pages or similar metrics is generated.

[0033] In embodiments in which the processing system 100 employs a page migration scheme, the analysis process of block 406 includes a page migration candidate analysis, represented by block 416. To this end, the data aggregation / reporting module 204 evaluates statistics and other data regarding the page 120 collected during one or more page table walk iterations to determine whether the page 120 is a candidate for page migration, and if so, sets a migration priority for the page 120. By way of example, the data aggregation / reporting module 204 may analyze (at block 414) the collected memory page usage data to determine a relative access frequency to the corresponding page 120, such that if the page 120 has a relatively high access frequency but is currently located in a relatively slower access memory 106, then the page 120 may be identified as a high priority candidate for page migration to the faster access memory 106. As another example, the statistical analysis represented by block 408 may reveal that a subset of pages 120 of the higher power consumption memory 106 are accessed less frequently compared to the average frequency of page accesses of the memory 106, and thus each page 120 of this subset is identified as a candidate for migration to another memory 106 that may be set to a lower power state for a longer period of time due to the infrequent access and the priority of such migration set according to a policy specified by the OS / HV 110. As mentioned above, in some embodiments, the candidates for pages 120 for page migration and their migration priority are set by the MPU 108 configuring the RTM field 318 and migration priority field 320 of the accessed PTE 302, respectively. In other embodiments, the data aggregation / reporting module 204 maintains a data structure that includes identifiers of the pages 120 identified as candidates for page migration and corresponding identifiers of their determined migration priorities.

[0034] In some embodiments, accessing a PTE 302 at block 404 triggers the PTE management module 206 to perform one or more PTE management operations with respect to the accessed PTE 302 at block 420. These operations may include one or more PTE management operations traditionally performed by the OS / HV via the MMU, additional PTE management operations that are an extension or addition to traditional PTE management functions, or a combination thereof. Illustratively, as represented by block 422, the management operations of block 420 may include setting or clearing various status / attribute bits (or fields) of the PTE 302 being accessed. For example, if the accessed field 308 is set when the PTE 302 is accessed, the field is cleared so that the data aggregation / reporting module 204 can determine whether the corresponding page 120 has been accessed again since the most recent page table walk iteration. The dirty field 310 is similarly set or cleared when the PTE 302 is accessed according to policies set by the OS / HV 110. As another example, as represented by block 424, the PTE management module 206 performs filtering of the statistics generated in block 406 based on a policy from the OS / HV 110. To illustrate, a policy may specify that a particular region of a particular memory 106, or an entire particular memory 106, is to be excluded from the analysis, and the PTE management module 206 signals the data aggregation / reporting module 204 to avoid including the analysis data collected from the corresponding PTE 302. As represented by block 426, the PTE management operations performed in block 420 include management of orphan pages.

[0035] As yet another example, as represented by block 428, the PTE management module 206 detects and possibly corrects an error in the PTE 302 or the corresponding page table 126. To illustrate, if the dirty field 310 of the PTE 302 is set but the accessed field 308 of the PTE 302 is cleared, this indicates an inconsistency because if the corresponding page 120 was modified, then by definition it would also have been accessed. In response to detecting this error, the PTE management module 206, in some embodiments, attempts to correct the error, such as by accessing each memory location in the page to determine whether the memory location was modified; if at least one modified memory location is identified in the page, the PTE management module 206 concludes that the accessed field 308 was erroneously cleared and sets the accessed field 308 accordingly. However, if no modified memory locations are identified in the page, the PTE management module 206 concludes that the dirty field 310 was erroneously set and corrects the PTE 302 by clearing the dirty field 310. In other embodiments, the PTE management module 206 identifies PTEs 302 or page tables 126 that appear to have errors but do not attempt to correct certain types of errors (as indicated by policy) and, for such errors, generates a list of potentially corrupted PTEs 302 / page tables 126 and provides the list to the OS / HV 110 to address and correct, or issues an interrupt or other exception to trigger the OS / HV 110 to address the detected errors.

[0036] As described above, page allocation in the processing system 100 relies on the use of one or more free lists 132, which designate physical pages in the memory subsystem 104 that are "free" or available for allocation. As represented by block 430, in one embodiment, the PTE management module 206 operates to manage the free lists 132. This includes, for example, using LRU / MRU information, memory page usage information, and other statistical information collected by the data aggregation / reporting module 204 to identify pages 120 that have not been referenced recently or have been referenced relatively infrequently and whose data has not been modified, and marking such pages as available for allocation and inserting them into the free lists 132 according to a specific policy. Additionally, to ensure page security, when a physical page in the memory 106 becomes available on the free lists 132, the MPU 108 uses its own DMA functionality or DMA engine 118 to overwrite the physical page in the memory 106 with a specific value (e.g., all zeros).

[0037] In some embodiments, MPU 108 maintains free list 132, but it is OS / HV 110, along with MMU 114, that allocates pages using pages marked as available in free list 132. However, in other embodiments, PTE management module 206 of MPU 108 operates to perform some or all of the page allocations using free list 132 on behalf of OS / HV 110. Illustratively, MMU 114 signals MPU 108 when a page needs to be allocated, which then selects a physical page from free list 132 that is available for allocation, and then updates page table structure 124 to include the selected page's PTE 302 in preferred page table 126, and signals MMU 114 that the page has been allocated and page table 126 has been updated accordingly.

[0038] As described above, the primary processor 102 typically uses the TLB 130 to provide rapid PTE access for virtual to physical address translation of a small subset of the PTEs in the page table structure 124, where the PTEs in the TLB 130 typically represent the most recently accessed or most frequently accessed pages 120. Thus, when the MPU 108 modifies a PTE 302 in the page table structure 124 whose copy is stored in the TLB 130, the PTE management module 206 updates the TLB 130 to reflect the changes made to the corresponding PTE 302 in the page table structure 124, as represented by block 432. For example, when a page 120 is moved to the free list 132, the MPU 108 accesses the TLB 130 to invalidate the corresponding copy of the PTE 302 in the TLB 130. As another example, if a status / attribute field is changed or if a page is migrated to another memory 106 and therefore has a new physical base address, MPU 108 accesses the copy of the PTE in TLB 130 to make these same changes. As yet another example, when MPU 108 is used to allocate a page, MPU 108 also updates TLB 130 to include an entry for the page being allocated, according to policies specified by OS / HV 110.

[0039] In some embodiments, the processing system 100 has the ability to employ power throttling through selective activation and deactivation of various memories 106 of the memory subsystem 104. Thus, as represented by block 434, the PTE management module 206, either independently or at the direction of the OS / HV 110, identifies circumstances in which such power throttling is available and then directly controls placing the memories 106 in or out of a data-preserving powered-down state or provides a recommendation to the OS / HV 110, which in turn determines whether or not to implement power management of the memories 106. By way of example, the PTE management module 206 may identify from the analyzed page data 134 generated by the data aggregation / reporting module 204 that most or all of the pages 120 of a particular memory 106 are accessed relatively infrequently, consistent with a policy, in which case the PTE management module 206 may directly place the memory 106 in a data-preserving, low-power state or provide a recommendation to the OS / HV 110 to do so. Furthermore, when doing so, the PTE management module 206 configures the PTE 302 of page 120 of memory 106 with a read-only bit so that any attempt to write to memory 106 causes a page fault that is intercepted by OS / HV 110, which causes OS / HV 110 (or MPU 108) to restore memory 106 to full power.

[0040] After suitable analysis and management actions are performed using the accessed PTE 302 in blocks 406 and 420, respectively, the data aggregation / reporting module 204 determines in block 436 whether a reporting trigger has occurred, such as a signal from the OS / HV 110 requesting updated page information, the expiration of a timer, the page table walker 202 reaching the last PTE 302 of the page table structure 124, and the initiation of a context switch. If a trigger is present, in block 438 the data aggregation / reporting module 204 provides information regarding the page table walk, analysis, and management performed by the MPU 108 since the last reporting instance, such as a representation of the analyzed page data 134, the updated free list 132, page migration recommendations, and page table error reports. The OS / HV 110 then acts upon this reporting information, such as by attempting to correct the identified page table errors, allocating pages based on the updated free list 132, and initiating page migration based on the recommendations. In other embodiments, OS / HV 110 evaluates the information and then instructs MPU 108 to take one or more page management actions based on the evaluation, such as instructing MPU 108 to perform a particular page migration, correct a particular page table error, etc. If there are no reporting requirements or concurrently with the reporting process, method 400 returns to block 404, where page table walker 202 selects and accesses the next PTE 302 in the currently traversed page table structure 124, and the next iteration of method 400 is performed for this selected and accessed PTE.

[0041] 5 illustrates an example method 500 of page migration in memory subsystem 104 supported at least in part by MPU 108, according to some embodiments. Method 500 begins at block 502 with selection of a page 120 that has been marked as a candidate for page migration, either via a page migration candidate list or via setting the RTM field 318 (FIG. 3) of the corresponding PTE 302. At block 504, the page migration module 208 analyzes the candidate page 120 in light of page migration guidance or other policies provided by OS / HV 110 to determine whether the candidate page 120 is to be in fact migrated to another memory 106. This analysis may include, for example, an evaluation of page migration priority taking into account a weighting or hierarchy of page migration types (e.g., page migration for speed is prioritized over page migration for load balancing, which is prioritized over page migration for wear leveling, etc.), and an evaluation taking into account the estimated benefit of the page migration (e.g., if X>>Y, then migrating a page with page access frequency X to a faster memory is more likely to provide a performance benefit than migrating a page with page access frequency Y). If the result of the analysis is to avoid migrating the candidate page, then in some embodiments, the selected page 120 is removed as a page migration candidate at block 506, such as by clearing the RTM field 318 of the corresponding PTE entry 302 or by removing the page 120 from the page migration candidate list.

[0042] Alternatively, if the determination is to migrate the page, then at block 508, the data stored in the original location of the page 120 in the source memory 106 is copied to the identified location in the destination memory 106, where the destination memory 106 is selected based on any of a variety of factors related to the motivation for the page migration, such as a memory 106 with a faster access time being selected if the goal of the page migration is to improve performance, or a memory 106 with a larger storage capacity being selected if the motivation is to clear space in the faster source memory 106. The original page location is then overwritten with a specific data pattern for security purposes.

[0043] Before, during, or after transferring the page 120 via copying the page's data, the page migration module 208 accesses the dirty field 310 of the PTE 302 associated with the migrating page 120 to determine whether the migrating page 120 contains modified data, at block 510. If so, then, at block 512, the page migration module 208 performs a write-back operation using the DMA engine of the MPU 108 or the DMA engine 118 of the primary processor 102 to store the modified page in the mass storage device from which the page originated.

[0044] At block 514, the PTE management module 206 updates the PTE 302 associated with the migrating page to reflect its new location in the destination memory 106. This includes, for example, updating the physical base address field 304 (FIG. 3) to reflect the new location, updating the type field 314 (FIG. 3) and the location field 316 (FIG. 3), and clearing the RTM field 318 and the migration priority field 320. If a copy of the PTE 302 also exists and is valid in the TLB 130, the PTE management module 206 updates the TLB 130 in a similar manner. Since the migration of the page 120 frees a physical page in the source memory 106 and associates a physical page in the destination memory 106, at block 516 the page migration module 208 updates the free list 132 to remove the new location of the migrating page 120 from the free list 132 and insert the old location of the migrating page 120 into the free list 132. Additionally, in some embodiments, at block 518, the data aggregation / reporting module 204 operates to update any portions of the analyzed page data 134 affected by the page migration, such as by transferring statistical data associated with the migrated page from portions associated with the source memory 106, source memory space, or other source group, to portions associated with the destination memory 106, destination memory space, or other destination group.

[0045] In some embodiments, the aforementioned apparatus and techniques are implemented in a system that includes one or more integrated circuit (IC) devices (also referred to as integrated circuit packages or microchips), such as the MPU 108 described above with reference to FIGS. 1-5. Electronic design automation (EDA) and computer-aided design (CAD) software tools are used to design and manufacture these IC devices. These design tools are typically represented as one or more software programs. The one or more software programs include code executable by a computer system that operates the computer system to operate on the code representing the circuits of the one or more IC devices to perform at least a portion of the processing to design or adapt a manufacturing system for manufacturing the circuits. This code may include instructions, data, or a combination of instructions and data. The software instructions representing the design or manufacturing tools are typically stored in a computer readable storage medium accessible by the computing system. Similarly, the code representing one or more phases of the design or manufacture of the IC devices may be stored in or accessed from the same or different computer readable storage medium.

[0046] A computer-readable storage medium includes any non-transitory storage medium or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tape, magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or micro-electromechanical systems (MEMS) based storage media. The computer-readable storage medium (e.g., system RAM or ROM) may be internal to the computing system, the computer-readable storage medium (e.g., a magnetic hard drive) may be permanently attached to the computing system, the computer-readable storage medium (e.g., an optical disk or Universal Serial Bus (USB)-based flash memory) may be removably attached to the computing system, or the computer-readable storage medium (e.g., network-accessible storage (NAS)) may be coupled to the computer system via a wired or wireless network.

[0047] In some embodiments, certain aspects of the above techniques may be implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored in or tangibly embodied on a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by one or more processors, operate the one or more processors to perform one or more aspects of the above techniques. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device such as a flash memory, a cache, a random access memory (RAM), or one or more other non-volatile memory devices, etc. The executable instructions stored on the non-transitory computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats that can be interpreted or executed by one or more processors.

[0048] According to one aspect, a system includes a primary processor connectable to a memory subsystem having at least one memory. The primary processor is for executing system software that uses memory address translation based on one or more page tables stored in the memory subsystem. The system further includes a coprocessor connectable to the memory subsystem. The coprocessor is for performing iterations of a page table walk through one or more page tables maintained in the memory subsystem and for performing one or more page management operations on behalf of the system software based on the iterations of the page table walk. In some embodiments, the one or more page management operations include an operation by the coprocessor generating analysis page data for memory pages represented by at least a subset of page table entries accessed by the coprocessor during the page table walk, the analysis page data based on values ​​of one or more fields of the page table entries, the analysis page data including at least one of: a number of pages having a particular status or attribute, a statistical representation of pages having a particular status or attribute, or a list of pages identified as having a particular status or attribute. In some embodiments, the one or more page management operations further include at least one of an operation by the coprocessor of maintaining a free list of pages available for allocation based on the analyzed page data; an operation by the coprocessor of identifying at least one page as a candidate for page migration based on the analyzed page data; an operation by the coprocessor of identifying at least one of a least recently used (MRU) page or a least recently used (LRU) page based on the analyzed page data; and an operation by the coprocessor of identifying memory of the memory subsystem as a candidate for setting to a low power state based on the analyzed page data.

[0049] In some embodiments, the one or more page management operations include at least one of an operation by the coprocessor modifying one or more fields of a page table entry or an operation by the coprocessor invalidating or inserting a page table entry into the page table. The one or more page management operations may further include an operation by the coprocessor modifying a translation lookaside buffer of the primary processor in response to the modification of one or more fields of the page table entry or in response to the invalidating or inserting a page table entry into the page table.

[0050] In some embodiments, the one or more page management operations include an operation by the coprocessor evaluating a page table entry or a page table accessed during the page table walk for errors. In some embodiments, the one or more page management operations include an operation by the coprocessor migrating a page from one memory of the memory subsystem to another memory of the memory subsystem. In some embodiments, the one or more page management operations include at least one of selecting a page for inclusion in a free list of pages available for allocation based on a page table entry of a page accessed during the page table walk or allocating a page from the free list based on a request for a page allocation from the primary processor. In some embodiments, the system software is for specifying one or more policies for page management to the coprocessor, and the coprocessor is for performing the one or more page management operations based on the one or more policies. In some embodiments, the coprocessor is for performing at least one of the one or more page management operations in response to a command from the system, or the coprocessor is configured to perform at least one of the one or more page management operations independent of the system software. In other embodiments, the coprocessor is configured to perform at least one of one or more page management operations in coordination with system software using one or more flags to prevent simultaneous access attempts by both the primary processor and the coprocessor. In some embodiments, the coprocessor has limited general purpose processing capabilities compared to the primary processor.

[0051] According to another aspect, a coprocessor is connectable to the primary processor and connectable to a memory subsystem having at least one memory. The coprocessor performs iterations of a page table walk of a set of one or more page tables and generates analyzed page data representative of the set of one or more page tables based on each page table entry of at least a subset of the page table entries of the one or more page tables accessed during the page table walk. In some embodiments, the analyzed page data includes at least one of a number of pages having a particular status or attribute, a statistical representation of pages having a particular status or attribute, or a list of pages identified as having a particular status or attribute.

[0052] According to yet another aspect, a method of page management in a processing system having a primary processor, a coprocessor, and a memory subsystem having at least one memory is provided, the method including executing system software on the primary processor, the execution of the system software including performing address translation based on one or more page tables maintained in the memory subsystem, performing, on the coprocessor, iterations of a page table walk through the one or more page tables, and performing, on behalf of the system software, one or more page management operations based on the iterations of the page table walk on the coprocessor.

[0053] In addition to the above, it should be noted that not all activities or elements described in the summary description are required, some of the specific activities or devices may not be required, one or more additional activities may be performed, and one or more additional elements may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will appreciate that various changes and modifications can be made without departing from the scope of the invention as set forth in the claims. Thus, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the invention.

[0054] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and features by which any benefit, advantage, or solution may occur or be manifested are not to be construed as critical, essential, or essential features of any or all claims. Moreover, the specific embodiments described above are illustrative only, as the disclosed invention may be modified and practiced in different but similar manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as set forth in the appended claims. It is therefore apparent that the specific embodiments described above may be altered or modified, and all such variations are considered to be within the scope of the disclosed invention. Accordingly, the protection sought herein is set forth in the appended claims.

Claims

1. a primary processor connectable to a memory subsystem having at least one memory, the primary processor configured to execute system software that uses memory address translation based on one or more page tables stored in the memory subsystem; a coprocessor connectable to the memory subsystem, the coprocessor configured to perform one or more page management operations on behalf of the system software, including performing iterations of page table walks through one or more page tables maintained in the memory subsystem and maintaining a free list of pages available for allocation based on the iterations of the page table walks; Processing system.

2. The one or more page management operations include: generating, by the coprocessor, analysis page data for memory pages represented by at least a subset of the page table entries accessed by the coprocessor during the page table walk, the analysis page data being based on values ​​in one or more fields of the page table entries; The processing system of claim 1 .

3. the analysis page data includes at least one of a number of pages having a particular status or attribute, a statistical representation of pages having the particular status or attribute, or a list of pages identified as having the particular status or attribute; The processing system of claim 2 .

4. The one or more page management operations include: identifying, by the coprocessor, at least one page as a candidate for a page transition based on the analyzed page data; determining, by the coprocessor, at least one of a least recently used (MRU) page or a least recently used (LRU) page based on the analyzed page data; identifying, by the coprocessor, memory of the memory subsystem as a candidate for being placed in a lower power state based on the analyzed page data; At least one of The processing system of claim 2 .

5. The one or more page management operations include: modifying one or more fields of a page table entry by said coprocessor; or an operation by said coprocessor of invalidating or inserting a page table entry into a page table; At least one of The processing system of claim 1 .

6. The one or more page management operations include: modifying a translation lookaside buffer of the primary processor in response to a modification by the coprocessor of one or more fields of a page table entry, or in response to an invalidation or insertion into a page table of a page table entry; The processing system of claim 5.

7. The one or more page management operations include: evaluating, by the coprocessor, page table entries or page tables accessed during the page table walk for errors. The processing system of claim 1 .

8. The one or more page management operations include: migrating a page from one memory of the memory subsystem to another memory of the memory subsystem by the coprocessor; The processing system of claim 1 .

9. The one or more page management operations include: selecting a page in a free list of pages available for allocation based on a page table entry of the page accessed during said page table walk; or allocating pages from the free list based on a request for a page allocation from the primary processor; At least one of The processing system of claim 1 .

10. the system software is configured to specify to the coprocessor one or more policies regarding page management; the coprocessor is configured to perform the one or more page management operations based on the one or more policies. The processing system according to any one of claims 1 to 9.

11. the coprocessor is configured to perform at least one of the one or more page management operations in response to commands from the system software. The processing system according to any one of claims 1 to 9.

12. the coprocessor is configured to perform at least one of the one or more page management operations independent of the system software. The processing system according to any one of claims 1 to 9.

13. the coprocessor is configured to perform at least one of the one or more page management operations in coordination with the system software using one or more flags to prevent simultaneous access attempts by both the primary processor and the coprocessor. The processing system according to any one of claims 1 to 9.

14. the coprocessor has limited general-purpose processing capabilities compared to the primary processor; The processing system according to any one of claims 1 to 13.

15. a coprocessor connectable to a primary processor and connectable to a memory subsystem having at least one memory, performing an iterative page table walk of a set of one or more page tables; generating analyzed page data representative of the set of one or more page tables based on each page table entry of at least a subset of the page table entries of the one or more page tables accessed during the page table walk; [0023] The present invention is configured to Coprocessor.

16. the analysis page data includes at least one of a number of pages having a particular status or attribute, a statistical representation of pages having the particular status or attribute, or a list of pages identified as having the particular status or attribute; The coprocessor of claim 15.

17. The coprocessor includes: maintaining a free list of pages available for allocation based on the analyzed page data; identifying at least one page as a candidate for a page transition based on the analyzed page data; determining at least one of a least recently used (MRU) page or a least recently used (LRU) page based on the analyzed page data; identifying memory of the memory subsystem as a candidate for being placed in a lower power state based on the analyzed page data; an operation to modify one or more fields of a page table entry; an operation for invalidating or inserting a page table entry into the page table; modifying a translation lookaside buffer of the primary processor based on a corresponding change to one or more fields of a page table entry or at least one of invalidating or inserting a page table entry into a page table; evaluating a page table entry or page table accessed during said page table walk for errors; 4. The method of claim 1, further comprising:

17. A coprocessor according to claim 15 or 16.

18. 1. A method of page management in a processing system comprising a primary processor, a coprocessor, and a memory subsystem having at least one memory, comprising: executing system software on the primary processor, the execution of the system software performing address translation based on one or more page tables maintained in the memory subsystem; performing, in the coprocessor, iterative page table walks through the one or more page tables; performing, on behalf of the system software, one or more page management operations in the coprocessor, including maintaining a free list of pages available for allocation based on the iterations of the page table walk; method.

19. The one or more page management operations include: generating analysis page data for memory pages represented by at least a subset of the page table entries accessed by the coprocessor during the page table walk, the analysis page data being based on values ​​in one or more fields of the page table entries.

20. The method of claim 18.

20. The one or more page management operations include: identifying at least one page as a candidate for a page transition based on the analyzed page data; determining at least one of a least recently used (MRU) page or a least recently used (LRU) page based on the analyzed page data; identifying memory of the memory subsystem as a candidate for being placed in a lower power state based on the analyzed page data; At least one of 20. The method of claim 19.

21. The one or more page management operations include: Modifying one or more fields of a page table entry; Invalidating a page table entry in the page table; inserting a page table entry into a page table; modifying a translation lookaside buffer of said primary processor in response to a change in one or more fields of a page table entry or in response to an invalidation or insertion into a page table of a page table entry; evaluating a page table entry or page table accessed during said page table walk for errors; migrating pages from one memory of the memory subsystem to another memory of the memory subsystem; selecting a page in a free list of pages available for allocation based on a page table entry of a page accessed during said page table walk; allocating pages from the free list based on requests for page allocations from the primary processor; At least one of 20. The method of claim 18.

22. the system software specifying one or more policies regarding page management to the coprocessor, the one or more page management actions being performed by the coprocessor based on the one or more policies; 22. The method of any one of claims 18 to 21.

23. performing the one or more page management operations includes performing at least one of the one or more page management operations in response to a command from the system software.

22. The method of any one of claims 18 to 21.

24. performing the one or more page management operations includes performing at least one of the one or more page management operations independent of commands from the system software.

22. The method of any one of claims 18 to 21.

Images