Migration of pages of memory accessible by I / O devices

Abstract

The electronic device includes a memory, a processor executing software entities, a page migration engine (PME), and an input / output memory management unit (IOMMU). The software entities and the PME perform operations to prepare for migrating a page of memory accessible by at least one IO device in the memory, and the software entities and the PME set migration state information in a page table entry for the page of memory and information in a reverse map table (RMT) entry involved in migrating the page of memory based on the operations being performed. The IOMMU controls use of the information in the page table entry and controls execution of memory accesses for the page of memory based on the migration state information in the page table entry and the information in the RMT entry. The PME migrates the page of memory upon completing the operations to prepare for migrating the page of memory.

Description

[Background technology]

[0001] (Related Technology) Some electronic devices include a processor (e.g., a central processing unit, etc.) that performs calculations, control, memory access, and other operations, along with a memory in which data is stored (wherein "data" is used to collectively refer to information such as program code instructions, input or result data for and from computational operations, constant values, etc.). Many such electronic devices also include input-output (IO) devices such as network interface devices, disk controllers, etc. IO devices may interact with the processor and memory to perform various operations. For example, a network interface device may store data received over a network in memory and then signal the processor that the data stored in memory is awaiting processing.

[0002] In some cases, data stored in a memory in an electronic device may be stored in the memory in a way that makes access to the data inefficient. For example, a memory may be implemented using a combination of two different types of memory, one of which is a faster access but more expensive type of memory and the other of which is a slower access but less expensive type of memory. Inefficiencies may arise in such memories when frequently accessed, high priority and / or time-critical data is stored in the slower type of memory or vice versa. As another example, inefficiencies may arise when data is stored in a remote portion of memory, such as a portion of memory in a remote socket of a non-uniform memory access (NUMA) electronic device.

[0003] Because inefficiencies in accessing data in memory can affect the overall performance of an electronic device, some electronic devices include a migration mechanism to dynamically rearrange or "migrate" data in memory so that the data is stored in a more appropriate location in memory. For example, data in memory can be organized into contiguous blocks of data or "pages of memory" of a specified size (e.g., 4KiB, 2MiB, etc.) for memory management and migration purposes. The migration mechanism can monitor accesses of the pages of memory and selectively migrate the pages of memory based on specified patterns or characteristics of the accesses of the pages of memory. For example, in an electronic device having two different types of memory as described above, if a page of memory stored in the slower type of memory is accessed frequently enough, the migration mechanism can migrate the page of memory from the slower type of memory to the faster type of memory. In some electronic devices, the migration mechanism migrates the pages of memory automatically and autonomously, such that the pages of memory can be migrated without communicating information about the migration to each of the accessing entities (e.g., processors, IO devices, etc.) that have access to the pages of memory. In other words, the migration mechanism migrates pages of memory "secretly" without each of the accessing entities necessarily having direct knowledge of, providing approval for, and / or being involved in the migration.

[0004] In some electronic devices, some or all IO devices do not provide support for migrating pages of memory. For example, the IO devices may be older or simpler IO devices that do not include software and / or hardware support for handling faults (e.g., page faults, etc.) that may occur if a page of memory accessible to such an IO device is to be migrated and the IO device attempts to access the page of memory or information associated with the page of memory (e.g., address translation information, etc.) during the migration, the IO device and / or other functional blocks and devices within the electronic device (e.g., processor, etc.) may crash, experience errors, or behave in an unexpected or undesirable manner. [Brief description of the drawings]

[0005] [Figure 1] FIG. 1 is a block diagram illustrating an electronic device according to some embodiments. [Diagram 2] FIG. 2 is a block diagram illustrating functional blocks and devices within an operating system and an electronic device according to some embodiments. [Diagram 3] FIG. 2 is a block diagram illustrating functional blocks and devices within a hypervisor and an electronic device in accordance with some embodiments. [Figure 4] FIG. 2 is a block diagram illustrating a page table according to some embodiments. [Diagram 5] FIG. 1 is a block diagram illustrating metadata in a page table entry in accordance with some embodiments. [Figure 6] FIG. 2 is a block diagram illustrating a reverse map table according to some embodiments. [Figure 7] FIG. 1 is a state diagram illustrating transition states according to some embodiments. [Figure 8] FIG. 1 is a flow diagram illustrating a process for migrating a page of memory according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] The following description is presented to enable any person skilled in the art to make and use the described embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the described embodiments will be readily apparent to those skilled in the art, and the general principles described herein may be applied to other embodiments and applications. Thus, the described embodiments are not limited to the embodiments shown, but are to be accorded the widest scope consistent with the principles and features described herein.

[0007] (term) In the following description, various terms are used to describe the embodiments. Below is a simplified general description of some of the terms. It should be noted that the terms may have important further aspects that are not described herein for clarity and conciseness, and therefore the description is not intended to be limiting of the terms.

[0008] Functional Block: A functional block refers to a set of interrelated circuitry, such as integrated circuitry, discrete circuitry, etc. The circuitry is "interrelated" in that the circuit elements in the circuitry share at least one characteristic. For example, the circuits may be contained in, fabricated on, or otherwise coupled to a particular integrated circuit chip, substrate, circuit board, or portion thereof, may be involved in performing particular operations (e.g., computational operations, control operations, storage operations, etc.), may be controlled by a common control element and / or a common clock, etc. The circuitry of a functional block may have any number of circuit elements, from a single circuit element (e.g., a single integrated circuit logic gate or discrete circuit element) to millions or billions of circuit elements (e.g., integrated circuit memory). In some embodiments, the functional block performs operations "in hardware" using circuitry that performs the operations without executing program code.

[0009] Data: Data is a generic term referring to information that can be stored in memory and / or used in computation, control and / or other operations. Data includes information such as actual data (e.g., results of computation or control operations, outputs of processing circuitry, inputs for computation or control operations, variable values, sensor values, etc.), files, program code instructions, control values, and / or other information. Data can be organized into logical blocks (e.g., 4KiB or 2MiB pages of memory, etc.) that contain one or more of the above types of information for operations such as loading data into memory, migrating data within memory, etc.

[0010] Access: Access as applied to interactions with data stored in memory (e.g., main memory, cache memory, etc.), i.e., "memory access," refers to all forms of interaction that may be performed for, on, with, and / or with the data and corresponding metadata and / or control values. For example, access may include reading or loading data from memory, writing or storing data to memory, invalidating or deleting data in memory, reading or writing metadata to data in memory, changing the status, coherency state, or permissions for data in memory, etc. Data is "accessible" to a given functional block or device if a particular type of memory access is permitted / authorized for that functional block or device.

[0011] (Virtual Memory) In the described embodiment, the electronic device uses virtual memory techniques to handle data access by software entities (e.g., application programs, operating systems, device drivers, virtual machines, etc.) executing within the electronic device or by input / output (IO) devices (e.g., network interface devices, Peripheral Component Interface Express (PCIe) bus devices, disk controllers, etc.). Generally, when data is first accessed by a software entity or an IO device, a block or page of memory of a predetermined size (e.g., 4KiB, 2MiB, etc.) containing the data is copied from a mass storage device (e.g., a disk drive or non-volatile semiconductor memory) to an available physical location in memory (e.g., main memory) within the electronic device, or a page of memory is newly created in memory (e.g., to store the results of a calculation or other operation, etc.). The electronic device tracks the physical location of the pages of the software entity or IO device so that the software entity and the IO device do not need to track the physical location of the pages in memory. Instead of accessing memory using addresses based on the physical location (or physical address) of data in memory, software entities and IO devices access memory using "virtual" addresses in a virtual address space, which is a local address space specific to the corresponding software entity or IO device. From the perspective of the software entity or IO device, the virtual address indicates the actual physical location where the data is stored in memory, and memory accesses are made by the software entities and IO devices accordingly using the virtual addresses. However, the virtual addresses may not map directly to the physical addresses of the physical locations where the data is stored in pages in memory. As part of keeping track of the physical locations of pages, the electronic device translates the virtual addresses used by the software entities and IO devices in memory access requests into the physical addresses where the data is actually located.The electronic device then uses the physical addresses to perform memory accesses for the software entities and IO devices.

[0012] To enable the above-mentioned virtual address to physical address translation, the electronic device includes a page table. The page table is a record stored in the memory of the electronic device that includes entries, or "page table entries," that have virtual address to physical address translation information and other information about pages of data stored in the memory. In other words, the page table includes a mapping between a virtual address and a corresponding physical address for each page of data present in the memory. Upon receiving a request to access memory at a given virtual address from a software entity or an IO device, the electronic device obtains the corresponding physical address information from the page table by performing a page table walk, during which the page table is searched for a page table entry that provides a physical address associated with the virtual address. For example, upon receiving a request to access memory at a given virtual address from an operating system, a memory management unit (MMU) in a central processing unit (CPU) core in the electronic device may perform a page table walk to obtain the corresponding physical address information from the page table, and use the physical address information to process the memory access. As another example, upon receiving a request from an IO device to access memory at a given virtual address, an input-output memory management unit (IOMMU) in the electronic device may perform a page table walk to obtain corresponding physical address information from the page table and process the memory access using the physical address information.

[0013] Because the page table walk described above is relatively slow, it is desirable to avoid performing a page table walk. Therefore, an electronic device includes a translation lookaside buffer (TLB), which is a local cache used to store a limited number of copies of address translation information (i.e., information based on a page table entry) obtained during a page table walk. For example, a CPU may include a TLB used to locally store a copy of information based on a page table entry used for memory access for a software entity (or a copy of the page table entry itself). As another example, an IOMMU may include a TLB used to locally store a copy of information based on a page table entry used for memory access for an IO device (or a copy of the page table entry itself). During operation, a CPU core or an IOMMU first attempts to obtain a cached page table entry from a corresponding TLB to perform a virtual-to-physical address translation. If a copy of the corresponding page table entry is not present in the TLB (i.e., a "miss" occurs), the CPU core or an IOMMU performs a page table walk to obtain the desired page table entry. The CPU core or IOMMU may then cache a copy of the retrieved page table entry in its respective TLB for later use.

[0014] In some embodiments, the IOMMU and IO devices support a communication protocol for address translation, such as PCI Express (PCIe) Address Translation Services (ATS). In the case of ATS, input / output (IO) devices may include a corresponding local cache or “address translation cache” (ATC) that each IO device uses to store copy information based on the page table entry (or the page table entry itself). When an IO device desires a local copy of the information based on the page table entry (or, more simply, a copy of the entire page table entry), the IO device requests the information based on the page table entry from the IOMMU. The IOMMU verifies that the IO device is authorized to have the information, and then provides the information based on the page table entry to the IO device. The IO device then stores the information based on the page table entry in the ATC for later use. Thus, using the ATC allows the IO device to avoid having to repeatedly request information based on the page table entry from the IOMMU. In some embodiments, for example, in the case of a “secure” ATS, the IOMMU does not provide the system physical address from the page table entry to the requesting IO device, but instead provides a guest physical address. In these embodiments, the IOMMU provides guest physical addresses that must be translated by the IOMMU to system physical addresses for memory accesses, in order to avoid providing IO devices with system physical addresses that can be used by the IO device to access memory directly (without specific security checks by the IOMMU).

[0015] (Virtual Machines and Hypervisors) In some embodiments, a processor in an electronic device executes one or more virtual machines. A virtual machine is a software entity that emulates or otherwise interfaces with a processor and other functional blocks and devices (e.g., memory, IO devices, etc.) in an electronic device to provide support for running software programs. For example, a virtual machine may provide support for running one or more instances of an operating system, called a guest operating system. The guest operating system, in turn, provides support for running other software programs, such as applications, databases, etc.

[0016] In some embodiments, the processor also executes a hypervisor. A hypervisor is a software entity that performs operations to control and manage the execution of virtual machines. For example, the hypervisor may start and initialize the virtual machines, help control the virtual machines' access to functional blocks and devices within the electronic device (e.g., directing which areas of memory and / or IO devices the virtual machines are allowed to access), terminate or shut down the virtual machines, etc. The configuration of the virtual machines and hypervisor as found in some embodiments is shown in FIG. 3.

[0017] (page table) In some embodiments, the guest operating system, the IO device, the IOMMU, and the hypervisor in the electronic device use virtual memory techniques for memory access. Programs running under the guest operating system, the guest operating system itself, and the IO device can use local addresses for memory access. For example, the programs running under the guest operating system can access memory using guest virtual addresses, and the IO device can access memory using guest physical addresses. As described above, the local addresses in memory accesses from the programs, the guest operating system, and the IO device should be translated before the data can be accessed in memory. For example, the guest virtual address in memory accesses from the programs can be translated to a guest physical address. However, the guest physical address is itself virtual, i.e., the guest operating system's view of the organization of data in memory, and thus the guest physical address is translated to the system physical address where the data is located in memory. As another example, the guest physical address in memory accesses from the IO device can be translated to the system physical address where the data is located in memory.

[0018] To support programs, and in some embodiments guest operating systems, and IO devices that use local addresses, the electronic device uses a set of page tables for address translation. The page tables include a guest page table for each guest operating system that is used to translate guest virtual addresses in memory accesses from the programs to guest physical addresses. The page tables also include nested (i.e., host, system, etc.) page tables that are used to translate guest physical addresses in memory accesses from the guest operating systems to system physical addresses. The page tables further include an IO page table that is used to translate guest physical addresses in memory accesses from the IO devices to system physical addresses. Note, however, that in some embodiments, the information in the IO page table is included / combined with information in the nested page tables, and the nested page tables are used for all translations of guest physical addresses to system physical addresses.

[0019] As an example of the use of page tables described above to translate addresses, in some embodiments, upon receiving a memory access request from a program that includes a guest virtual address via a guest operating system, the processor sequentially uses the guest page table and the nested page table to perform the translation. In other words, the processor (e.g., the guest operating system) first translates the guest virtual address to a guest physical address using the guest page table. The processor (e.g., the hypervisor) then translates the guest physical address to a system physical address using the nested page table. Upon obtaining the system physical address, the processor performs the memory access using the system physical address. The processor may also cache the system physical address in a TLB within the processor.

[0020] As another example of the use of page tables described above to translate addresses, in some embodiments, upon receiving a memory access request from an IO device that includes a guest physical address, the IOMMU uses the IO page table to perform the translation. In other words, the IOMMU uses the IO page table to translate the guest physical address to a system physical address. Upon obtaining the system physical address, the IOMMU performs the memory access using the system physical address. The IOMMU may also cache the system physical address in a TLB within the IOMMU.

[0021] (Reverse Map Table) In the described embodiments, the hypervisor (and / or other software entities, functional blocks, and / or devices) can modify information in the nested page tables. For example, the hypervisor can modify the mapping of guest physical addresses to system physical addresses in the nested page tables. Such modifications are typical and necessary when properties of a page of memory change, such as when a page of memory is moved within memory, when ownership of a page of memory changes, etc. Because the hypervisor can modify information in the nested page tables, if the hypervisor maliciously or erroneously modifies information in a page table entry, the electronic device may experience unintended or unexpected behavior. For example, if the hypervisor remaps a page of memory of a guest operating system to a malicious page of memory, the hypervisor may cause the guest operating system to execute undesirable, and possibly malicious, program code. Thus, the described embodiments perform operations to ensure that information in the nested page tables is not unexpectedly or unacceptably modified by the hypervisor. For these operations, the described embodiments use, among other things, a reverse map table (RMT) to ensure that the hypervisor (and / or another entity) is not remapping the translation from guest physical addresses in the nested page tables to system physical addresses. A reverse map table is a record in memory that contains information that can be used to determine, among other things, which entity owns a page of memory (e.g., the guest operating system or the hypervisor to which the page of memory is assigned) and whether a system physical address obtained during a table walk of the nested page tables for a guest physical address can be assigned to that guest physical address.In other words, a reverse map table can be used to ensure that the system physical address of a given page in memory matches only one guest physical address at a time, thereby preventing accidental or malicious remapping of such addresses. An example reverse map table is shown in Figure 6.

[0022] (memory page migration) In the described embodiments, a page of memory may be migrated within the memory. For migration, the page of memory is physically moved within the memory, so that the page is relocated from a first location in the memory to a second location in the memory. For example, if the page of memory is an 8KiB page of memory and is initially located at address A to A+8KiB-1, the page may be migrated to address B to B+8KiB-1. In some embodiments, the page of memory may be migrated to relocate the page of memory and / or the memory itself to a location in the memory that is more suitable for the page of memory. For example, a frequently accessed or higher priority page of memory may be migrated from a first type of memory having a slower access speed to a second type of memory having a faster access speed, or vice versa. As another example, the page of memory may be migrated within the memory to avoid wear and tear on memory circuitry within the memory. As yet another example, the page of memory may be migrated within the memory to place the page of memory closer to other pages of memory being accessed. As yet another example, in some embodiments, the memory is configured for non-uniform memory access (NUMA), where multiple portions of the memory are each located near a respective processing node, and pages of the memory may be migrated between portions of the memory within a node.

[0023] In some embodiments, the decision to migrate a page of memory is made by a software entity and / or functional block or device within the electronic device based on one or more migration rules (i.e., guidelines, standards, thresholds, etc.) that identify the conditions under which the page of memory should be migrated. For example, an operating system or hypervisor executed by a processor within the electronic device may determine that a page of memory should be migrated based on the frequency or type of accesses made to the page of memory. As another example, a memory management unit functional block within the processor may determine that a page of memory should be migrated from a slower access memory to a faster access memory based on a priority identified in a memory access request for the page of memory.

[0024] Although a processor is capable of migrating pages of memory within a memory, i.e., physically moving individual bytes of a page of memory within a memory, using a processor to perform the migration may consume processor resources that are better used for other operations (e.g., computational operations, other types of memory access, etc.). Thus, in some embodiments, the migration of a page of memory is offloaded from the processor to a page migration engine functional block or device. The page migration engine is capable of, and may be optimized to, perform the migration of a page of memory given an identification of the page of memory and a destination location within the memory. In some embodiments, the page migration engine includes a direct memory access (DMA) functional block that performs operations to physically move the page of memory within the memory.

[0025] (overview) In a described embodiment, the electronic device includes a processor and one or more input / output (IO) devices (IO devices may include functional blocks or actual devices within and / or connected to the electronic device, such as network interfaces, disk controllers, graphics processing units, etc.). The electronic device also includes memory used to store data for use by the processor and / or IO devices. The electronic device further includes at least one page migration engine functional block that performs operations to copy, move, and otherwise manipulate data stored in the memory. As part of these operations, the page migration engine performs operations associated with migrating pages of memory (e.g., 4 KiB pages of memory), which involves relocating pages of memory within the memory according to migration rules. The electronic device also includes an input / output memory management unit (IOMMU) that manages memory access by the IO devices. In a described embodiment, the IOMMU performs operations to control IO device access of pages of memory that are to be or are being migrated, and to control use of information from IO page table entries of such pages of memory. The operations performed by the IOMMU are directed to ensuring that IO devices, and more generally electronic devices, do not experience errors or faults that could occur if an IO device accesses a page of memory that has been migrated (or encounters a missing page of memory) and / or uses inaccurate information from the page table entry for such a page of memory.

[0026] In the described embodiment, two different records are used by the IOMMU to control access by IO devices to pages of memory that are prepared for migration or are being migrated. The records include an IO page table and a reverse map table (RMT), each of which includes respective information used to determine whether a page of memory is being migrated and therefore whether the page of memory and / or the respective IO page table entry may be accessed. The information in the IO page table includes the migration information in the IO page table entry. In other words, the IO page table entry includes migration state information along with the translation information and metadata described above. The migration state information includes an indication of whether the page of memory associated with the page table entry is being prepared for migration or whether the page of memory is being migrated (or not). For example, in some embodiments, the migration state information includes two bits, and each possible combination of the two bits identifies a corresponding migration state from among a set of migration states. In a first migration state identified by 00, i.e., a "non-migration" migration state, the page of memory has not been migrated. In the second and third migration states, identified by 01 and 10 respectively, the respective preparations for migrating the page of memory are taking place. In the fourth migration state, identified by 11, the final preparations for migrating the page of memory are taking place or the page of memory is in the process of migrating (i.e., is actually being relocated in memory). The information in the RMT includes the migration information in the RMT entry and a lock indicator. In other words, the RMT entry includes the RMT migration state information and a lock indicator along with information for determining whether nested page table entries have been remapped. Similar to the migration state information in the page table entries in the IO page table, the RMT migration state information includes an indication of whether the page of memory associated with the page table entry is being prepared for migrating or whether the page of memory has been migrated (or not). The lock indicator includes an indication of whether the respective RMT entry is locked.

[0027] In some embodiments, the processor executes a software entity that, in combination with the page migration engine, prepares a page of memory for migration and performs at least some of the operations to handle the migration of the page of memory. For example, the software entity may be or include a hypervisor, an operating system, an application program, etc. Among the operations performed by the software entity and / or the page migration engine to prepare the page of memory for migration are setting migration state information of the page of memory in an IO page table and / or information in an RMT entry involved in the migration of the page of memory. For example, as the functional blocks and devices in the electronic device progress through each of the transition states described above, the software entity or the page migration engine may set the migration state information and / or information in the RMT entry accordingly. Another operation performed by the software entity and / or the page migration engine is to invalidate existing copies of information from the IO page table entry for migrating the page stored in a cache in the IOMMU or IO device. Further operations performed by the page migration engine are, after migrating the page of memory, updating the IO page table entry for the page of memory to identify the new location in memory for the page of memory, and setting migration state information in the IO page table entry and information in the RMT entry to indicate a non-migration state.

[0028] In some embodiments, the IOMMU enforces access restrictions on pages of memory and IO page table entries based on the migration state information in the IO page table entry and / or the information in the RMT entry. For example, if the migration state information in the IO page table entry and / or the information in the RMT entry for a source page of memory indicates that preparations are being made to migrate the page of memory or that the page of memory is being migrated, the IOMMU may prevent itself and IO devices from storing or using copies of information from the IO page table entry in local caches. As another example, if the migration information in the IO page table entry and / or the information in the RMT entry for a source page of memory indicates that the page of memory is being prepared for migration or is being migrated, the IOMMU may ensure that IO device accesses of the page of memory are handled accordingly. That is, the IOMMU may ensure that existing memory accesses of the page of memory are completed before the page of memory is migrated and that newly encountered memory accesses of the page of memory are stalled (i.e., held, buffered, etc.) until after the page of memory is migrated. In contrast, if the migration information in the IO page table entry for the source page of memory and / or the information in the RMT entry indicates that the page of memory is in a non-migratory state (and therefore has not been migrated), then the IOMMU may allow normal access of the page of memory and the IO page table entry.

[0029] In some embodiments, some or all of the IO devices support Address Translation Services (ATS), such as PCI Express (PCIe) Address Translation Services, which is a standard by which an IO device can request information from an IOMMU from an IO page table entry that is cached locally in a local cache or entry in an "Address Translation Cache (ATC)" in the IO device. As part of the above-described operations to prevent copies of information from IO page table entries from being stored in caches in the IO device, the IOMMU can reject / deny ATS requests for information from IO page table entries associated with pages of memory that are to be or are being migrated.

[0030] For the above operations, the IOMMU uses information in the IO page table and the RMT to process memory accesses by IO devices and IO page table accesses to migrate pages of memory. However, memory accesses by other functional blocks and devices in the electronic device to move pages of memory can be handled differently. For example, when preparing an IO device accessible page of memory for migration, the page of memory can simply be marked as missing in the nested page table (and the corresponding information is invalidated in the TLB in the processor). Since the processor (unlike a specific IO device) includes a mechanism for handling missing pages, the processor will proceed normally to handle the missing page, and will then continue memory accesses after the page of memory has been migrated and the nested page table updated accordingly. In addition, pages of memory that have not been prepared for migration or have not been migrated and their corresponding page table entries may be accessed normally, even if they are accessible to the IO device, subject to memory access ordering rules, etc.

[0031] In some embodiments, when migrating a page of memory that is not IO device accessible, but instead is only processor accessible, some or all of the mechanisms described above can be used, but adjusted for operations involving processor accessible pages. For example, in some embodiments, a software entity or page migration engine can mark the page of memory as "not present" in a nested page table (and invalidate the corresponding one in the TLB in the processor). As in the above case, the processor will handle the missing page of memory using a page fault mechanism. Additionally or alternatively, the software entity and / or page migration engine can lock the RMT entry of the migrating processor-only page of memory, and the accessing processor can use an RMT check / verification mechanism to avoid accessing the migrating page of memory.

[0032] By handling the migration of pages of memory accessible to IO devices, the described embodiments allow such pages of memory to be more freely migrated within memory despite the presence of IO devices that may not themselves properly handle migration of pages of memory. Migrating these pages of memory can help ensure that the pages of memory are in desired locations within memory. This can improve the operation of the memory, which in turn improves the operation of processors and IO devices that use the pages of memory in memory.

[0033] (Electronic Devices) FIG. 1 illustrates a block diagram of an electronic device 100 according to some embodiments. As seen in FIG. 1, the electronic device 100 includes a processor 102, a memory 104, a mass storage device 106, input / output (IO) devices 108-110, an input / output (IO) hub 112, and a page transition engine 114. All of the processor 102, the memory 104, the mass storage device 106, the IO devices 108-110, the IO hub 112, and the page transition engine 114 are implemented in "hardware," i.e., using corresponding integrated circuits, discrete circuits, and / or devices. For example, in some embodiments, the processor 102, the memory 104, the mass storage device 106, the IO devices 108-110, the IO hub 112, and the page transition engine 114 are implemented in integrated circuits on one or more semiconductor chips, in combination with discrete circuits and / or devices, or in discrete circuits and / or devices. In FIG. 1, electronic device 100 is partially shaded to allow the various diagrammatic elements to be more easily distinguished.

[0034] Processor 102 is a functional block that performs calculations, memory access, control, and / or other operations in electronic device 100. Processor 102 includes cores 116-118, each of which includes one or more central processing unit (CPU) cores, graphics processing unit (GPU) cores, embedded processors, application specific integrated circuits (ASICs), and / or other computing functional blocks.

[0035] The processor 102 includes cache memory, or "cache," which is a functional block used to store copies of data that may be used by the cores 116-118 to perform various operations. As seen in FIG. 1, the caches in the processor 102 include first level (L1) caches 120-122 (L1 120 and L1 122) in the cores 116-118, respectively. Each of the L1 caches 120-122 includes memory circuitry for storing data and control circuitry for handling accesses of data stored in the memory circuitry. The processor 102 also includes a shared second level (L2) cache 124 and a third level (L3) cache 126, each of which includes memory circuitry for storing data and control circuitry for handling accesses of data stored in the memory circuitry.

[0036] Processor 102 includes a platform security processor (PSP) 128, which is a functional block used to perform security-related operations in electronic device 100. For example, in some embodiments, PSP 128 includes a CPU core, an ASIC, and / or a microcontroller. PSP 128 includes circuitry designed to be secure against specified malicious or erroneous behavior of other functional blocks and devices within processor 102 and / or software entities executed by processor 102. Thus, PSP 128 may be used to secure the operation of other functional blocks, devices, and / or software entities that are susceptible to such behavior. In other words, PSP 128 may perform operations associated with enabling a trusted execution environment in electronic device 100. To this end, PSP 128 may perform operations for encryption / decryption (e.g., key generation, data encryption / decryption, etc.), registration and / or authentication of hardware and / or software entities, access permission verification, etc. In some embodiments, PSP 128 performs at least some of the operations described herein to handle transitions of pages accessible to IO devices 108-110. For example, a software entity may communicate a request to migrate a page of memory to PSP 128, which then verifies that the migration of the page of memory is permitted (i.e., that the software entity is allowed to move a particular page of memory to a destination location in memory) before forwarding the request to migrate the page of memory to page migration engine 114 for subsequent processing.

[0037] Memory 104 is a functional block used to store data for other functional blocks in electronic device 100. For example, in some embodiments, memory 104 is the "main" memory in electronic device 100. Memory 104 includes memory circuitry for storing data and circuitry for handling access of data stored in the memory circuitry.

[0038] In some embodiments, memory 104 includes two or more different types of memory arranged such that a different portion of the set of addressable locations in memory 104 is in each type of memory. For example, in some embodiments, half of the addressable locations are in a first type of memory and are therefore implemented using the first type of memory circuitry, and half of the addressable locations are in a second type of memory and are therefore implemented using the second type of memory circuitry. The use of two types of memory is illustrated in FIG. 1 by memory type 130 and memory type 132. For example, memory type 130 may be a fifth generation double data rate synchronous dynamic random access memory (DDR5 DRAM) or another type of memory, and memory type 132 may be a 3D XPoint memory or another type of memory. Each type of memory has respective memory characteristics. For example, memory characteristics may include some or all of the following: access speed / latency, access bandwidth, data protection / verification / error correction, reliability / mean time between failures, power consumption, heat, data addressability (e.g., byte or block addressability), data compatibility / specifications, monetary cost, design complexity, etc.

[0039] The mass storage device 106 is a functional block and / or device that stores data used by other functional blocks and devices in the electronic device 100. For example, the mass storage device 106 may be or include a semiconductor memory, a disk drive, an optical drive, etc. The mass storage device 106 includes circuits and / or devices that retain stored data even when power to the mass storage device 106 is cut off (or otherwise reduced), and thus serves as a non-volatile "long-term" storage device for data. At run-time (i.e., when the electronic device 100 is operating), copies of data are retrieved from the mass storage device 106 and stored in the volatile memory 104 (and possibly one or more caches) for subsequent access by functional blocks in the electronic device 100. For example, data may be retrieved / read from the mass storage device 106 in blocks or pages of a predetermined size (e.g., 4KiB, 2MiB, 1GiB, etc.) and in pages stored in the memory 104, i.e., "pages of memory." Additionally, pages of memory generated by other functional blocks and devices within electronic device 100 may be stored within memory 104 (e.g., to store computation results, files, configuration values, etc.) and eventually written out to mass storage device 106. While a page of memory is stored in memory 104, it may be accessible to the other functional blocks and devices in accordance with the access rules and permissions for the page of memory and / or the data therein.

[0040] Returning to the processor 102, the memory management unit (MMU) 134 is a functional block that processes memory access requests and requests for information from page tables. When data is to be accessed by a functional block in the processor 102, the functional block sends the memory access request to the MMU 134. For example, a software entity (e.g., a program, an operating system, etc.) being executed by the core 116 may cause a load / store unit in a processing circuit in the core 116 to send a memory access request (e.g., to load or store data, etc.) to the MMU 134. The MMU 134 then sends a corresponding memory access request to one or more of the L2 cache 124, the L3 cache 126, and the memory 104 to satisfy or resolve the memory access request. For example, when data is to be loaded, the MMU 134 may retrieve the data from the L2 cache 124, the L3 cache 126, or the memory 104 and forward the data to the requesting functional block. This may mean loading one or more memory pages containing the data from mass storage device 106 into memory 104 if the data is not already present in memory 104.

[0041] MMU 134 includes table walker (TW) 136, which is a functional block that performs operations related to obtaining address translation information and other information from page tables via page table walks. For example, as described in more detail above, in some embodiments, electronic device 100 uses virtual memory to allow software entities (e.g., executed by cores 116-118) to access memory using virtual addresses that are translated to physical addresses. Upon receiving a memory access request from a software entity having a virtual address, table walker 136 performs a page table walk operation to translate the virtual address to the physical address of the page where the data is located in memory 104. During a page table walk operation, table walker 136 may check page table entries and / or other records to ensure that the functional block and / or software entity requesting the respective memory access is authorized to perform such access, i.e., is permitted to access memory at a physical address, etc., and that the page tables and other records used for the translation have not been tampered with or impermissibly altered. In some embodiments, the table walker 136 checks the migration state information in the page table entry as part of the operations described herein to process page migrations.

[0042] In some embodiments, table walker 136 uses two separate page tables to perform address translation. In some of these embodiments, the two page tables include guest page table 148 and nested page table 150. Each of guest page table 148 and nested page table 150 is a record that includes entries, or “page table entries,” for storing address translation information and other information for a respective page of memory in memory 104. For example, guest page table 148 may include page table entries having information for pages of memory currently in memory 104 that have been accessed by a corresponding guest operating system. In some embodiments, guest page table 148 includes guest virtual address to guest physical address translation information for pages of memory and respective other information (i.e., metadata, etc.). As another example, nested page table 150 may include page table entries having information for pages of memory currently in memory 104 that have been accessed by a guest operating system and / or another software entity or functional block. Nested page table 150 (interchangeably referred to as a system page table, a host page table, etc.) contains guest physical address to system physical address translation information for pages of memory, as well as other information (i.e., metadata, etc.). In some embodiments, among the other information stored in page table entries in nested page table 150 is migration state information used to handle migration of pages of memory, as described herein.

[0043] 1 as an example, in some embodiments, electronic device 100 includes a separate guest page table for each guest operating system (there can be multiple guest operating systems) that address translation is used by / for that guest operating system. Additionally, while guest page table 148 and nested page table 150 are shown in different types of memory, there is no requirement that guest page table 148 and nested page table 150 be stored in a particular type of memory, and guest page table 148 and nested page table 150 can be stored anywhere in memory 104. Also, while shown as a single entity in FIG. 1, in some embodiments guest page table 148 and / or nested page table 150 are or include multiple separate page sub-tables that are used in combination to perform virtual to physical address translations. For example, in some embodiments, one or both of guest page table 148 and nested page table 150 are hierarchical page tables that include two or more layers or levels of subtables (and may include multiple subtables at each layer), where each subtable except the final subtable indicates the next subtable in the hierarchy to be searched for address translation information, and the final subtable contains the address translation information. In these embodiments, performing a table walk includes traversing each subtable to arrive at the final subtable where the address translation information is stored.

[0044] MMU 134 includes a translation lookaside buffer (TLB) 138, which is a functional block used to store copies of information obtained from page table entries (e.g., from guest page table 148 and / or nested page table 150). TLB 138 includes memory circuitry for caching to store copies of page table entries or portions thereof obtained during a page table walk by table walker 136 (or from other sources). To avoid performing a page table walk, MMU 134 obtains address translation information and / or other page table entry information from TLB 138, when possible. However, if the address translation information and / or other page table entry information is not present in TLB 138, table walker 136 performs a table walk in guest page table 148 and / or nested page table 150 to obtain the information. In some embodiments, the specified information in TLB 138 is invalidated before the corresponding page is migrated, as described herein.

[0045] The input / output (IO) devices 108-110 are functional blocks or devices that interact with the processor 102 and other functional blocks and devices within the electronic device 100. Thus, the processor 102 and other functional blocks may receive “input” data from or send “output” data to the IO devices 108-110. The IO devices 108-110 may also interact with functional blocks or devices external to the electronic device 100. For example, the I / O devices 108-110 may include network interface devices, disk controllers, devices coupled to corresponding wired or wireless buses or interfaces (e.g., Peripheral Controller Interface Express (PCIe) bus, Universal Serial Bus (USB), WiFi network, etc.), graphics processing units, etc. The specific operations performed by each of the IO devices 108-110 depend on the nature of the respective IO device. For example, if IO device 108 is a disk controller, then IO device 108 may retrieve data from a disk (e.g., mass storage device 106) and write data to memory 104, or vice versa. As another example, if IO device 110 is a network interface device, then IO device 110 may store in memory 104 data received over a network (not shown), retrieve from memory 104 data to be sent over the network to a receiving device, provide data to or retrieve data from processor 102, etc.

[0046] IO hub 112 is a functional block or device that performs operations to interface between IO devices 108-110 and other functional blocks within electronic device 100. In other words, IO hub 112 performs operations to route communications and other data between IO devices 108-110 and functional blocks such as processor 102, memory 104, etc. The particular communications routed by IO hub 112 and the operations performed to route the communications depend on the nature of the IO devices 108-110, but may include memory access, data communications, control and configuration communications, etc.

[0047] IO hub 112 includes an input / output memory management unit (IOMMU) 140, which is a functional block that performs operations to process memory accesses by IO devices 108-110. When data in memory 104 is accessed by an IO device, the IO device sends a corresponding memory access request. IOMMU 140 intercepts the memory access request from the IO device, processes the request internally to determine whether the memory access request can proceed, and then sends a corresponding request to memory 104 to access the data (assuming the memory access request can proceed).

[0048] Among the operations performed by IOMMU 140 to process memory access requests from IO devices 108-110 are performing address translations for memory accesses to IO devices 108-110 (i.e., to obtain a physical address in memory associated with a virtual address used by IO devices 108-110 in a memory access request). IOMMU 140 includes input-output table walker (IOTW) 142, which is a functional block that performs operations related to obtaining address translations from page tables (e.g., page table walks, etc.). The operations performed by input-output (IO) table walker 142 are similar to those performed by table walker 136, even though the memory accesses are from IO devices 108-110 (rather than processor 102). In some embodiments, along with performing address translations, IO table walker 142 checks page information, such as metadata and migration state information, to determine whether memory accesses from IO devices 108-110 are permitted for a page of memory and / or whether the page of memory has been migrated as described herein. In these embodiments, IO table walker 142 may block or otherwise handle memory accesses to avoid inappropriate memory accesses or memory accesses to pages that are being migrated.

[0049] In some embodiments, IOMMU 140 uses an IOMMU-specific page table, IO page table 152, to perform address translation. IO page table 152 is a record that contains entries, or "page table entries," for storing address translation information and other information for each page of memory in memory 104. For example, IO page table 152 may include page table entries with information for pages of memory currently in memory 104 that have been accessed by an IO device or by IOMMU 140 itself. IO page table 152 includes guest physical address to system physical address translation information for pages of memory, as well as other information for each (i.e., metadata, etc.). (In some embodiments, IOMMU 140 obtains address information from a guest operating system that owns the page of memory, such that the contents of IO page table 152 include the guest physical addresses.) In some embodiments, among other information stored in the page table entries in IO page table 152 is migration state information that is used to handle migration of pages of memory, as described herein. Although shown as a single entity in FIG. 1, IO page table 152, like guest page table 148 and nested page table 150, may include a hierarchy of separate sub-tables.

[0050] IOMMU 140 also includes a translation lookaside buffer (TLB) 144, which is a functional block used to store copies of information obtained from page table entries in IO page table 152 (or elsewhere). TLB 144 includes memory circuitry for caching to store copies of page table entries or portions thereof obtained during a page table walk by table walker IO table walker 142 (or from other sources). To avoid performing a page table walk, IOMMU 140 obtains address translation information and / or other page table entry information from TLB 144, when possible. However, if the address translation information and / or other page table entry information is not present in TLB 144, IO table walker 142 performs a table walk to obtain the information. In some embodiments, the specified information in TLB 144 is invalidated before the corresponding page is migrated, as described herein.

[0051] In some embodiments, some or all of IO devices 108-110 include a local cache for storing copies of information obtained from page table entries. For example, in some embodiments, some or all of IO devices 108-110 and IOMMU 140 provide support for Peripheral Component Express (PCIe) Address Translation Services (ATS), under which IOMMU 140 can provide copies of information obtained from page table entries in IO page tables 152 to IO devices 108-110 for caching within IO devices 108-110. An example of such a cache is shown as Address Translation Cache (ATC) 146 in IO device 108. ATC 146 includes memory circuitry for caching that stores copies of page table entries or portions thereof obtained during a page table walk by IO table walker 142 (or from other sources). To avoid performing a page table walk, IO device 108 obtains address translation information and / or other page table entry information from ATC 146, when possible. However, if the address translation information and / or other page table entry information is not present in ATC 146, IO table walker 142 performs a table walk to obtain the information (or uses cached information from TLB 144). In some embodiments, the specified information in ATC 146 is invalidated before the corresponding page is migrated, as described herein.

[0052] In some embodiments, although shown as a single entity in FIG. 1, IOMMU 140 includes multiple individual IOMMUs (e.g., multiple instances of an IOMMU functional block). For example, in some embodiments, electronic device 100 includes at least two IOMMU functional blocks, each controlling memory access by a subset of the set of IO devices in electronic device 100. In these embodiments, these IOMMUs work together or through a “master” IOMMU or other functional block to ensure that the IOMMUs use information in the page tables to properly access the memory of the IO devices, including when there are pages of memory in transition as described herein. For example, the IOMMUs may broadcast requests and acknowledgments, such as invalidating TLBs, completing or stalling memory accesses, over a communication link to which all the IOMMUs are connected, to ensure that all of the IOMMUs are synchronized before performing operations such as allowing a given IOMMU to access information in a page table entry or allowing a memory access from an IO device to proceed. In some of these embodiments, all IOMMUs use a single shared page table (eg, IO page table 152), while in other embodiments, each IOMMU uses a separate page table.

[0053] In some embodiments, electronic device 100 includes a reverse map table (RMT) 154 in memory 104. Reverse map table 154 is a record used (e.g., by table walker 136 and IO table walker 142) to detect when changes are made to nested page table 150 and otherwise protect pages of memory from unauthorized access by guest operating systems, hypervisors, and / or IO devices. Reverse map table 154 includes several entries, one entry for each page of memory in memory 104 that may be allocated for use by one or more virtual machines. For example, if memory 104 includes 32 GiB of memory that may be allocated in 4 KiB pages to virtual machines, the reverse map table includes 8,388,608 entries. An entry in reverse map table 154 may store information such as an identifier of a guest physical address associated with a system physical address, an identifier of a permission level for accessing a virtual machine, an identifier of an owner of a page of memory, and / or other information associated with the corresponding page. An example of a reverse map table is shown in Figure 6 and described in more detail below. In some embodiments, when performing a table walk, table walker 136 and / or IO table walker 142 checks the corresponding entry in reverse map table 154 to ensure that the information in nested page table 150 is used safely (i.e., has not been unexpectedly modified by the hypervisor) and that a given memory access can be performed on the respective page of memory.

[0054] The page migration engine 114 is a functional block that performs operations associated with accessing data in the memory 104. The page migration engine 114 may be used to offload memory access operations from other functional blocks in the electronic device 100. For example, in some embodiments, the page migration engine 114 offloads, and thus receives and performs, memory accesses on behalf of the cores 116-118, thereby allowing the cores 116-118 to avoid performing memory access operations. In some embodiments, the page migration engine 114 includes, among other functional blocks, one or more direct memory access (DMA) functional blocks that perform operations to move pages of memory in the memory 104. Among the memory access operations performed by the page migration engine 114 may be to migrate pages of memory in the memory 104. To migrate a page of memory, upon receiving a request from one of the cores 116-118 (e.g., from an operating system executed by the core 116) or from another source, such as the PSP 128, identifying the page of memory and the new location in memory where the page of memory is to be stored, the page migration engine 114 moves the page of memory to the new location (i.e., physically moves individual bits / bytes within the page of memory to the new location in one or more data move operations). In addition to migrating the page of memory, the page migration engine 114 also performs operations to process the migration of pages of memory that are accessible to IO devices, such as operations to update the IO page tables 152 and reverse map tables 154, as described herein.

[0055] 1 as a single entity, in some embodiments, page transition engine 114 includes multiple individual page transition engines (e.g., multiple instances of a page transition engine functional block). For example, in some embodiments, page transition engine 114 includes at least two page transition engines, each arranged to perform memory accesses, including migrating pages of memory as described herein, on behalf of other functional blocks in electronic device 100. In these embodiments, page transition tasks / requests are distributed among two or more page transition engines according to a distribution scheme (e.g., round robin, by memory address or location in memory, etc.), the page transition engines may throttle accesses to avoid designated memory accesses, etc.

[0056] In some embodiments, communication paths are coupled between functional blocks in electronic device 100, as indicated by arrowed lines between the functional blocks. The communication paths include one or more buses, wires, guides, and / or other connections, possibly involving controllers, fabric elements (e.g., switches, routers, etc.). The communication paths are used to route commands, data, control signals, and / or other information between the functional blocks. For example, in some embodiments, an interconnect fabric is coupled between IO hub 112, processor 102 (e.g., MMU 134), IOMMU 140, and memory 104. Note that some communication paths are not shown in FIG. 1 for clarity.

[0057] Although electronic device 100 is illustrated in FIG. 1 with a particular number and arrangement of functional blocks and devices, in some embodiments electronic device 100 includes a different number and / or arrangement of functional blocks and devices. For example, in some embodiments processor 102 includes a different number of cores. As another example, in some embodiments a different number and / or arrangement of caches are present within processor 102. As yet another example, in some embodiments IO device 108 does not include an address translation cache, or both IO devices 108-110 include a translation cache. As yet another example, while shown included in electronic device 100, in some embodiments one or both of IO devices 108-110 are external IO devices, such as IO devices coupled to electronic device 100 via a network, a wired or wireless connection, or the like. Generally, in the described embodiments electronic device 100 includes a number and / or arrangement of functional blocks sufficient to perform the operations described herein.

[0058] 1 is simplified for purposes of explanation. However, in some embodiments, the electronic device 100 and / or the processor 102 include additional or different elements and mechanisms for performing the operations described herein and other operations. For example, the electronic device 100 and / or the processor 102 may include power functional blocks or devices, human interface functional blocks or devices, etc.

[0059] In some embodiments, functional blocks shown separately in FIG. 1 are implemented together. For example, in some embodiments, IO hub 112 and some or all of IO devices 108-110 are incorporated within / with processor 102, such as fabricated on the same integrated circuit chip. In other words, in such embodiments, IO hub 112 and IO devices 108-110 may be integrated with processor 102 (e.g., as a "system on a chip" or otherwise). Alternatively, in some embodiments, functional blocks shown as part of other functional blocks may be separate. For example, in some embodiments, PSP 128 is implemented separately from processor 102.

[0060] Electronic device 100 may be or may be included in any electronic device that performs computational operations. For example, electronic device 100 may be or may be included in a desktop computer, a laptop computer, a wearable electronic device, a tablet computer, a smartphone, a server, an artificial intelligence device, a virtual or augmented reality device, a network device, a toy, an audiovisual device, a home appliance, a controller, a vehicle, and / or combinations thereof.

[0061] (operating system) In some embodiments, a processor in an electronic device executes several software entities that cause the processor to perform various operations. For example, in some embodiments, the software entities include an operating system. Figure 2 shows a block diagram illustrating functional blocks and devices in an operating system and an electronic device according to some embodiments. Note that Figure 2 is simplified and generally illustrates the relationship between the operating system and the electronic device hardware. In some embodiments, some or all of the elements shown in Figure 2 are not present and / or are located differently.

[0062] As seen in FIG. 2, an operating system 200 interfaces between electronic device hardware 202 (i.e., between functional blocks and devices of electronic device 100, such as processor 102) and a set of programs 204-208. For example, operating system 200 may be an operating system such as Windows® from Microsoft (Redmond, WA), macOS® from Apple, Inc. (Cupertino, CA), and programs 204-208 may be productivity applications, scientific computing applications, web browsers, and the like, respectively. IOMMU 210 interfaces between IO devices 212 and electronic device hardware 202 (e.g., memory 104, and the like). In some embodiments, the functional blocks and devices of FIG. 2, i.e., electronic device hardware 202, IOMMU 210, and IO devices 212, are similar to hardware (e.g., processor 102, memory 104, page transition engine 114, and the like), IOMMU 140, and IO devices 108-110, respectively, in electronic device 100, as shown in FIG. 1.

[0063] (Virtual Machines and Hypervisors) FIG. 3 illustrates a block diagram showing functional blocks and devices within a hypervisor and an electronic device according to some embodiments. As seen in FIG. 3, there are three virtual machines (VMs) 300-304, each of which runs a corresponding guest operating system (GUEST OS) 306-310 and one or more programs (PRGRMS) 312-316. A hypervisor 318 interfaces between a host operating system 320 and the virtual machines 300-304. The host operating system 320 interfaces between the electronic device hardware 322 and the hypervisor 318. An IOMMU 324 interfaces between an IO device 326 and the electronic device hardware 322. In some embodiments, the functional blocks and devices of FIG. 3, i.e., electronic device hardware 322, IOMMU 324, and IO devices 326, are similar to the hardware (e.g., processor 102, memory 104, page migration engine 114, etc.), IOMMU 140, and IO devices 108-110, respectively, in electronic device 100 as shown in FIG. 1.

[0064] 3 is a simplified schematic illustrating the relationship of a hypervisor to electronic device hardware. However, in some embodiments, some or all of the elements illustrated in FIG. 3 are not present and / or are located differently. For example, in some embodiments, hypervisor 318 communicates more directly with electronic device hardware 322 rather than communicating with electronic device hardware 322 through host operating system 320.

[0065] (page table entry) As discussed above, in some embodiments, functional blocks and devices within an electronic device (e.g., processor 102, IOMMU 140, etc.) use one or more page tables (e.g., guest page table 148, nested page table 150, and / or IO page table 152) to perform address translation and for other operations. FIG. 4 illustrates a block diagram of a page table 400 according to some embodiments. The page table 400 illustrated in FIG. 4 is presented as a general example of a page table. In some embodiments, some or all of nested page table 150 and IO page table 152 are located inside page table 400 (i.e., contain similar information as page table 400). In some embodiments, guest page table 148 is similarly located, although in some cases, the metadata in the page table entry does not include transition state information.

[0066] As seen in FIG. 4, the page table 400 includes several page table entries 402 (one of which is highlighted using dashed lines), each of which may store an address translation 404 along with corresponding metadata 406. When a page of memory (i.e., a block of data of a specified size, such as 4 KiB, 2 MiB, etc.) is retrieved from mass storage (e.g., mass storage 106) and stored in memory (e.g., memory 104) or newly created in memory, a corresponding page table entry 402 is added to the page table 400 along with the address translation 404 and metadata 406. Thus, if a page of memory is available in memory 104 and is being accessed by a software entity, functional block, or device associated with the page table 400, the page table 400 should include a corresponding page table entry 402. Recall that the page table entries 402 are added to the page table 400 to enable tracking of mappings between addresses used by accessing entities (e.g., software entities, functional blocks, and devices) and addresses where the data is located in memory.

[0067] In some embodiments, the address translation 404 in each page table entry 402 includes information that can be used to determine a physical address associated with a respective page of memory. In other words, each address translation 404 includes information that can be used to identify a location in memory 104 of a page of memory for an accessing software entity, functional block, or device. For example, in some embodiments, the address translation 404 includes virtual address to physical address translation information, such as guest virtual address to guest physical address translation information. As another example, in some embodiments, the address translation 404 includes physical address to physical address translation information, such as guest physical address to system physical address translation information. In some embodiments, each address translation 404 includes a first field having some or all of one or more virtual or physical addresses (e.g., N bits out of M bits of each virtual address, where N≦M) and a second field having some or all of the physical addresses to which the one or more virtual or physical addresses map / match.

[0068] The metadata 406 includes information associated with, characterizing, controlling, and / or otherwise relating to a corresponding address translation 404. When translation information for a page of memory is added to a page table entry 402 in the page table 400, metadata is obtained, generated, etc., and added to the page table entry 402. FIG. 5 illustrates a block diagram of the metadata 406 for a page table entry, according to some embodiments. As seen in FIG. 5, the metadata 406 includes validity 500, permission 502, control 504, and migration state information (MIG STATE INF) 506. The validity 500 includes one or more values ​​related to the validity of the page table entry 402, the address translation 404 in the page table 402, and / or the corresponding page of memory in the memory 104. For example, the validity 500 may include one or more bits indicating whether the page table entry 402 itself is valid / accessible and / or whether the corresponding page in the memory 104 is valid / accessible.

[0069] Permissions 502 include one or more values ​​related to access permissions for a corresponding page in memory 104. For example, permissions 502 may include one or more bits indicating whether the corresponding page in memory 104 has read-only or read-write permission set, whether the page is accessible only with a particular privilege (e.g., administrator, user, kernel), and / or other permission information.

[0070] The controls 504 include one or more values ​​related to the usage of the page table entry 402 and / or the corresponding page in the memory 104. For example, the controls 504 may include a page size indicator, a dirty indicator (if the corresponding page in the memory 104 has been modified and therefore does not match the associated block of memory in the mass storage device 106), an access indicator (if the page in the memory 104 has been accessed one or more times), a write-through indicator (if modifications to a page in the memory 104 are immediately reflected in the associated block of memory on the mass storage device 106), and / or other values.

[0071] The migration state information 506 includes one or more values ​​that indicate whether the associated page is being prepared to be migrated or whether the associated page is being migrated. For example, in some embodiments, the migration state information includes a sufficient number of bits to uniquely identify each possible migration state for the corresponding page of memory. For example, the migration state information may be stored in three bits if there are six migration states, with a first migration state for a non-migration state for a page that is not being migrated, a sixth migration state for a page that is being migrated, and four intermediate migration states for various preparatory operations to prepare the page of memory for migration. Functional blocks or devices that access the page table entry may use the migration state information 506 to determine which operations are permitted (or not permitted) for the page of memory and / or the information in the page table entry 402 of the corresponding page table. For example, in some embodiments, if the migration state indicates that a page of memory is being prepared to be migrated or that a page of memory is being migrated, the accessing functional block or device may determine that a copy of information from the page table entry is not allowed to be cached in a local cache and / or that some or all memory accesses to the page of memory are restricted (e.g., should be stalled until after the page is migrated).

[0072] In some embodiments, one or both of the address translation 404 and the metadata 406 may be modified / changed, updated, etc. after being added to the page table 400. For example, if a page is moved from a first location to a second location in memory 104, the address translation 404 in the corresponding page table entry 402 may be updated. As another example, in an embodiment in which the metadata 406 includes permissions 502 (e.g., read / write permissions), the permissions 502 may be updated to indicate read-only, read / write, etc. permissions for the corresponding page in memory 104. As yet another example, the transition state information 506 may be updated as functional blocks and devices progress through transition states to transition a page of memory.

[0073] As described above, when attempting to obtain address information or metadata, MMU 134 (e.g., table walker 136), IOMMU 140 (e.g., IO table walker 142), or another entity within electronic device 100 may perform a page table walk. During a page table walk, MMU 134, IOMMU 140, etc., searches page table 400 to find a page table entry 402 in which a corresponding address translation 404 is held, if such a page table entry 402 exists. Upon encountering such a page table entry 402, MMU 134, IOMMU 140, etc., obtains the address from page table entry 402. For example, MMU 134 may obtain a system physical address associated with the guest physical address, IOMMU 140 may obtain a system physical address associated with the guest physical address, etc. If MMU 134, IOMMU 140, etc., is unable to find a corresponding page table entry 402, an error handling action is performed (e.g., a page fault is issued and then handled, etc.).

[0074] In the described embodiments, functional blocks and devices (e.g., IOMMU, page migration engine, etc.) in the electronic device use the information in page table 400 to perform operations to prevent IO devices from encountering problems with missing or unavailable page table entries or corresponding pages of memory due to migration of the corresponding pages of memory. For example, in some embodiments, when determining an address translation for a memory access request, IO table walker 142 checks migration state information 506 in each page table entry 402 to determine whether the address translation 404 is allowed to be cached by IOMMU 140 and / or IO devices 108-110 and / or whether the memory access should be stalled due to the current migration state.

[0075] While page table 400 is shown with a particular number of page table entries 402, some embodiments include a different number of page table entries 402 (as indicated by the ellipsis in FIG. 4). Additionally, while page table entries 402 and metadata 406 are shown with a particular arrangement and type of information, in some embodiments, one or both of page table entries 402 and metadata 406 include different information. For example, in some embodiments, at least some of the metadata described above is stored in a location other than page table entries 402. For example, in some embodiments, a separate in-memory data structure is used to store transition state information. Generally, page table 400 includes sufficient information to enable functional blocks and devices within an electronic device to store and retrieve address translation information and metadata, as described herein.

[0076] Although page table 400 is shown as a single table, in some embodiments, page table 400 is organized differently. For example, in some embodiments, page table 400 is implemented using two or more sub-tables arranged in a hierarchy. In these embodiments, a search of each sub-table directs a search functional block or device to the next sub-table in the hierarchy until a final sub-table is reached, where address translation 404 and metadata 406 are stored in a corresponding page table entry 402.

[0077] (reverse map table entry) As mentioned above, in some embodiments, functional blocks and devices within an electronic device (e.g., MMU 134, IOMMU 140, etc.) use a reverse map table to verify the correctness of entries in a page table and for other operations. FIG. 6 shows a block diagram illustrating a reverse map table 600 according to some embodiments. Although reverse map table 600 is illustrated in FIG. 6 as including certain information, in some embodiments, there may be a different arrangement or type of information. In general, reverse map table 600 includes sufficient information to perform the operations described herein. It should be noted that in some embodiments, reverse map table 154 is located within reverse map table 600 (i.e., includes similar information as reverse map table 600).

[0078] In the following paragraphs, the reverse map table 600 is described in sufficient detail to understand the operations of using the reverse map table in this description. However, certain details of the reverse map table and the operations for maintaining and using the reverse map table are omitted for clarity and brevity. A more detailed description of the reverse map table, as well as some examples of operations on and using the reverse map table, can be found in U.S. Patent No. 10,509,736, issued December 17, 2019, the contents of which are incorporated herein by reference.

[0079] As seen in FIG. 6, the reverse map table 600 includes a number of entries 602 (the entries 602 are highlighted in FIG. 6 using dashed lines). Each entry 602 includes information about a corresponding page in memory, and in some embodiments, each allocatable page in memory has a respective reverse map table entry. In some embodiments, the entries in the reverse map table 600 are indexed using a system physical address associated with each page of memory, such that each entry 602 is associated with a distinct system physical address. For example, a first entry in the reverse map table 600 may be associated with a page of memory at the lowest system physical address (e.g., address A in the address space of the memory), a second entry in the reverse map table 600 may be associated with a page of memory at the second lowest system physical address (e.g., address A+4KiB-1 for a 4KiB page of memory), and so on.

[0080] In the reverse map table 600, each entry 602 may be used to store, for a respective page of memory, a guest identifier (ID) 604, a guest physical address (PHY ADDR) 606, page information (INFO) 608, an assigned indicator (ASSIGNED) 610, a lock indicator (LOCK) 612, migration state information (MIG STATE INF) 614, and other information (OTHER) 616. The guest identifier 604 is an identifier associated with the guest operating system to which the corresponding page of memory is assigned. Thus, when the corresponding page of memory is allocated for use by a particular guest operating system, the identifier of the particular guest operating system is recorded in the guest identifier 604. For example, the guest identifier 604 may hold an address space identifier (ASID), an ID string, a name, and / or another value that identifies the guest operating system. For a page of memory not assigned to a particular guest operating system, guest identifier 604 may identify the page as being allocated to the hypervisor, shared among multiple guest operating systems, or allocated in some other manner.

[0081] Guest physical address 606 is a value representing a guest physical address associated with the system physical address of entry 602. For example, when a page at a given system physical address is allocated for use by a guest operating system, assigned to a guest operating system, etc., the guest physical address used by the guest operating system to address the page is recorded in the corresponding entry 602 in reverse map table 600. In this manner, a record is created of the particular guest physical address used by the guest operating system to which each page is allocated. This information allows a table walker (e.g., table walker 136 or IO table walker 142) to check a system physical address obtained during a walk of a nested page table or IO page table to determine whether the system physical address maps to an expected guest physical address. This can allow detection of whether the mapping has been maliciously or erroneously altered by the hypervisor or another entity.

[0082] Page information 608 includes information about a page of memory, such as a subpage count for the page of memory (e.g., the number of 4KiB pages in a 2MiB page, etc.) and / or the size of the page of memory (e.g., in bytes, etc.). Page information 608 can be used to determine whether a memory access to a page of memory is directed to an appropriately sized page of memory (i.e., that a memory access of a 2MiB page is not addressing a 4KiB page, etc.).

[0083] The allocated indicator 610 is an indicator of whether the entry 602 is currently allocated to one or more guest operating systems. The allocated indicator 610 is used to prevent use of information from an entry 602 in the reverse map table 600 that is not currently allocated but may contain information, is initialized but contains no actual information, etc. In some embodiments, the allocated indicator 610 is set if the page is currently allocated to one or more guest operating systems and is unset / cleared when the page is deallocated from all guest operating systems.

[0084] The lock indicator 612 is an indicator of whether the entry 602 is locked. When the lock indicator 612 is set, the entry 602 is considered locked, and when the lock indicator 612 is unset / cleared, the entry 602 is considered free or available. When the entry 602 is locked by a given software entity, functional block, or device in the electronic device 100, other software entities, functional blocks, and devices are prevented from accessing the entry 602 (e.g., reading, writing / updating, etc. the entry 602). In contrast, software entities, functional blocks, or devices may be permitted to access an available (i.e., unlocked) entry 602 (possibly after they themselves have set the lock indicator 612). For example, in an embodiment in which the lock indicator 612 is a single bit and is initially clear (e.g., set to zero), an entity may set the lock indicator 612 (e.g., to one), access the entry 602 one or more times, and then clear the lock indicator 612 when it is finished accessing the entry 602. Using lock indicator 612, the described embodiments can avoid race conditions and other conditions that may occur when concurrent reads and / or writes occur to entries 602. Additionally, lock indicator 612 can be used in combination with other information, software entities, functional blocks and / or devices to prevent a specified access from transitioning a page of memory and / or a respective page table entry.

[0085] The migration state information 614 includes one or more values ​​that indicate whether the associated page of memory is being prepared to be migrated or whether the associated page of memory is being migrated. For example, in some embodiments, the migration state information 614 includes a sufficient number of bits to identify each migration state of a set of migration states of the corresponding page of memory. Continuing with the six migration state example above, for example, the migration state information may be stored in three bits. A functional block or device accessing an entry 602 may use the migration state information 614 to determine which operations are permitted or not permitted on the page of memory and / or the information in the corresponding page table entry. For example, in some embodiments, if the migration state information 614 indicates that the page of memory is being prepared to be migrated or that the page of memory is being migrated, the accessing functional block or device may determine that a copy of the information from the page table entry is not permitted to be cached in a local cache and / or that some or all memory accesses to the page of memory are restricted. In some embodiments, the information maintained in transition state information 614 may be organized differently and / or include different values ​​than the transition information in transition state information 506. Some examples of the different information maintained in transition state information 614 and transition state information 506 are presented below with respect to FIG.

[0086] Other information 616 includes information for and / or about the page of memory, the guest operating system to which the page is allocated, entry 602 itself, and / or other information useful in controlling access to entry 602 and / or the page of memory. For example, in some embodiments, other information 616 includes a verified indicator that is an indicator of whether entry 602 has been verified by a guest operating system (i.e., a guest operating system to which the corresponding page of memory is allocated). As another example, in some embodiments, other information 616 includes a permission level block having a set of permissions for the corresponding page of memory. As yet another example, in some embodiments, other information 616 includes a shared page indicator that is an indicator of whether the corresponding page is shared by two or more guest operating systems. Verified indicators, permission level blocks, shared page indicators, and / or other metadata (e.g., immutable indicators, etc.) that may be included in other information 616 are described in more detail in U.S. Pat. No. 10,509,736, the contents of which are incorporated herein by reference as noted above.

[0087] (Transition state) In some embodiments, software entities, functional blocks and / or devices within the electronic device (e.g., IOMMU 140, page migration engine 114, processor 102, operating system 200, and / or hypervisor 318) progress through a set of transitional states as pages of memory are prepared for migration and then migrated. In each of the transitional states, apart from an initial non-migration state, respective operations are performed to prepare the pages of memory for migration and / or to migrate the pages of memory. Requests for memory access of the pages of memory and / or information in the respective page table entries to be or being migrated are processed by functional blocks and devices within the electronic device (e.g., IOMMU 140, page migration engine 114, etc.) based on the current migration state of the pages of memory. Generally, the access requests are processed such that the software entities, functional blocks and / or devices continue to operate correctly while the operations of each transitional state are being performed. For example, IO devices (eg, IO devices 108-110) and / or processors (eg, processor 102) are protected from errors and faults that may result from accessing pages of memory and / or respective page table entries that are in transition.

[0088] FIG. 7 illustrates a state diagram showing transition states, according to some embodiments. The transition states illustrated in FIG. 7 are examples of transition states that software entities, functional blocks, and / or devices in an electronic device progress through as a page of memory is prepared for migration and then the page of memory is migrated. Although a particular number and arrangement of transition states and transitions between transition states are illustrated in FIG. 7, in some embodiments, a different number and / or arrangement of transition states and / or different transitions between transition states may be used. For example, some transition states in FIG. 7 include two or more actions (e.g., transition state 708 in which both a lock indicator and transition state information are set in the reverse map table entry) that may otherwise be associated with separate transition states.

[0089] In Figure 7, the transitions between the transition states are shown using arrow lines between the transition states. A first set of transitions between the transition states are shown using thick lines and other transitions between the transition states are shown using thin lines. In the following description, the first set of transitions between the transition states will be described, followed by the other transitions between the transition states. However, it should be noted that the use of different line thicknesses is merely for clarity in describing the transition states and the transitions between the transition states.

[0090] As seen in FIG. 7, the migration states include migration states (MS) 700-712. In migration state 700, which is a "non-migration" migration state, the page of memory has not been migrated and therefore will remain in its current location in memory. Thus, migration state 700 is a migration state in which the page of memory is maintained until the page of memory is migrated (or, more generally, the page of memory is held until the page of memory is migrated). In migration state 700, migration state information in a page table entry in an IO page table entry for the page of memory (e.g., migration state information 506) is set to a respective bit sequence, value, etc., that represents or identifies that the page of memory is in migration state 700. For example, assuming that the migration state information in an IO page table includes two bits, the migration state information in an IO page table entry for the page of memory may be set to 700 to represent or identify migration state 00. In FIG. 7, the migration state information for the IO page table entry is shown as PTE MS:0 in the callout for migration state 700 (the respective callouts for each of migration states 700-712 are shown using dashed lines).

[0091] In some embodiments, a nested page table (e.g., nested page table 150) includes migration state information within a page table entry. In these embodiments, similar to how migration state information is set in an IO page table, migration state information is set in the nested page table entry to identify the current migration state of each page of memory. However, for clarity and brevity, the setting of migration state information within a nested page table entry is not described with respect to FIG. 7. Thus, all references to "page table entries" in the following description of FIG. 7 refer to IO page table entries unless otherwise specified. Note that in some embodiments, the IO page table and nested page table are combined, such that all of the information described herein as being found in an IO page table is instead found in a nested page table, and the electronic device may not include a separate IO page table. In these embodiments, operations described herein involving an IO page table instead involve a nested page table.

[0092] In the migration state 700, the RMT migration state information in the reverse map table (RMT) entry for each of the source page of memory and the destination page of memory (e.g., migration state information 614) is set to a bit sequence, value, etc., that represents or identifies the page of memory as being in the migration state 700. For example, assuming the migration state information includes two bits, the RMT migration state information in each of the RMT entries may be set to 00 to represent or identify the migration state 700. As used herein, a "source page of memory" and a "destination page of memory" are, respectively, the location in memory where the page of memory to be migrated is currently located and the location in memory to which the page of memory should be relocated during the migration. Recall that the reverse map table includes an RMT entry for each page of memory (i.e., associated with each location / physical address where a page of memory may be stored), and thus the RMT migration state information is found in the reverse map table entry associated with the location in memory where the page of memory to be migrated is currently located (the source page of memory) and the location in memory to which the page of memory should be relocated (the destination page of memory). In FIG. 7, the RMT migration state information in the RMT entries for the source page of memory RMT[S] and the destination page of memory RMT[D] is shown in the callout of migration state 700 as RMT[S]MS:0 and RMT[D]MS:0, respectively.

[0093] Along with the RMT migration state information, in the migration state 700, a lock indicator (e.g., lock indicator 612) in the RMT entry for each of the source page of memory and the destination page of memory is cleared (i.e., unset, deasserted, etc.). This indicates that each of the RMT entries for the source page of memory and the destination page of memory are now unlocked and therefore may be accessed normally (i.e., without any restrictions due to the migration of the page of memory). In Figure 7, the lock indicators in the RMT entries for the source page of memory and the destination page of memory are shown in the callout for the migration state 700 as RMT[S]LK:MT[S]LK:0 and RMT[D]LK:0, respectively.

[0094] In the migrating state 700, the IOMMU allows accesses of the page of memory and the respective information in the page table entry from the IO device to proceed normally, because the migration state information in the page table entry and the RMT migration state information in the RMT entry identify the page of memory as being in the migrating state 700. That is, because no preparations have been made to migrate the page of memory, and the page of memory has not been migrated, accesses of the page of memory and the information in the page table entry are not prevented, stalled, or otherwise restricted by the IOMMU. Thus, in the migrating state 700, the IOMMU uses the information from the page table entry normally, and performs memory accesses of the page of memory normally.

[0095] In some embodiments, access of the page of memory and its respective page table entry is not restricted in the transition state 700 due to the transition of the page of memory, although other restrictions on access of the information within the page of memory and the page table entry may still be enforced by the IOMMU. For example, if the page of memory is not accessible to IO devices because it is a private page of memory that is not normally accessible to IO devices, then being in the transition state 700 does not change such access restrictions.

[0096] From transition state 700, the functional blocks, devices, and software entities proceed to transition state 702 (as indicated by the bold arrowed line between transition states 700 and 702). For example, upon encountering a request to migrate a page of memory (e.g., forwarded from a platform security processor (e.g., PSP 128)) or otherwise determining that a page of memory should be migrated, the software entity (e.g., hypervisor, operating system, etc.) notifies the page migration engine to initiate operations to migrate the page of memory, i.e., proceed to transition state 702. Upon entering transition state 702, the page migration engine sets migration state information in a page table entry for the page of memory to a bit sequence, value, etc., that represents or identifies transition state 702. For example, assuming the transition state information includes two bits, the transition state engine may set the transition state information to 10 to represent or identify transition state 702. In FIG. 7, the page table entry's migration state information is shown as PTE MS:2 in the callout for transition state 702.

[0097] Note that in the embodiment shown in FIG. 7, migration state information 1 is not used in the migration state page table entry, and therefore PTE MS:1 is not shown in the migration state of FIG. 7. In this embodiment, migration state information value 1 is reserved for uses not described with respect to FIG. 7. More specifically, in the embodiment shown in FIG. 7, the IOMMU and IO device operate in a secure ATS mode in which the IOMMU provides a guest physical address to the IO device in response to an ATS request, and therefore does not provide a system physical address to the IO device. In the secure ATS mode, the IOMMU provides only a guest physical address, since the guest physical address in the memory access request is translated to a system physical address by the IOMMU, and the IOMMU provides an opportunity for the IOMMU to check information in the page table entry and / or RMT for the memory access request. That is, the IOMMU can check the migration state for each page of memory, and the memory access can be processed as described with respect to FIG. 7. In contrast, in some embodiments, in (non-secure) ATS mode, if an IO device is provided with a system physical address and therefore may access memory without the IOMMU performing the checks described above, the software entity may set the migration state information of a page of memory to 1 to inform the IOMMU (and other entities) that the software entity is initiating operations to migrate the page of memory. In these embodiments, the software entity may invalidate the system physical address held by the IO device in the ATC to avoid accessing a page of memory that is being prepared for migration and / or is being migrated.

[0098] There is no change in the RMT migration state information or lock indicator in the RMT entry of the source page of memory or the destination page of memory between migration state 700 and migration state 702. This is because the RMT migration state information and lock indicator have not yet changed in migration state 702 and will change in subsequent migration states as the page of memory continues to be prepared for migration. In Figure 7, the RMT migration state information in the RMT entries of the source page of memory and the destination page of memory are shown in the callouts of migration state 702 as RMT[S]MS:0 and RMT[D]MS:0, respectively. Additionally, in Figure 7, the lock indicator in the RMT entries of the source page of memory and the destination page of memory are shown in the callouts of migration state 702 as RMT[S]LK:0 and RMT[D]LK:0, respectively. Note that because the lock indicator in the RMT entry for each of the source page of memory and the destination page of memory remains cleared, the RMT entries for each of the source page and destination page remain unlocked and therefore may still be accessed normally.

[0099] In the migrating state 702, initial preparations are made to migrate a page of memory in memory. The preparations include preventing the IOMMU from caching (i.e., storing) a new copy of information from the page table entry for the page of memory in a local cache in the IOMMU and / or IO device (e.g., TLB 144, ATC 146, etc.). Because a subsequent migration of the page of memory would cause the copy of information from the page table entry to be inaccurate due to the page of memory being located at a new physical address in memory, the copy of information from the page table entry is prevented from being stored in the local cache in the IOMMU or IO device until the page of memory is migrated and the page table entry is updated. Thus, in the migrating state 702, the migration state information in the page table entry for the source page of memory, the RMT migration state information, and the lock indicator in the RMT entry identify the page of memory as being in the migrating state 702, so the IOMMU will not obtain a copy of the information from the page table entry to cache such a copy in a local cache within the IOMMU itself and / or to provide such a copy to an IO device for storage in the IO device's local cache. In other words, when the IOMMU encounters the combination of PTE MS:2, RMT[S]LK:0, and RMT[S]MS:0 during a page table walk and reverse map table check of a memory access request for a page of memory, the IOMMU determines that the page of memory is in the migrating state 702 and restricts use of the copy of information from the page table entry as described.

[0100] In some embodiments, if the IOMMU does not "provide" a copy of information from a page table entry to the IO device, as described above, the IOMMU denies an ATS request from the IO device to copy information from a page table entry. For example, in some embodiments, the IOMMU can respond to an ATS request by informing the requesting IO device that the page of memory does not exist, allowing the IO device to proceed with the operation to make the page exist.

[0101] In the migrating state 702, in contrast to certain subsequent migrating states, the IOMMU allows memory accesses to proceed other than the newly encountered memory access for which the RMT check is not performed. For example, the IOMMU may allow both existing and newly encountered memory accesses for which the RMT check is performed to proceed. The memory access for which the RMT check is performed proceeds normally because the RMT entry for the source page of memory is not yet marked as locked in the migrating state 702. In other words, upon encountering the combination of PTE MS:2, RMT[S]LK:0 and RMT[S]MS:0 during the page table walk and reverse map table check of a memory access request for a page of memory, the IOMMU determines that the page of memory is in the migrating state 702 and allows the newly encountered memory access to proceed. Allowing the memory access to proceed clears / completes the memory access for the page of memory in preparation for migrating the page of memory (because a page of memory should not be migrated with a memory access outstanding) without unnecessarily delaying the operation of the IO device.

[0102] In the migrating state 702, in contrast to the other types of memory accesses described above, the IOMMU stalls newly encountered memory accesses from IO devices to the page of memory for which no RMT check is performed. Thus, in the migrating state 702, because the migration state information in the page table entry and the RMT entry of the source page identifies the page of memory as being in the migrating state 702, the IOMMU stalls newly encountered memory accesses for which no RMT check is performed until the migration of the page of memory is completed (e.g., back to the migrating state 700). For example, the IOMMU may hold the newly encountered memory accesses for which no RMT check is performed in a buffer, memory element, etc. (e.g., by storing information about the memory access request from the IO device), and then release the held memory accesses from the buffer, memory element, etc. after the migration of the page of memory is completed.

[0103] In some embodiments, in preparation for and / or when entering the transition state 702, the software entities and / or page migration engines update the nested page tables (i.e., page tables used by the cores 116-118 but not by the IO devices) to identify the page of memory as not present. For example, in some embodiments, the software entities may modify the validity information in each nested page table entry (e.g., validity 500) and / or other information in the nested page table entry to identify the page of memory as missing. With the nested page tables configured in this manner, the cores 116-118 (and / or other software entities, functional blocks, etc. that use the nested page tables) consider the page to be "missing," i.e., not yet loaded from the mass storage device 106 or otherwise not present in memory, and do not perform a memory access for the page of memory. Instead, the cores 116-118 use well-known page fault mechanisms to handle missing pages of memory. In these embodiments, the eventual "loading" of the page of memory in the page fault mechanism is actually a relocation of the page of memory being migrated, and thus in some embodiments, the resolution of such page faults is stalled until the page of memory is migrated.

[0104] From transition state 702, the functional blocks, devices, and software entities progress to transition state 704 (as indicated by the bold arrowed line between transition states 702 and 704). In other words, after performing the operations of transition state 702, the page migration engine can transition to transition state 704. Because transition state 704 uses the same transition state information as transition state 702, when entering transition state 704, the page migration engine leaves the transition state information in the page table entry for the page of memory set to the same bit sequence, value, etc. In FIG. 7, the page table entry transition state information is shown as PTE MS:2 in the callout for transition state 704. Additionally, in transition state 704, the RMT transition state information in the RMT entries for each of the source page of memory and the destination page of memory is unchanged from transition state 702. In Figure 7, the RMT migration state information in the RMT entries for the source page of memory and the destination page of memory are shown in the callout to migration state 704 as RMT[S]MS:0 and RMT[D]MS:0, respectively. In addition, the lock indicator in the RMT entry for the destination page of memory has not changed from migration state 702 to migration state 704. In Figure 7, the lock indicator in the RMT entry for the destination page of memory is shown in the callout to migration state 704 as RMT[D]LK:0. This indicates that the RMT entry for the destination page remains unlocked and therefore may still be accessed normally.

[0105] In the migrating state 704, in preparation for migrating a page of memory, the page migration engine locks the RMT entry of the source page of memory. Locking the RMT entry of the source page of memory prevents accesses (of accesses that perform an RMT check) of the RMT entry itself and / or the source page of memory. In some embodiments, when locking the RMT entry of the source page of memory, the page migration engine performs a compare and swap (CAS) and updates the lock indicator from 0 (cleared / unlocked) to 1 (set / locked). In FIG. 7, the lock indicator in the RMT entry of the source page of memory is shown in the callout to the migrating state 704 as RMT[S]LK:1.

[0106] In some embodiments, in the migration state 704, i.e., when RMT[S]LK:1 and RMT[S]MS:0, the IOMMU stalls all memory access requests to the source page of memory because the RMT entry is indicated as locked and there is no indication yet in the RMT entry that the page of memory is being migrated. In other words, when the combination of PTE MS:2, RMT[S]LK:1, and RMT[S]MS:0 is encountered during the page table walk and reverse map table check of a memory access request to a source page of memory in the migration state 704, the IOMMU stalls all newly encountered memory accesses of the source page of memory. In addition, when a particular accessing entity (e.g., cores 116-118) finds a locked RMT entry for the source page of memory during an RMT check prior to accessing the source page of memory, it faults (e.g., page faults, etc.) or otherwise handles the locked RMT entry and does not access the source page of memory.

[0107] In the migrating state 704, the IOMMU continues to prevent caching (i.e., storing) of a new copy of information from the page table entry for the page of memory in a local cache in the IOMMU and / or IO device because the migration state information in the page table entry and the RMT entry for the source page identifies the page of memory as being in the migrating state 704. That is, similar to the migrating state 702, the IOMMU does not allow a copy of information from the page table entry to be cached as described above.

[0108] From transition state 704, the functional blocks, devices, and software entities progress to transition state 706 (as indicated by the thick arrowed line between transition states 704 and 706). Because there is no change in the transition state information in the page table entry between transition states 704 and 706, the transition state information in the page table entry is not changed. In FIG. 7, the transition state information in the page table entry is shown in the callout of transition state 706 as PTE MS:2. In transition state 706, the lock indicator in the RMT entry of the source page is also not changed from transition state 704, which is shown in the callout of transition state 706 in FIG. 7 as RMT[S]LK:1. Additionally, the lock indicator and the RMT transition state information in the RMT entry of the destination page are not changed, which are shown in FIG. 7 as RMT[D]MS:0 and RMT[D]LK:0 in the callout of transition state 706.

[0109] In the migrating state 706, the page migration engine sets the RMT migration state information in the RMT entry of the source page to identify the source page as being in the migrating state 706. For example, in some embodiments, the page migration engine may perform a compare and swap (CAS) to update the RMT migration state information in the RMT entry of the source page of memory from 0 to 1. In Figure 7, the RMT migration state information in the RMT entry of the source page of memory is shown in the callout for the migrating state 706 as RMT[S]MS:1.

[0110] Because the migration state information in the page table entry and the RMT entry for the source page identifies the page of memory as being in the migrating state 706, the IOMMU stalls all newly encountered memory accesses of the first type until the migration of the page of memory is complete (e.g., back to the migrating state 700). For example, in some embodiments, the first type memory accesses include memory accesses such as reads, compares, and swaps that cause a response (e.g., returned data, etc.) from the memory to the requesting IO device, as opposed to second type memory accesses such as writes, invalidations, etc., that do not cause a response. The IOMMU may hold all newly encountered first type memory accesses in a buffer, memory element, etc., and then release the held memory accesses after the migration of the page of memory is complete. In other words, when the combination of PTE MS:2, RMT[S]LK:1, and RMT[S]MS:1 is encountered during the page table walk and reverse map table check of a memory access request for a page of memory, the IOMMU determines that the page of memory is in a transition state 706 and stalls all memory accesses of the first type to the page of memory. However, in the transition state 706, the IOMMU allows memory accesses other than the newly encountered memory access of the first type to proceed. For example, the IOMMU may allow an existing memory access of the first type to proceed. In addition, the IOMMU may allow both an existing memory access of the second type and the newly encountered memory access to proceed. This may include releasing the memory access that was stalled in the transition state 704, as described above. In some embodiments, newly encountered first type memory accesses are stalled separately from newly encountered second type memory accesses (which are stalled in transition states 710-712, as described below) to avoid contentions, deadlock conditions, and / or other problems that may arise between the first type memory accesses and the second type memory accesses when the memory accesses complete in preparation for migrating a page of memory.

[0111] In the migrating state 706, the IOMMU continues to prevent caching (i.e., storing) of a new copy of information from the page table entry for the page of memory in local caches within the IOMMU and / or IO device because the migration state information in the page table entry and the RMT entry for the source page identifies the page of memory as being in the migrating state 706. That is, similar to the migrating states 702-704, the IOMMU does not allow copies of information from the page table entry to be cached as described above.

[0112] From transition state 706, the functional blocks, devices, and software entities progress to transition state 708 (as indicated by the bold arrowed line between transition states 706 and 708). Because there is no change in the transition state information in the page table entry between transition states 706 and 708, the transition state information in the page table entry is not changed. In Figure 7, the transition state information in the page table entry is shown in the callout for transition state 708 as PTE MS:2.

[0113] In the Migrate state 708, the page migration engine prepares the RMT entry of the destination page of memory for migration of the page of memory. For this operation, the page migration engine first locks the RMT entry of the destination page of memory to prevent access of the RMT entry itself and / or the destination page of memory. For example, in some embodiments, the page migration engine may perform a compare and swap (CAS) and update a lock indicator in the RMT entry of the destination page of memory from 0 (cleared / unlocked) to 1 (set / locked). In FIG. 7, the lock indicator in the RMT entry of the destination page of memory is shown in the callout to the Migrate state 708 as RMT[D]LK:1.

[0114] After locking the RMT entry of the destination page of memory, the page migration engine partially overwrites the RMT entry of the destination page of memory with a copy of information from the RMT entry of the source page of memory. For example, in some embodiments, the page migration engine saves a copy of information from the RMT entry of the source page of memory that is returned in response to the CAS operation described above (i.e., to set the locked indicator and the RMT migration state information to the migrated state 706). The page migration engine then overwrites the RMT entry of the destination page of memory with the saved copy of information from the RMT entry of the source page of memory. For example, in some embodiments, the page migration engine copies information such as the guest ID (e.g., guest ID 604), guest physical address (e.g., guest physical address 606), page information (e.g., page information 608), etc., from the RMT entry of the source page of memory to the RMT entry of the destination page of memory. The page migration engine then removes (e.g., deletes, overwrites, sets to default values, etc.) the information from the RMT entry of the source page. The page migration engine transfers information from the RMT entry of the source page to the RMT entry of the destination page by overwriting the RMT entry with information from the RMT entry of the destination page and deleting information from the RMT entry of the source page. The transfer switches ownership, access control and other records from the source page of memory to the destination page of memory and removes ownership, access control and other records from the source page of memory (so that no two pages of memory are impermissibly referenced by the source RMT entry and the destination RMT entry).

[0115] During the above-described overwrite operation on the RMT entry of the destination page, the page migration engine does not copy the RMT migration state information from the RMT entry of the source page of memory to the RMT entry of the destination page of memory. Instead, the page migration engine sets the RMT migration state information in the RMT entry for the destination page of memory to identify the destination page of memory as being in the migrated state 708. For example, in some embodiments, the page migration engine may set the RMT migration state information in the RMT entry of the destination page of memory to 2. In FIG. 7, the RMT migration state information in the RMT entry of the destination page of memory is shown in the callout to the migrated state 708 as RMT[D]MS:2. When set to 2, the RMT migration state information, along with the lock indicator being set, causes the IOMMU to process memory accesses from the IO device as described above.

[0116] In the migration state 708, the page migration engine invalidates any existing copies of information from the page table entry in the local cache in the IOMMU and in the TLB in the cores 116-118 (recall that the existing copies of information from the page table entry in the local cache in the IO device were previously invalidated in migration state 702). For example, the page migration engine may issue an invalidation request to the IOMMU causing the IOMMU to invalidate any existing copies of information from the page table entry in the local cache. In addition, the page migration engine similarly invalidates any existing copies of information from the RMT entries from the source page of memory and the destination page of memory in the local cache in the IOMMU and in the TLB in the cores 116-118. In the migration state 708, the IOMMU continues to prevent the IOMMU and / or IO device from storing copies of information from the page table entry of the page of memory in their local caches, as described above for migration states 702-706.

[0117] From transition state 708, the functional blocks, devices, and software entities progress to transition state 710 (as indicated by the bold arrowed line between transition states 708 and 710). Because there is no change in the transition state information in the page table entry between transition states 708 and 710, the transition state information in the page table entry is not changed. In Figure 7, the transition state information for the page table entry is shown in the callout for transition state 710 as PTE MS:2.

[0118] In the migrating state 710, the page migration engine finishes preparing the RMT entry of the source page of memory for migration of the page of memory. For this operation, the page migration engine sets the RMT migration state information in the RMT entry of the source page to identify the source page as being in the migrating state 710. For example, in some embodiments, the page migration engine may set the RMT migration state information in the RMT entry of the source page of memory to 2. In FIG. 7, the RMT migration state information in the RMT entry of the source page of memory is shown in the callout to the migrating state 710 as RMT[S]MS:2. When set to 2, the RMT migration state information causes software entities and functional blocks, such as cores 116-118, to prevent access to the source page of memory.

[0119] Because the migration state information in the page table entry and the RMT entry for the source page identifies the page of memory as being in the migration state 710, the IOMMU stalls newly encountered memory accesses of both the first type and the second type until the migration of the page of memory is completed (e.g., back to the migration state 700). For example, the IOMMU may hold all newly encountered memory accesses of the first type and the second type in a buffer, memory element, etc., and then release the held memory accesses after the migration of the page of memory is completed. In other words, upon encountering the combination of PTE MS:2, RMT[S]LK:1, and RMT[S]MS:2 during the page table walk and reverse map table check of a memory access request for a page of memory, the IOMMU determines that the page of memory is in the migration state 710 and stalls both the first type and the second type memory accesses to the page of memory. However, in the migration state 710, the IOMMU allows memory accesses other than the newly encountered memory accesses to proceed. For example, the IOMMU may allow existing memory accesses of the first and second types to proceed. In the transition state 710, the IOMMU continues to prevent the IOMMU and / or IO devices from storing copies of information from the page table entries of the page of memory in their local caches, as described above for transition states 702-708.

[0120] From transition state 710, the functional blocks, devices, and software entities progress to transition state 712 (as indicated by the thick arrowed line between transition states 710 and 712). When entering transition state 712, the page migration engine sets migration state information in the page table entry of the page of memory to a bit sequence, value, etc., that represents or identifies the transition state 712. For example, assuming the migration state information includes two bits, the migration state information may be set to 11 to represent or identify the transition state 712. In transition state 712, the page migration engine leaves the RMT migration state information and the lock indicator in the RMT entries of the source page of memory and the destination page of memory unchanged, since these values ​​are the same between transition states 710 and 712. Thus, in FIG. 7 , the migration state information in the page table entry is shown as PTE MS:3, and the RMT migration state information and lock indicators in the RMT entries for the source page of memory and the destination page of memory are shown in the migration state 712 callout as RMT[S]LK:1, RMT[S]MS:2, RMT[D]LK:1 and RMT[D]MS:2.

[0121] In the migrating state 712, the memory is prepared to migrate the page of memory and then the page of memory is migrated. For these operations, the IOMMU continues to stall both the first and second type of memory accesses to the page of memory because the page table entry and RMT migration state information in the RMT entry of the source page of memory and the destination page of memory indicate that the page of memory is in the migrating state 712. The IOMMU also continues to prevent the IOMMU and / or IO devices from storing copies of information from the page table entry of the page of memory in their local caches, as described above for the migrating states 702-710. Additionally, the IOMMU completes all existing / outstanding memory accesses to the page of memory. In other words, all remaining memory accesses that were in progress before entering the migrating state 712 are completed. All existing / outstanding memory accesses are completed, so that memory accesses of the page of memory do not access the page of memory during the migration (i.e., because stale / incorrect data may be accessed, etc.).

[0122] Preparation for migrating the page of memory is completed after existing / outstanding memory accesses are completed and therefore all remaining accesses to the page of memory are stalled in preparation for migrating the page of memory. Thus, the page migration engine migrates the page of memory in the memory. For example, the page migration engine may move the page of memory from a first / source location / physical address in the memory to a second / destination location / physical address in the memory (e.g., a location specified in the request to migrate the page of memory). In some embodiments, the move in the memory is from a first type of memory (e.g., a memory of a slower access type) to a second type of memory (e.g., a memory of a faster access type) or vice versa. Although the migration state 712 is shown as a single migration state, in another example, the migration state may be divided into a migration state 712A, where existing / outstanding memory accesses are completed, and a migration state 712B, where the page of memory is migrated / moved in the memory and cleanup operations (as described below) are performed in preparation for returning the page of memory to the migration state 700.

[0123] After migrating the page of memory, the page migration engine updates (or causes another functional block or software entity to update) the page table entry to identify the physical address at which the page of memory is stored (i.e., the physical address of the second location in memory). In other words, the page migration engine updates the address translation information for the page of memory such that guest physical addresses used by IO devices in the electronic device can be correctly mapped to the physical addresses at which the page of memory is stored. The page migration engine also sets (if necessary) any associated metadata in the page table entry based on the second location in memory.

[0124] With the page of memory successfully migrated, the functional blocks, devices, and software entities return to the migrated state 700 (as indicated by the thick arrowed line between migrated state 712 and 700). As part of returning to the migrated state 700, the page migration engine sets the migration state information in the page table entry of the page of memory to a bit sequence, value, etc., that represents or identifies the migrated state 700. For example, assuming the migration state information includes two bits, the migration state information may be set to 00 to represent or identify the migrated state 700. In FIG. 7, the migration information in the page table entry is shown in the callout of the migrated state 700 as a PTE. The page migration engine also sets the RMT migration state information and lock indicators in the RMT entries of the source page of memory and the destination page of memory to represent or identify the migrated state 700. 7, the RMT migration state information and lock indicators in the RMT entries of the source page of memory and the destination page of memory are shown in the callouts of the migration state 700 as RMT[S]MS:0, RMT[S]:0, RMT[D]MS:0, and RMT[D]LK:0. Recall that the RMT entry of the destination page has the appropriate access and security information for the migrated page of memory because the other information in the RMT entry of the source page has been transferred to the RMT entry of the destination page. In addition, the RMT entry of the source page is cleared of information for the moved page of memory.

[0125] As described above, migrating state 700 is a "non-migratory" migration state. Because the migration state information in the page table entry and the RMT entry for the destination page of memory identifies the page of memory as being in migrating state 700, the IOMMU allows accesses of the page of memory and the information in the page table entry, respectively, from the IO device to proceed normally. That is, because no preparations have been made to migrate the page of memory, and the page of memory has not been migrated, accesses of the information in the page of memory and the page table entry are not prevented, stalled, or otherwise restricted by the IOMMU.

[0126] In some embodiments, when returning to the migration state 700 (i.e., after updating the page table entry and the RMT entry), the page migration engine signals to the software entity that the migration is complete. The software entity then resumes normal operation of the page of memory. For example, the software entity may update the nested page table to identify the page of memory as present. For example, in some embodiments, the software entity may change the validity information in the nested page table entry (e.g., validity 500 to identify the page of memory as present) and / or other information in the nested page table entry. The software entity also updates the page table entry in the nested page table to identify the system physical address where the page of memory is stored. The software entity may then release the page fault mechanism to allow processing of any page faults that occur based on the page of memory marked as missing in the nested page table.

[0127] Upon returning to the transition state 700, the IOMMU releases the stalled memory accesses to the page of memory that were previously stalled during the previous transition state. For example, in some embodiments, the stalled memory accesses are held by the IOMMU while the IOMMU polls / periodically reads the page table entry. When the IOMMU finds that the page table entry identifies the page of memory as being in the transition state 700, the IOMMU begins to retrieve the stalled memory accesses in first-in-first-out order from the buffer or memory element in which they were stored and executes each of the memory accesses. In some embodiments, the IOMMU executes each of the stalled memory accesses before other newly encountered memory accesses of the page of memory (the newly encountered memory accesses may themselves be held and then subsequently released in the order in which they were received or in another order). In addition, the IOMMU resumes caching information from the page table entries in its local cache. In other words, the IOMMU may itself cache a copy of information from the page table entries and provide a copy of information from the page table entries to the requesting IO device. The page migration engine also returns a response to the requesting software entity indicating that the page of memory has been successfully migrated, thereby completing the migration of the page of memory.

[0128] The example of FIG. 7 assumes that the migration attempt is successful and therefore all operations to prepare the page to be migrated and then migrate the page are successful. However, if any of the operations fail or an undesirable or unexpected condition or error is encountered, the migration of the page may be terminated. Upon termination of the migration, the functional blocks, devices, and software entities return from the current migration state (which may be any of the migration states 702-712) to the migration state 700. This is illustrated by the corresponding thin arrowed lines between the migration states 702-710 and the migration state 700, and the thick arrowed line between the migration state 712 and the migration state 700. Upon termination of the migration of the page of memory, the software entity, the page migration engine, or another functional block or device restores the migration state information in the page table entry of the page of memory and the RMT entries of the source page of memory and the destination page of memory to a bit sequence, value, etc., that represents or identifies the migration state 700. For example, assuming that the migration state information includes two bits, the migration state information may be set to 00 to represent or identify the migration state 700, etc. In the event of returning to the migration state 700 due to a failure to migrate a page of memory, the IOMMU releases any stalled memory accesses to the page of memory, resumes caching copies of information from the page table entry in local caches in the IOMMU and IO devices, and more generally removes the effects of preparations to migrate the page of memory. In addition, the page migration engine (and / or another functional block or device) communicates the failure of the migration to a software entity. The software entity can then execute an error handling routine (or ignore the failure if the migration was voluntary).

[0129] In some embodiments, despite the restrictions on memory accesses of the migrating pages of memory and the caching of information from the respective page table entries in the local cache, accesses to non-migrating pages of memory and their respective page table entries continue normally. That is, because no provision has been made to migrate the non-migrating pages of memory and the non-migrating pages of memory are not migrated, accesses to the non-migrating pages of memory and the information in the respective page table entries are not prevented, stalled, or otherwise restricted by the IOMMU. In other words, the IOMMU uses the information from such page table entries normally and performs memory accesses of such pages of memory normally. In some embodiments, the migration state information and RMT entry of the non-migrating pages are set to indicate that the non-migrating pages are in the migratory state 700. However, certain memory accesses of the non-migrating pages of memory may be held up for ordering reasons and / or other reasons based on the migration of the pages of memory, but generally may proceed as long as the non-migrating pages of memory and their respective page table entries remain correct in memory.

[0130] In some embodiments, functional blocks, devices, and software entities may transition between transition states in a different order. For example, in some embodiments, transition states may be skipped. An example of a transition state skip is illustrated in FIG. 7 by the “Transition State Skip” label from transition state 700 (each of the possible skips between transition states is not labeled for clarity). Note that such skips may include skips from any suitable transition state to any other acceptable transition state, so long as the transition state information and the RMT entries of the source and / or destination pages of memory are correctly updated. For example, in some embodiments, a page of memory that is only accessible to the processor (and thus not accessible to IO devices) may be transitioned directly from transition state 700 to transition state 712. In these embodiments, the processor includes a page fault mechanism for handling missing pages of memory, and the software entities and / or page migration engine may simply mark the page of memory as missing in the respective page table entries and proceed with the migration of the page of memory. In some embodiments, the processor uses different page tables than the IOMMU, and thus the page of memory is marked as "missing" during the transition in the processor's page table, which causes the page of memory to be processed by the processor using the processor's page fault mechanism. However, in some embodiments, the processor uses the same page tables as the IOMMU, although the available indicators for the processor and the IOMMU are different. In these embodiments, the processor's available indicator may be set such that the page of memory is processed by the processor using the processor's page fault mechanism, but the page of memory remains "available" for the IOMMU to process (such as via stalling memory accesses) as described herein.

[0131] While in the example shown in FIG. 7 only a single page of memory is migrated, in some embodiments preparation for migrating and / or migrating multiple pages of memory may occur in parallel. In other words, using the migration state information described above and the information in the RMT of each page of memory, preparation for migrating and / or migration of two or more pages of memory may occur at the same or similar times, contemporaneously, etc. It should be noted that the individual pages of memory in this case are in different migration states, and thus migration of a page of memory may be dynamically initiated (e.g., when a software entity determines that the respective page of memory is to be migrated) and completed in parallel with preparation for migrating and / or migration of other pages of memory. Thus, each page of memory is individually identifiable via its page table entry and respective RMT entry as being in one of the migration states or in a non-migration state, and operations for handling access of the page of memory and / or respective page table entries are performed as described herein.

[0132] (The process that handles migration of pages of memory) In the described embodiments, software entities (e.g., hypervisor 318, operating system 200, etc.) and a page migration engine (e.g., page migration engine 114) and an IOMMU (e.g., IOMMU 140) perform operations to control memory access of a page of memory and associated page table entries while the page of memory is being prepared for migrating and while the page of memory is being migrated within a memory (e.g., memory 104). FIG. 8 illustrates a flow diagram that illustrates a process for migrating a page of memory according to some embodiments. FIG. 8 is presented as a general example of operations performed by a requestor in some embodiments. However, in some embodiments, a requestor performs different operations and / or performs operations in a different order.

[0133] As can be seen from Figure 8, the process begins when the software entities and page migration engine, while performing operations to prepare to migrate an IO device accessible page of memory in memory, set migration state information in a page table entry of the page of memory and information in a reverse map table (RMT) entry associated with the migration of the page of memory based on the operations being performed (step 800). Generally, for this operation, when the software entities and page migration engine perform their respective operations to prepare to migrate a page of memory, the software entities and page migration engine update the migration information in the page table entry of the page of memory and the information in the RMT entry involved in the migration of the page of memory to indicate the latest / current migration state. Recall that the RMT entry involved in migrating a page of memory includes an RMT entry for a source page of memory (i.e., the page of memory itself) at a first location and an RMT entry for a destination page of memory at a second location to which the page of memory is to be migrated. In some embodiments, setting information in an RMT entry involved in the migration of a page of memory includes setting a lock indicator and / or RMT migration state information based on the operation being performed (note that some or all of these may not need to change between migration states, as shown above with respect to FIG. 7).

[0134] As discussed above, in migration states 702-712, the software entity and / or page migration engine performs operations to prepare to migrate a page of memory, and then in migration state 712 (which, as discussed above, may be divided into migration state 712A, in which a second type of existing / outstanding memory access is completed, and migration state 712B, in which the page of memory is migrated / moved in memory and cleanup operations are performed), the page migration engine performs operations to migrate the page of memory. Thus, setting the migration state information in the page table entry for the page of memory and the information in the RMT entry involved in the movement of the page of memory based on the operation being performed involves setting the migration state information and the information in the RMT entry involved in the movement of the page of memory to identify the migration state of the page of memory. For example, after receiving a request to migrate a page of memory, while (or before) performing initial operations to prepare to migrate the page of memory, the software entity may set the migration state information to migration state 702, such as by setting a bit sequence or value indicative of migration state 702. However, in the migrating state 702, the information in the RMT entries involved in migrating pages of memory remains unchanged from the migrating state 700 (see, for example, the callout for the migrating state 702 in FIG. 7).

[0135] In some embodiments, the particular operations performed by the software entities and IO devices depend on the current migration state of the page of memory. For example, in some embodiments, among the operations performed by the software entities are operations to invalidate (via corresponding commands to the IOMMU) any existing copies of information from page table entries in caches in at least one IO device. In some embodiments, the software entities perform this operation in the migration state 702. As another example, in some embodiments, among the operations performed by the page migration engine are operations to invalidate any existing copies of information from page table entries in caches in the IOMMU. In some embodiments, the page migration engine performs this operation in the migration state 708. As yet another example, in some embodiments, among the operations performed by the page migration engine are operations to complete any memory accesses to the page of memory that are not stalled by the IOMMU before migrating the page of memory. In some embodiments, the page migration engine performs these operations in the migration states 702-712. The sequence / order in which these operations may be performed by the software entities and the page migration engine is described above with respect to FIG. 7.

[0136] The IOMMU controls the use of information from the page table entry based on the migration state information in the page table entry and the information in the RMT entry involved in migrating the page of memory (step 802). For this operation, when the IOMMU (or rather the IOMMU table walker) performs a table walk, it obtains the migration state information and the information from the RMT entry of the source page of memory and checks the current values ​​of the migration state information and the information from the RMT entry of the source page of memory (e.g., the lock indicator and the RMT migration state information). If the migration state information and the information in the RMT entry for the source page of memory indicate that the page of memory is being prepared to be migrated and / or that the page of memory is being migrated, the IOMMU restricts the use of the information from the page table. For example, in migration states 702-712, the IOMMU may prevent caching copies of information from the page table entry in a cache in the IOMMU itself or in the IO device. In contrast, if the migration state information and the information in the RMT entry of the source page of memory, such as in migration state 700, indicate that the page is not being migrated, the IOMMU may use the information from the page table as normal. Thus, in the non-migratory state, the IOMMU itself can cache a copy of the information from the page table entry, and can provide IO devices with a copy of the information from the page table entry (subject to any other existing access restrictions).

[0137] The IOMMU also controls execution of memory accesses to the page of memory based on the migration state information in the page table entry and the information in the RMT entry involved in the migration of the page of memory (step 804). For this operation, the IOMMU obtains the migration state information and the information in the RMT entry for the source page of memory, as described above, and checks the current value of the migration state information and the information in the RMT entry for the source page of memory. If the migration state information and the information in the RMT entry for the source page of memory indicate that the page of memory is being prepared for migration and / or that the page of memory is being migrated, the IOMMU can restrict memory accesses to the page of memory. For example, in migration states 702-712, the IOMMU can stall designated newly encountered memory accesses to the page of memory, but existing memory accesses are allowed to proceed to completion before the page of memory is migrated. In contrast, if the migration state information for the source page of memory and the information in the RMT entry for the source page of memory indicates that the page is in another migration state, such as migration state 700, then the IOMMU may allow memory accesses to the page of memory to proceed normally. Thus, in these migration states, the IOMMU performs memory accesses to the page of memory in the typical manner, subject to any other existing access restrictions.

[0138] While FIG. 8 presents an example of setting up and using the migration state information in the page table entry and the information in the RMT entry as a page of memory is "prepared" to be migrated, in some embodiments the migration state information in the page table entry and the information in the RMT entry generally reside in the page table entry and RMT, respectively, and are used to "control" access to the page of memory and the information in the corresponding page table entry at other times (i.e., other migration states). For example, for a page of memory where the "preparation" described in FIG. 8 is complete and the page of memory is actually being migrated (i.e., is in the process of being moved from one location in memory to another), the migration state information in the page table entry and the information in the RMT entry are used by the IOMMU to control access to the page of memory and the information in the respective page table entry. In some embodiments, migration of the page of memory is performed in the migration state 712, with the above-described control over access to the page of memory and the information in the respective page table entry. As another example, for pages of memory that have not been migrated, and therefore preparations have not been made to migrate the page of memory, the migration state information in the page table entry and the information in the RMT entry are used by the IOMMU to control access of the page of memory and the information in the respective page table entries. In some embodiments, operations on non-migrating pages of memory are performed in the migration state 700, with the above-described control (or lack thereof) over access of the page of memory and the information in the respective page table entries.

[0139] (Migration of processor-only pages of memory) In some embodiments, the pages of memory may be “processor-only” pages of memory, and thus pages of memory that are accessible only by the processors (e.g., cores 116-118) and not accessible to IO devices. Because such pages of memory are not accessible to IO devices, at least some of the operations described above for migrating these pages of memory are performed differently. In some embodiments, to migrate a processor-only page of memory, a software entity (e.g., an operating system, a hypervisor, etc.) first updates the page table entry (e.g., in a nested page table) for the page of memory to identify that the page is not present. For example, the software entity may set or clear respective validity information, etc., in the page table entry. The software entity then instructs the page migration engine to initiate operations to migrate the page of memory.

[0140] Upon receiving instructions from the software entity, the page migration engine sets a lock indicator in the RMT entry for the source page of memory (i.e., the location in memory where the page of memory is stored before migration) and the destination page of memory (i.e., the location in memory where the page of memory is stored after migration) to identify each RMT entry as locked. Recall that other software entities and functional blocks should not further access a given RMT entry and / or the associated page of memory upon finding the lock indicator set. The page migration engine also invalidates any cached copies of the RMT entry for the source page of memory and / or the RMT entry for the destination page of memory in a cache (e.g., TLB) within the processor to ensure that the processor does not erroneously use stale copies of these RMT entries. Because copies of the RMT entry for the source page of memory and / or the RMT entry for the destination page of memory are not present in the cache within the processor, the processor must check each RMT entry itself during any memory access operation during the migration, and will therefore find a current version of each RMT entry with the lock indicator set.

[0141] After locking the RMT entries of the source page of memory and the destination page of memory and clearing the copy of the RMT entries from the cache, the page migration engine migrates the page of memory. In other words, the page migration engine moves data from the source page of memory to the destination page of memory. After moving the page of memory, the page migration engine transfers information from the RMT entry of the source page of memory to the RMT entry of the destination page of memory and clears the RMT entry of the source page of memory. By transferring information as described, the page migration engine "migrates" the control, access, and security information of the page of memory to the RMT entry of the destination page of memory and removes the information of the page of memory from the RMT entry of the source page of memory (to avoid impermissible duplication of information). The page migration engine also unlocks the RMT entry of the destination page of memory by clearing the lock indicator in the RMT entry of the destination page of memory.

[0142] After the above operations are completed, the page of memory has been successfully migrated. Thus, the page migration engine informs the software entity that the page of memory has been migrated, i.e., the migration is complete. The software entity updates the page table entry for the page of memory to reflect the migration, e.g., by setting the new physical address of the page of memory associated with the destination page of memory. The page migration engine also sets the page table entry to identify the page of memory as existing.

[0143] Although certain operations for migrating a processor-only page of memory are described above, in some embodiments, different operations are performed and / or the operations are performed in a different order. For example, in some embodiments, the page migration engine (not a software entity) makes the updates to the page table entry. In these embodiments, the software entity (or another software entity or functional block) may simply request the page migration engine to migrate the page of memory, and the page migration engine may perform the operations to migrate the page of memory. As another example, in some embodiments, the page table entry itself is not marked as not present (and the page table entry may remain unchanged during the migration). In these embodiments, setting the RMT entry as locked is sufficient to prevent undesired processor access of the page of memory during migration.

[0144] In some embodiments, at least one electronic device (such as, for example, electronic device 100) uses code and / or data stored in a non-transitory computer-readable storage medium to perform some or all of the operations described herein. More specifically, at least one electronic device reads the code and / or data from the computer-readable storage medium and executes the code and / or uses the data when performing the described operations. The computer-readable storage medium may be any device, medium, or combination thereof that stores code and / or data used by the electronic device. For example, the computer-readable storage medium may include, but is not limited to, volatile and / or non-volatile memory, including flash memory, random access memory (e.g., eDRAM, RAM, SRAM, DRAM, DDR4 SDRAM, etc.), non-volatile RAM (e.g., phase-change memory, ferroelectric random access memory, spin-transfer random access memory, magnetoresistive random access memory, etc.), read-only memory (ROM), and / or magnetic or optical storage media (e.g., disk drives, magnetic tapes, CDs, DVDs, etc.).

[0145] In some embodiments, one or more hardware modules perform the operations described herein. For example, hardware modules may include, but are not limited to, one or more central processing units (CPUs) / CPU cores, graphic processing units (GPUs) / GPU cores, application specific integrated circuits (ASICs) chips, field programmable gate arrays (FPGAs), compressors or encoders, computational units, embedded processors, accelerated processing units (APUs), controllers, requesters, completers, network communication links, and / or other functional blocks. When circuitry (e.g., integrated circuit elements, discrete circuit elements, etc.) within such hardware modules is activated, the circuitry performs some or all of the operations. In some embodiments, hardware modules include general purpose circuits, such as execution pipelines, computational or processing units, that perform operations upon execution of instructions (e.g., program code, firmware, etc.). In some embodiments, hardware modules include application specific or dedicated circuitry that performs operations, and in some cases, circuitry that performs some or all of the operations "in hardware" without executing instructions.

[0146] In some embodiments, data structures representing some or all of the functional blocks and circuit elements described herein (e.g., electronic device 100 or some portion thereof) are stored in a non-transitory computer-readable storage medium including a database or other data structure that can be read by the electronic device and used directly or indirectly to fabricate hardware including the functional blocks and circuit elements. For example, the data structure can be a behavioral level description or a register-transfer level (RTL) description of the hardware functionality in a high-level design language (HDL) such as Verilog or VHDL. The description can be read by a synthesis tool that can synthesize the description to generate a netlist including a list of transistors / circuit elements from a synthesis library that represents the functionality of the hardware including the functional blocks and circuit elements described above. The netlist can then be placed and routed to generate a data set that describes the geometric shapes to be applied to a mask. The mask can then be used in various semiconductor processing steps to fabricate a semiconductor circuit or circuits (e.g., integrated circuits) corresponding to the functional blocks and circuit elements described above. Alternatively, the database on the computer accessible storage medium may be a netlist (with or without a synthesis library) or a data set, or Graphic Data System (GDS) II data, as desired.

[0147] In this description, variables or unspecified values ​​(i.e., general descriptions of values ​​without specific examples of values) are represented by letters such as N, M, X, etc. As used herein, despite the possible use of similar letters in different places in this description, the variables and unspecified values ​​in each instance are not necessarily the same, i.e., there may be variables and values ​​intended for some or all of the general variables and unspecified values. In other words, the examples of specific values ​​of N, as well as any other letters used to represent variables and unspecified values ​​in this description, are not necessarily related to each other.

[0148] As used herein, the phrase "et al." or "etc." is intended to indicate one and / or instance, i.e., the equivalent of "at least one of" the elements in the list with which the others are associated. For example, in a statement that "the electronic device performs a first operation, a second operation, etc.", the electronic device performs at least one of the first operation, the second operation, and the other operation. In addition, the elements in the list with which the others are associated are merely examples from a set of examples, and at least some of the examples may not appear in some embodiments.

[0149] The foregoing descriptions of the embodiments have been presented for purposes of illustration and description only. They are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed. Accordingly, many modifications and variations will be apparent to those skilled in the art. Further, the above disclosure is not intended to limit the embodiments. The scope of the embodiments is defined by the appended claims.

Claims

1. 1. An electronic device comprising: Memory, an input / output memory management unit (IOMMU); a processor for executing the software entities; a page transition engine; the software entity and the page migration engine are configured to perform operations to prepare a page of memory accessible by at least one IO device for migration within the memory; the software entity and the page migration engine set migration state information in a page table entry for the page of memory and information in a reverse map table (RMT) entry involved in migrating the page of memory based on the operation being performed; the IOMMU is configured to control use of information in the page table entries and to control performance of memory accesses of the pages of memory based on the migration state information in the page table entries and information in the RMT entries. Electronic devices.

2. The RMT entry involved in migrating a page of memory includes: a source location within said memory where a page of said memory is to be placed; an RMT entry for a destination location in said memory to which said page of memory is to be migrated; The electronic device of claim 1.

3. When setting information in the RMT entry, the page transition engine: configured to set RMT transition state information and a lock indicator for each of the RMT entries based on the operation being performed. The electronic device of claim 1.

4. The software entity and the page migration engine, when setting the migration state information in the page table entry and the information in the RMT entry, the software entity and the page migration engine are configured to set the migration state information and / or information in the RMT entry to a predetermined value; The predetermined value, when read by the IOMMU, by not storing a copy of the information of the page table entry in a cache in the IOMMU, and by not providing a copy of the information of the page table entry to the at least one IO device to be stored in a cache in the at least one IO device; having the IOMMU control the use of information in the page table entries; The electronic device of claim 1.

5. The software entity and the page migration engine, when performing the operations to prepare to migrate a page of the memory, the software entity invalidating existing copies of the page table entry's information in a cache in the at least one IO device; the page migration engine is configured to invalidate existing copies of the page table entry's information in a cache in the IOMMU; The electronic device according to any one of claims 1 to 4.

6. When the page migration engine sets the migration state information in the page table entry and the information in the RMT entry, configured to set the migration state information and information in the RMT entry to a first value; The first value, when read by the IOMMU, by stalling newly encountered memory accesses of a first type to the page of memory but allowing existing memory accesses of the first type and a second type to the page of memory and newly encountered memory accesses of the second type to proceed; having the IOMMU control execution of memory accesses of the pages of memory; The electronic device of claim 1.

7. When the page migration engine sets the migration state information in the page table entry and the information in the RMT entry, configured to set the migration state information and information in the RMT entry to a second value; The second value, when read by the IOMMU, by stalling newly encountered memory accesses of both the first type and the second type to the page of memory, but allowing existing memory accesses of the first type and the second type to the page of memory to complete. having the IOMMU control execution of memory accesses of the pages of memory; The electronic device of claim 6.

8. When the page migration engine performs the operations to prepare to migrate a page of the memory, configured to ensure that the first type and the second type of memory accesses to non-stalled pages of the memory are completed before migrating the pages of the memory. An electronic device according to any one of claims 1 to 4 and 7.

9. the page migration engine configured to perform operations to migrate pages of the memory; The operation includes: moving a page of the memory from a first location to a second location within the memory; After said movement is completed, updating information in the page table entry such that the page of memory is identified as being stored at the second location within the memory; and setting the migration state information and information in the RMT entry to a predetermined value which, when read by the IOMMU, causes the IOMMU to resume normally performing memory accesses of the pages of memory using information in the page table entry normally. The electronic device of claim 1 .

10. the RMT entries include an RMT entry for a source location in the memory where the page of memory is to be placed, and an RMT entry for a destination location in the memory where the page of memory is to be migrated; When the page transition engine sets the information in the RMT entry, overwriting the specified information in the RMT entry for the destination location with a copy of the specified information obtained from the RMT entry for the source location; and configured to remove the specified information from an RMT entry relating to the source location. An electronic device according to any one of claims 1 to 4, 6, 7 and 9.

11. 1. A method for migrating pages of memory in a memory in an electronic device, comprising: When performing an operation to prepare a page of memory accessible by at least one IO device for migrating within the memory, setting migration state information in a page table entry for the page of memory and information in a reverse map table (RMT) entry involved in migrating the page of memory based on the operation being performed; and controlling use of the information in the page table entry and performance of memory accesses of the page of memory based on the migration state information in the page table entry and information in the RMT entry. method.

12. Setting migration state information in the page table entry and information in the RMT entry includes: setting the migration state information and information in the RMT entry to a first value; The first value, when read by the IOMMU, by stalling newly encountered memory accesses of a first type to the page of memory but allowing existing memory accesses of the first type and a second type to the page of memory and newly encountered memory accesses of the second type to proceed; having the IOMMU control execution of memory accesses of the pages of memory; 12. The method of claim 11.

13. Setting migration state information in the page table entry and information in the RMT entry includes: setting the migration state information and information in the RMT entry to a second value; The second value, when read by the IOMMU, by stalling newly encountered memory accesses of both the first type and the second type to the page of memory, but allowing existing memory accesses of the first type and the second type to the page of memory to complete. having the IOMMU control execution of memory accesses of the pages of memory; 13. The method of claim 12.

14. Preparing to migrate the page of memory includes: ensuring that the first type and the second type of memory accesses to non-stalled pages of the memory are completed before migrating the pages of the memory.

14. The method of claim 13.

15. performing an operation to migrate a page of the memory; The operation includes: moving a page of the memory from a first location to a second location within the memory; After said movement is completed, updating information in the page table entry such that the page of memory is identified as being stored at the second location within the memory; and setting the migration state information and information in the RMT entry to a predetermined value which, when read by the IOMMU, causes the IOMMU to resume normally performing memory accesses of the pages of memory using information in the page table entry normally. The method of any one of claims 11 to 14.

Images