I / O memory management unit access to guest operating system buffers and logs

Abstract

An electronic device comprising: a processor that executes a guest operating system; a memory having a guest portion that is reserved for storing data and information to be accessed by the guest operating system; and an input-output memory management unit (IOMMU). In the guest portion, the IOMMU writes information to a guest buffer and / or log that is used to transfer information from the IOMMU to the guest operating system. The IOMMU also reads information from the guest portion in a guest buffer and / or log that is used to transfer information from the guest operating system to the IOMMU.

Description

Background Art

[0001] Related technologies

[0002] Some electronic devices (e.g., servers or desktop computers, etc.) support "virtualization" of electronic device hardware such as input-output (I / O) devices. Virtualization involves an intermediate entity on or in an electronic device, thereby providing a software instance (e.g., an application, etc.) executing on the electronic device with the illusion that the software instance can directly access the electronic device hardware when in fact the intermediate entity intercepts / redirects or otherwise assists the access made by the software instance. For example, a common intermediate entity is a "virtual machine". A virtual machine is a software entity, which usually comes with platform hardware support, abstracts the electronic device hardware and emulates or presents known interfaces of the electronic device hardware, thereby enabling the software instance to execute on various types and arrangements of underlying electronic device hardware, which may include electronic device hardware that would otherwise be incompatible with the software instance. In some electronic devices, the virtual machine provides support for executing one or more operating system instances (called "guest" operating systems), each of which can view the (virtualized) resources of a specific machine, such as memory, central processing unit (CPU), etc. The guest operating system in turn provides an environment for executing other software instances such as productivity applications, databases, etc.

[0003] In some electronic devices, virtual machines are managed and controlled by a software entity called a hypervisor. The hypervisor can start or initialize virtual machines; control, monitor and assist virtual machines in accessing electronic device hardware; terminate or shut down virtual machines; provide specific virtualized resources at expected locations or using specific hardware programming models; arbitrate access to physical resources required by multiple virtual machines; direct external interface input and output (e.g., from a user or from a network to an appropriate virtual machine); and so on. FIG1 presents a block diagram showing virtual machines and hypervisors. As can be seen in FIG1 , there are three virtual machines (VMs) 100, under each of which a guest operating system (guest OS) 102 and one or more programs (PRGRMS) 104, such as a database, software application, etc., are executed. The virtual machines 100 communicate with a hypervisor 106, which is connected between a host operating system (host OS) 108 and the virtual machines 100. The host operating system 108 provides an interface between the electronic device hardware 110 and the hypervisor 106. Furthermore, the hypervisor 106 is coupled between the virtual machine 100 and an input-output management unit (IOMMU) 112 , where the IOMMU 112 functions as a memory management unit and controller for the IO device hardware 114 .

[0004] One of the operations performed by the hypervisor is handling communications between the electronic device hardware and the guest operating system (or more broadly, the virtual machine). For example, the hypervisor can translate, redirect, or otherwise assist in communications between the guest operating system and the input-output management unit (IOMMU), such as by providing memory addresses for direct memory accesses of the I / O device to the memory being used by the virtual machine. Communications handled by the hypervisor include communications such as writes by the IOMMU to the peripheral page request (PPR) log and event log, and writes by the guest operating system to the command buffer. The PPR log, event log, and command buffer writes are described in detail in the AMD I / O Virtualization Technology (IOMMU) Specification, Revision 3.00, December 2016, the entire contents of which are incorporated herein by reference.

[0005] FIG2 presents a block diagram showing typical communications between a guest operating system and an IOMMU handled by a hypervisor. In FIG2 , many elements are shown as dots or spots; these elements are logs, buffers, etc. stored in a memory (e.g., the main memory of an electronic device) and thus accessed via typical memory access techniques. The elements in FIG2 , together with the guest operating system 102, the hypervisor 106, and the IOMMU 112, include a guest peripheral page request (PPR) log 200, a guest command buffer (CMD BUF) 202, and a guest event log 204, which are structures (e.g., lists, tables, etc.) in memory for storing communications from and to the guest operating system 102. In addition, the elements include guest pointers (PTRS) / status registers (REGS) 206, which are a set of locations in memory for storing pointers to guest operating system structures and status information associated with the guest operating system. Elements also include an IOMMU peripheral page request (PPR) log 208, an IOMMU command buffer 210, and an IOMMU event log 212, which are structures (e.g., lists, tables, etc.) in memory for storing communications to and from the IOMMU 112. Elements also include IOMMU memory mapped input-output (MMIO) pointer / status registers (REGS) 214 in the IOMMU 112, which are a set of registers in the IOMMU 112 for storing pointers to various IOMMU 112 structures and status information associated with the IOMMU 112.

[0006] In operation, and using commands as an example, the guest operating system 102 writes commands destined for the IOMMU 112 into the guest command buffer 202 (i.e., writes to the next available location in the buffer in memory that stores commands from the guest operating system 102). The hypervisor 106 (as shown via the dashed line in FIG. 2 ) detects the guest operating system's write to the guest command buffer 202, retrieves and processes the command (e.g., replaces the guest domain ID and / or guest device ID in the command with the corresponding host domain ID and / or device ID, etc.), and stores the processed command in the IOMMU command buffer 210. The hypervisor 106 also updates the tail pointer in the IOMMU MMIO pointer / status register 214 for the IOMMU command buffer 210 to indicate the newly written command (e.g., increments the tail pointer to the next location in the IOMMU command buffer 210). The IOMMU 112 then uses the command buffer head pointer, etc., to retrieve the command from the IOMMU command buffer 210 and execute the command, which causes the IOMMU 112 to perform the corresponding action. The hypervisor 106 performs similar operations to cause the IOMMU 112 to write to the IOMMU peripheral page request log 208 and the IOMMU event log 212 (e.g., replacing the host device ID with the guest device ID, etc.). Because the memory reads and writes, pointer updates, and other operations performed by the hypervisor 106 have long latency, using the hypervisor 106 to intervene between the guest operating system 102 and the IOMMU 112 can cause delays in processing communications, keep the processor busy, and increase traffic on the memory bus of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG1 presents a block diagram illustrating a virtual machine and a hypervisor.

[0008] FIG2 presents a block diagram illustrating communications between a guest operating system and an IOMMU handled by a hypervisor.

[0009] Figure 3 A block diagram illustrating a virtual machine and a hypervisor according to some embodiments is presented.

[0010] Figure 4 A block diagram illustrating an electronic device according to some embodiments is presented.

[0011] Figure 5 A block diagram is presented showing portions of memory accessed by an IOMMU according to some embodiments.

[0012] Figure 6 A block diagram illustrating values ​​stored in a guest copy of an IOMMU MMIO register according to some embodiments is presented.

[0013] Figure 7 A block diagram is presented that illustrates communications between a guest operating system and an IOMMU handled by the IOMMU according to some embodiments.

[0014] Figure 8 A flow diagram is presented that illustrates a process during which an IOMMU accesses a memory location in a guest portion of memory for a guest operating system according to some embodiments.

[0015] Fig. 9 A flow diagram is presented that illustrates a process during which an IOMMU reads from a guest buffer or log in a guest portion of memory according to some embodiments.

[0016] Fig.10 A flow diagram is presented that illustrates a process during which an IOMMU writes to a guest buffer or log in a guest portion of memory according to some embodiments.

[0017] Like reference numerals refer to like figure elements throughout the drawings and description. DETAILED DESCRIPTION

[0018] The following description is presented to enable any person skilled in the art to make and use the described embodiments, and is provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications. Therefore, the described embodiments are not limited to the embodiments shown, but are to be given the widest scope consistent with the principles and features disclosed herein.

[0019] the term

[0020] In the following description, various terms are used to describe the embodiments. The following is a simplified and general description of one of these terms. Note that the term may have important additional aspects, which are not listed herein for clarity and brevity, and therefore the description is not intended to limit the term.

[0021] Functional Block: A functional block refers to a group, set, and / or collection of one or more interrelated circuit elements (such as integrated circuit elements, discrete circuit elements, etc.). Circuit elements are "interrelated" because the circuit elements share at least one property. For example, the interrelated circuit elements may be included in a particular integrated circuit chip or a portion thereof, manufactured on a particular integrated circuit chip or a portion thereof, or otherwise coupled to a particular integrated circuit chip or a portion thereof, may be involved in the execution of a given function (computing or processing function, memory function, etc.), may be controlled by a common control element and / or common block, etc. A functional block may include any number of circuit elements, from a single circuit element (e.g., a single integrated circuit logic gate) to millions or billions of circuit elements (e.g., integrated circuit memory).

[0022] Virtualization, virtual machines, and hypervisors

[0023] The described embodiments support "virtualization" of electronic device hardware such as memory, input-output (IO) devices, etc. Virtualization generally involves an intermediate entity on or in an electronic device, thereby providing a software instance executed on the electronic device with the illusion that the software instance can directly access the electronic device hardware when in fact the intermediate entity intercepts / redirects, translates, or otherwise assists the access performed by the software instance. For example, the intermediate entity can present to the software instance an electronic device register set, a memory location set, an electronic device setting set, and other function block sets, which appear to the software instance as the actual device registers, memory locations, etc. of the electronic device, but instead are only copies presented by the intermediate entity. In this case, the intermediate entity receives, intercepts, or otherwise obtains access to the copy of the electronic device hardware, and performs corresponding interactions with the actual electronic device hardware on behalf of the software instance. Virtualization of electronic device hardware has many benefits, such as enabling different electronic devices to use different arrangements of electronic device hardware, different addresses, locations, or identifiers of electronic device hardware, etc., while the software instance is presented with the same interface as the electronic device hardware via the intermediate entity. In addition, the intermediate entity can determine whether to allow or block access to the electronic device hardware by a given software instance, and thus the virtualization of the electronic device hardware implements protection of the electronic device hardware (or portions thereof) and / or software instances executed on the electronic device. By controlling access as described above, the intermediate entity can share the electronic device hardware between multiple software instances and / or provide each software instance with exclusive access to portions of the electronic device hardware.

[0024] In the described embodiments, the intermediate entity includes a "virtual machine". A virtual machine is a software entity that abstracts electronic device hardware and presents a known interface to actual or simulated electronic device hardware to a software instance. Abstracting the hardware enables the software instance to execute on various types and arrangements of underlying electronic device hardware, which may include electronic device hardware that would otherwise be incompatible with the software instance. In the described embodiments, the virtual machine provides support for executing one or more operating system instances (referred to as "guest" operating systems). The guest operating system, in turn, provides an environment for executing other software programs such as applications, databases, etc.

[0025] In the described embodiment, the virtual machine is managed and controlled by a software entity called a hypervisor. The hypervisor can start or initialize the virtual machine; control, monitor and assist the virtual machine's access to the electronic device hardware; terminate or shut down the virtual machine, etc. Figure 3 A block diagram illustrating a virtual machine and a hypervisor according to some embodiments is presented. Figure 3 As can be seen in FIG. 1 , there are three virtual machines (VMs) 300, under each of which a guest operating system (guest OS) 302 and one or more programs (PRGRMS) 304, such as a database, software application, etc., are executed. The virtual machines 300 communicate with a hypervisor 306, which is connected between a host operating system (host OS) 308 and the virtual machines 300. The host operating system 308 provides an interface between the electronic device hardware 310 and the hypervisor 306. Different from the one shown in FIG. 1 for the existing electronic device, in FIG. Figure 3 314, the IOMMU 312 is coupled directly between the guest operating system 302 and the IO device hardware 314 without the intervention of the hypervisor 306 (as indicated by the thicker line between the IOMMU 312 and the guest operating system 302). Thus, unlike existing electronic devices, in the described embodiments, the hypervisor 306 is not responsible for performing at least some of the operations described to handle communications between the guest operating system 302 and the IOMMU 312, as described herein. However, note that some communications occur between the IOMMU 312 and the hypervisor 306, as indicated by the line between the hypervisor 306 and the IOMMU 312. Further, note that in some embodiments, the host operating system 308 is not present and the hypervisor 306 communicates more directly with the electronic device hardware 310.

[0026] Overview

[0027] In the described embodiments, the electronic device includes a processor, a memory (e.g., a main memory), a plurality of input-output (IO) devices (e.g., a network interface device, a disk controller, etc.), and an input-output memory management unit (IOMMU) connected between the processor and the IO devices. The processor executes a hypervisor, one or more virtual machines, and a guest operating system in the virtual machine. Each of the guest operating systems is allocated a guest portion of memory (a continuous or discontinuous area or block of memory) that is reserved for storing data and information to be accessed by the guest operating system. In the described embodiments, the IOMMU performs operations to handle communications between the guest operating system and the IOMMU.

[0028] As part of handling communications between the guest operating systems and the IOMMU, rather than using a single IOMMU copy of certain buffers and logs as in existing systems, the IOMMU directly accesses each guest operating system's buffers and logs in a corresponding guest portion of memory. In other words, rather than accessing copies of buffers and logs maintained by the IOMMU (and hypervisor) in existing systems, the IOMMU accesses a single copy of the buffers and logs used by each of the guest operating systems. For example, in some embodiments, each guest operating system maintains buffers and logs in a corresponding guest portion of memory, including a command buffer, an event log, and / or a peripheral page request (PPR) log, and the IOMMU reads and / or writes information directly to memory locations (e.g., entries) in each guest operating system's buffers and logs in that guest operating system's portion of memory.

[0029] In existing systems where the IOMMU uses a single copy of the buffer and log, the IOMMU maintains an IOMMU memory-mapped input-output (MMIO) register set for storing information about the buffer and log. For example, the IOMMU MMIO register includes information such as a pointer set (e.g., a head and / or tail pointer of each log or buffer, etc.) and a control value set (e.g., a size indicator and / or configuration value of each log or buffer, etc.) for the buffer and log of the buffer and log. Since in the described embodiment, the IOMMU accesses the buffer and log in different guest parts of the memory separately (i.e., at different memory addresses and possibly with different control values), a single IOMMU MMIO register set is not enough. Therefore, in the described embodiment, the IOMMU is provided with a separate copy of the IOMMU MMIO register set for each guest operating system for accessing the buffer and log in the corresponding guest part of the memory. In other words, in some embodiments, if the system supports N guest operating systems, the IOMMU is provided with N copies of the IOMMU MMIO register (i.e., a pointer set and a control value set for the buffer and log), and each guest operating system has one copy.

[0030] Due to the number of registers required to track the buffers and logs of all possible guest operating systems (which may be very large), a separate copy of the IOMMU MMIO registers does not actually exist in the IOMMU (that is, the IOMMU does not include a hardware circuit for a copy of the IOMMU MMIO registers). Alternatively, a separate copy of the IOMMU MMIO register set of the guest operating system is stored in the IOMMU backup storage area, which is a portion of the main memory reserved for storing data and information of the IOMMU. In the described embodiment, the IOMMU "virtualizes" the IOMMU MMIO registers, or presents to the guest operating system (and other entities, such as hypervisors, etc.) that the guest operating system can access the IOMMU MMIO registers via a specified IOMMU MMIO address. However, there is no register at the IOMMU MMIO address. Alternatively, the IOMMU intercepts the access of the guest operating system to the IOMMU MMIO registers at the IOMMU MMIO address, and redirects the access of the IOMMU MMIO registers to the corresponding location in the IOMMU backup storage area. For example, in some embodiments, the IOMMU MMIO addresses include addresses from a specified address range provided by the IOMMU in the IOMMU interface / aperture that are used to map communications to copies of IOMMU registers in the IOMMU backing store.

[0031] In operation, the IOMMU performs many operations to access the guest buffer or log (or more generally, the corresponding memory location) in the guest portion of the memory. For example, the IOMMU can read information written to the guest buffer or log by the guest operating system (or another entity, including the IOMMU itself) from the guest buffer or log. In some embodiments, the operation of the IOMMU reading the guest buffer or log begins when the guest operating system writes information to the guest buffer or log in the corresponding guest portion of the memory. For example, the guest operating system can write commands to the command buffer in the guest portion of the memory, and the commands, when processed by the IOMMU, cause the IOMMU to perform corresponding actions. The guest operating system then transmits memory writes to update the command buffer tail pointer received, intercepted, or otherwise detected by the IOMMU in the IOMMU MMIO register (and / or other command buffer IOMMU MMIO register). The IOMMU then performs corresponding writes to the IOMMU MMIO register copy of the guest operating system in which the command buffer tail pointer is stored in the IOMMU backup storage area. Continuing with this example, the IOMMU may write updated data such as a new address or offset of a command buffer tail pointer to a copy of a command buffer tail pointer register of the guest operating system in the IOMMU backup storage area. In addition, the IOMMU monitors such writes from the guest operating system, such as monitoring writes to the IOMMU MMIO registers, etc. The IOMMU then uses information from the command buffer head pointer directly or indirectly to read commands from the guest command buffer in the guest portion of the memory. The IOMMU then processes the command so that the IOMMU performs the corresponding action. After processing the command, the IOMMU updates the command buffer head pointer in the IOMMU MMIO register copy of the guest operating system in the IOMMU backup storage area, and may update other command buffer information of the guest operating system in the IOMMU backup storage area. For example, the IOMMU may update the command buffer head pointer in the IOMMU MMIO register copy to indicate that the command has been processed and is therefore no longer present in the command buffer in the guest portion of the memory (such as by advancing the command buffer head pointer to indicate the next entry in the command buffer).

[0032] In addition to reading information from the guest buffer or log, the IOMMU can also write information to be read by the guest operating system (or another entity, including the IOMMU itself) to the guest buffer or log. For example, the IOMMU can write information about the event to the event log in the guest portion of the memory for the guest operating system. In some embodiments, when the information is written to the guest buffer or log of a given guest operating system, the IOMMU first determines the location of the guest buffer or log (or the entry therein) in the guest portion of the memory for the given guest operating system. For this operation, the IOMMU determines the system physical address of the location information of the guest buffer or log in the IOMMU backup storage area of ​​the IOMMU MMIO register copy of the guest operating system. Continuing with this example, the IOMMU determines the system physical address of the storage event log tail pointer of the IOMMU MMIO register copy of the guest operating system in the IOMMU backup storage area. The IOMMU then obtains the location information of the guest buffer or log of the guest operating system from the IOMMU MMIO register copy at the system physical address (e.g., the system physical address of the entry, entry counter, etc.). The IOMMU then uses the location information to write information to a given guest buffer or log in the memory. For example, the IOMMU can use the location information to write information about the event to an event log in the guest portion of the guest operating system of the memory. The IOMMU then updates the associated IOMMU MMIO register copy in the IOMMU backing store to indicate that the write has occurred (e.g., advances the command buffer tail pointer to the next entry in the command buffer in the guest portion of the memory). The IOMMU also sends an internal interrupt to the guest operating system, which indicates that the IOMMU has written information to the guest buffer or log. The guest operating system then performs operations to read information from the guest buffer or log in the guest portion of the memory based on the interrupt (these operations are described in detail below).

[0033] By directly accessing (i.e., performing memory reads and writes) buffers and logs in the guest portion of memory for the guest operating system, the IOMMU is able to avoid relying on the hypervisor to intervene / process access to the guest buffers and logs as in existing systems. Moving these operations from the hypervisor (implemented in software) to the IOMMU (implemented in hardware) speeds up the operation, requires less memory system bandwidth, and reduces the load on the computational functional blocks in the processor, which improves the overall performance of the electronic device. Improved performance of the electronic device leads to higher user satisfaction.

[0034] Electronic Devices

[0035] Figure 4 A block diagram illustrating an electronic device 400 according to some embodiments is presented. Figure 4 As can be seen, electronic device 400 includes processor 402 , memory 404 , mass storage device 406 , input-output (IO) devices 408 - 412 , input-output (IO) hub 414 , and memory controller 416 .

[0036] The processor 402 is a functional block that performs computing operations in the electronic device 400. The processor 402 includes two cores 418-420, each of which includes one or more computing mechanisms, such as a central processing unit (CPU) core, a graphics processing unit (GPU) core, an embedded processor, an application-specific integrated circuit (ASIC), and / or other computing mechanisms. The processor 402 also includes a memory management unit (MMU) 422, which is a functional block that performs operations associated with address translation (e.g., page table traversal, translation lookaside buffer lookup, etc.), memory access protection, etc. to perform memory access by the cores 418-420.

[0037] The memory 404 is a functional block that performs operations of a memory (e.g., a "main" memory) in the electronic device 400. The memory 404 includes: memory circuits, such as one or more of dynamic random access memory (DRAM), double data rate synchronous DRAM (DDR SDRAM), and / or other types of memory circuits for storing data and instructions used by other functional blocks in the electronic device 400; and control circuits for handling access (e.g., reading, writing, checking, deleting, invalidating, etc.) to the data and instructions stored in the memory circuits.

[0038] The mass storage device 406 is a functional block and / or device that performs the operation of a large-capacity non-volatile storage element to store data and instructions for use by other functional blocks in the electronic device 400. The mass storage device 406 may be or include a large-capacity semiconductor memory (e.g., flash memory, etc.), a disk drive (hard disk drive, etc.), an optical drive, etc. A copy of the data and instructions stored in the mass storage device 406 is obtained and stored in the memory 404 for use by other functional blocks in the electronic device 400. For example, in some embodiments, data and / or instructions are retrieved from the mass storage device 406 in blocks or "pages" of a given size (e.g., 4kB, 2MB, etc.) and the pages are stored in the memory 404 for access by other functional blocks. In addition, pages can be newly created at available locations in the memory 404 (e.g., for storing calculation results, etc.).

[0039] IO devices 408-412 are functional blocks and / or devices that perform corresponding IO operations. The specific nature of the IO operations performed by each of the IO devices 408-412 depends on the nature of the IO devices. For example, IO devices 408-412 may include human-machine interface devices, network interface devices, audio / visual processing or providing devices, GPUs, sensor devices, disk controllers, peripheral component interfaces (PCI) devices, universal serial bus (USB) devices, etc., and each IO device performs associated operations, such as receiving input from people (e.g., keyboards, mice, etc.), receiving or sending data on a network, etc. IO devices 408-412 provide data and / or instructions to other functional blocks in the electronic device 400 or consume data and / or instructions from them. For example, in some embodiments, IO devices 408-412 access (i.e., read, write, invalidate, etc.) data in memory pages in the guest memory 428 (i.e., the portion of the memory reserved for a given guest operating system).

[0040] The IO hub 414 is a functional block that performs the operation of an input-output hub, which is connected between the IO devices 408-412 and other functional blocks in the electronic device 400 (e.g., the processor 402, the memory 404, etc.). The operations performed by the IO hub 414 include the following operations: ensuring that communications to the IO devices 408-412 reach the intended IO devices, communications from the IO devices 408-412 correctly reach other functional blocks, preventing the IO devices 408-412 from making unauthorized access to other functional blocks, and vice versa, etc. In some embodiments, the IO hub 414 is connected between buses using different communication standards (such as between the peripheral component interface express (PCIe) bus and the HyperTransport between, etc.), and thereby convert or translate the associated communications.

[0041] The IO hub 414 includes an IOMMU 424, which is a functional block that performs operations for enabling the IO devices 408-412 to access data and / or instructions in the memory 404, communicates with the processor 402 (and the guest operating system executed thereby), etc. In these embodiments, when an IO device (e.g., IO device 408) in the memory 404 wants to access data and instructions, the IO device sends a memory access request (e.g., a direct memory access request or DMA) to the IOMMU 424. The IOMMU 424 then sends a corresponding request to the memory 404 to satisfy the memory access request. For example, in some embodiments, if data is to be retrieved based on the memory access request, the IOMMU 424 obtains the data from the memory 404 (or, if the data is not present in the memory 404, the mass storage device 406) and forwards the data to the IO device that issued the request. In some embodiments, IOMMU 424 includes page tables, translation lookaside buffers, and / or other functional blocks for translating "virtual" or local memory addresses used by IO devices 408-412 into physical addresses in memory 404 where data is actually located.

[0042] In the described embodiments, the IOMMU 424 communicates with the guest operating system executed by the kernel 418-420 in the virtual machine, and vice versa. For example, in some embodiments, the IOMMU 424 (or the IO device 408-412 via the IOMMU 424) transmits events and peripheral page requests (PPRs) to the guest operating system. In these embodiments, the IOMMU 424 reports events such as IO page faults (representing page table traversals of the IO device 408-412), IOMMU 424 hardware errors, etc. to the guest operating system via a shared guest event log in the memory 404. In addition, in these embodiments, the IOMMU 424 forwards PPRs from peripheral devices (IO devices) to the guest operating system, and the PPRs perform memory page service operations (i.e., operations on or associated with pages in the memory 404 that can be accessed by the guest operating system) using the well-known address translation service or ATS standard via the shared guest PPR log in the memory 404. As another example, in some embodiments, the guest operating system transmits commands to the IOMMU 424. In these embodiments, the guest operating system issues commands to the IOMMU 424 via the shared guest command buffer in memory 404 to control the IOMMU 424 and / or IO devices 408-412, such as completion waits (which act as command barriers that force earlier commands to complete before the IOMMU 424 proceeds), device table entry invalidations, IOMMU 424 translation lookaside buffer entry invalidations, etc.

[0043] In some embodiments, the IOMMU 424 provides an interface to the guest operating system that includes memory-mapped locations, registers, etc. for communicating with the IOMMU 424. For example, in some embodiments, the IOMMU 424 provides a set of memory-mapped input-output (MMIO) memory locations to which the guest operating system can write values ​​so that the IOMMU 424 will receive the values. In some embodiments, the interface is virtualized in that the memory locations, registers, etc. are not used to store values, as assumed by the guest operating system, but are merely presented, such as by the IOMMU 424. In these embodiments, the IOMMU 424 can receive values ​​from the guest operating system (e.g., addressed to an IOMMU MMIO address, etc.) via the interface, but a separate copy of the value is stored in the memory location, register, etc. of each guest operating system using the IOMMU backing store 426 and / or other locations in the memory 404. The memory accessed by the IOMMU 424 to communicate with the guest operating system and other entities (e.g., processor 402, etc.) is described in more detail below.

[0044] In some embodiments, although not in Figure 4 4, but the IOMMU 424 includes a local cache memory for storing copies of data or information for the IOMMU 424. For example, in some embodiments, the cache memory is used to store copies of recently used or frequently used data from the IOMMU backing store 426, such as the value set of the IOMMU MMIO registers for a given guest operating system and / or individual value sets therefrom. The cache memory is smaller than the IOMMU backing store 426, and therefore may have only the capacity (i.e., memory locations) to store a portion (and possibly a small portion) of the data and information stored in the IOMMU backing store 426.

[0045] The guest memory 428 is a portion of the memory 404 used by the corresponding guest operating system to store data and information to be used by the guest operating system (e.g., one or more continuous or non-contiguous pages or blocks of memory). In general, the guest operating system and / or other entities can use the guest memory 428 to store any form of data and information used by the guest operating system and / or other entities. In some embodiments, the guest memory 428 is protected and only certain entities are allowed to access the guest memory 428. For example, the corresponding guest operating system, hypervisor, security processor, and / or operating system in the electronic device 400 can "protect" the guest memory 428 by limiting access to the guest memory 428 to the corresponding guest operating system and other specified devices and / or functional blocks. In some embodiments, the guest memory 428 is encrypted or otherwise made inaccessible to undesirable entities. In some embodiments, the guest memory 428 is used to store guest event logs, guest peripheral page request (PPR) logs, and guest command buffers, which are data structures (e.g., tables, lists, etc.) used to communicate between the guest operating system and the IOMMU. The guest event log, guest peripheral page request (PPR) log, and guest command buffer are described in more detail below.

[0046] In some embodiments, communication paths are coupled between various functional blocks (processor 402, memory controller 416, memory 404, etc.) in the electronic device 400, as shown by arrow lines between elements. The communication paths include one or more buses, wires, conductors, and / or other connections that may be connected to controllers, structural elements (switches, routers, etc.), circuit elements, etc. The communication paths are used to route commands, data, control signals, and / or other information between functional blocks. For example, in some embodiments, a coherent bus structure or interconnect is coupled between the IO hub 414, the processor 402 (e.g., MMU 422), and the memory 404. Note that for clarity, some communication paths in the electronic device 400 are not shown in FIG. Figure 4 Shown in.

[0047] In some embodiments, Figure 3 The electronic device hardware 310 in the embodiment includes functional blocks and devices such as processor 402 and memory 404, and the IO device hardware 314 includes functional blocks and devices such as IO devices 408-412. In these embodiments, Figure 3 The IOMMU 312 and Figure 4 The IOMMU 424 in performs at least some of the same operations.

[0048] The electronic device 400 is shown using a specific number and arrangement of elements (e.g., functional blocks and devices such as a processor 402, a memory 404, etc.) and communication paths. However, the electronic device 400 is simplified for purposes of illustration, and in some embodiments, a different number or arrangement of elements and / or communication paths are present in the electronic device 400. For example, the electronic device 400 may include a power subsystem, a display, etc. In general, the electronic device 400 includes sufficient elements and communication paths to perform the operations described herein.

[0049] The electronic device 400 may be or may be included in any electronic device that performs computing operations. For example, the electronic device 400 may be or may be included in an electronic device such as a desktop computer, a laptop computer, a wearable electronic device, a tablet computer, a smart phone, a server, an artificial intelligence device, a virtual or augmented reality device, a network device, a toy, an audio-visual device, a home appliance, a controller, a vehicle, etc. and / or a combination thereof.

[0050] Portions of memory accessed by the IOMMU

[0051] In some embodiments, the IOMMU accesses data and information in different portions of memory (e.g., memory 404) in order to perform the operations described herein. In some of these embodiments, the portions of memory include an IOMMU backing store (e.g., IOMMU backing store 426), a guest memory (e.g., guest memory 428), and / or a hypervisor memory. Figure 5 A block diagram is presented showing a portion of memory accessed by an IOMMU according to some embodiments. Figure 5 Presented as an example, but in some embodiments, the memory and / or different portions of the memory store different types and / or arrangements of information. Generally, the memory includes sufficient information to implement the operations described herein.

[0052] like Figure 5 As can be seen in the figure, the IOMMU backing store 500 includes an ID translation table 502. In general, the ID translation table 502 includes information used by the IOMMU to translate or convert guest domain IDs and / or device IDs in communications from a guest operating system to the IOMMU (or vice versa, communications from the IOMMU to the guest operating system) to host domain IDs and / or device IDs. Domain IDs and device IDs are described in detail in the AMD I / O Virtualization Technology (IOMMU) Specification, Revision 3.00, December 2016, which is incorporated herein by reference as described above.

[0053] In some embodiments, the ID translation table 502 includes a separate table for domain IDs (shown as domain ID mapping table 504) and a separate table for device IDs (shown as device ID mapping table 506), but separate tables are not required (and therefore all translations can be included in a single table). The domain ID mapping table 504 includes a set of entries, each entry for storing an identification or indication of a guest domain ID associated or related to a specified host domain ID. The device ID mapping table 506 includes a set of entries, each entry for storing an identification or indication of a guest device ID associated or related to a specified host device ID. In operation, when a guest or host domain ID and / or device ID is to be translated or converted in a communication, the IOMMU performs a lookup in the ID translation table 502 (i.e., domain ID mapping table 504 and / or device ID mapping table 506) to obtain the corresponding translation or conversion.

[0054] The IOMMU backing store 500 also includes guest controls 508. Typically, the guest controls 508 include copies of values ​​in or from interface registers and control registers of a guest operating system stored in the electronic device. For each supported guest operating system, the guest controls 508 include copies of guest interface registers and / or guest operating system control registers (or at least the values ​​therein) that control the interaction between the IOMMU and the guest operating system. For example, for each guest operating system, the guest controls 508 may include mapping control registers for communicating a domain ID and / or device ID mapping of the guest operating system to the IOMMU.

[0055] The IOMMU backing store 500 also includes a guest memory mapped input and output (MMIO) 510. Generally, the guest MMIO 510 includes pointers and control information for accessing the guest operating system's buffers and logs (e.g., guest command buffers, guest event logs, and guest PPR logs) in the guest portion of memory 404 (e.g., guest memory 428). More specifically, for each supported guest operating system, the guest MMIO 510 includes a copy of the values ​​used to control access to the buffers and logs in the guest portion of memory 404. For example, in some embodiments, the IOMMU supports (can interact with, handle communications with, etc.) 2 N guest operating systems, where N = 10, 16 or other values, and thus guest MMIO 510 includes up to 2 values N In the described embodiment, the values ​​used to control access are similar to the values ​​stored in the IOMMU MMIO registers in existing devices, but a separate set of values ​​is maintained for each supported guest operating system and refers to the guest portion of memory 404 for that guest operating system (rather than a single copy in the IOMMU in existing devices).

[0056] Figure 6 A block diagram is presented that illustrates values ​​stored in the guest MMIO 510 according to some embodiments. For example, in Figure 6 , the values ​​of a single complete IOMMU MMIO register set for a given guest operating system and the values ​​of portions of two adjacent IOMMU MMIO register sets above and below the single complete value set are shown. Figure 6 By way of illustration, as described above, in some embodiments, the IOMMU backing store 426 includes multiple separate value sets, each value set being associated with a given guest operating system. Figure 6 This is presented as an example, but in some embodiments, guest MMIO 510 stores different values ​​or values ​​in a different arrangement. Typically, guest MMIO 510 and / or another entity stores enough information to access buffers and logs in the guest portion of memory as described herein.

[0057] The values ​​in each value set in the guest MMIO 510 can generally be grouped into values ​​associated with a guest command (CMD) buffer, a guest event log, and a guest PPR log of the guest operating system. For a guest command buffer, the values ​​include a command tail pointer 600, which is a pointer or other reference indicating the end or most recently written entry of the guest command buffer in the corresponding guest portion of memory 404. In other words, the command tail pointer 600 stores a pointer or reference indicating the location where the end or most recently written entry of the command buffer is located, such as a memory address, for example, some or all bits of a physical address in the guest portion of memory 404. The values ​​of the guest command buffer also include a command head pointer 602, which is a pointer or other reference indicating the head or base of the command buffer, that is, the location in the corresponding guest portion of memory 404 where the command buffer begins and / or the location where the buffered commands begin. The values ​​of the guest command buffer also include a command length 604, which is a value indicating the current size or maximum size of the command buffer, such as the number of entries or bytes. The value of the guest command buffer also includes command control 606, which is a bit set or bit sequence that includes multiple bits (or combinations thereof) that indicate the configuration of the command buffer in the guest operating system and / or the corresponding guest portion of memory 404. For example, in some embodiments, command control 606 includes bits that indicate whether certain interrupts related to the command buffer are enabled, whether command buffering or processing is enabled (or disabled / paused), what types of commands exist or are allowed to exist in the command buffer, etc. For example, in some embodiments, command control 606 includes control values ​​for command buffers similar to those described in the AMD I / O Virtualization Technology (IOMMU) Specification, Revision 3.00, which is incorporated herein by reference as described above, such as CmdWaitInt, CmdBufRun, CmdWaitInteEn, CmdBufEn, and / or other values.

[0058] In some embodiments, the command buffer is implemented as a ring buffer by using a command head pointer 602 to point to the next entry / position to be read in the command buffer and using a command tail pointer 600 to indicate the last / most recently added entry / position in the command buffer (which can be the same entry / position when only one command is stored in the command buffer). In these embodiments, as commands are read from the entry and processed, the command head pointer 602 advances one entry until the command head pointer 602 and the command tail pointer 600 indicate the same entry. In some of these embodiments, the command length 604 stores the maximum length of the command buffer. Using pointers to implement a ring buffer is well known in the art, so it will not be described in detail herein.

[0059] For the event log, although the values ​​in the guest MMIO 510 (i.e., event tail pointer 608, event head pointer 610, event length 612, and event control 614) are similar in functionality to those values ​​described above for the command buffer, these values ​​are used to access the event log in the corresponding guest portion of memory 404. The same is true for the PPR log, where although the values ​​in the guest MMIO 510 (i.e., PPR tail pointer 616, PPR head pointer 618, PPR length 620, and PPR control 622) are similar in functionality to those values ​​described above for the command buffer, these values ​​are used to access the PPR log in the corresponding guest portion of memory 404.

[0060] In some embodiments, fewer than the supported number (and may be much fewer) of guest operating systems may be executed in the electronic device 400 at any given time. In some of these embodiments, the IOMMU backing store 500 is dynamically adjusted to include sufficient space for storing the guest MMIO 510, etc. In other words, when the guest operating system is initialized / active, the hypervisor and / or another entity may allocate, append, or activate space in the IOMMU backing store 500 (i.e., in the guest MMIO 510) for storing the IOMMU MMIO register values ​​of the guest operating system. In some of these embodiments, the IOMMU backing store 500 does not have additional empty / unused space allocated for virtual machines that do not yet exist (i.e., have not yet been initialized) in the electronic device 400.

[0061] In some embodiments, the hypervisor performs at least some initialization operations for the IOMMU backup storage area 500. For example, in some embodiments, the hypervisor allocates memory, such as continuous or scattered memory pages, in the memory 404 for storing the IOMMU backup storage area. As described above, the memory 404 is a general memory used by various functional blocks (e.g., kernels 418-420, etc.) in the electronic device 400 to store data and information, and is not just a local memory in the IOMMU. Therefore, the operation involves the hypervisor allocating space in the "main" memory of the electronic device 400 for storing the IOMMU backup storage area. As another example, in some embodiments, the hypervisor writes the initial value to a copy of the IOMMU MMIO register in the backup storage area of ​​each active guest operating system via the IOMMU. Therefore, the hypervisor writes the initial value of the pointer and the control value of each active guest operating system to the corresponding copy of the IOMMU MMIO register in the IOMMU backup storage area, such as when the guest operating system starts.

[0062] In some embodiments, only a subset of available IOMMU MMIO registers is included in the copy of the IOMMU MMIO registers in the IOMMU backup storage area. In these embodiments, the IOMMU itself can provide other registers. For example, in some embodiments, the registers storing IOMMU control values ​​(e.g., IOMMU enable / disable, etc.), page table base addresses, etc. in the IOMMU are provided by the IOMMU. Typically, the IOMMU MMIO registers for the buffer and log in the part for the guest operating system that is not used to access the memory are not presented as copies, but a single copy of each register is provided by the IOMMU. In some of these embodiments, the hypervisor simulates these IOMMU registers of the guest operating system, but the access to the registers in the IOMMU is received by the IOMMU and handled in the IOMMU.

[0063] return Figure 5 , the guest memory 512 includes a guest event log 514, a guest peripheral page request (PPR) log 516, and a guest command buffer 518 for the guest operating system. Generally, the guest event log 514, the guest PPR log 516, and the guest command buffer 518 are memory structures (e.g., lists, tables, buffers, etc.) for storing corresponding events, PPR requests, and commands for access by the IOMMU and / or the guest operating system. For example, the guest event log 514 can be a table or a ring buffer including a plurality of (e.g., 24, 50, etc.) entries, each entry being used to store information (e.g., a bit pattern) representing a specific event occurring in the IOMMU, which is to be processed by the corresponding guest operating system. In operation, the IOMMU transmits events and PPRs to the guest operating system via corresponding logs in the guest event log 514 and the guest PPR log 516 in the guest memory 512. In addition, the guest operating system transmits commands to the IOMMU via corresponding command buffers in the guest command buffer 518 in the guest memory 512.

[0064] In some embodiments, each guest operating system active in the electronic device 400 is associated with a corresponding separate guest portion of memory (i.e., multiple pages in the memory 404) that includes a guest event log, a peripheral page request log, and a guest command buffer used by the guest operating system and accessible by the IOMMU. Figure 5 Additional guest memories 520 - 522 are shown behind guest memory 512 .

[0065] The hypervisor memory 524 includes a device table 526. In general, the device table 526 is a table in which device-related information of devices (which may be actual / physical devices or virtual devices) associated with and / or coupled to the electronic device is stored. The device table 526 includes a set of entries, each of which can be used to store information about the corresponding device, such as pointers to page tables and interrupt tables, control and configuration values, function indicators, mode indicators, domain IDs, security information and settings, etc. In addition, in the described embodiment, and unlike in existing device tables, each entry in the device table 526 includes a device ID and a guest identifier of a guest operating system that communicates with, is responsible for, or is otherwise associated with the device. In operation, in addition to using the device table to determine information about the device, the IOMMU also uses the device ID and / or guest identifier to translate or convert the guest device ID to the host device ID.

[0066] In some embodiments, some or all of the IOMMU backing store 500, guest memory 512, and hypervisor memory 524, and / or portions thereof, are not contiguous, but are stored in different areas or locations of memory. For example, the base address of the guest event log 514 (and therefore the guest event log itself) may be located in memory away from the guest PPR log 516. Thus, the guest event log 514 may not be as Figure 5 An adjacent visitor PPR log 516 is shown.

[0067] In some embodiments, the IOMMU includes a private address map that includes pointers, references, and / or other indications of the locations in memory of various data and information accessed by the IOMMU in memory. For example, in some embodiments, the IOMMU private address map includes pointers or references to various copies of the IOMMU MMIO registers of the guest operating system, and / or the starting point / base address of the set of copies of the IOMMU MMIO registers in the IOMMU backup storage area 500. In these embodiments, before accessing the data and information in memory, the IOMMU performs a lookup in the private address map for the location of the data and information.

[0068] In some embodiments, the IOMMU backing store 500 and / or portions thereof (control bits, etc.) are accessible via the IOMMU (e.g., by sending requests to the IOMMU) or are inaccessible to other entities in the electronic device 400. For example, at least some data and information in the IOMMU backing store 500 may be accessed by other entities via writing to and reading from corresponding IOMMU MMIO registers.

[0069] IOMMU and guest operating system communication

[0070] In the described embodiment, an IOMMU (eg, IOMMU 424) handles communications between the IOMMU (or IO devices serviced thereby) and the guest operating system. Figure 7 A block diagram is presented showing communications between a guest operating system 700 and an IOMMU 702 handled by the IOMMU 702 according to some embodiments. Figure 7 , but other embodiments use different numbers or arrangements of elements. Generally, in the described embodiments, IOMMU 702 includes or accesses sufficient elements to implement the operations described herein. Figure 7 In the diagram, many elements are shown as dots or spots; these elements are logs, buffers, etc. stored in memory (e.g., stored in IOMMU backing store 500, stored in guest memory 512, etc.) and accessed by IOMMU 702, guest operating system 700, and / or other entities using typical memory access techniques. In some embodiments, guest operating system 700, IOMMU 702, and hypervisor 704 are accessed in a manner similar to that of a guest operating system 700. Figure 3 The guest operating system 302, IOMMU 312, and hypervisor 306 in FIG. 3 are organized in a similar manner, but this is not required.

[0071] like Figure 72 for existing systems, in the described embodiments, the IOMMU 702 and the guest operating system 700 communicate with each other more directly. In other words, the IOMMU 702 and the guest operating system 700 communicate with each other via the guest event log 514, the guest PPR log 516, and the guest command buffer (BUFF) 518 in the memory (i.e., the guest portion of the memory for the guest operating system 700 (e.g., the guest memory 428)). In addition, the guest operating system 700 and the IOMMU 702 use the guest control 508 and the guest MMIO 510 to specify how the communication will be performed. For example, in some embodiments, the IOMMU 702 uses a pointer in the guest MMIO 510 to determine the location of the guest event log 514, the guest command buffer 518, and the guest PPR log 516 of the guest operating system 700 in the memory. The hypervisor 704 does not intervene and is not otherwise involved in some or all of the operations used to complete these communications. For example, the hypervisor 704 does not perform operations such as translating domain IDs and device IDs for these communications, accessing pointers in the guest MMIO 510, and / or accessing buffers and logs in the guest portion of memory. Instead, the IOMMU 702 performs these operations. Since the IOMMU 702 translates domain IDs and device IDs, accesses guest buffers and logs, etc., the described embodiments avoid using the hypervisor 704 to handle at least part of the communication between the guest operating system 700 and the IOMMU 702, which may mean that the communication is completed faster, thereby causing less load on the processor 402 and memory 404, etc.

[0072] In operation, using commands as an example, the guest operating system 700 writes the invalidate_IOMMU_pages command to the guest command buffer 518. The command, when processed by the IOMMU 702, causes the IOMMU 702 to invalidate a series of entries in the IOMMU translation cache as specified by the domain ID in the command. In other words, the guest operating system performs a memory write in the corresponding guest portion of memory to update the next open / available entry in the guest command buffer 518 to include information for the invalidate_IOMMU_pages command (i.e., bits representing the command). The guest operating system 700 then sends a write command to the IOMMU to update (e.g., advance, etc.) the command buffer tail pointer (e.g., command tail pointer 600) in the corresponding IOMMU MMIO register to indicate that the guest operating system 700 has written the command to the command buffer. The IOMMU 702 detects the guest operating system 700 writing to the command buffer tail pointer, such as via snooping for writes to addresses in the corresponding guest command buffer, detecting changes in the value of the buffer tail pointer, receiving a write command from the guest operating system 700, etc. Upon detecting a write of the command buffer tail pointer, the IOMMU 702 uses the value of the command buffer head pointer (e.g., command head pointer 602) to retrieve the command from the command buffer in the guest portion of memory and prepare the command for processing (e.g., replacing the host domain ID associated with the guest domain ID in the command, etc.). The IOMMU 702 then processes the command, causing the IOMMU 702 to invalidate the series of entries in the translation cache of the IOMMU 702 indicated by the host domain ID. The IOMMU 702 performs at least some similar operations: the IOMMU 702 writes to the guest PPR log 516 and the guest event log 514, although in reverse, as the IOMMU 702 typically writes to these logs while the guest operating system 700 reads the logs.

[0073] Although the hypervisor 704 does not participate in certain portions of the communication between the guest operating system 700 and the IOMMU 702, such as the translation of the guest domain ID to the host domain ID, the hypervisor 704 and the guest operating system 700 and / or the IOMMU 702 may exchange communications associated with the communication between the guest operating system 700 and the IOMMU 702 separately, or the hypervisor 704 may otherwise participate in ensuring that the guest operating system 700 and / or the IOMMU 702 properly handle the communication. For example, the hypervisor 704 may determine directly (e.g., via communication) or indirectly (e.g., via snooping memory accesses) that the IOMMU 702 or the guest operating system 700 has performed a specified operation (e.g., writing to a buffer or log) and may perform operations such as sending an interrupt signal to the guest operating system 700 and / or the IOMMU 702, updating a shared memory location used as a flag, etc. As described above, the hypervisor 704 may also initialize the IOMMU backing store, etc.

[0074] The IOMMU accesses the guest buffer and logs the process in the guest portion of memory

[0075] In the described embodiments, an IOMMU (e.g., IOMMU 702) performs operations for accessing (e.g., reading and writing) guest buffers and logs in a guest portion of memory, which are used to transfer information between the IOMMU and a guest operating system (e.g., guest operating system 700). For example, in some embodiments, the IOMMU accesses entries in a guest command buffer, a guest event log, and / or a guest peripheral page request (PPR) log in a corresponding guest portion of memory for the guest operating system. The IOMMU directly accesses the guest buffers and logs of each guest operating system because the IOMMU uses typical memory access operations to access the guest buffers and logs in the memory portion of each guest operating system without requiring a hypervisor to handle the memory access operations. Figure 8 A flow chart is presented showing a process during which an IOMMU accesses a buffer or log in a guest portion of memory for a guest operating system according to some embodiments. Note that Figure 8 The operations shown are presented as general examples of operations performed by some embodiments. The operations performed by other embodiments include different operations, operations performed in a different order, and / or operations performed by different entities or functional blocks.

[0076] Figure 8 The operations shown are general in scope. Figures 9 and 10 A more specific example is presented in , including an IOMMU reading from a buffer or log in a guest portion of memory for a guest operating system ( Fig. 9) and the IOMMU writes to a buffer or log in the guest portion of memory for the guest operating system ( Fig.10 ).

[0077] like Figure 8 As can be seen in FIG, the IOMMU performs information access in a guest buffer or log in the guest portion of the memory for the guest operating system, and the guest buffer or log is used to transfer information between the IOMMU and the guest operating system (step 800). For this operation, the IOMMU first determines that the access will be performed. For example, after receiving an indication that information is written to a guest buffer or log by a guest operating system or another entity, the IOMMU may determine that information is to be read from a specific guest buffer or log. The IOMMU then determines the address of the guest buffer or log in the guest portion of the memory (i.e., the specific memory location of the entry to be accessed). The IOMMU then performs access to the buffer or log in the guest portion of the memory. In order to access the buffer or log, the IOMMU uses the address of the buffer or log to perform a typical memory access operation. By directly accessing the buffer or log in the guest portion of the memory, rather than using its own set of buffers and logs (e.g., IOMMU buffers and logs such as the IOMMU PPR log 208), the IOMMU is able to communicate with the guest operating system without the involvement of the hypervisor.

[0078] Fig. 9 A flow chart is presented showing a process according to some embodiments during which a guest operating system writes to a guest buffer or log in the guest portion of memory and the IOMMU reads from it. Note that Fig. 9 The operations shown are presented as general examples of operations performed by some embodiments. Operations performed by other embodiments include different operations, operations performed in a different order, and / or operations performed by different entities or functional blocks. For example, Fig. 9 Commands in a guest command buffer are used as an example, however, in other embodiments, the IOMMU reads from a different buffer or log, such as a guest PPR log or a guest event log. In these embodiments, the IOMMU may perform some or all of the same operations to read from different guest buffers or logs.

[0079] Fig. 9The operation in begins when a guest operating system (e.g., guest operating system 700) writes a command to a command buffer (e.g., guest command buffer area 518) in a guest portion of memory (e.g., guest memory 428) (step 900). For example, in some embodiments, the guest command buffer is a ring buffer in the guest portion of memory that includes a plurality of entries, and the operation involves the guest operating system writing the command (i.e., a bit sequence representing the command) to the next available entry in the guest command buffer.

[0080] Because the guest operating system has added the command that needs to be processed by the IOMMU (e.g., IOMMU 702) to the guest command buffer, the guest operating system notifies the IOMMU of the command. More specifically, the guest operating system sends a write request for updating the command buffer tail pointer, the write request pointing to the guest virtual address of the IOMMU MMIO register storing the command buffer tail pointer (step 902). In the described embodiment, the guest operating system uses a local or "virtual" address to access the IOMMU MMIO register (and may not know the actual IOMMU address). The guest virtual address is translated by the memory management unit (MMU) executing the guest operating system in the processor into the system physical address (i.e., IOMMU MMIO address) where the MMIO register in the IOMMU is located (e.g., by performing a page table traversal in the page table of the guest operating system, etc.). As described above, the IOMMU virtualizes the IOMMU MMIO register, so the system physical address used by the MMU is not the address where the copy of the IOMMU MMIO register of the guest operating system is located, but the address of the IOMMU in the interface / aperture that is recognized by the IOMMU as an access to the MMIO register. Thus, after translation in the MMU, the MMU forwards the write request to the IOMMU associated with the guest virtual address at the IOMMU MMIO address (ie, system physical address) (step 904).

[0081] Because the IOMMU MMIO address is only an address provided by the IOMMU for receiving access requests from the guest operating system, the IOMMU also performs a translation to determine the physical address of the copy of the command buffer tail pointer of the guest operating system stored in the memory based on the write request (step 906). During this operation, the IOMMU uses the identification of the guest operating system to determine the physical address in the memory, which specifies which copy of the IOMMU MMIO registers to access in the IOMMU backup storage area. For this process, the IOMMU first calculates the IOMMU virtual address for the copy of the IOMMU MMIO registers of the guest operating system based on the identifier of the guest operating system and the system physical address provided by the MMU using an algorithm, table, etc. The IOMMU then uses one or more page tables to translate the IOMMU virtual address into the system physical address of the copy of the IOMMU MMIO registers of the guest operating system stored in the IOMMU backup storage area.

[0082] The IOMMU then stores the data from the write request at the physical address in the memory, thereby updating the copy of the command buffer tail pointer of the guest operating system (step 908). In other words, the IOMMU stores data such as one or more bits of the update address of the command buffer tail pointer, entry identifiers, run counts, etc. in the memory location where a copy of the command buffer tail pointer associated with the guest operating system is stored in the IOMMU backup storage area. The update causes the command buffer tail pointer to directly or indirectly indicate that the command has been written to the guest command buffer and is therefore waiting for IOMMU processing. In some embodiments, the IOMMU, the guest operating system, and / or another entity also sets other pointers or indicators to indicate that the command is waiting for processing in the guest command buffer or the guest operating system provides a command. For example, in some embodiments, the IOMMU includes a set of command waiting bits, one bit for each supported guest operating system, and sets (or clears) the command waiting bit for the guest operating system to indicate that the command is waiting for processing in the guest command buffer.

[0083] The IOMMU, based on detecting a write request (or otherwise determining that a command is pending), uses data from a copy of the command buffer head pointer to retrieve a command from a guest command buffer in the guest portion of memory (step 910). For this operation, the IOMMU uses data from a copy of the command buffer head pointer directly or indirectly, possibly together with other pointers or information (e.g., guest command buffer base pointer, etc.) from other IOMMU MMIO register copies of the guest operating system in the backing store to determine the system physical address of the entry in the guest command buffer where the command is stored. For this operation, the IOMMU may perform translation, etc., along with the determined address (e.g., using a corresponding page table) to determine the system physical address of the entry in the guest command buffer. The IOMMU then reads the command from the system physical address, which may involve an operating system, a hypervisor, a memory management unit, etc., which allows the memory read to occur - assuming that the IOMMU is allowed to access a specific address in the guest portion of memory. At the end of this operation, the IOMMU has a command retrieved from the guest portion of memory (e.g., with the various bits of the command), and is therefore ready to execute the command.

[0084] The IOMMU then processes the command (step 912). For this operation, in some embodiments, command processing logic in the IOMMU processes the command, which causes the IOMMU to perform operations on the command. Fig. 9 , but after processing the command, the IOMMU updates the system status to indicate that the command has been processed. For example, in some embodiments, the IOMMU updates the command buffer head pointer in the copy of the corresponding IOMMU MMIO register of the guest operating system in the IOMMU backup storage area, and may update other command buffer information of the guest operating system in the IOMMU backup storage area.

[0085] Fig.10 A flow chart is presented showing a process according to some embodiments during which the IOMMU writes to a guest buffer or log in the guest portion of memory and the guest operating system reads from it. Note that Fig.10 The operations shown are presented as general examples of operations performed by some embodiments. Operations performed by other embodiments include different operations, operations performed in a different order, and / or operations performed by different entities or functional blocks. For example, Fig.10 Using events in a guest event log as an example, however, in other embodiments, the IOMMU writes to a different buffer or log, such as a guest PPR log or a guest command buffer. In these embodiments, the IOMMU may perform some or all of the same operations to write to a different guest buffer or log.

[0086] Fig.10 The operations in begin when an IOMMU (e.g., IOMMU 702) determines a system physical address of a guest event log (e.g., guest event log 514) in a guest portion of memory for a guest operating system (e.g., guest memory 428) (step 1000). For this operation, the IOMMU calculates, looks up, or otherwise determines a system physical address of a copy of an IOMMU MMIO register of the guest operating system in a backing store that holds a guest event log tail pointer based on an identifier of the guest operating system (e.g., domain ID). The system then obtains the guest event log tail pointer from the system physical address and uses the value of the guest event log tail pointer (and possibly values ​​from other copies of the IOMMU MMIO registers of the guest operating system, such as an event log base pointer, etc.) directly or indirectly as the system physical address of the guest event log. In some embodiments, the IOMMU performs one or more operations to calculate a system physical address based on the value of the event log tail pointer and / or use one or more page tables to translate the value of the event log tail pointer.

[0087] The IOMMU then uses the system physical address to write the event to a guest event log in the guest portion of memory for the guest operating system (step 1002). For example, in some embodiments, the guest event log is a ring buffer in the guest portion of memory that includes multiple entries, and the operation involves the IOMMU writing the event (i.e., a sequence of bits representing the event) to the next available entry in the guest event log. The IOMMU uses typical memory access operations to write the event to the guest event log. In some embodiments, writing to the system physical address in the guest portion of memory involves an operating system, a hypervisor, a memory management unit, etc., which allows the memory write to occur - assuming that the IOMMU is allowed to access a specific address in the guest portion of memory.

[0088] Because IOMMU has added the event to the guest event log that needs guest operating system to process, IOMMU notifies the guest operating system of the event. In some embodiments, notifying the guest operating system includes updating the copy of the event log tail pointer of the guest operating system in the IOMMU backup storage area and sending an interrupt to the guest operating system to notify both of the guest operating system. Therefore, IOMMU determines the physical address of the copy of the event log tail pointer of the guest operating system in the IOMMU backup storage area (step 1004). During this operation, IOMMU uses the identification of the guest operating system and one or more IOMMU private address tables and / or page tables to determine the physical address in the memory, which identification specifies which copy of the IOMMU MMIO register is accessed in the IOMMU backup storage area. IOMMU then stores data at the physical address, thereby updating the copy of the event log tail pointer of the guest operating system (step 1006). In other words, IOMMU stores data such as one or more bits of the update address of the event log tail pointer, entry identifiers, run counts, etc. in the memory location where the copy of the event log tail pointer associated with the guest operating system is stored in the IOMMU backup storage area. The update causes the event log tail pointer to directly or indirectly indicate that the event has been written to the event log and is therefore awaiting processing by the guest operating system.

[0089] The IOMMU also sends an interrupt to the guest operating system indicating that an event is pending in the guest event log (step 1008). For this operation, the IOMMU can use an internal interrupt mechanism that passes interrupts between the IOMMU and the appropriate guest operating system, for example, through a virtual advanced programmable interrupt controller (vAPIC) associated with the guest operating system.

[0090] In response to receiving the interrupt, the guest operating system sends a read request to the IOMMU to read data from the event log header pointer, the read request pointing to the guest virtual address of the IOMMU MMIO register storing the event log header pointer (step 1010). In the described embodiment, the guest operating system uses a local or "virtual" address to access the IOMMU MMIO register (and may not know the actual and / or virtual IOMMU address). The guest virtual address is translated by the memory management unit (MMU) in the processor executing the guest operating system into the system physical address (i.e., the IOMMU MMIO address) at which the MMIO register in the IOMMU is located (e.g., by performing a page table walk in the page table of the guest operating system, etc.). As described above, the IOMMU virtualizes the IOMMU MMIO register, so the system physical address used by the MMU is not the address at which the guest operating system's copy of the IOMMU MMIO register is located, but rather the address of the IOMMU in the interface / aperture that is recognized by the IOMMU as an access to the MMIO register. Thus, after translation in the MMU, the MMU forwards the read request to the IOMMU associated with the guest virtual address at the IOMMU MMIO address (ie, system physical address) (step 1012).

[0091] Because the IOMMU MMIO address is simply an address provided by the IOMMU for receiving access requests from the guest operating system, the IOMMU also performs a translation to determine the physical address of the copy of the event log header pointer of the guest operating system stored in the memory based on the read request (step 1014). During this operation, the IOMMU uses the identification of the guest operating system to determine the physical address in the memory, which specifies which copy of the IOMMU MMIO registers to access in the IOMMU backup storage area. For this process, the IOMMU first calculates the IOMMU virtual address for the copy of the IOMMU MMIO registers of the guest operating system based on the identifier of the guest operating system and the system physical address provided by the MMU using an algorithm, table, etc. The IOMMU then uses one or more page tables to translate the IOMMU virtual address into the system physical address of the copy of the IOMMU MMIO registers of the guest operating system stored in the IOMMU backup storage area.

[0092] The IOMMU then reads the data for the read request from the memory location at the physical address in the memory. In other words, the IOMMU reads one or more bits of the address of the event log header pointer, an entry identifier, a run count, and other data in the memory location of the copy of the event log header pointer associated with the guest operating system stored in the IOMMU backup storage area. The event log header pointer directly or indirectly indicates the next event that has been written to the guest event log and is waiting for the guest operating system to process. In some embodiments, the IOMMU, the guest operating system, and / or another entity also sets other pointers or indicators to indicate that there are events in the guest event log waiting to be processed or that the IOMMU provides an event. For example, in some embodiments, the guest operating system and / or the hypervisor include a set of event wait bits, one bit for each supported guest operating system, and the event wait bit is set (or cleared) for the guest operating system to indicate that there are events in the guest event log waiting to be processed. The IOMMU then returns the data from the event log header pointer to the guest operating system (step 1016).

[0093] The guest operating system then uses the data to perform one or more subsequent operations to process the event in the guest event log (step 1018). For example, in some embodiments, the guest operating system's copy of the event log head pointer stores the address of the location of the event log entry in the guest portion of memory, and the guest operating system uses the address directly or indirectly to retrieve the event from the guest event log before processing the event.

[0094] In some embodiments, the electronic device (e.g., electronic device 400 and / or a portion thereof) uses code and / or data stored on a non-transitory computer-readable storage medium to perform some or all of the operations described herein. More specifically, when performing the described operations, the electronic device reads code and / or data from a computer-readable storage medium and executes the code and / or uses the data. A computer-readable storage medium may be any device, medium, or combination thereof that stores code and / or data for use by an electronic device. For example, a computer-readable storage medium may include, but is not limited to, volatile and / or non-volatile memory, including flash memory, random access memory (e.g., eDRAM, RAM, SRAM, DRAM, DDR4 SDRAM, etc.), read-only memory (ROM), and / or magnetic or optical storage media (e.g., disk drives, tapes, CDs, DVDs, etc.).

[0095] In some embodiments, one or more hardware modules perform the operations described herein. For example, a hardware module may include, but is not limited to, one or more processors / cores / central processing units (CPUs), application specific integrated circuits (ASIC) chips, neural network processors or accelerators, field programmable gate arrays (FPGAs), computing units, embedded processors, graphics processors (GPUs) / graphics cores, pipelines, accelerated processing units (APUs), caches / cache controllers, memory / memory controllers, function blocks, and / or other programmable logic devices. When such hardware modules are activated, the hardware modules perform some or all of the operations. In some embodiments, a hardware module includes one or more general-purpose circuits, which are configured by executing instructions (program code, firmware, etc.) to perform operations.

[0096] In some embodiments, a data structure representing some or all of the structures and mechanisms described herein (e.g., the electronic device 400, the IOMMU 424, and / or a portion thereof) is stored on a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium comprising a database or other data structure that can be read by the electronic device and used directly or indirectly to manufacture hardware including the structures and mechanisms. For example, the data structure can be a behavioral level description or a register transfer level (RTL) description of the hardware functionality in a high-level design language (HDL) such as Verilog or VHDL. The description can be read by a synthesis tool that can synthesize the description to generate a netlist that includes a list of gate / circuit elements from a synthesis library that represents the functionality of the hardware including the above structures and mechanisms. The netlist can then be placed and routed to generate a data set describing the geometry to be applied to the mask. The mask can then be used in various semiconductor manufacturing steps to produce one or more semiconductor circuits (e.g., integrated circuits) corresponding to the above structures and mechanisms. Alternatively, the database on the computer accessible storage medium may be a netlist (with or without a synthesis library) or a data set (as desired), or Graphic Data System (GDS) II data.

[0097] In this specification, variables or unspecified values ​​(i.e., general descriptions of values ​​without specific examples of values) are represented by letters such as N. As used herein, although similar letters may be used in different places in this specification, variables and unspecified values ​​are not necessarily the same in each case, and some or all of the general variables and unspecified values ​​may be expected to have different variable amounts and values. In other words, in this specification, N and any other letters used to represent variables and unspecified values ​​are not necessarily related to each other.

[0098] As used herein, the expression "etc." is intended to present one and / or circumstances, that is, an equivalent of "at least one" of the elements in the list associated with "etc.". For example, in the sentence "the electronic device performs a first operation, a second operation, etc.", the electronic device performs at least one of the first operation, the second operation, and the other operation. In addition, the elements in the list associated with "etc." are merely examples from a set of examples, and in some embodiments, at least some of the examples may not appear.

[0099] The previous description of the embodiments is presented for the purpose of illustration and description only. The previous description is not intended to be exhaustive or to limit the embodiments to the disclosed form. Accordingly, many modifications and variations will be apparent to practitioners in the art. In addition, the above disclosure is not intended to limit the embodiments. The scope of the embodiments is defined by the appended claims.

Claims

1. An electronic device for accessing a guest buffer and a log, the electronic device comprising: a processor, the processor executing a guest operating system; a memory having a guest portion, the guest portion being reserved for storing data and information to be accessed by the guest operating system; as well as An input-output memory management unit IOMMU, the IOMMU being configured to: In the guest portion, writing information to a guest buffer and / or log for transferring information from the IOMMU to the guest operating system; and Information in a guest buffer and / or log used to transfer information from the guest operating system to the IOMMU is read from the guest portion.

2. The electronic device of claim 1 , wherein the IOMMU writing information to a given guest buffer or log in the guest portion comprises: The IOMMU determines a system physical address of the given guest buffer or log in the memory; the IOMMU writing information to the given guest buffer or log in the memory at the system physical address; and The IOMMU sends an indication to the guest operating system that the IOMMU has written the information to the given guest buffer or log.

3. The electronic device of claim 2, wherein the IOMMU writing information to the given guest buffer or log in the guest portion further comprises: The IOMMU determines, based on an identifier of the guest operating system, a system physical address of a copy of an IOMMU MMIO register of the guest operating system in an IOMMU backing store; The IOMMU obtains the system physical address from the copy of the IOMMU MMIO register of the guest operating system in the IOMMU backing store; as well as The IOMMU updates the copy of the IOMMU MMIO registers of the guest operating system in the IOMMU backing store to indicate the write to the guest buffer or log after writing the information to the guest buffer or log.

4. The electronic device of claim 2, wherein sending the indication to the guest operating system comprises: The IOMMU sends an internal interrupt to the guest operating system via a virtual copy of the guest operating system's interrupt controller, the internal interrupt notifying the guest operating system that the IOMMU has written to the given guest buffer or log. 5 . The electronic device of claim 2 , wherein the guest operating system reads the information from the guest buffer or log at the system physical address in the guest portion based on the indication.

6. The electronic device of claim 5, wherein the guest operating system reading the information from the guest buffer or log in the guest portion comprises: The guest operating system obtains information from the guest operating system's copy of the IOMMU MMIO registers in the IOMMU backing store via the IOMMU; as well as The guest operating system uses the information from the copy of the IOMMU MMIO registers to read the information from a guest buffer or log in the guest portion.

7. The electronic device of claim 1 , wherein the IOMMU reading information from a given guest buffer or log in the guest portion comprises: The IOMMU receives a memory write request from the guest operating system to update an IOMMU MMIO register associated with the given guest buffer or log, the guest operating system having separately written to the given guest buffer or log; The IOMMU determines a system physical address of the given guest buffer or log in the guest portion based on the update; and The IOMMU reads information from the given guest buffer or log in the memory at the system physical address.

8. The electronic device according to claim 7, wherein the IOMMU is further configured to: determining a system physical address of a copy of the IOMMU MMIO registers of the guest operating system in an IOMMU backing store based on the memory write request and an identifier of the guest operating system; and A corresponding update is performed on the copy of the IOMMU MMIO registers of the guest operating system in the IOMMU backing store.

9. The electronic device of claim 1, wherein the guest buffer and / or log comprises some or all of the following: Command buffers; Log events; and Peripheral page request PPR log.

10. The electronic device according to claim 1, wherein: The processor executes at least one other guest operating system, and the memory has a separate guest portion reserved for storing data and information to be accessed by the at least one other guest operating system; and The IOMMU is configured as: writing information in a corresponding guest portion to a guest buffer and / or log for transferring information from the IOMMU to the at least one other guest operating system; and Information in a guest buffer and / or log for transferring information from the at least one other guest operating system to the IOMMU is read from the corresponding guest portion.

11. A method for accessing a guest buffer and a log in an electronic device, the electronic device comprising: a processor, the processor executing a guest operating system; a memory having a guest portion, the guest portion being reserved for storing data and information to be accessed by the guest operating system; and an input-output memory management unit IOMMU, the method comprising: In the guest portion, the IOMMU writes information to a guest buffer and / or log for transferring information from the IOMMU to the guest operating system; as well as The IOMMU reads information from the guest portion for transferring information from the guest operating system to a guest buffer and / or log of the IOMMU.

12. The method of claim 11 , wherein the IOMMU writing information to a given guest buffer or log in the guest portion comprises: The IOMMU determines a system physical address of the given guest buffer or log in the memory; the IOMMU writing information to the given guest buffer or log in the memory at the system physical address; and The IOMMU sends an indication to the guest operating system that the IOMMU has written the information to the given guest buffer or log.

13. The method of claim 12, wherein the IOMMU writing information to a given guest buffer or log in the guest portion further comprises: The IOMMU determines, based on an identifier of the guest operating system, a system physical address of a copy of an IOMMU MMIO register of the guest operating system in an IOMMU backing store; The IOMMU obtains the system physical address from the copy of the IOMMU MMIO register of the guest operating system in the IOMMU backing store; as well as The IOMMU updates the copy of the IOMMU MMIO registers of the guest operating system in the IOMMU backing store to indicate the write to the guest buffer or log after writing the information to the guest buffer or log.

14. The method of claim 12, wherein sending the indication to the guest operating system comprises: The IOMMU sends an internal interrupt to the guest operating system via a virtual copy of the guest operating system's interrupt controller, the internal interrupt notifying the guest operating system that the IOMMU has written to the given guest buffer or log.

15. The method according to claim 12, further comprising: The guest operating system reads the information from the guest buffer or log in the guest portion at the system physical address based on the indication.

16. The method of claim 15, wherein the guest operating system reading the information from the guest buffer or log in the guest portion comprises: The guest operating system obtains information from the guest operating system's copy of the IOMMU MMIO registers in the IOMMU backing store via the IOMMU; as well as The guest operating system uses the information from the copy of the IOMMU MMIO registers to read the information from a guest buffer or log in the guest portion.

17. The method of claim 11, wherein the IOMMU reading information from a given guest buffer or log in the guest portion comprises: The IOMMU receives a memory write request from the guest operating system to update an IOMMU MMIO register associated with the given guest buffer or log, the guest operating system having separately written to the given guest buffer or log; The IOMMU determines a system physical address of the given guest buffer or log in the guest portion based on the update; and The IOMMU reads information from the given guest buffer or log in the memory at the system physical address.

18. The method of claim 17, wherein the IOMMU is further configured to: determining a system physical address of a copy of the IOMMU MMIO registers of the guest operating system in an IOMMU backing store based on the memory write request and an identifier of the guest operating system; and A corresponding update is performed on the copy of the IOMMU MMIO registers of the guest operating system in the IOMMU backing store.

19. The method of claim 11, wherein the guest buffer and / or log includes some or all of the following: Command buffers; Log events; and Peripheral page request PPR log.

20. The method of claim 11, wherein: The processor executes at least one other guest operating system, and the memory has a separate guest portion reserved for storing data and information to be accessed by the at least one other guest operating system; and The IOMMU is configured as: writing information in a corresponding guest portion to a guest buffer and / or log for transferring information from the IOMMU to the at least one other guest operating system; and Information in a guest buffer and / or log for transferring information from the at least one other guest operating system to the IOMMU is read from the corresponding guest portion.

Images