Address translation prefetch for input / output devices

Abstract

In one example of the present technology, an input / output memory management unit (IOMMU) of a computing device is configured to: receive a prefetch message including a virtual address from a central processing unit (CPU) core of a processor of the computing device; performing a page traversal on the virtual address through a page table stored in a main memory of the computing device to obtain a prefetched translation of the virtual address to the physical address; and storing the prefetched translation of the virtual address to the physical address in a translation lookaside buffer (TLB) of the IOMMU.

Description

Background Art

[0001] In direct memory access (DMA), input / output devices attached to a computer system communicate directly with memory without involving the CPU - for example, without temporarily storing data in registers of the CPU during transfers between the memory and the input / output devices. Direct memory access significantly improves the performance of these computer systems because the CPU is relieved of the task of copying data between input / output (I / O) devices and can perform other tasks for the user, and because the data being copied is not subject to any data transfer bottlenecks associated with passing data to or from the CPU.

[0002] A typical computer system includes physical memory, where corresponding memory addresses are used to access (e.g., read from and write to) various portions of the memory. Memory locations within the physical memory are often referred to using physical addresses (PAs). Computer operating systems include memory management features, where software applications running on the operating system access memory through virtual addresses that are mapped to corresponding physical addresses. The memory management unit (MMU) of the central processing unit (CPU) provides hardware support for performing address translations between virtual addresses and physical addresses.

[0003] A DMA controller for an input / output device (I / O device) uses PAs to identify locations in memory that the I / O device will read from or write to using DMA requests, where the host operating system can provide these PAs directly to the I / O device. However, software applications that utilize the I / O device typically provide them using virtual addresses rather than PAs. Therefore, a computer system may also include an input / output memory management unit (IOMMU) or a system memory management unit (SMMU) that connects a DMA-enabled I / O bus to main memory, where the IOMMU or SMMU translates between virtual addresses and PAs.

[0004] In the context of computer virtualization, a hypervisor running on a host operating system manages the execution of one or more virtualized or simulated computer systems, which are referred to as client virtual machines, and in which client virtual machines execute client operating systems. The client virtual machine accesses the portion of the memory allocated to it through its allocated virtual address space, wherein the address in the space may be referred to as a client physical address (GPA). Therefore, the client operating system allocates a virtual address space to a software application running on the client operating system, wherein the virtual address space is mapped to the client physical address space. This results in an additional indirection layer between the (client) virtual address seen by the application and the actual physical address in the host physical memory (e.g., from the client virtual address to the client physical address, and from the client physical address to the host physical address). Similarly, address translation must be performed so that the I / O device can continue to use the application virtual address or the client virtual address, translate the virtual address into the GPA and HPA (i.e., the physical address of the host), and issue a DMA request accordingly, wherein the GPA is translated into the HPA, in order to access the correct portion of the host physical memory.

[0005] It is with respect to these and other considerations that examples are presented. Furthermore, although relatively specific problems have been discussed, it should be understood that the examples should not be limited to solving the specific problems identified in this background. Summary of the invention

[0006] This summary is provided to introduce a series of concepts in a simplified form, which are further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended as an aid in determining the scope of the claimed subject matter.

[0007] Aspects of the present technology relate to using direct memory access technology to move data between I / O devices and physical memory to improve the address translation performance for input / output devices (I / O devices). Some aspects of the present technology relate to starting an address translation cache (e.g., stored in an I / O device) so that when a device issues a direct memory access request, the address translation is available at low latency. Some aspects of the present technology relate to input / output memory management unit (IOMMU) cache management strategies that reduce wasted work by preferentially retaining cached translation data that has not yet been used. Some additional aspects of the present technology relate to using a system cache (e.g., the last level cache of a central processing unit) to selectively cache translations that are not suitable for other caches. These modifications reduce the delay associated with performing page walks to perform address translations between virtual addresses and physical addresses by increasing the probability of a cache hit in the address translation cache.

[0008] The details of one or more aspects are described in the following drawings and specific embodiments. Other features and advantages will be apparent by reading the following specific embodiments and examining the associated drawings. It is to be understood that the following specific embodiments are only illustrative and do not limit the claimed invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings, which are incorporated into and constitute a part of this disclosure, illustrate various aspects of the present invention. In the drawings:

[0010] Figure 1 is a block diagram of a computing device including a processor, a memory, and an input / output device that performs direct memory access (DMA) operations using the memory according to some examples of the present technology.

[0011] Figure 2A is a timing diagram that depicts the communication timing between an input / output (I / O) device, an I / O memory management unit (IOMMU), main memory (or dynamic random access memory or DRAM), and a host processor core in the case of a DMA read from main memory to an I / O device.

[0012] Figure 2B is a timing diagram that depicts the communication timing between the I / O device, IOMMU, main memory, and the host processor core in the case of a DMA write from the I / O device to the main memory.

[0013] Figure 3A is a flow chart of a method 300 for performing address translation prefetching by an IOMMU as initiated by a CPU core according to one example of the present technology.

[0014] Figure 3B is a timing diagram according to an example of the present technology, which depicts the communication timing between the I / O device, IOMMU, main memory, and the host processor core in the case of DMA reading from main memory to the I / O device with address translation prefetching by the IOMMU.

[0015] Figure 3C is a timing diagram according to an example of the present technology, which depicts the communication timing between the I / O device, IOMMU, main memory, and host processor core in the case of DMA writes from the I / O device to the main memory using address translation prefetching by the IOMMU.

[0016] Figure 3D is a flow chart of a method for performing address translation prefetch and write push on a cache of an I / O device by an IOMMU according to an example of the present technology.

[0017] Figure 3E is a timing diagram of an example according to the present technology, which depicts the communication timing between the I / O device, IOMMU, main memory, and host processor core when DMA writes are performed from the I / O device to the main memory using address translation prefetch by the I / O device, and write push is performed to the address translation cache of the I / O device.

[0018] Figure 4A is a flow chart of a method for performing address translation prefetch initiated by a controller of an I / O device according to an example of the present technology.

[0019] Figure 4B is a timing diagram of an example according to the present technology, which depicts the communication timing between the I / O device, IOMMU, main memory, and host processor core when DMA writes are performed from the I / O device to the main memory using address translation prefetch by the I / O device.

[0020] Figure 5 is a block diagram illustrating example physical components of a computing device in which various aspects of the invention may be practiced.

[0021] Fig. 6A and Figure 6B is a simplified block diagram of a mobile computing device in which various aspects of the invention may be practiced. DETAILED DESCRIPTION

[0022] The following specific embodiments refer to the accompanying drawings. Wherever possible, the same reference numerals are used in the accompanying drawings and the following description to refer to the same or similar elements. Although various aspects of the present invention can be described, modifications, adjustments, and other implementations are possible. For example, the elements shown in the accompanying drawings can be replaced, added, or modified, and the methods described herein can be modified by replacing, reordering, or adding stages to the disclosed methods. Therefore, the following specific embodiments do not limit the present invention, but the proper scope of the present invention is defined by the attached claims. Examples can take the form of hardware implementation, or complete software implementation, or a combination of software and hardware implementations. Therefore, the following specific embodiments will not be considered to be restrictive.

[0023] A typical computer system includes physical memory (e.g., random access memory or RAM), where various portions of the memory are accessed (e.g., read from and written to) using corresponding memory addresses. RAM stores currently running computer programs or software applications—such as a web browser, a computer game, machine learning training software, a word processor, etc., and also stores data loaded by these programs—such as web pages, 3D models and images in computer games, training data sets and statistical models for machine learning, documents loaded in a word processor, etc.

[0024] Computer systems also typically include or are attached to input / output devices (I / O devices), such as graphics cards (e.g., 3D graphics accelerators), data storage devices (e.g., hard drives, solid-state drives, etc.), network interface controllers for connecting to a computer network (e.g., Wi-Fi or Ethernet controllers), special-purpose accelerators (e.g., data compression / decompression accelerators, data encoding / decoding accelerators such as for error detection codes and error correction codes (such as hardware controllers for redundant arrays of independent drives or RAID), artificial intelligence accelerators for training machine learning models or computing predictions or inferences based on machine learning models), webcams, keyboards, mice, touch screens, printers, etc. I / O devices can be connected to a computer system via an interconnect such as Peripheral Component Interconnect Express (PCIe) or other types of system interconnects.

[0025] Some I / O devices are capable of performing direct memory access (DMA), which allows these devices to interact directly with physical memory without involving the CPU. This improves the performance of the computer system because the CPU is relieved of the task of copying data between input / output (I / O) devices and can therefore perform other tasks (e.g., executing other computer programs or responding to user input). In addition, the speed at which data can be transferred between I / O devices and physical memory is not limited by bandwidth bottlenecks associated with transferring data to or from the CPU.

[0026] Memory locations within physical memory are often referred to as physical addresses (PAs). The DMA controller of an I / O device uses PAs to identify locations in memory that the I / O device will read from or write to using DMA requests, where the host operating system typically provides these PAs directly to the I / O device.

[0027] Computer operating systems also typically include memory management features so that software applications (e.g., a web browser or a computer game) access memory through virtual addresses that are mapped to corresponding physical addresses (PAs). Translations between virtual memory addresses and PAs can be accelerated by the CPU's hardware memory management unit (MMU).

[0028] Software applications utilizing DMA-enabled I / O devices typically provide virtual addresses rather than PAs to the I / O devices (e.g., because the underlying PAs are hidden from the application by the operating system). Thus, in order for the I / O device to issue a DMA request with the correct PA of the location in the physical memory, address translation must also be performed. In some cases, the computer system also includes an input / output memory management unit (IOMMU), or more generally, a system memory management unit (SMMU) that acts as a middleware between the I / O interconnect and the main memory that supports DMA, wherein the IOMMU or SMMU translates between the virtual address and the PA. As used herein, the term input / output memory management unit (IOMMU) will also be used to include a similar or more general system memory management unit (SMMU) class that performs a similar function of translating between virtual addresses and physical addresses as a middleware between the I / O interconnect and the main memory that supports DMA. In some computer systems, the IOMMU and the MMU are integrated into the same physical semiconductor package as one or more CPUs. For clarity, examples of the present technology will be presented below in the context of a computer system having separate IOMMU and MMU components, with the understanding that the present technology is similarly applicable to situations where the MMU translates virtual addresses to the PA for applications (as in a typical MMU) and for I / O devices (as would be performed by the IOMMU), thereby including or integrating functionality that would otherwise be performed by a separate IOMMU.

[0029] In the case of executing an application in a non-virtualized environment, the application may provide a virtual address to a DMA-enabled I / O device, and wherein the IOMMU assists in translating the virtual address to a physical address for performing the DMA operation.

[0030] In the case of a virtualized environment, two address translations may be required: from a guest virtual address (GVA) visible to an application running on a guest virtual machine to a guest physical address (GPA) associated with the guest virtual machine (e.g., in a virtual address space assigned to the guest virtual machine), and then to a host physical address (HPA) associated with the host. In the context of virtualization, a physical address (PA) may also be referred to as a host physical address (HPA) to clarify that the address is different from a guest physical address (GPA). As described in the background, in the context of a virtual machine, a hypervisor running on a host operating system manages the execution of one or more guest virtual machines, which run a corresponding guest operating system. A guest virtual machine accesses a portion of a host memory assigned to it through its assigned virtual address space, and the addresses in this space may be referred to as guest physical addresses (GPA). Therefore, a software application running in a guest operating system is provided with a guest virtual address space (GVA space) mapped onto a guest physical address space (GPA space). In these cases, an address translation must be performed so that the guest virtual address (GVA) is translated to the GPA, and wherein the GPA is then translated to the HPA. This translation also applies to the use of I / O devices by the guest virtual machine so that the I / O device can issue DMA requests using the appropriate HPA for the portion of host memory allocated to the application running in the guest virtual machine. The IOMMU can also accelerate this multi-stage address translation.

[0031] Address translation typically involves looking up page tables located in host memory (page walks) to obtain or look up translations from GVA to GPA and from GPA to HPA. Because page tables are organized hierarchically, page walks typically involve traversing multiple levels of page tables stored in main memory before obtaining the HPA returned to the I / O device so that the I / O device (eventually) can issue a DMA request based on the returned HPA, where each step may require a round trip between the MMU or IOMMU performing the page walk and main memory. I / O devices typically encounter hundreds of nanoseconds of latency when accessing host memory. Therefore, performing these page walks through host memory to perform address translations can introduce significant latency before the I / O device receives the HPA required to implement the DMA request.

[0032] To reduce this latency (whether in the absence of virtualization and performing a translation from a virtual address to a physical address, or in the case of translating a GVA to a GPA and a GPA to an HPA), in some cases, the host computer system provides a translation cache (similar to a translation lookaside buffer or TLB inside a processor core or CPU core) in hardware (e.g., included in a semiconductor package with an IOMMU and a CPU core) to save frequently used address translations in a faster memory structure (e.g., static random access memory or SRAM) to significantly reduce the latency of page traversal. Appropriately sizing the TLB can be challenging, especially when many virtual functions across multiple devices share the same TLB. A conservatively sized (e.g., sized larger to increase the likelihood of a cache hit) TLB can be area intensive (e.g., consuming a large amount of physical space in the processor core due to a larger amount of SRAM) and expensive. On the other hand, a smaller TLB may be ineffective even when there are a small number of traffic flows (e.g., multiple DMA data flows between an I / O device and host memory).

[0033] As another technique for reducing latency, the Peripheral Component Interconnect Express (PCIe) specification defines a mechanism called Address Translation Service (ATS) that allows an I / O device to request that an address translation be returned to the device. Thus, the translation can be stored internally in a device-side Address Translation Cache (ATC) to reduce reliance on the host-side cache.

[0034] The use of client virtual address (GVA) further aggravates this problem. The use model for "shared virtual memory" (SVM) involves the host processor and I / O interaction using GVA, without the pre-translation performed by the host operating system. This means that the I / O device receives the GVA for DMA operation. These GVAs must be translated to GPA, and then subsequently translated to HPA. This is generally referred to as 2-stage translation. Each level of translation can have its own TLB structure for cache address translation. Each level of the page table hierarchy can have a separate TLB. For simplicity, this paper includes a plurality of caches supporting hierarchical page tables and a plurality of translation stages to the discussion of translation cache.

[0035] While host-side caches (e.g., TLBs) and device-side caches (e.g., ATCs) provide some improvements to latency by caching previously obtained address translation results, these systems can still sometimes introduce significant latency. One problem arises from the lack of locality in I / O access streams. For example, a computer system may have an application workload in which there are hundreds of active DMA streams that may be distributed across multiple virtual machines running on the same computer system. In addition, each stream may have an arbitrary (e.g., random) step size, which causes the degree of locality within a single page to vary (e.g., memory accesses to nearby or local locations in memory may reuse the same address translation, but accesses to more distant locations may require additional address translations). Even when there is some locality, a 4KB region of memory requiring a single translation can be accessed in only about 64ns using a x16 PCIe Gen5 device (with a peak bandwidth of 64GB / s in each direction) - from the I / O device's perspective, this data transfer time is comparable to or typically less than the DRAM latency incurred in accessing the translation (e.g., accessing the address translation will take about as long as transferring the data itself). Input / output access characteristics, especially when a device is shared by many VMs, will tend to have low locality and therefore may incur translation cache misses, and these DMA requests will also be limited by memory latency due to the need to perform a page walk through pages stored in host memory to perform the address translation. The translation cache miss penalty can create stalls in the data movement pipeline to the I / O device and therefore limit I / O device throughput.

[0036] As such, aspects of the present technique are directed to proactively reducing or minimizing address translation cache misses, or equivalently, increasing the frequency of translation cache hits, in order to reduce delays caused by address translation, thereby increasing I / O device throughput when using DMA.

[0037] Figure 1 is a block diagram of a computing device according to some examples of the present technology, the computing device including a processor, a memory, and an input / output device that performs direct memory access (DMA) operations using the memory. Figure 1As shown in FIG. 1 , computing device 100 includes processor 110 including one or more central processing unit (CPU) cores 112, host cache 114, memory management unit (MMU) 116 and its associated cache or translation lookaside buffer (TLB) 117 (in some processor designs, the MMU is located within the CPU core), and input / output memory management unit (IOMMU) 118 and its associated cache or translation lookaside buffer (TLB) 119. The one or more CPU cores may be configured to operate according to an instruction set such as x86 or The host cache 114 (which may also be referred to herein as a system cache or a last-level cache) stores data and / or program instructions, wherein the host cache 114 is located near the CPU core 112 (e.g., with a low latency and high bandwidth connection to the CPU core 112) and is separate from the CPU core 112, wherein access to the host cache 114 is shared by the CPU cores 112. A single CPU core 112 may include a lower level cache (e.g., a separate level 1 or L1 cache) within each CPU core in the CPU core 112, wherein these lower levels of cache dedicated to a particular CPU core 112 (non-shared) are not considered part of the shared host cache 114. The host cache 114 may store blocks read from the main memory 130 or physical memory and working data to be written to (or flushed to) the main memory 130.

[0038] The memory controller 132 of the main memory 130 is connected to the processor 110 via the interconnect 140 or communication bus. The memory controller 132 of the main memory 130 is configured to receive commands from the interconnect 140 and is configured to read from or write to a specific location in the main memory 130. As discussed above, locations in the main memory 130 can be accessed by the memory controller 132 based on their physical addresses (PA), where an address refers to a page of memory (each page is typically several kilobytes in size). The operating system provides a memory management function by providing a virtual address space to applications, and by abstracting how virtual addresses in the virtual address space are mapped to physical addresses in the physical address space, where software applications or programs access memory locations in the main memory 130 based on virtual addresses. The mapping or translation between virtual addresses and physical addresses is typically stored in a page table 134, which is stored in the main memory 130. When the CPU core 112 accesses an instruction or data from a particular memory location (as specified by a virtual address), the virtual address is translated into a physical address in order to identify an appropriate location within the main memory 130 for writing the data, such as the first memory page 136 or the second memory page 138. The first memory page 136 and the second memory page 138 are identified by the controller 132 based on the corresponding first physical address 137 and the corresponding second physical address 139. The memory management unit (MMU) 116 of the processor 110 performs the translation from the virtual address to the physical address by performing a page walk, i.e., performing (possibly multiple) lookups through the page table 134 in the main memory 130. As described above, in the case of virtualization (virtual machines) and / or shared virtual memory (SVM), additional lookups may be required, such as performing a translation from a guest virtual address (GVA) to a guest physical address (GPA) to a host physical address (HPA).

[0039] Performing a page walk is a time-consuming process because the MMU 116 communicates with the main memory 130 via the interconnect 140, which has a relatively high latency (e.g., compared to the clock rate of the one or more CPU cores 112), where the latency is due to the inherent latency of the memory 130 and due to the latency of traversing the interconnect 140. To reduce the latency of performing a translation from a virtual address (or guest virtual address in the case of virtualization) to a PA (or HPA in the case of virtualization), the MMU 116 may also include a cache or translation lookaside buffer (TLB) 117 that caches the translations, thereby avoiding the latency associated with performing a page walk if the address translations are already cached in the TLB 117.

[0040] Taking a network interface controller (NIC) (e.g., a Wi-Fi network card or an Ethernet network card) as an example of an I / O device 150, when the computing device 100 sends data over the network, the data packets can be copied from the main memory 130 to the I / O device 150 so that the I / O device 150 can transmit the data packets over the network. Similarly, when the NIC receives data packets from the network, these data packets are copied to the main memory 130 so that the data can be used by the software application running on the computing device 100.

[0041] In many computer systems, I / O devices are memory mapped, where writes to and reads from the I / O devices are performed based on reserved memory addresses similar to the HPA, but where the HPA in the reserved range is directed to various I / O devices rather than specific pages in main memory 130.

[0042] Direct memory access (DMA) technology allows input / output devices (I / O devices) to read and write directly to main memory without passing the data through the CPU core 112 .

[0043] Without using DMA, the processor 110 mediates the transfer of data between the main memory 130 and the I / O device 150. For example, when writing data to an I / O device, the CPU core 112 may control the reading of the data from a specified address in the main memory 130 into a register in the CPU core 112, and then copy the data from the register to a memory-mapped location corresponding to the I / O device 150 (e.g., a memory buffer in the controller 152 of the I / O device 150). Similarly, when transferring data from the I / O device 150 to the main memory 130, the CPU core 112 may copy the data from the memory buffer in the controller 152 to the register of the CPU core 112, and may copy the data from the register of the CPU core 112 to the specified address in the main memory 130.

[0044] In contrast, when using DMA, the processor 110 may send a DMA stream setup command to the controller 152 of the DMA-capable I / O device, where the command specifies a memory location in the main memory 130. The I / O device 150 may then directly perform a write to the specified memory location in the main memory 130, or perform a read from the specified memory location, without intermediately storing the data in the processor 110 (e.g., in a register of the CPU core 112).

[0045] Because the DMA stream setup command is issued by a software application running on the processor 110, the processor 110 usually makes a system call that causes the operating system to translate the virtual address provided by the software application into a physical address, where the physical address will be sent to the device later. However, using a system call to the operating system to perform address translation usually causes significant software processing overhead. Therefore, in some methods for DMA, the memory location specified in the DMA stream setup command is a virtual address (e.g., a customer virtual address). Therefore, address translation is performed to translate the virtual address provided by the software application into a PA (or HPA) corresponding to an actual memory location in the main memory 130, which will be accessed by the I / O device 150 in a manner similar to that for an application running on the processor. Some computer systems also include an input / output memory management unit (IOMMU) 118, which is configured to translate the virtual address used by the I / O device 150 into a PA by performing a page walk on the page table 134, which effectively eliminates the software overhead associated with translating addresses in some computer systems that use operating system-level system calls to perform address translation. Like MMU 116 , IOMMU 118 may also have a cache or translation lookaside buffer (TLB) 119 that stores or caches previously determined translations between virtual addresses and PAs, thereby reducing or avoiding delays associated with performing a full page walk through page table 134 .

[0046] In some systems, the I / O device controller 152 also includes an address translation cache (ATC) 154 that is also configured to cache translations from virtual addresses to PAs so that the I / O device controller 152 does not need to request address translations from the IOMMU 118 .

[0047] Figure 2A is a timing diagram that depicts the DMA stream setup commands and the communication timing between the input / output (I / O) device, the I / O memory management unit (IOMMU), the main memory (or dynamic random access memory or DRAM), and the host processor core in the case of a DMA read from main memory to an I / O device.

[0048] like Figure 2AAs shown in , the CPU core 210 transmits a DMA stream setup command to an I / O device 250 (e.g., a network interface controller or NIC). The DMA stream setup command specifies the virtual address of data in main memory (or DRAM) 230 to be written to the I / O device (e.g., for transmission to the network 270). When the DMA controller of the I / O device 250 attempts to execute the command (e.g., retrieve data from a location in main memory 230), the DMA controller needs to translate the virtual address into a physical address (PA) and may first check a cache (e.g., an address translation cache ATC) in the I / O device 250. Figure 2A In the scenario shown in , it is assumed that this is a new request and therefore the required translation for the current virtual address does not exist in the ATC, so attempting to find it in the cache will result in an ATC miss. Due to the ATC miss, the DMA controller of the I / O device 250 requests an address translation from the IOMMU 218.

[0049] exist Figure 2A , when the IOMMU 218 receives a translation request, it also checks the IOMMU's cache (e.g., a translation lookaside buffer or TLB). As described above, assuming this is the first time this particular virtual address is looked up, the attempted lookup in the TLB results in an IOMMU TLB miss. Therefore, the IOMMU performs a page walk at 234 through the page table stored in main memory (or DRAM) 230. As previously discussed, performing a page walk may require multiple lookups that require multiple round trips between the IOMMU 218 and DRAM 230, as indicated by Figure 2A After the page traversal is completed, the IOMMU 218 has calculated or obtained the translation of its input virtual address to the physical address (PA), and the IOMMU 218 returns the translation to the I / O device 250.

[0050] Using the translated or translated address (the obtained PA associated with the original virtual address), I / O device 250 issues a DMA read request directly to DRAM 230. DRAM 230 retrieves the data at the PA specified in the DMA read request and returns it to I / O device 250 for further use (e.g., sending the data as a network packet on network 270).

[0051] like Figure 2A As shown in , a significant amount of time is spent performing the translation from the virtual address to the PA before a DMA read can be performed.

[0052] Figure 2Bis a timing diagram that depicts the DMA stream setup command and the communication timing between the I / O device, IOMMU, main memory, and CPU core in the case of DMA write from the I / O device to the main memory. Figure 2A In a manner, CPU core 210 transmits a DMA stream setup command to I / O device 250 (e.g., a network interface controller or NIC). The DMA stream setup command specifies a virtual address in main memory 230 where data is to be written from the I / O device (e.g., where the I / O device 250 stores data received from network 270).

[0053] In this case, there may be a time delay between when the DMA stream setup command is sent to I / O device 250 and when data is ready to be written from I / O device 250 to DRAM 230 (e.g., while waiting for data packets from network 270 to arrive). When such data does arrive, the DMA controller of I / O device 250 may attempt to perform the DMA write, but first needs to translate the virtual address specified in the DMA stream setup command into a physical address in DRAM 230. Therefore, the DMA controller may first attempt to find the address in its cache (e.g., ATC). As previously described, in Figure 2A In the scenario shown in FIG. 1 , it is assumed that this is a new request and therefore the required translation for the current virtual address does not exist in the ATC, so the attempted lookup in the cache results in an ATC miss. Due to the ATC miss, the DMA controller of the I / O device 250 requests an address translation from the IOMMU 218. Figure 2A In the manner described, the IOMMU performs a page walk at 234 to translate the virtual address to the corresponding PA, and returns the translation to the I / O device 250, which uses the translated address (PA or HPA) to perform a DMA write of the data.

[0054] like Figure 2A As shown in , a significant amount of time is spent performing the translation from the virtual address to the PA before a DMA write can be performed.

[0055] Therefore, aspects of the present technology relate to systems and methods for reducing delays associated with address translations by pre-fetching address translations so as to fill one or more caches with the translations. In more detail, some aspects relate to using the expected translation to start the IOMMU TLB, the address translation cache (ATC) of the I / O device, or both. In some examples of the present technology, the startup is initiated by the CPU core itself when providing a command (e.g., a DMA stream setup command) to the I / O device. In some examples of the present technology, the startup is completed by the controller of the I / O device that pre-fetches the translation. Some examples of the present technology include multiple methods, and one or more methods are applied depending on the capabilities of the I / O device and / or the characteristics of the typical business flow associated with the I / O device.

[0056] Figure 3A is a flow chart of a method 300 for performing address translation prefetching by an IOMMU as initiated by a CPU core according to an example of the present technology. Figure 1 As shown in , the method is performed by IOMMU 118.

[0057] Figure 3B is a timing diagram of an example according to the present technology, which depicts the communication timing between the DMA stream setting command and the I / O device, IOMMU, main memory, and CPU core in the case of DMA reading from the main memory to the I / O device using address translation prefetch by the IOMMU.

[0058] refer to Figure 3A and Figure 3B At operation 301, the IOMMU 318 receives a DMA stream setup command including a virtual address from the CPU core 310. Figure 3B In the example shown in , a prefetch message including a virtual address is sent to IOMMU 318 at the same time (or shortly thereafter) as a DMA stream setup command is sent to I / O device 350. In some examples, the same DMA stream setup command is transmitted to IOMMU 318 as the prefetch message, which includes the virtual address to IOMMU 318 (e.g., the DMA stream setup command is broadcast to both IOMMU 318 and I / O device 350). In some embodiments, different messages containing the same virtual address are sent to I / O device 350 and IOMMU 318.

[0059] At operation 303, IOMMU 318 pre-fetches a physical address corresponding to the virtual address contained in the pre-fetch message, such as by performing a page walk through a page table stored in DRAM 330 at 334, and stores the resulting translation in a cache (TLB) of IOMMU 318 at operation 305 (e.g., Figure 1 18 (the TLB 119 of the IOMMU 118 shown in FIG. 1 ).

[0060] After receiving the DMA stream setup command, the DMA controller of the I / O device 350 attempts to perform a DMA read based on the virtual address specified in the DMA stream setup instruction. The DMA controller begins by checking its local cache (address translation cache or ATC) for cache translations for the specified virtual address. Figure 3B In the example shown in , this is the first time a particular virtual address is used by I / O device 350, and therefore there will be no cached translation in the ATC, resulting in an ATC miss (cache miss). Therefore, the DMA controller of I / O device 350 requests an address translation from IOMMU 318, which receives the request at operation 307. Figure 3B , the IOMMU 318 receives the translation request before the page traversal 334 is complete, and therefore the IOMMU 318 waits until it receives the final translation before responding to the I / O device 350 at operation 309. However, because the page traversal 334 begins earlier (e.g., in response to the translation request received from the CPU core 310), the time delay or latency between the translation request from the I / O device 350 and the response from the IOMMU 318 is shorter than it would be if no prefetching were performed (compare FIG. 1 ). Figure 3B The time between "request translation" with prefetch and response "translation" in Figure 2B In other cases, depending on the behavior of I / O device 350, the translation request may be sent to IOMMU 318 later, such as after page walk 334 is complete, so that prefetching of the address translation results in an immediate IOMMU TLB hit for the prefetched result.

[0061] In this way, examples of the present technology reduce latency and speed up the calculation or retrieval of the translation from a virtual address (or guest virtual address) to a corresponding physical address (PA) by prefetching the translation associated with the corresponding virtual address (e.g., in the case of machine virtualization, looking up the HPA based on the corresponding GVA or GPA) for performing a DMA read operation from main memory to an I / O device.

[0062] Figure 3C is a timing diagram of an example of the present technology, which depicts the DMA stream setup command and the communication timing between the I / O device, the IOMMU, the main memory, and the CPU core in the case of DMA write from the I / O device to the main memory using address translation prefetching by the IOMMU. According to an example, the IOMMU performs (such as Figure 3C 3B ) is similar to the operation performed when prefetching is used to perform address translation of a read request, and thus will be combined with Figure 3A The example method shown in describes operations performed by an IOMMU according to one example.

[0063] Similar to Figure 3B The method described in this way, refer to Figure 3A and Figure 3C At operation 301, the IOMMU 318 receives a DMA stream setup command including a virtual address from the CPU core 310. Figure 3B , the DMA stream setup command is sent to the IOMMU 318 at the same time (or shortly thereafter) as the DMA stream setup command is sent to the I / O device 350. In some examples, the same DMA stream setup command is transmitted to the IOMMU 318 to be used as the DMA stream setup command to the I / O device 350 (e.g., broadcast to both the IOMMU 318 and the I / O device 350). In some embodiments, different messages containing the same virtual address are sent to the I / O device 350 and the IOMMU 318.

[0064] At operation 303, the IOMMU 318 pre-fetches a physical address corresponding to the virtual address, such as by performing a page walk through a page table stored in DRAM 330 at 334, and stores the resulting translation in a cache (TLB) of the IOMMU 318 (e.g., Figure 1 18).

[0065] After receiving the DMA stream setup command, the DMA controller of the I / O device 350 starts a process for writing data from the I / O device 350 to the main memory 330 at the location specified by the DMA stream setup command. Figure 3C In the example shown in FIG. 3 where the I / O device 350 is a network interface controller (NIC), writing is initiated by receiving a network data packet from the network 370 at the I / O device 350. However, the present technology is not limited thereto, and data may be received by the I / O device 350 or generated by the I / O device 350 to be written to the main memory 330. For example, a graphics processing unit (GPU), an artificial intelligence (AI) accelerator, or other computing device may generate a computation result in response to input data and write the computed result to the main memory 330.

[0066] When a DMA write is initiated, the DMA controller attempts to find the address translation for the virtual address specified in the DMA stream setup command in a local cache (e.g., ATC) local to the I / O device 350. Figure 3C In the specific example of , this is the first time a particular virtual address is used by I / O device 350, and therefore there will be no cached translation in the ATC, resulting in an ATC miss (cache miss). Therefore, the DMA controller of I / O device 350 requests an address translation from IOMMU 318, which receives the request at operation 307. Figure 3C In the specific example shown in , the IOMMU 318 receives the translation request from the I / O device 350 after the page traversal 334 is completed, and therefore this results in an IOMMU cache hit (translation lookaside buffer or TLB hit) so that the IOMMU can immediately respond to the I / O device with the address translation. In other cases, the IOMMU may receive the translation request before the page traversal 334 is completed, so that the IOMMU must wait before it can respond to the translation request (similar to Figure 3B )

[0067] In the examples presented above, the IOMMU 318 is described as storing the pre-fetched PA in a cache, such as a translation lookaside buffer associated with the IOMMU 318, e.g. Figure 1 1 and 12. The TLB 119 shown in FIG. 1 is associated with the IOMMU 118. However, the present technology is not limited thereto.

[0068] In some examples, in IOMMU 318, the resulting translation is stored in a system-level cache (such as Figure 1 14) of the processor 110 shown in the figure, instead of or in addition to storing the translation in the TLB 119. In more detail, while a translation cache miss is inevitable when a translation is first accessed (on demand or by prefetching), there may be a capacity miss or a conflict miss. A capacity miss occurs when there is a shared translation cache and their combined working set does not fit in the cache for many business flows (e.g., there are more active translations than can fit in the TLB 119 of the IOMMU 118). A conflict miss occurs when translations required by the same or different business flows are "mapped" into the same physical location of the TLB. A conflict miss is a function of the specific addresses associated with the translations and the behavior of the hash function that is applied to these addresses to map them to the limited space within the cache.

[0069] In order to mitigate the effects of such cache misses, some aspects of the embodiments utilize available system-level cache or host cache. Herein, the term system cache is intended to reflect a large, logical cache shared across all CPU cores 112 and I / O devices, which are parts of the same coherent domain. In some examples of the present technology, when the translation is extracted into the translation cache, the translation is stored in the system-level cache (e.g., stored in both the TLB and the system-level cache, such as the host cache 114). In some examples of the present technology, when the translation is removed from the translation cache, the translation is allocated (e.g., stored or copied) to the system-level cache, such as the host cache 114. In some examples, the allocation (or writing) of two types of addresses to the system-level cache is selectively performed by the IOMMU 118 based on the attributes associated with the address translation, wherein the IOMMU 118 selectively stores the translation into the system-level cache in a manner that maintains cache coherence, data integrity, and program functionality. In some examples, these attributes include cache allocation attributes, such as "allocate on read" or "allocate on read". An IOMMU TLB miss will result in a read from memory. If the read is marked with the "allocate" attribute, the read will not only return the data to the IOMMU, but also allocate a copy of the data in the system-level cache.

[0070] In such embodiments where the translation is also stored in the system-level cache, in the event that a lookup of an address in the TLB results in an IOMMU TLB miss (e.g., the virtual address is not available in the IOMMU's TLB), the IOMMU first checks the system-level cache (e.g., host cache 114) for the cached translation and, in the case of a system-level cache hit, returns the cached translation or, in the case of a system-level cache miss, performs a page walk to retrieve the translation from a page table in main memory and thereafter stores the retrieved translation (e.g., physical address) in the IOMMU TLB and / or the system-level cache, depending on space availability, as guided by attributes associated with the translation.

[0071] In this way, examples of the present technology reduce latency and speed up the calculation or retrieval of a translation from a virtual address (or guest virtual address) to a corresponding physical address (PA) for performing a DMA write operation from an I / O device to a main memory by prefetching the PA associated with the corresponding virtual address.

[0072] In the above Figure 3A , Figure 3B and Figure 3CIn the example described and shown in , IOMMU 318 pre-fetches a translation of a virtual address and waits for a translation request from I / O device 350 before sending the translation to the I / O device. In some examples of the present technology, IOMMU 318 transmits the translated address to I / O device 350 to populate a local cache at I / O device 350 (e.g., ATC of I / O device 350) in a "write push" to the cache.

[0073] Figure 3D is a flow chart of a method 380 for performing address translation prefetch and write push for a cache of an I / O device by an IOMMU according to an example of the present technology. Figure 3E is a timing diagram of an example according to the present technology, which depicts the DMA stream setting command and the communication timing between the I / O device, IOMMU, main memory, and CPU core when DMA writes from the I / O device to the main memory are performed using address translation prefetching by the IOMMU, and write push is performed to the address translation cache (ATC) of the I / O device. Figure 3D The write push to the ATC of the I / O device 350 is shown in the context of a DMA write. However, the present technology is not limited thereto, and may also be applied to, for example, the case of a DMA read operation.

[0074] Similar to Figure 3B and Figure 3C In the manner described in the above, at operation 381, the IOMMU 318 receives a DMA stream setup command including a virtual address from the CPU core 310. Figure 3D , the translation request is sent to the IOMMU 318 at the same time (or shortly thereafter) as the DMA stream setup command is sent to the I / O device 350. In some examples, the same DMA stream setup command is transmitted to the IOMMU 318 to be used as the translation request to the IOMMU 318 (e.g., broadcast to both the IOMMU 318 and the I / O device 350).

[0075] At operation 383, the IOMMU 318 pre-fetches a physical address corresponding to the virtual address, such as by performing a page walk through a page table stored in DRAM 330 at 334, and optionally stores the resulting translation in a cache (TLB) of the IOMMU 318 (e.g., Figure 1 18).

[0076] At operation 387 , the IOMMU transmits the translation to the I / O device 350 (eg, transmits the translation without receiving a translation request from the I / O device 350 ) to cause the I / O device 350 to write-push the translation to the ATC of the I / O device 350 .

[0077] exist Figure 3E , the write push of the address translation to the ATC of I / O device 350 occurs before the arrival of the data packet from network 370. Thus, when I / O device 350 is ready to perform a DMA write, the necessary translation (e.g., the PA corresponding to the virtual address specified in the DMA stream setup command received by the I / O device) is already present in the ATC, and thus the lookup of the virtual address in the local cache by the DMA controller results in an ATC hit, and the DMA write with the translated address can be performed immediately (e.g., without the additional delay associated with sending a translation request to the IOMMU).

[0078] Although Figure 3E The timing associated with DMA writes and write pushes of translations of the ATC of the I / O device 350 is shown, but the present technology is not limited to this and may also be applied to write pushes of translations of the ATC of the I / O device 350 in the case of DMA reads.

[0079] The above about Figure 3A , Figure 3B , Figure 3C , Figure 3D ,and Figure 3E The described examples relate to prefetching of address translation performed by the IOMMU. However, examples of the present technology are not limited thereto. For example, in some examples, a controller (e.g., a DMA controller) of an I / O device is configured to perform prefetching of address translation.

[0080] Figure 4A is a flow chart of a method for performing address translation prefetch initiated by a controller of an I / O device according to an example of the present technology. Figure 4B is a timing diagram of an example according to the present technology, which depicts the communication timing between the DMA stream setup command and the I / O device, IOMMU, main memory, and CPU core when DMA writes from the I / O device to the main memory are performed using address translation prefetching by the I / O device.

[0081] exist Figure 4A and Figure 4BIn the example shown in , prefetching of address translation is initiated by the I / O device 450, and therefore receiving a DMA stream setup command from the CPU core 410 is sufficient for the I / O device (e.g., there is no need to transmit a separate DMA stream setup instruction or other message containing the virtual address directly from the CPU core 410 to the IOMMU 418).

[0082] At operation 401, the I / O device 450 (e.g., a DMA controller of the I / O device 450) receives a DMA stream setup command including a virtual address, wherein the DMA stream setup command may have been sent by the CPU core 410. At operation 403, the I / O device 450 pre-fetches the translation of the virtual address into a physical address by sending a translation request to the IOMMU 418. Operation 403 is considered a pre-fetch because the I / O device 450 has not yet performed a DMA operation (e.g., a DMA read or a DMA write) using the DMA stream associated with the virtual address. For example, the DMA controller of the I / O device 450 may transmit a translation request in response to receiving a DMA stream setup command (as opposed to requesting a translation in response to determining that data is to be written to main memory using a DMA stream or that data is to be read from main memory using a DMA stream).

[0083] IOMMU 418 may check its cache (e.g., TLB and system-level cache, if applicable) to determine if there is a cache hit. If not, then Figure 4B As shown in the example of , the IOMMU performs a page walk 434 through the main memory DRAM 430 to obtain a translation (e.g., a physical address corresponding to a virtual address), which the IOMMU 418 returns to the I / O device 450. At operation 405, the I / O device stores the pre-fetched translation (e.g., the pre-fetched physical address) in an address translation cache (ATC).

[0084] Thus, when the I / O device 450 needs to execute a DMA command at operation 407 (such as a DMA write due to the arrival of a data packet from the network 470, where the I / O device 450 is a network interface controller), the ATC is enabled with the necessary address translation as specified by the DMA stream setup command. In this way, the I / O device can perform the DMA operation at operation 409 using a pre-fetched physical address retrieved from a local cache (e.g., ATC) that is looked up based on a virtual address (ATC hit) without incurring the delays associated with performing address translation (e.g., traversing the interconnect 140 to access the IOMMU and, in some cases, performing a page walk through a page table stored in main memory). For example, if Figure 2B as shown in Figure 3CIn the case of prefetching translations in the IOMMU without further pushing writes to the ATC as shown in the example of Figure 4B The delay between the arrival of packet data in and the execution of DMA data writing is compared with the corresponding delay without performing prefetching.

[0085] Some examples of the present technology involve selecting between different methods for starting a cache, such as an address translation cache (ATC) of an I / O device, a translation lookaside buffer (TLB) of an IOMMU, and / or a system-level cache of a processor. As described above, in some examples, translations are associated with attributes, and the attributes are used to select between different prefetching methods. For example, Figure 4A and Figure 4B As shown in the example of , when there is more time between the DMA stream setup command and the first DMA operation performed by the I / O device according to the DMA stream setup command, the prefetch initiated by the I / O device is more effective. For example, in the case of an immediate DMA read based on the DMA stream setup command, such as Figure 2A In the example shown in the timing diagram of , implementing prefetching by the I / O device will result in essentially no reduction in latency. Figure 3B As shown in the timing diagram of , in this particular case of a DMA read operation, prefetching at the IOMMU can reduce latency because the IOMMU can start the translation process (e.g., page walk) while the I / O device 250 processes the DMA stream setup command and prepares and sends the request translation.

[0086] Thus, in some examples, the CPU core selects between different methods of performing prefetching of address translations, such as by determining whether to send an address translation prefetch command directly to the IOMMU and / or whether to direct a DMA controller of the I / O device to perform prefetching of address translations, where the determination is based on properties such as whether the I / O device is capable of performing prefetching, whether the DMA stream is a DMA read stream or a DMA write stream, the expected time between a DMA stream setup command and the first DMA operation, etc.

[0087] Prefetched translations are expected to be kept in the translation cache until they are used. Some comparable cache eviction policies (or translation cache replacement policies) keep frequently used translations and discard the least recently used translations (least recently used or LRU policies). However, assuming that each DMA stream setup command creates a DMA stream that will eventually be used, prefetching translations without utilizing address translations and then removing them from the cache results in wasted work performing page walks, since these page walks must be performed again when the DMA stream is eventually used.

[0088] Thus, some examples of the present technology involve explicit retention of prefetched translations (e.g., protection from eviction) until those prefetched translations have been used at least a threshold number of times, where the threshold number is greater than zero, where the particular threshold number can be specified based on properties of a DMA stream (e.g., based on typical access patterns). The number of times the prefetched translations have been used or accessed is monitored and recorded by a controller that manages a particular cache, such as a DMA controller for an I / O device in the case where addresses are stored in an ATC, and an IOMMU in the case where addresses are stored in an IOMMU TLB or system-level cache (e.g., by storing a count of the number of accesses as metadata in the cache). After the prefetched translations have been used at least a specified threshold number of times, a standard cache eviction policy is used, such as based on a least recently used (LRU) policy.

[0089] Thus, examples of the present technology improve the performance of direct memory access (DMA) operations between an I / O device and main memory by pre-fetching address translations to reduce or avoid delays associated with performing address translations, such as delays due to traversals in an interconnect (e.g., a PCIe interconnect), and delays due to performing page traversals through page tables stored in main memory. Some examples of the present technology involve initiating a pre-fetch from a CPU core by transmitting a message to an IOMMU to pre-fetch address translations for a particular virtual address, and storing the address translations in a translation lookaside buffer (TLB) or other cache, wherein the IOMMU begins performing page traversals before receiving a translation request for the virtual address from the I / O device. Some examples of the present technology involve initiating a pre-fetch from an I / O device in response to receiving a DMA stream setup command rather than in response to preparing to execute a DMA command. Some examples of the present technology involve a cache eviction policy that protects pre-fetched translations from being evicted from the cache before being accessed a specified number of times.

[0090] Figure 5 6 and the associated description provide a discussion of various operating environments in which examples of the present invention may be practiced. Figure 5 The devices and systems shown and discussed in FIG. 6 are for purposes of example and explanation only and are not limiting of the wide variety of computing device configurations that may be used to practice various aspects of the invention described herein.

[0091] Figure 5is a block diagram illustrating the physical components (i.e., hardware) of an example computing device 500 with which the present disclosure may be practiced. The computing device components described below may be applicable to the computing device 100 described above. In a basic configuration, the computing device 500 may include at least one processing unit 502 and a system memory 504. Depending on the configuration and type of the computing device, the system memory 504 may include, but is not limited to, volatile storage devices (e.g., random access memory), non-volatile storage devices (e.g., read-only memory), flash memory, or any combination of these memories. The system memory 504 may include an operating system 505 and one or more program modules 506 that are suitable for running software applications 550, such as an aggregator 108, an application 115 utilizing direct memory access, or a client application 120. For example, the operating system 505 may be suitable for controlling the operation of the computing device 500. In addition, various aspects of the present invention may be practiced in conjunction with a graphics library, other operating systems, or any other application program, and are not limited to any particular application or system. Figure 5 This basic configuration is shown by those components within dashed line 508. Computing device 500 may have additional features or functionality. For example, computing device 500 may also include additional data storage devices (removable and / or non-removable), such as, for example, magnetic disks, optical disks, or tapes. Such additional storage devices may be stored in a variety of formats. Figure 5 , which is illustrated by a removable storage device 509 and a non-removable storage device 510.

[0092] As described above, a number of program modules and data files may be stored in system memory 504. When program module 506 executes on processing unit 502, processes may be performed including, but not limited to, configuration and use of DMA streams between I / O devices and the main memory (or DRAM) portion of system memory 504. Other program modules may be used in accordance with examples of the present invention, and these program modules may include applications such as email and contacts applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided applications, and the like.

[0093] In addition, examples of the present invention may be practiced in circuits including discrete electronic components, in packaged or integrated electronic chips containing logic gates, in circuits utilizing a microprocessor, or on a single chip containing electronic components or a microprocessor. For example, examples of the present invention may be practiced via a system on a chip (SOC) wherein Figure 5Each component or many components of the components shown in can be integrated into a single integrated circuit. Such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") on a chip substrate as a single integrated circuit. When operated via the SOC, the functions described herein for pre-fetching the execution address translation can be operated via application-specific logic integrated with other components of the computing device 500 on a single integrated circuit (chip). The examples disclosed herein can also be practiced using other technologies capable of performing logical operations, such as, for example, AND gates (AND), OR gates (OR), and NOT gates (NOT), which include, but are not limited to, mechanical, optical, fluid, and quantum technologies. In addition, various aspects of the present invention can be practiced in a general-purpose computer or in any other circuit or system.

[0094] The computing device 500 may also have one or more input devices 512, such as a keyboard, a mouse, a pen, a voice input device, a touch input device, etc. Output device(s) 514, such as a display, a speaker, a printer, etc., may also be included. The above devices are examples, and other devices may be used. The computing device 500 may include one or more communication connections 516 that allow communication with other computing devices 518. Examples of suitable communication connections 516 include, but are not limited to, RF transmitter, receiver, and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0095] As used herein, the term computer-readable medium may include computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media for storage of information (such as computer-readable instructions, data structures, or program modules) implemented in any method and technology. System memory 504, removable storage device 509, and non-removable storage device 510 are all examples of computer storage media (i.e., memory storage devices). Computer storage media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage device, cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, or any other product that can be used to store information and can be accessed by computing device 500. Any such computer storage medium may be part of computing device 500. Computer storage media do not include carrier waves or other propagated data signals.

[0096] Communication media may be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" may describe a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0097] Fig. 6A and Figure 6B A mobile computing device 600 is shown, such as a mobile phone, smartphone, tablet computer, laptop computer, etc., upon which various aspects of the present invention may be practiced. Fig. 6A , an example of a mobile computing device 600 for implementing these aspects is shown. In a basic configuration, the mobile computing device 600 is a handheld computer having both input elements and output elements. The mobile computing device 600 typically includes a display 605 and one or more input buttons 610, which allow a user to enter information into the mobile computing device 600. The display 605 of the mobile computing device 600 can also be used as an input device (e.g., a touch screen display). If included, an optional side input element 615 allows further user input. The side input element 615 can be a rotary switch, a button, or any other type of manual input element. In alternative examples, the mobile computing device 600 can include more or fewer input elements. For example, in some examples, the display 605 may not be a touch screen. In an alternative example, the mobile computing device 600 is a portable telephone system, such as a cellular phone. The mobile computing device 600 can also include an optional keypad 635. The optional keypad 635 can be a physical keypad or a "soft" keyboard generated on the touch screen display. In various aspects, output elements include a display 605 for displaying a graphical user interface (GUI), a visual indicator 620 (e.g., a light emitting diode), and / or an audio transducer 625 (e.g., a speaker). In some examples, the mobile computing device 600 includes a vibration transducer for providing tactile feedback to the user. In yet another example, the mobile computing device 600 includes input and / or output ports for sending or receiving signals to or from external devices, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., an HDMI port).

[0098] Figure 6B6 is a block diagram illustrating the architecture of one example of a mobile computing device. That is, mobile computing device 600 may include system (i.e., architecture) 602 to implement some examples. In one example, system 602 is implemented as a "smartphone" capable of running one or more applications (e.g., browser, email, calendar, contact manager, messaging client, game, and media client / player). In some examples, system 602 is integrated into a computing device, such as an integrated personal digital assistant (PDA) and wireless phone.

[0099] One or more application programs 650 may be loaded into the memory 662 and run on or in association with the operating system 664. Examples of application programs include a phone dialer, an email program, a personal information management (PIM) program, a word processing program, a spreadsheet program, an Internet browser program, a messaging program, and the like. The system 602 also includes a non-volatile storage area 668 within the memory 662. The non-volatile storage area 668 may be used to store persistent information that should not be lost when the system 602 is powered off. The application programs 650 may use and store information in the non-volatile storage area 668, such as emails or other messages used by the email application, and the like. A synchronization application (not shown) also resides on the system 602 and is programmed to interact with a corresponding synchronization application residing on a host computer to keep the information stored in the non-volatile storage area 668 synchronized with the corresponding information stored at the host computer. As should be appreciated, other applications may be loaded into the memory 662 and run on the mobile computing device 600.

[0100] System 602 has a power supply 670, which may be implemented as one or more batteries. Power supply 670 may also include an external power source, such as an AC adapter or a powered docking cradle that replenishes or charges the batteries.

[0101] The system 602 may also include a radio 672 that performs transmit and receive radio frequency communication functions. The radio 672 facilitates wireless connectivity between the system 602 and the "outside world" via a communications carrier or service provider. Transmissions to and from the radio 672 are performed under the control of the operating system 664. In other words, communications received by the radio 672 may be distributed to applications via the operating system 664, and vice versa.

[0102] Visual indicator 620 can be used to provide visual notification, and / or can be used to produce an audio interface 674 of audible notification via audio transducer 625. In the example shown, visual indicator 620 is a light emitting diode (LED), and audio transducer 625 is a speaker. These devices can be directly coupled to power supply 670 so that when these devices are activated, even if processor 660 and other components may be turned off to save battery power, they will remain turned on for a duration specified by the notification mechanism. LED can be programmed to remain turned on indefinitely until the user takes action to indicate the power-on state of the device. Audio interface 674 is used to provide audible signals to the user and receive audible signals from the user. For example, in addition to being coupled to audio transducer 625, audio interface 674 can also be coupled to a microphone to receive audible input, such as to facilitate telephone conversations. System 602 can also include video interface 676, which implements the operation of onboard camera 630 to record still images, video streams, etc.

[0103] The mobile computing device 600 implementing the system 602 may have additional features or functionality. For example, the mobile computing device 600 may also include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage Figure 6B , shown by non-volatile storage area 668.

[0104] As described above, data / information generated or captured by mobile computing device 600 and stored via system 602 may be stored locally on mobile computing device 600, or data may be stored on any number of storage media that may be accessed by the device via radio 672 or via a wired connection between mobile computing device 600 and a separate computing device associated with mobile computing device 600, such as a server computer in a distributed computing network such as the Internet. As should be appreciated, such data / information may be accessed via mobile computing device 600, via radio 672, or via a distributed computing network. Similarly, such data / information may be conveniently transferred between computing devices for storage and use according to well-known data / information transfer and storage methods, including email and collaborative data / information sharing systems.

[0105] According to an example of the present technology, an input / output memory management unit (IOMMU) of a computing device is configured to: receive a pre-fetch message including a virtual address from a central processing unit (CPU) core of a processor of the computing device; perform a page traversal on the virtual address through a page table stored in a main memory of the computing device to obtain a pre-fetched translation of the virtual address to a physical address; and store the pre-fetched translation of the virtual address to the physical address in a translation lookaside buffer (TLB) of the IOMMU. The IOMMU may also be configured to: receive a translation request including a virtual address from an input / output device; and respond to the translation request using the pre-fetched translation of the virtual address to the physical address.

[0106] The IOMMU may also be configured to push the pre-fetched translation of the virtual address to the input / output device identified in the pre-fetch message without receiving a translation request from the input / output device. The IOMMU may control the input / output device to store the pre-fetched translation in an address translation cache of the input / output device.

[0107] The IOMMU may also be configured to: monitor a usage count of the prefetched translation stored in the TLB; and protect the prefetched translation from being evicted from the TLB when the usage count of the prefetched translation is less than a threshold count, wherein the threshold count is greater than zero.

[0108] The IOMMU may be further configured to store the prefetched translation in a host cache of the computing device.The IOMMU may be further configured to store the prefetched translation in the host cache in response to detecting that the prefetched translation is removed from the TLB.

[0109] According to one example of the present technology, an input / output device includes: a controller configured to perform a direct memory access (DMA) operation using a main memory of a computing device connected to the input / output device, the controller configured to: receive a DMA stream setup command including a virtual address; transmit a translation request to a system memory management unit (SMMU) to receive a pre-fetched translation of the virtual address to a physical address; and store the pre-fetched translation of the virtual address to the physical address in an address translation cache of the input / output device.

[0110] The controller may be further configured to: look up a pre-fetched translation of the virtual address to the physical address in the address translation cache; and perform a DMA operation using a main memory of the computing device based on the pre-fetched translation.

[0111] The controller may be further configured to: monitor a usage count of the prefetched translation stored in the address translation cache; and protect the prefetched translation from being evicted from the address translation cache when the usage count of the prefetched translation is less than a threshold count, wherein the threshold count is greater than zero.

[0112] According to an example of the present technology, a computing device includes: a processor, the processor including: a central processing unit (CPU) core; and a system memory management unit (SMMU) including a translation lookaside buffer (TLB); a main memory storing a page table; and an input / output (I / O) device connected to the processor and the main memory via an interconnect, the CPU core being configured to: transmit a direct memory access (DMA) stream setup command to the I / O device, the DMA stream setup command including a virtual address; the SMMU being configured to: prefetch a translation of the virtual address into a physical address; and store the prefetched translation of the virtual address into the physical address in the TLB; the I / O device being configured to: receive the DMA stream setup command; receive a prefetched translation of the virtual address from the SMMU; and perform a DMA operation using the main memory based on the prefetched translation of the virtual address.

[0113] The CPU core may be configured to transmit a prefetch message to the SMMU, the prefetch message including the virtual address included in the DMA stream setup command.

[0114] The SMMU may be configured to prefetch translations of virtual addresses into physical addresses in response to a prefetch message.

[0115] The SMMU may also be configured to: monitor a usage count of the prefetched translation stored in the TLB; and protect the prefetched translation from being evicted from the TLB when the usage count of the prefetched translation is less than a threshold count, wherein the threshold count is greater than zero.

[0116] The processor may include a host cache, and the SMMU may be further configured to store the prefetched translation in the host cache of the computing device. The SMMU may be further configured to store the prefetched translation in the host cache based on one or more attributes associated with the prefetched translation.

[0117] The SMMU may be configured to write-push the prefetched translations to the I / O device via the interconnect, and the I / O device may be configured to store the prefetched translations write-pushed by the SMMU in an address translation cache of the I / O device.

[0118] The I / O device may be configured to: in response to receiving a DMA stream setup command, send a translation request including a virtual address to the SMMU; and store the pre-fetched translation in an address translation cache (ATC) local to the I / O device.

[0119] The I / O device may also be configured to: look up a pre-fetched translation of the virtual address to the physical address in the ATC; and perform the DMA operation using the main memory of the computing device based on the pre-fetched translation.

[0120] According to one example of the present technology, a non-transitory computer-readable medium stores instructions that, when executed by a processor, cause the processor to enable a translation lookaside buffer (TLB) of an input / output memory management unit (IOMMU) by: sending a DMA stream setup command by a CPU core of the processor to an input / output (I / O) device, the DMA stream setup command including a virtual address; and sending a prefetch message including the virtual address from the CPU core of the processor to the IOMMU, so that the IOMMU: prefetches a translation of the virtual address into a corresponding physical address from a main memory; and stores the translation of the virtual address into the corresponding physical address in the TLB of the IOMMU.

[0121] For example, the above text refers to the block diagram and / or operation diagram of the method, system, and computer program product according to various aspects of the present invention, and describes various aspects of the present invention. The function / action noted in the box may not occur in the order shown in any flow chart. For example, depending on the function / action involved, the two blocks displayed continuously can actually be executed substantially simultaneously, or these blocks can sometimes be executed in reverse order. In addition, as used herein and in the claims, the phrase "at least one element in element A, element B, or element C" is intended to convey any one of the following: element A, element B, element C, element A and element B, element A and element C, element B and element C, and element A, element B and element C.

[0122] The description and illustration of the one or more examples provided in this application are not intended to limit or restrict the scope of the invention as claimed in any way.

[0123] The aspects, examples, and details provided in this application are considered to be sufficient to convey ownership and enable others to make and use the best mode of the claimed invention. The claimed invention should not be interpreted as being limited to any aspect, example, or detail provided in this application. Whether it is shown and described in combination, or shown and described individually, various features (structures and methods) are intended to be selectively included or omitted to produce examples with specific feature sets. Having provided the description and illustration of the present application, those skilled in the art can envision changes, modifications, and alternative examples that fall within the spirit of the broader aspects of the overall inventive concept embodied in this application, which changes, modifications, or alternative examples do not depart from the broader scope of the claimed invention.

Claims

1. An input / output memory management unit (IOMMU) of a computing device, wherein the IOMMU is configured to: receiving a prefetch message including a virtual address from a central processing unit (CPU) core of a processor of the computing device; performing a page walk of the virtual address through a page table stored in a main memory of the computing device to obtain a pre-fetched translation of the virtual address to a physical address; and The pre-fetched translation of the virtual address to the physical address is stored in a translation lookaside buffer (TLB) of the IOMMU.

2. The IOMMU according to claim 1, further configured to: receiving a translation request including the virtual address from an input / output device; and The translation request is responded to with the prefetched translation of the virtual address to the physical address.

3. The IOMMU of claim 1 , further configured to: write-push the pre-fetched translation of the virtual address to the input / output device identified in the pre-fetch message if no translation request is received from the input / output device.

4. The IOMMU of claim 3, wherein the IOMMU controls the input / output device to store the pre-fetched translation in an address translation cache of the input / output device.

5. The IOMMU according to claim 1, further configured to: monitoring a number of times the prefetched translation stored in the TLB is used; and The prefetched translation is protected from eviction from the TLB when the number of times the prefetched translation is used is less than a threshold number of times, wherein the threshold number of times is greater than zero.

6. The IOMMU of claim 1 , further configured to store the pre-fetched translation in a host cache [114] of the computing device.

7. The IOMMU of claim 6, further configured to store the prefetched translation in the host cache in response to detecting that the prefetched translation is moved out of the TLB.

8. A computing device comprising: A processor, the processor comprising: Central processing unit (CPU) core; and A system memory management unit (SMMU), wherein the SMMU includes a translation lookaside buffer (TLB); Main memory storing a page table; and input / output I / O devices connected to the processor and the main memory via an interconnect, The CPU core is configured as: Transmitting a direct memory access (DMA) stream setting command to the I / O device, wherein the DMA stream setting command includes a virtual address; The SMMU is configured as: pre-fetching the virtual address to physical address translation; and storing the pre-fetched translation of the virtual address to the physical address in the TLB; The I / O devices are configured as: Receiving the DMA stream setting command; receiving the prefetched translation of the virtual address from the SMMU; and A DMA operation is performed using the main memory based on the prefetched translation of the virtual address. 9 . The computing device of claim 8 , wherein the CPU core is configured to transmit a prefetch message to the SMMU, the prefetch message including the virtual address included in the DMA stream setup command.

10. The computing device of claim 9, wherein the SMMU is configured to prefetch the translation of the virtual address to the physical address in response to the prefetch message.

11. The computing device of claim 8, wherein the SMMU is further configured to: monitoring a number of times the prefetched translations stored in the TLB are used; and The prefetched translation is protected from eviction from the TLB when the number of times the prefetched translation is used is less than a threshold number of times, wherein the threshold number of times is greater than zero.

12. The computing device of claim 8, wherein the processor comprises a host cache [114], and wherein the SMMU is further configured to: The prefetched translations are stored in the host cache of the computing device.

13. The computing device of claim 12, wherein the SMMU is further configured to store the prefetched translation in the host cache based on one or more attributes associated with the prefetched translation.

14. The computing device of claim 8, wherein the SMMU is configured to push the pre-fetched translations to the I / O device via the interconnect, and Wherein the I / O device is configured to store the pre-fetched translations write-pushed by the SMMU in an address translation cache of the I / O device.

15. The computing device of claim 8, wherein the I / O device is configured to: In response to receiving the DMA stream setup command, sending a translation request including the virtual address to the SMMU; storing the pre-fetched translation in an address translation cache ATC local to the I / O device; looking up the prefetched translation of the virtual address to the physical address in the ATC; and A DMA operation is performed using the main memory of the computing device based on the prefetched translation.

Images