Memory access policies in peripheral devices based on memory usage characteristics

Abstract

A system includes a processing device and a peripheral device. The processing device is configured to allocate a memory region in a memory. The peripheral device is configured to set a memory access policy in response to usage characteristics of the memory region and to access data in the memory region using direct memory access (DMA) according to the memory access policy.

Description

FIELD OF THE INVENTION

[0001] The present invention relates generally to computing systems and, more particularly, to memory access by peripheral devices. BACKGROUND OF THE INVENTION

[0002] Computing systems often include a peripheral device connected to a host via a peripheral bus. Peripherals can include, for example, network adapters, storage devices, accelerators, and graphics processing units (GPUs). Peripheral buses, also referred to as system buses, can include, for example, Peripheral Component Interconnect Express (PCIe), Advanced Extensible Interface (AXI), Compute Express Link (CXL), Nvlink, or Nvlink Chip-to-Chip (Nvlink-C2C). In many computing systems, a peripheral device is capable of directly accessing memory regions in a system's memory after an initial memory registration using direct memory access (DMA). SUMMARY OF THE INVENTION

[0003] The invention is defined by the claims. To illustrate the invention, aspects and embodiments that may or may not fall within the scope of the claims are described herein.

[0004] An embodiment of the present invention described herein provides a system including a processing device and a peripheral device. The processing device is configured to allocate a memory region in a memory. The peripheral device is configured to set a memory access policy in response to usage characteristics of the memory region and to access data in the memory region using direct memory access (DMA) according to the memory access policy.

[0005] In one embodiment, the processing device is configured to provide context information describing the memory region to the peripheral device, and the peripheral device is configured to access data in the memory region according to the context information. In some embodiments, the memory access policy specifies a caching policy for caching portions of the data or portions of context information describing the memory region in the peripheral device. In some embodiments, the memory access policy specifies a prefetching policy for prefetching portions of the data or portions of context information describing the memory region in the peripheral device.

[0006] In example embodiments, the usage characteristics include one or more of: a pattern of addresses characterizing access to the memory region, an access frequency characterizing access to the memory region, an access direction characterizing access to the memory region, a location of the memory region, and whether the memory region is pinned or unpinned.

[0007] In some embodiments, the peripheral device is configured to derive the usage characteristics by tracking memory access transactions performed in the memory region. Additionally or alternatively, the processing device is configured to generate an indication indicating the usage characteristics of the memory region and provide the indication to the peripheral device, and the peripheral device is configured to adjust the memory access policy in response to the indication.

[0008] In some embodiments, the processing device is configured to select the hint from a defined set of hints, and the peripheral device is configured to select the memory access policy from a defined set of memory access policies. In an exemplary embodiment, one or both of the processing device and the peripheral device are configured to adaptively modify one or more of the following: one or more of the hints, one or more of the memory access policies, and a mapping between the hints and the memory access policies.

[0009] In one disclosed embodiment, the memory access policy and a mapping between the hint and the memory access policy are internal to the peripheral device and are not accessible to the processing device. In another embodiment, one or both of the processing device and the peripheral device are configured to provide an application programming interface (API) for specifying one or more of the following: one or more of the hints, one or more of the memory access policies, and a mapping between the hint and the memory access policy. In yet another embodiment, the hint is an ad hoc hint that is valid for a defined period of time or for one or more memory access transactions to be performed in the memory region.

[0010] Additionally, according to one embodiment of the present invention, a method is provided including allocating a memory region in a memory using a processing device. In a peripheral device, a memory access policy is set in response to usage characteristics of the memory region, and data in the memory region is accessed using direct memory access (DMA) according to the memory access policy.

[0011] In one embodiment, the method further comprises providing context information describing the memory region from the processing device to the peripheral device, wherein accessing data in the memory region is performed according to the context information. In some embodiments, the memory access policy specifies a caching policy for caching portions of the data or portions of context information describing the memory region in the peripheral device. In some embodiments, the memory access policy specifies a prefetching policy for prefetching portions of the data or portions of context information describing the memory region in the peripheral device.

[0012] In exemplary embodiments, the usage characteristics include one or more of: a pattern of addresses characterizing access to the memory region, an access frequency characterizing access to the memory region, an access direction characterizing access to the memory region, a location of the memory region, and whether the memory region is pinned or unpinned.

[0013] In some embodiments, the method further comprises deriving the usage characteristics by the peripheral device by tracking memory access transactions performed in the memory region. Additionally or alternatively, the method further comprises generating an indication indicating the usage characteristics of the memory region and providing an indication from the processing device to the peripheral device, and wherein setting the memory access policy is performed in response to the indication.

[0014] Generating the hint comprises selecting the hint from a defined set of hints, and wherein setting the memory access policy comprises selecting the memory access policy from a defined set of memory access policies. In an exemplary embodiment, the method further comprises adaptively modifying one or more of: one or more of the hints, one or more of the memory access policies, and a mapping between the hints and the memory access policies.

[0015] In one disclosed embodiment, the memory access policy and a mapping between the hint and the memory access policy are internal to the peripheral device and are not accessible to the processing apparatus. In another embodiment, the method further comprises providing an application programming interface (API) for specifying one or more of the following: the hint; the memory access policy; and a mapping between the hint and the memory access policy. In yet another embodiment, the hint is an ad hoc hint that is valid for a defined period of time or for one or more memory access transactions to be performed in the memory region.

[0016] Any feature of one aspect or embodiment may be applied to other aspects or embodiments in any appropriate combination. In particular, any feature of a method aspect or method embodiment may be applied to a device aspect or device embodiment, and vice versa.

[0017] The present invention will be more fully understood from the following detailed description of the embodiments thereof, taken together with the drawings, in which: BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram schematically illustrating a computing system comprising a host, a peripheral device, and a memory according to an embodiment of the present invention; the Fig. 2 and Fig.3 are flowcharts illustrating methods for setting a memory access policy in the peripheral device of Fig. 1 schematically illustrate embodiments of the present invention; and Fig. 4 is a block diagram schematically illustrating additional details of the host and peripheral device and setting memory access policies according to an embodiment of the present invention. DETAILED DESCRIPTION OF EMBODIMENTS OVERVIEW

[0018] In some computing systems, a host or other processing device provides a peripheral with the capability of directly accessing memory regions using DMA. For example, a network adapter can read data directly from system memory to generate outgoing packets for transmission. Similarly, a network adapter can write data received in incoming packets directly to system memory. Other types of peripherals, such as storage devices or graphics accelerators, can also read and write data to and from the system using DMA, without involving the host in the data transfer.

[0019] In one example implementation, the host and peripheral perform a process called "memory registration" to allow the peripheral to directly access a memory region. As part of the memory registration process, the host provides the peripheral with metadata, called a "context," that describes the memory region. The context may include, for example, a mapping between virtual addresses and physical addresses to be used by the peripheral. Once the memory region has been registered, the host typically issues memory access commands (e.g., read and write commands) to the peripheral, but the actual data transfer is performed directly between the peripheral and memory.

[0020] Different memory regions may differ significantly from one another in the manner in which they are used by the host and, consequently, in the manner in which the peripheral device is required to access them. For example, some memory regions may be accessed in a sequential manner, other memory regions may be accessed with some periodic or otherwise predictable pattern, and still other memory regions may be accessed in a random manner. Some memory regions are predominantly read from, whereas others are predominantly written to. Some memory regions are frequently accessed, whereas others are rarely or only once accessed.In the present context, characteristics that indicate the manner in which a memory region is accessed, such as the examples above, are referred to herein as "usage characteristics" of a memory region. Additional examples are discussed further below.

[0021] The usage characteristics of memory regions can be valuable to the peripheral when managing memory access, for example, when deciding on operations such as caching and prefetching. For example, data caching provides a significant performance improvement in a frequently accessed memory region. On the other hand, in a memory region that is read only once, data caching does not improve performance and unnecessarily wastes memory resources. As another example, prefetching data is highly effective in a sequentially accessed memory region, but is virtually useless if the memory is accessed randomly.

[0022] Typically, peripheral devices have no prior information regarding the usage characteristics of the memory regions they are required to access. This lack of information prevents peripheral devices from optimizing their memory access operations.

[0023] Embodiments of the present invention described herein provide improved techniques for memory access by peripheral devices. In the disclosed embodiments, the peripheral device is made aware of the usage characteristics of a memory region. The peripheral device sets a memory access policy for accessing the memory region based on the usage characteristics.

[0024] The peripheral device may obtain information regarding the usage characteristics of a memory region in various ways. In some embodiments, the host (or any other processing device external to the peripheral device) sends the peripheral a "hint" indicating the usage characteristics. Various types of hints are described herein. In other embodiments, the peripheral device learns the usage characteristics regardless of any hints by tracking memory access transactions performed in the memory region. The latter embodiments may be used with legacy hosts that do not support the disclosed techniques.

[0025] In various embodiments, the peripheral device may set various types of memory access policies, e.g., caching policies and / or prefetching policies, based on the usage characteristics of the memory region.

[0026] In some embodiments, the peripheral device includes (i) a list of possible hints, (ii) a list of possible memory access policies, and (iii) a mapping between the hints and the policies. Upon receiving a particular hint from the host, the peripheral device selects the appropriate memory access policy based on the mapping. The hints, the policies, and / or the mapping between them may be user-configurable and / or vary over time.

[0027] In some embodiments, the policies and policy hint mapping are not disclosed to the host. Such embodiments are useful, for example, for maintaining privacy and data security in a peripheral device serving multiple different hosts.

[0028] The disclosed techniques enable a peripheral device to align memory access operations (e.g., caching and prefetching) with actual usage characteristics per memory region. As a result, the peripheral device's computational and memory resources can be used more efficiently. SYSTEM DESCRIPTION

[0029] Fig.Figure 1 is a block diagram schematically illustrating a computing system 20 according to an embodiment of the present invention. System 20 includes a processing device, in this example a host 24, a peripheral device 28, and a memory 32, all connected to a peripheral bus 36.

[0030] In the embodiment of Fig. 1, the host 24 includes a central processing unit (CPU) 40 that performs the various computing tasks of the host, and a bus interface (bus I / F) 44 that connects the host to the bus 36. The peripheral device 28 includes circuitry 48 that performs the various processing tasks of the peripheral device, a bus I / F 52 that connects the peripheral device to the bus 36, and a cache memory 56.

[0031] Peripheral device 28 may, for example, include a network adapter that connects host 24 to a communications network (not shown). Examples of network adapters include InfiniBand™ Host Channel Adapter (HCA) and Ethernet Network Interface Controller (NIC). Alternatively, the peripheral device may include a storage device, such as a solid-state drive (SSD), a storage controller or storage accelerator, a graphics accelerator, such as a graphics processing unit (GPU), an accelerator that offloads certain computational tasks to the host, and / or any other suitable type of peripheral device.

[0032] The peripheral bus 36 may include, for example, a Peripheral Component Interconnect Express (PCIe) bus, an Advanced Extensible Interface (AXI) bus, a Compute Express Link (CXL) bus, an NvLink or NvLink Chip-to-Chip (NvLink C2C) bus, or any other suitable type of peripheral bus. The peripheral bus 36 is also referred to as a system bus.

[0033] In some embodiments, the CPU 40 of the host 24 defines one or more memory regions 60 in the memory 32. The CPU 40 then instructs the peripheral device 28 to access (e.g., read and / or write to) these memory regions using DMA. In the present example, two regions 60 are defined, labeled "REGION X" and "REGION Y."

[0034] For example, consider an embodiment in which peripheral device 28 is a network adapter that connects host 24 to a network. In this embodiment, a driver in CPU 40 may store data and metadata for outgoing packets in a memory region 60 and request peripheral device 28 to construct and send the packets. In response to the request, circuitry 48 of peripheral device 28 reads the data and / or metadata from the specified memory region 60 using DMA (i.e., directly over bus 36, without involving host 24) and constructs and sends the packets.

[0035] As another example, in an embodiment where peripheral device 28 is a memory device, CPU 40 may request the peripheral device to read specific data from memory and write the data to one of regions 60 in memory 32. In response, circuitry 48 of peripheral device 28 retrieves the requested data from memory and writes the data to the specified memory region 60 in memory 32 using DMA. Other examples of memory regions to which references may refer are work queues and completion queues.

[0036] To enable the peripheral device 28 to directly access a given region 60, the CPU 40 and the circuitry 48 perform a memory registration operation. As part of a memory registration, the CPU 44 sends the circuitry 48 a context 64 for the memory region 60 being registered. The context 64 includes various metadata describing the memory region. The context may, for example, include a virtual-to-physical address mapping to be used when accessing the memory region. The example of Fig.1 shows two contexts 64, labeled "CONTEXT X" and "CONTEXT Y," which describe the memory regions 60 "REGION X" and "REGION Y," respectively. Circuitry 48 stores contexts 64 locally. Once a memory region 60 has been registered in peripheral device 28, circuitry 48 is able to directly access the corresponding memory region 60 using DMA, including performing the necessary address translations.

[0037] The types of address translations (address mappings) specified in contexts 64 may vary depending on the system implementation. In some embodiments, an access to memory 32 involves a single translation of a virtual address (VA) to a physical address (PA). In these embodiments, a given context 64 typically stores a single type of address mapping. In other embodiments, e.g., in a virtualized environment, an access to memory 32 involves two address translations—a first translation from a guest virtual address (GVA) to a guest physical address (GPA), followed by a second address translation from the GPA to a host physical address (HPA, also known as a physical machine address (MPA) or physical system address (SPA). In such embodiments, a given context 64 typically stores both types of address mapping.

[0038] In some embodiments, the peripheral device in a virtualized environment may be required to access memory 32 using one of two modes of operation. ▪ Input / Output Memory Management Unit Translation (IOMMU Translation): In this mode, the host 24 includes an IOMMU. The peripheral accesses pinned memory over bus 36 using the GPA. The access is marked as UNTRANSLATED, signaling the IOMMU that additional translation is required. The IOMMU, in turn, translates the GPA to an HPA and completes the DMA access. ▪ Address Translation Services (ATS): In this mode, an ATS-capable peripheral can request translations from the IOMMU through an appropriate protocol (e.g., the ATS protocol for PCIe). The peripheral then caches the translations supplied by the IOMMU in an Address Translation Cache (ATC). Before performing the DMA access, the peripheral queries the ATC to obtain the HPA of the required access and performs a PCIe request marked as "TRANSLATED."

[0039] When accessing a given memory region 60, the circuitry 48 of the peripheral device 28 may perform operations such as caching and / or prefetching. In a typical prefetching operation, the circuitry 48 predicts one or more addresses in the region 60 from which data is likely to be read next and reads these addresses before receiving an explicit request from the host 24. In a typical caching operation, the circuitry 48 caches selected portions of data in the cache 56 to service future requests for data from the cache 56 instead of from the memory 32. Because the size of the cache 56 is limited, the circuitry 48 typically evicts selected portions of data from the cache 56 using appropriate eviction criteria.

[0040] In some embodiments, caching and prefetching may be applied to portions of contexts 64, not just data portions. For example, in many practical cases, the size of a context 64 is large, e.g., due to the size of the virtual-to-physical address mapping. In such cases, part or even all of the context 64 may be stored in memory 32. Only selected portions of the context (e.g., individual address translations) may be prefetched and / or cached in cache 56 as needed.

[0041] The host 24 may access different memory regions 60 in different ways, i.e., with different usage characteristics. Several non-limiting examples of usage characteristics include the following: ▪ The address pattern used to access the memory region. Examples of patterns can include a sequential and cyclic pattern, a cyclic but non-sequential pattern, a periodic stride pattern, or a random pattern. ▪ The frequency with which the memory region is accessed, for example, one-time access, low frequency access (infrequent access), or high frequency access (frequent access). ▪ The memory location in which the memory region is located. Examples of locations may include a neighboring peer device (a device located under the same bus switch), a non-neighboring peer device (a device located under the root complex but not under the same bus switch), a system location (e.g., system memory), or local memory (within the peripheral, not requiring communication over bus 36). ▪ The type or types of one or more memory access operations performed, for example, read-only, write-only, mixed read and write, predominantly read, or predominantly write. ▪ Mutable / Immutable - An indication of whether the system is expected to change memory translations or not. Characteristics can include, for example, pinned (immutable) access or unpinned (mutable) access.

[0042] Alternatively, any other suitable usage characteristics may be used. In some embodiments, the host 24 defines and stores usage characteristic records 68 for the various memory regions 60. The example of Fig. 1 shows two usage characteristic data sets 68, labeled “USE-CHAR. X” and “USE-CHAR. Y,” which are defined for the storage regions 60 “REGION X” and “REGION Y,” respectively.

[0043] To enable the peripheral device 28 to optimize its memory access operations (e.g., caching and / or prefetching), the CPU 40 provides hints to the circuitry 48 indicating the usage characteristics of the various memory regions 60. A hint provided for a particular memory region 60 may indicate a single usage characteristic (e.g., "access type is read-only") or a combination of usage characteristics (e.g., "access type is read-only and access pattern is random").

[0044] The hint may be provided to the peripheral device 28 as part of the initial memory registration of the memory region in question or at any other time. For a given memory region, the CPU 40 may send an updated hint that changes the usage characteristics of the region, if necessary. In some embodiments, a given hint may be an "ad hoc hint" valid for a specific period of time or for a specific upcoming transaction or group of transactions. An example of such an hint may be "the next N transactions are expected to be read transactions."

[0045] Upon receiving an indication for a particular memory region 60, circuitry 48 sets a memory access policy for that region based on the indication. In some embodiments, the memory access policy includes a caching policy that defines (i) criteria for caching pieces of data and / or metadata (e.g., pieces of context 64) in cache 56 and / or criteria for evicting pieces of data and / or metadata from cache 56. Additionally or alternatively, the memory access policy includes a prefetching policy that defines criteria for prefetching pieces of data and / or metadata from memory 32 to peripheral device 28.

[0046] As another example, when using ATS, circuitry 48 may prefetch address translation requests, in which case a hint-based prefetching policy may also be defined for this type of prefetching. Additionally or alternatively, a caching policy for caching address translations in the ATC may be defined based on hints. Further additionally or alternatively, the memory access policy may include any other suitable policy.

[0047] In various embodiments, the circuitry 48 may use any suitable technique for setting a memory access policy for a memory region 60 based on a received indication. In the embodiment of Fig.1, circuitry 48 includes a hint list 72 specifying a pool of possible hints and a policy list 76 specifying a pool of possible memory access policies. Circuitry 48 also stores a mapping (shown as arrows in the figure) that maps between hints on hint list 72 and policies on policy list 76. Generally, a given hint can be mapped to one or more policies, and a given policy can be mapped to one or more hints. Upon receiving a hint from CPU 40, circuitry 48 looks up hint list 72, finds the record of hint list 72 corresponding to the received hint, and sets the memory access policy (or policies) mapped to that hint record.

[0048] In some embodiments, circuitry 48 of peripheral device 28 may include a policy enforcer (not shown in the figure) configured to enforce the memory access policies.

[0049] The configuration of system 20, including the configurations of host 24, peripheral device 28, and memory 32, are example configurations chosen purely for conceptual clarity. Any other suitable configurations may be used in alternative embodiments. For example, in some embodiments, host 24 does not send notifications to peripheral device 28. Instead, circuitry 48 in peripheral device 28 learns the usage characteristics of one or more memory regions 60 by monitoring memory access transactions performed in memory 32. For example, circuitry 48 may track the addresses accessed in the memory region to determine whether the access pattern is sequential or random.As another example, the circuitry 48 may track the times or frequencies of access to the memory region to decide whether the memory region should be considered frequently accessed or infrequently accessed.

[0050] As another example, the memory regions in the embodiment of Fig. 1, and hints are sent from the CPU 40 of the host 24. In alternative embodiments, memory regions may be defined and hints may be sent from any other suitable processing device. Examples of processing devices include a CPU, a GPU, and a peer device such as a field-programmable gate array (FPGA). The embodiments described herein primarily relate to a host by way of example.

[0051] In still other embodiments, circuitry 48 may use a hybrid method that sets a memory access policy for a memory region based on a combination of one or more hints and self-learning. As another example, circuitry 48 may set one memory access policy based on hints and another memory access policy for the same memory region or for a different memory region based on self-learning.

[0052] As yet another example, the disclosed techniques do not mandate the use of a memory registry to define a memory region 60. For example, a memory region may simply be a set of addresses accessed by the peripheral device. In exemplary embodiments, the peripheral device 28 may access the memory 32 using techniques such as implicit on-demand paging (Implicit-ODP), unified virtual memory (UVM), or shared virtual addressing (SVA), all of which do not require memory registry. In such embodiments, the circuitry 48 in the peripheral device 28 may allocate a context 64 per address set having a particular usage characteristic. The boundaries of the memory region and / or the applicable usage characteristic may either be explicitly advertised or implicitly self-learned.

[0053] When schemes such as ODP, SVA, and UVM are used, a memory page may not be pinned to host memory and may thus be swapped through a disk and later moved to another memory location. When using such schemes, a hint may indicate how likely it is that the relevant memory is present on the host (or on a swap device). When using a swap device, circuitry 48 may issue an early paging request, e.g., using the PCIe Page Request Interface (PRI) or a vendor-specific event. Such an early request is also considered herein to be a type of prefetching, and prefetching policies for memory pages may thus be defined based on hints. PROCEDURE FOR SETTING SECURITY ACCESS POLICIES IN PERIPHERAL DEVICE

[0054] Fig.2 is a flowchart schematically illustrating a method for setting a memory access policy in the peripheral device 28 according to an embodiment of the present invention. The method begins with the CPU 40 of the host 24 registering a particular memory region 60 in a memory registration stage 80. As part of the memory registration process, the CPU 40 specifies a context 64 for the memory region. The context may include, for example, the range of addresses of the memory 32 contained in the memory region 60, any translation(s) of a virtual address to a physical address, and / or any other suitable metadata. The CPU 40 sends the context 64 to the circuitry 48 of the peripheral device 28.

[0055] In a hint stage 84, the CPU 40 sends the circuitry 48 an indication indicating one or more usage characteristics of the respective memory region. In a mapping stage 88, the circuitry 48 maps the received indication to a corresponding memory access policy. In an exemplary embodiment, the circuitry 48 uses a hint list 72, a policy list 76, and the mapping between them ( Fig. 1) to select the appropriate memory access policy. In a memory access stage 92, the circuits 48 access the relevant memory region (in memory 32) using the selected memory access policy.

[0056] Fig. 3 is a flowchart schematically illustrating a method for setting a memory access policy in the peripheral device 28 according to an alternative embodiment of the present invention. In contrast to the method of Fig.2 above includes the procedure of Fig. 3 self-learning of a usage pattern of a memory region instead of cues.

[0057] The procedure begins with a memory registration level 96, similar to level 80 of Fig. 2. In a usage pattern learning stage 100, the circuitry 48 of the peripheral device 28 tracks memory access transactions (e.g., read and / or write transactions) performed in the respective memory region. Based on the tracked transactions, the circuitry 48 derives one or more usage patterns for the memory region, e.g., whether the access pattern is random or sequential, whether the memory region is accessed once, rarely, or frequently, etc.

[0058] In a policy setting stage 104, circuitry 48 maps the derived usage pattern to an appropriate memory access policy. As an illustrative example, in response to determining that the memory region in question is accessed only once or infrequently, circuitry 48 may decide to set a "no caching" policy for the memory region. On the other hand, if the memory region appears to be accessed very frequently, circuitry 48 may prioritize caching data for that memory region.

[0059] In a memory access stage 108, the circuits 48 access the memory region using the selected memory access policy.

[0060] In various embodiments, the peripheral device 28 may perform any suitable mapping between cues (in the case of Fig. 1) or usage characteristics (in the case of Fig.3) and use storage access policies. Several non-limiting examples are given in the following table: Table 1: Example illustrations of notice / usage characteristics → guideline Note / usage characteristics Storage access policy Random access Do not prefetch Sequential access Prefetch Rarely accessed / accessed once Do not cache Frequently accessed Caching Latency sensitive Data / Control Low eviction priority, data cache stashing Context has been updated (memory region will probably be used soon) Prefetch Often not available Pre-issue of page requests ADDITIONAL DESIGNS AND VARIATIONS

[0061] Fig. Figure 4 is a block diagram schematically illustrating additional details of host 24 and peripheral 28 and the setting of memory access policies according to an embodiment of the present invention. In the present example, memory 32 (referred to as the "memory subsystem" in the figure) is part of host 24. The system is a virtualized system that includes both (i) an IOMMU 122 in host 24 and (ii) ATS using an ATC 136 in peripheral 28.

[0062] In the embodiment of Fig.4, the software running on the host 24 includes (i) one or more service users 110 and (ii) a service provider 114. A given service user 110 may include any suitable user software, e.g., an I / O driver, that issues memory access commands to the storage subsystem 32. Examples of service users include a virtual machine (VM), a hypervisor (HV) managing the VM, a bare metal operation, a container, etc. A service user 110 may have pass-through capabilities, i.e., it may issue certain control path and data path commands directly to the peripheral 28. The service provider 114 includes a privileged entity (e.g., an HV, a host driver, or on-device software, such as software running on a controller in the peripheral) that is authorized to access the peripheral 28 at the system level, e.g., B. at the level of the entire physical host.The service provider 114 may include, for example, a bare metal host driver or an HV device driver.

[0063] Service users 110 may create device-specific memory objects 118 that provide metadata regarding memory regions 60 to peripheral device 28. The metadata in objects 118 may include, for example, address ranges, virtual-to-physical address translations (e.g., VA-to-GPA or VA-to-HPA depending on the use case, as described above), ATS entries, Memory Translation Tables (MTTs), memory keys (mkeys), Process Address Space IDs (PASIDs), and / or any other suitable information. Peripheral device 28 uses the metadata in objects 118 and locally stored Memory Translation Tables (MTTs) 126 to map memory regions of the application's virtual memory space (or device driver's physical memory space) to physical memory space, whether HPA or GPA. In some embodiments, the address translations are defined using indirection, e.g.,using an mkey pointing to one or more other mkeys. This structure is labeled "KLM" in the figure.

[0064] In some embodiments, hints and usage characteristics are defined at a finer granularity than an entire memory region. For example, for a given memory region, different usage characteristics and / or different hints may be defined per VM, per bus transaction, per work queue element (WQE), per packet, per address range within a memory region, etc.

[0065] In some embodiments, some objects 118 may include hints indicating the usage characteristics of the memory regions they address, as described above. In some embodiments, service provider 114 may set memory access policies to be applied by peripheral device 28 to objects 118. In the present example, service provider 114 provides peripheral device 28 with hint list 72, policy list 76, and a hint → policy mapping that maps the hints to the policies.

[0066] As noted above, the peripheral device 28 may include a policy enforcer (not visible in the figure) that optionally enforces the memory access policies. The term "optional" in this context means that the policy enforcer has the option of overriding, modifying, or ignoring the request to enforce a particular policy. For example, the policy enforcer may be requested to enforce a particular policy following a hint, but at the same time learn that the actual usage characteristics of the memory region in question do not conform to the requested policy. In such a case, the policy enforcer may decide to enforce a different policy than requested or to refrain from enforcing any of the policies.The policy enforcer may apply such decisions to the single non-tuned memory region, to all memory regions supplied by the service user with that hint, to all hints supplied by the service user, and so on.

[0067] The policy enforcer may disclose the supported policies as capabilities to the service provider 114. In one embodiment, the service provider 114 specifies the hint list 72, the policy list 76, and the hint-policy mapping based on the supported policies as reported by the peripheral device 28.

[0068] An insertion in the lower part of Fig. Figure 4 lists examples of possible memory access policies and their attributes. Such examples include, but are not limited to, the following: ▪ Cache_enable: An indication of whether an entry (comprising context or data) should be cached in a low-level cache or not. ▪ Cache_id: An indication of which cache type to use for a specific object (e.g., per-priority cache, end-to-end cache - providing an end-to-end lookup for indirection objects, page table lookup cache, flat lookup cache, etc.). ▪ Caching policy - Define high / low eviction priority. ▪ Preprocessing - Performing some action involving the memory object before the actual transactions using the object are issued to the peripheral. Preprocessing actions may include prefetching contexts or relevant address translations, or performing ATS requests or paging requests. Note that the term "prefetching an address translation" in this context refers to either fetching a translation that has already been performed or performing a translation (e.g., servicing an ATS request) and returning the resulting translation.

[0069] Another example of a memory access policy is a policy that determines attributes of the system bus transactions themselves. For example, PCIe Transaction Layer Packets (TLP) contain processing hint (PH) and steering tag fields. These attributes can influence the target memory cache behavior. One example is "cache stashing": hints can be provided to inject data directly into L2 or L1 processing device caches instead of a system-level cache or DRAM. This type of policy is also considered a type of caching policy herein.

[0070] The configurations of the system 20, including the configuration of the host 24, the peripheral device 28 and the memory 32, as described in the Fig. 1 and Fig.4 are exemplary configurations chosen purely for conceptual clarity. Any other suitable configurations may be used in alternative embodiments. Elements not necessary for understanding the principles of the present invention have been omitted from the figures for clarity.

[0071] The various elements of host 24 and peripheral device 28 may be implemented in hardware, e.g., in one or more application-specific integrated circuits (ASICs) or FPGAs, in software, or using a combination of hardware and software elements. In some embodiments, certain functions, e.g., some or all of the functions of CPU 40 and / or circuitry 48, may be implemented using one or more general-purpose processors programmed in software to perform the functions described herein. The software may be downloaded to any of the processors in electronic form, e.g., over a network, or it may alternatively or additionally be provided and / or stored on non-transitory tangible media, such as magnetic, optical, or electronic storage.

[0072] Although the embodiments described herein primarily deal with address DMA transactions, the methods and systems described herein may also be used in other applications, such as remote DMA (RDMA). In such applications, the host 24 and memory 32 may be remote from each other (i.e., across a network), or the host 24 and peripheral 28 may be remote from each other, or the peripheral 28 and memory 32 may be remote from each other. As another example, in RDMA applications, the host that performs memory registration is not necessarily the same host that issues hints for that memory region.

[0073] It will therefore be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described above. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described above, as well as variations and modifications thereof that would become apparent to those skilled in the art upon reading the above description and that are not disclosed in the prior art.Documents incorporated by reference into the present patent application are to be considered an integral part of the application, except that if any terms in those incorporated documents are defined in a manner that contradicts the definitions given explicitly or implicitly in the present specification, only the definitions in the present specification should be considered.

[0074] It will be understood that aspects and embodiments described above are merely exemplary and that detailed modifications may be made within the scope of the claims.

[0075] Each device, method, and feature disclosed in the description and (if applicable) the claims and the drawings may be provided independently or in any adequate combination.

[0076] Reference numerals appearing in the claims are exemplary only and are not intended to limit the scope of the claims.

Claims

[1] System comprising: a processing device for allocating a memory region in a memory; and a peripheral device to set a memory access policy in response to usage characteristics of the memory region and to access data in the memory region using direct memory access (DMA) according to the memory access policy. [2] The system of claim 1, wherein the processing device is arranged to provide context information describing the memory region to the peripheral device, and wherein the peripheral device is arranged to access data in the memory region according to the context information. [3] The system of claim 1 or 2, wherein the memory access policy specifies a caching policy for caching portions of the data or portions of context information describing the memory region in the peripheral device. [4] The system of claim 1, 2 or 3, wherein the memory access policy specifies a prefetching policy for prefetching portions of the data or portions of context information describing the memory region in the peripheral device. [5] A system according to any preceding claim, wherein the usage characteristics comprise one or more of the following: a pattern of addresses that characterizes an access to the memory region; an access frequency that characterizes an access to the memory region; an access direction characterizing access to the memory region; a location in the storage region and whether the memory region is pinned or unpinned. [6] A system according to any preceding claim, wherein the peripheral device is arranged to derive the usage characteristics by tracking memory access transactions performed in the memory region. [7] A system according to any preceding claim, wherein the processing device is arranged to generate an indication indicating the usage characteristics of the memory region and to provide the indication to the peripheral device, and wherein the peripheral device is arranged to adjust the memory access policy in response to the indication. [8] The system of claim 7, wherein the processing device is arranged to select the hint from a defined set of hints, and wherein the peripheral device is arranged to select the memory access policy from a defined set of memory access policies. [9] The system of claim 8, wherein one or both of the processing device and the peripheral device are arranged to adaptively modify one or more of the following: one or more of the clues; one or more of the storage access policies and a mapping between the hints and the storage access policies. [10] The system of claim 7, wherein the memory access policy and a mapping between the hint and the memory access policy are internal to the peripheral device and are not accessible to the processing device. [11] The system of claim 7, wherein one or both of the processing device and the peripheral device are arranged to provide an application programming interface (API) for specifying one or more of the following: the note; the storage access policy and a mapping between the hint and the storage access policy. [12] The system of any one of claims 7 to 11, wherein the indication is an ad hoc indication valid for a defined period of time or for one or more memory access transactions to be performed in the memory region. [13] Method comprising: allocating a memory region in a memory using a processing device; and in a peripheral device, setting a memory access policy in response to usage characteristics of the memory region and accessing data in the memory region using direct memory access (DMA) according to the memory access policy. [14] The method of claim 13, further comprising providing context information describing the memory region from the processing device to the peripheral device, wherein accessing data in the memory region is performed according to the context information. [15] The method of claim 13 or 14, wherein the memory access policy specifies a caching policy for caching portions of the data or portions of context information describing the memory region in the peripheral device. [16] The method of claim 13, 14 or 15, wherein the memory access policy specifies a prefetching policy for prefetching portions of the data or portions of context information describing the memory region in the peripheral device. [17] A method according to any one of claims 13 to 16, wherein the usage characteristics comprise one or more of the following: a pattern of addresses that characterizes an access to the memory region; an access frequency that characterizes an access to the memory region; an access direction characterizing access to the memory region; a location in the storage region and whether the memory region is pinned or unpinned. [18] The method of any one of claims 13 to 17, further comprising deriving the usage characteristics by the peripheral device by tracking memory access transactions performed in the memory region. [19] The method of any one of claims 13 to 18, further comprising generating an indication indicating the usage characteristics of the memory region and providing an indication from the processing device to the peripheral device, and wherein setting the memory access policy is performed in response to the indication. [20] The method of claim 19, wherein generating the hint comprises selecting the hint from a defined set of hints, and wherein setting the memory access policy comprises selecting the memory access policy from a defined set of memory access policies. [21] The method of claim 20, further comprising adaptively modifying one or more of the following: one or more of the clues; one or more of the storage access policies and a mapping between the hints and the storage access policies. [22] The method of claim 19, wherein the memory access policy and a mapping between the hint and the memory access policy are internal to the peripheral device and are not accessible to the processing device. [23] The method of claim 19, further comprising providing an application programming interface (API) for specifying one or more of: the note; the storage access policy and a mapping between the hint and the storage access policy. [24] The method of any one of claims 19 to 23, wherein the indication is an ad hoc indication valid for a defined period of time or for one or more memory access transactions to be performed in the memory region.

Images