Address translation service management

Abstract

Address translation service management techniques are described. These techniques are based on metadata that may be used to provide a prompt as an insight to memory access, and based thereon, optimize the use of a translation lookaside buffer to control which entries are maintained in a queue and manage address translation requests.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. patent application serial number 17 / 949,716, filed on September 21, 2022, and entitled “Address Translation Service Management,” the entire disclosure of which is hereby incorporated by reference herein in its entirety. Background Art

[0003] Virtual memory is a technique for managing the use of shared physical memory and by doing so supports a variety of functions. Examples of this functionality include extending the amount of memory available to applications beyond the amount of memory actually available in physical memory, supporting heterogeneous system architectures, offloading memory management from applications and the operating system, using various types of memory without the knowledge of the application, supporting memory optimization, addressing memory fragmentation, and so on.

[0004] To this end, a memory management unit is used to manage the conversion between virtual addresses and physical addresses. However, conventional techniques have encountered challenges due to the increasing functionality of systems and devices that employ virtual memory. Examples include delays caused by systems with multiple virtual machines, which hinder the operation of devices that implement these systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The detailed description is given with reference to the accompanying drawings.

[0006] Figure 1 is a block diagram of a non-limiting example system configured to employ address translation service management techniques.

[0007] Figure 2 is configured to Figure 1 A block diagram of a non-limiting example system in which a memory management agent and a translation lookaside buffer are implemented as part of a heterogeneous system architecture.

[0008] Figure 3 is shown as an accelerator Figure 2 A block diagram of a non-limiting example system of a configuration of a parallel processing system.

[0009] Figure 4 is a block diagram illustrating a non-limiting example system for utilizing reuse distance information for translation prefetching techniques by an address translation agent.

[0010] Figure 5 is a block diagram illustrating a non-limiting example system for irregular prefetching enabled by data structure analysis.

[0011] Figure 6Depicted is a procedure in an example implementation in which a host generates a command packet to include metadata.

[0012] Figure 7 Depicted are procedures in an example implementation of receiving a command packet including metadata and using the metadata to control inclusion of entries in a translation lookaside buffer. DETAILED DESCRIPTION

[0013] Virtual memory is a technology used to extend the functionality of a device that can be used to manage data storage. To support this, the task of a memory management unit is to perform conversions between virtual addresses and physical addresses. In one example, this can be used to implement a heterogeneous system architecture in which a central processing unit and a parallel processing unit share physical memory using virtual addresses as part of a unified virtual memory space. Other examples can also be envisioned (for example) to extend the amount of storage available to applications, offload memory management, support the use of multiple different types of memory, support memory optimization, address memory fragmentation, and the like.

[0014] To support address translation, the memory management unit includes a translation lookaside buffer having entries that define translations between virtual addresses and physical addresses (i.e., mappings between addresses). The translation lookaside buffer can be configured in a variety of ways, examples of which include an address translation cache (ATC), a content addressable memory (CAM) table, a priority queue, and the like. However, conventional techniques are challenged to extend the functionality of devices that rely on address translation. Examples of this situation involve the use of unified virtual memory and demand paging for heterogeneous system architectures, which encounter delays in translation and hinder translation lookaside buffer (TLB) operations in real-world scenarios.

[0015] To address these issues, address translation service (ATS) management techniques are described. These techniques are based on metadata that can be used to provide hints as insight into memory accesses, and based on this, optimize the use of ATS requests to control which address translations are maintained in the translation lookaside buffer (TLB) and which memory pages are stored in physical memory.

[0016] In one example, a host (e.g., an operating system or application executing on a central processing unit) generates a command packet. The command packet in this example includes instructions (e.g., commands, memory instructions to load or write data, instruction sequences, etc.), virtual addresses, and metadata. The metadata describes memory access characteristics that can then be used to control the use of a memory management agent. Once generated, the command packet is sent by the host to a command queue of the accelerator.

[0017] The command packet is received by the command queue of the accelerator. For example, the command processor of the accelerator processes the command packet using firmware to interpret the fields including metadata. The metadata is then used by a memory management agent (e.g., as part of an address translation agent of a memory management unit) to control functions involving ATS. The functions include managing entries in the translation lookaside buffer, ATS requests, translation prefetches, and memory page migrations.

[0018] In one example, management is based on access patterns associated with memory accesses. For example, metadata may be configured to indicate irregular memory access patterns with high reuse distances. Therefore, entries associated with irregular memory access patterns are restricted from being stored in the TLB. This is because the likelihood that an entry will be reused is relatively low, and the likelihood of replacing entries in the queue with a higher likelihood of reuse increases accordingly. Similarly, metadata indicating dense memory access patterns causes the memory management agent to store corresponding entries in the TLB. Further, the metadata also supports pre-fetching entries into the TLB (as further described below), for example by exploiting insights into reuse distances in dense access patterns to obtain additional entries.

[0019] In another example, metadata is used by the memory management agent to implement quality of service considerations. For example, this can be used to support priority information, deadlines for individual tasks to meet corresponding service level agreements (SLAs), etc. In another example, the use of metadata by the memory management agent utilizes pre-fetch techniques for memory translations, thereby reducing translation latency and enabling accurate and timely access to entries in the TLB, thereby reducing translation overhead. In another example, the use of metadata by the memory management agent utilizes pre-fetch techniques for memory pages, thereby reducing page fault latency and enabling accurate and timely access to memory pages, thereby reducing page fault overhead.

[0020] In another example, ATS requests are managed to improve operational efficiency. For example, in a scenario where a device sends a request to an I / O memory management unit (IOMMU) to retrieve a request, these requests are batched to avoid the overhead of sending individual requests, such as by sending a single request that requires multiple conversions. Additionally, in another scenario, requests from multiple commands requesting access to the same page are batched. For example, this is performed in response to detecting when a request matches a request already in progress and combining the requests. The response to the initial request may also serve the second request. In another scenario, the initial request is delayed for a period of time to detect whether a subsequent request requests access to the same memory, and if so, the subsequent request is combined before being sent.

[0021] In a specific implementation, a command processor is also used to operate with the memory management agent. The command processor can be programmed to implement policies to assist the memory management agent in managing TLB, address translation service (ATS) requests, and page migration. These policies can be configured to support domain-specific optimization and device reuse (e.g., accelerator reuse) across applications.

[0022] In one such example, a memory management agent is implemented as part of a domain-specific accelerator, such as for machine learning. The domain-specific accelerator is configurable for targeted functionality, thereby supporting intelligent address translation management. However, in real-world scenarios, variations are encountered in virtual memory usage based on input data, individual core characteristics, etc. Therefore, configuration of a programmed command processor in conjunction with the operation of the memory management agent allows domain-specific firmware to customize address translation functionality for specific use cases through policies executed by the command processor. This also supports the ability of devices implementing these features to evolve as application functionality evolves, thereby improving device operation. Various other examples are also contemplated, which are described below and illustrated using corresponding figures.

[0023] In some aspects, the technology described herein relates to a method comprising: receiving, by a memory management agent, a command packet comprising a virtual address and metadata describing memory access characteristics; and controlling, by the memory management agent based on the metadata, which entries are maintained in a translation lookaside buffer (TLB) that supports translation between the virtual address and a physical address of a physical memory.

[0024] In some aspects, the techniques described herein relate to a method in which metadata describes usage of data associated with a virtual address.

[0025] In some aspects, the techniques described herein relate to a method in which metadata indicates an irregular memory access pattern and controlling includes limiting storage of entries associated with the irregular memory access pattern in a translation lookaside buffer.

[0026] In some aspects, the techniques described herein relate to a method in which metadata indicates a dense memory access pattern and control includes storing entries associated with the dense memory access pattern in a translation lookaside buffer.

[0027] In some aspects, the techniques described herein relate to a method in which metadata indicates a domain type and control is based on the domain type.

[0028] In some aspects, the techniques described herein relate to a method wherein controlling includes invalidating at least one of the entries in the translation lookaside buffer, replacing at least one of the entries in the translation lookaside buffer, or controlling when at least one of the entries is stored in the translation lookaside buffer.

[0029] In some aspects, the techniques described herein relate to a method in which controlling includes prefetching at least one of the entries to a translation lookaside buffer based on metadata.

[0030] In some aspects, the techniques described herein relate to a method in which controlling includes controlling which entries are maintained in a translation lookaside buffer based on quality of service.

[0031] In some aspects, the techniques described herein relate to a method wherein controlling includes controlling page migration, page placement, page prefetching, or page removal.

[0032] In some aspects, the techniques described herein relate to a method in which a virtual address is part of a unified virtual memory space in which a central processing unit and a parallel computing unit share physical memory as part of a heterogeneous system architecture.

[0033] In some aspects, the technology described herein relates to a device comprising: a translation lookaside buffer configured to store entries for translations between virtual addresses and physical addresses; and an address translation agent configured to manage which entries are maintained in the translation lookaside buffer based on metadata received via a command packet, the metadata describing memory access characteristics.

[0034] In some aspects, the technology described herein relates to a device that further includes a physical memory accessible via a physical address, and wherein the virtual address implements a unified virtual memory space in which a central processing unit and a parallel computing unit share the physical memory as part of a heterogeneous system architecture.

[0035] In some aspects, the techniques described herein relate to a device that further includes a command processor configured to receive a command packet having metadata, an instruction, and a virtual address and to send the metadata to an address translation agent.

[0036] In some aspects, the techniques described herein relate to a device in which a command processor is configured as a programmable microcontroller to implement at least one policy that controls sending of metadata to an address translation agent.

[0037]

[0013] In some aspects, the techniques described herein relate to a device in which an address translation agent is configured to manage address translation requests.

[0038] In some aspects, the techniques described herein relate to a device in which an address translation agent is configured to manage at least one address translation request by deferring sending of at least one of the address translation requests.

[0039] In some aspects, the techniques described herein relate to a device in which an address translation agent is configured to batch multiple address translation requests into a single request.

[0040] In some aspects, the techniques described herein relate to an apparatus in which a plurality of address translation requests correspond to a plurality of commands, respectively.

[0041] In some aspects, the technology described herein relates to a method comprising: generating, by a host, a command packet comprising a virtual address and metadata describing memory access characteristics; and sending, by the host, the command packet for receipt by a memory management agent, the metadata being configured to control the memory management agent's management of entries in a translation lookaside buffer that support translation of virtual addresses and physical addresses.

[0042] In some aspects, the techniques described herein relate to a method in which a command packet includes instructions configured to load data from a physical memory.

[0043] Figure 1 1 is a block diagram of a non-limiting example system 100 configured to employ metadata-based translation lookaside buffer (TLB) management techniques. System 100 includes a host 102, a memory management unit 104, and physical memory 106 (e.g., volatile or non-volatile memory). Host 102 in this example represents the functionality of initiating command packet 108. Command packet 108 may be configured for use in a variety of scenarios, examples of which include use in a point-to-point bus having a serial link connecting a corresponding device to a root complex, such as with respect to Figure 3 Further description.

[0044] The command packet 108 includes instructions 110, such as memory instructions to load data from or write data to the physical memory 106 based on the corresponding virtual address 112. To this end, the memory management agent 114 includes entries 116, which are shown as being maintained in a translation lookaside buffer (TLB) 118. The entries 116 define a translation between the virtual address 112 and the corresponding physical address 120 in the physical memory 106. In other words, the entries 116 define a mapping between the virtual address 112 and the corresponding physical address 120. This can be used to support a variety of functions, examples of which include, for example, Figure 2Further description of the heterogeneous system architecture.

[0045] In the example shown, the host 102 includes a metadata generation module 122. The metadata generation module 122 is configured to generate metadata 124 for inclusion as part of the command packet 108 (e.g., the payload of the packet). The metadata 124 describes memory access characteristics that can be used to describe characteristics of the data that is the subject of the command packet, provide insights into potential subsequent memory accesses, etc. In this way, the metadata 124 acts as a "hint" provided by the host 102 about memory accesses. The memory management agent 114 is configured to utilize the insights provided by the metadata 124 to manage which entries 116 are maintained in the TLB 118.

[0046] Examples of functionality that may be used to manage access based on metadata 124 are shown as a quality of service (QoS) management module 126, a prefetch management module 128, a pattern management module 130, and a request management module 132. QoS management module 126 is configured to implement quality of service considerations to, for example, support priority information, deadlines for tasks that meet corresponding service level agreements (SLAs), etc. Prefetch management module 128 supports prefetching entries 116 into TLB 118. Pattern management module 130 may be configured to manage entries based on patterns of memory accesses observed and / or likely to be encountered as part of subsequent memory accesses.

[0047] The request management module 132 represents functionality for managing address translation service requests, which are used to initiate corresponding translation operations. In the event that a device sends a request to an I / O memory management unit (IOMMU) to retrieve a request, multiple of these requests are batched by the request management module 132 to reduce the overhead otherwise encountered when sending individual requests, such as by batching multiple address translation requests into a single request. This can also be used to batch multiple address translation requests that correspond to, for example, multiple commands belonging to the same page. In one example, this can be performed by detecting when an address translation request matches an address translation request that is already "in progress" and combining these requests. In another example of request management, the sending of an address translation request is delayed to determine whether a subsequent request involves access to the same memory, and then the subsequent request is batched as described above.

[0048] Thus, the metadata 124 provides a degree of control to the host 102 to provide insight and improve address translation operations. Further, the memory management agent 114 is configured to reduce translation delays through the use of the metadata 124, as further described in the following examples.

[0049] Figure 2 is configured to Figure 1Block diagram of a non-limiting example system 200 in which a memory management agent and TLB are implemented as part of a heterogeneous system architecture. System 200 includes a device 202 that can be configured in a variety of ways. By way of example and not limitation, device 202 configurations include computing devices, servers, mobile devices (e.g., wearable devices, mobile phones, tablets, laptops), processors (e.g., graphics processing units, central processing units, and accelerators), digital signal processors, disk array controllers, hard drive host adapters, memory cards, solid state drives, wireless communication hardware connections, Ethernet hardware connections, switches, bridges, network interface controllers, and other device configurations. It should be understood that in various specific implementations, the device 202 and implementations may be configured in a variety of ways without departing from the spirit or scope of the described technology. Figure 1 The devices of the host 102, memory management unit 104, and physical memory 106 may be configured as any one or more of those listed just above and / or various other devices.

[0050] Device 202 is configured to implement a heterogeneous system architecture. In the example shown, the architecture includes a central processing unit 204, a parallel processing system 206, and physical memory 106 that are communicatively coupled to each other via a common bus system 208. Bus system 208 supports shared use of physical memory 106 and tasks by implementing a unified virtual memory space 210 of virtual addresses 212. For example, bus system 208 may be configured as a point-to-point bus with serial links connecting corresponding devices to a root complex, an example of which includes Peripheral Component Interconnect Express (PCIe).

[0051] The central processing unit 204 is configured to support the execution of an operating system 214 and an application 216 acting as the host 102 to initiate the command packet 108. The parallel processing system 206 can be configured in a variety of ways, examples of which include a graphics processing unit (GPU), an accelerator, a digital signal processor (DSP), an application specific integrated circuit (ASIC), etc. The parallel processing system 206 includes, for example, a parallel computing unit 218 that implements the logical operations of the parallel processing system 206.

[0052] In this example the unified virtual memory space 210 is implemented such that the central processing unit 204 and the parallel processing system 206 share page tables. This enables data exchange by sharing pointers in what is known as a "zero copy" operation, rather than copying data between devices as involved in other architectures, and thus improves device operation. However, as previously described, conventional support for unified virtual memory and demand paging results in slow translation and hinders translation lookaside buffer operation in some real-world scenarios.

[0053] To address this issue, the device 202 incorporates a memory management agent 114 as part of the parallel processing system 206. The memory management agent 114 supports a flexible mechanism that can be configured to address considerations on a domain-specific basis and to address long-term potential address translations in systems with a large number of virtual machines targeting the parallel processing system 206 (e.g., accelerators) in a heterogeneous system.

[0054] Figure 3 is shown as an accelerator Figure 2 1 is a block diagram of a non-limiting example system 300 of a configuration of a parallel processing system 206. The parallel processing system 206 includes an address translation agent 302 implementing a memory management agent 114, an address translation queue 118, a command processor 304, a command queue 306, and an accelerator compute unit 308, which are communicatively coupled 310 to each other (e.g., via a bus).

[0055] In this example, metadata 124 regarding the use of TLB 118 is embedded in command packet 108. In a specific implementation, command packet 108 is a Heterogeneous System Architecture (HSA) packet, but packets for use with any compute agent are also contemplated.

[0056] The command processor 304 processes the command packet 108 using the corresponding firmware to interpret the fields in the packet as metadata 124. The command processor 304 then sends information to the address translation agent 302 to help make an informed decision about providing address translation services for the device 202.

[0057] The address translation agent 302 is configured to manage the TLB 118. This includes controlling when entries 116 are invalidated, how entries 116 are replaced, and / or when entries 116 are cached. Because the accelerator often executes a kernel for a given domain, and therefore the programmer or intelligent runtime has insight into expected virtual memory usage, the metadata 124 is configured to provide hints that are passed to the address translation agent 302 to convey insight into how to manage the TLB 118.

[0058] For example, linear algebra accelerators can be used across a variety of domains that rely on matrix and vector multiplication (e.g., graph analysis, machine learning, scientific applications, etc.). However, these domains differ in the degree of regularity of memory access. Sparse linear algebra (SpLA) kernels, such as sparse matrix-vector multiplication (e.g., used in graphics and machine learning applications), have highly irregular memory access patterns due to the inherent sparsity of the matrix. However, dense linear algebra kernels are highly regular.

[0059] Therefore, when the SpLA kernel is dispatched, the command packet 108 embeds information as part of the metadata 124, which indicates to the address translation agent 302 that the workload has irregular memory accesses with a high reuse distance. The address translation agent 302 then dynamically determines in this example that the SpLA translations are restricted from caching because these entries are likely to replace other entries with corresponding translations that are more useful. On the other hand, the entries 116 corresponding to the dense kernel translations are cached because the entries 116 have the possibility of being reused.

[0060] In addition, in addition to the kernel type, the address translation agent 302 can also be configured to optimize based on the input data. For example, in a dense linear algebra kernel, the reuse distance of the pages in the virtual memory can be determined based on the size and structure of the matrix. For example, for a relatively large matrix with a large number of rows, an approximate row reuse distance can be determined. Therefore, the entry 116 of the conversion corresponding to the row no longer used is discarded from the TLB 118 to make room for the entry 116 of the conversion from the upcoming row. In a specific implementation, this technology is performed in tandem with the conversion prefetch.

[0061] Figure 4 is a block diagram illustrating a non-limiting example system 400 for utilizing reuse distance information for translation prefetching techniques by the address translation agent 302. In deep neural network (DNN) training, memory accesses involve different patterns based on whether a forward pass or a backward pass is encountered.

[0062] For example, in the virtual memory pages 402, the pages in the initial tier of the forward pass 404 have a higher reuse distance than the pages in the later tiers. Therefore, the indication of this pattern and scenario in the metadata 124 can be used by the address translation agent 302 to discard the entries 116 for the pages in the initial tier. The entries 116 for the pages in the later tiers are maintained by the address translation agent 302 in the address translation queue 118. For the reverse pass 406, the entries 116 for these pages are refined by the address translation agent 302, thereby optimizing address translation and device operation.

[0063] Figure 5 1 is a block diagram of a non-limiting example system 500 illustrating irregular prefetching enabled by data structure analysis. The address translation agent 302 may also be configured to make intelligent translation prefetch requests. Following the above example, dense linear algebra kernels typically use regular memory accesses, and since continuous row and column data of the corresponding matrix are accessed, they may be considered as streaming accesses to continuous memory pages. Therefore, the dispatch packet may be configured to include metadata 124 that notifies the address translation agent 302 that the kernel behaves as a streaming kernel. In response, the address translation agent 302 initiates prefetching of continuous pages from a computational accelerator near a page boundary as memory accesses.

[0064] Alternatively, due to sparsity, SpLA memory accesses are typically irregular, thus making these accesses more difficult to predict. For example, operations on matrices stored in compressed sparse row (CSR) format are highly irregular due to the seemingly random access pattern to column vectors. However, CSR data structures are typically static and known. Therefore, in this example, the address translation agent 302 directly interprets the CSR column index array as an advance prefetch translation.

[0065] In the example shown, a first column, a second column, a third column, a fourth column, and a fifth column of data 502(1), 502(2), 502(3), 502(4), 502(5) are shown. Entries for the first column, the third column, and the fourth column 502(1), 502(3), 502(4) are maintained in the TLB 118. The address translation agent 302 then determines that the second column 502(2) and the fifth column 505(5) are located at page boundaries, and based on this, initiates a prefetch 504 of these columns.

[0066] Similarly, for the DNN training example above, entries describing translations of pages used in later layers are kept in TLB 118, and entries with translations corresponding to earlier layers are discarded. Additionally, in the reverse pass 406, entries corresponding to earlier layer page translations are pre-fetched in time.

[0067] In a virtualized accelerator environment, several processes / kernels are usually executed simultaneously on the accelerator at the same time. Therefore, when focusing on quality of service, managing TLB 118 requests for fairness and priority can be used to achieve the benefits of performance and throughput. As an example, priority information or deadlines for tasks that meet corresponding service level agreements (SLAs) are provided to address translation agent 302 via metadata 124. The QoS management module 126 of address translation agent 302 then uses this information from metadata 124 to schedule and batch the entries 116 corresponding to the conversion requests.

[0068] For example, for tasks with high priority or short SLA deadlines, these requests are eagerly scheduled. In addition, the memory management agent 114 can use the priority along with other information as previously described to manage the TLB 118. For example, entries 116 corresponding to higher priority requests are given priority by the address translation agent 302 for caching in the TLB 118. In the case of tasks with lower priority or with longer SLA deadlines, the address translation agent 302 batches the entries 116 corresponding to these translation requests. For example, the address translation agent 302 opens a window for translation requests, and when the SLA deadline is approaching (e.g., based on a threshold), the memory management agent 114 closes the window and sends the entries corresponding to each of the translation requests in a single request.

[0069] The functionality of the memory management agent 114 may be configured using fixed function hardware. Figure 3 As described, the functionality of the memory management agent 114 may also be configured in part by including a programmable command processor 304 (e.g., using a RISC-V CPU that executes domain-specific firmware). For example, the command processor 304 may be configured as a heterogeneous system architecture agent, where the primary heterogeneous system architecture agent uses an Architecture Queuing Language (AQL) agent dispatch packet to send a service request to the command processor 304.

[0070] While general purpose and programmable accelerators (e.g., GPUs) can be configured to support a variety of use cases, domain specific accelerators are configured for targeted functionality, thus supporting intelligent address translation queue 118 management. However, slight variations in virtual memory usage are encountered in real world scenarios based on input data or individual core characteristics. Therefore, the programmability of the command processor 304 combined with the operation of the memory management agent 114 of the address translation agent 302 supports the use of domain specific firmware on the command processor 304 to customize the TLB 118 functionality for specific use cases. In addition, as applications evolve, the programmability of the command processor 304 also evolves.

[0071] Thus, metadata-based TLB management techniques support improved management of address translation queue 118 entries 116 and reduced translation latency. These techniques also support improved accuracy and timeliness by using prefetching of entries and corresponding translations, which reduces translation overhead. Further, the ability to program the policy 312 of the command processor 304 to operate in conjunction with the memory management agent 114 supports domain-specific optimizations and accelerator reuse across applications.

[0072] Figure 6A process 600 in an example implementation in which a command packet is generated by a host to include metadata is depicted. A command packet is generated by the host. The command packet includes a virtual address and metadata describing memory access characteristics (block 602). The command packet may also be configured to include instructions, instruction series, commands, and the like. For example, a command packet 108 is received by a memory management unit 104 from a host 102. The command packet 108 includes metadata 124.

[0073] The command packet is sent by the host for receipt by the memory management agent. The metadata is configured to control the management of entries in the TLB by the memory management agent. These entries support translation of virtual addresses to physical addresses (block 604). For example, a command packet 108 is sent by the host 102 to the memory management unit 104. The command packet 108 includes metadata 124 as described above. The metadata 124 can be used by the memory management agent 114 of the memory management unit 104 to control which entries 116 are maintained in the TLB 118. An example of this is further described below.

[0074] Figure 7 A process 700 in an example implementation of receiving a command packet including metadata and using the metadata to control inclusion of entries in a TLB is depicted. A command packet is received by a memory management agent. The command packet includes a virtual address and metadata describing characteristics of a memory access (block 702), and may also include instructions, commands, instruction sequences, etc. For example, a command packet 108 with metadata 124 is received by a memory management agent 114 of a memory management unit 104 (e.g.) from a host 102.

[0075] The memory management agent controls which entries are maintained in a translation lookaside buffer (TLB) based on metadata. The entries support translations between virtual addresses and physical addresses of physical memory (block 704). For example, the memory management agent 114 controls which entries 116 are maintained in the TLB 118 based on metadata 124. Examples of functions that do this are shown as a QoS management module 126, a prefetch management module 128, and a mode management module 130. As another example, page migration is triggered based on metadata. Page migration, page placement, and page prefetch are used to move actual memory pages corresponding to translations to a device's memory (e.g., physical memory) in anticipation of future use. Similarly, for example, if the page being removed has a high reuse distance, the page may be removed from the device memory to make room for a page of increased usefulness.

[0076] It should be understood that many variations are possible based on the disclosure herein. Although the above features and elements are described in particular combinations, each feature or element may be used alone without the other features and elements, or in various combinations with or without the other features or elements.

[0077] The various functional units shown in the figures and / or described herein (including the host 102 and the device 202 where appropriate) are implemented in any of a variety of different ways, such as hardware circuits, software or firmware executed on a programmable processor, or any combination of two or more of hardware, software and firmware. The provided method is implemented in any of a variety of devices such as a general-purpose computer, a processor or a processor core. For example, suitable processors include general-purpose processors, special-purpose processors, conventional processors, digital signal processors (DSPs), graphics processing units (GPUs), parallel acceleration processors, multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) circuits, any other type of integrated circuit (IC) and / or state machines.

[0078] In one or more specific implementations, the methods or processes provided herein are implemented in a computer program, software, or firmware incorporated into a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROM disks), and digital versatile disks (DVDs).

[0079] Although the systems and techniques have been described in language specific to structural features and / or methodological acts, it should be understood that the systems and techniques defined in the appended claims are not necessarily limited to the specific features or acts described. These specific features and acts are disclosed as example forms of implementing the claimed subject matter.

Claims

1. A method, comprising: receiving, by a memory management agent, a command packet including a virtual address and metadata describing characteristics of a memory access; as well as Which entries are maintained in a translation lookaside buffer (TLB) are controlled by the memory management agent based on the metadata, the entries supporting translation between the virtual addresses and physical addresses of physical memory. 2 . The method of claim 1 , wherein the metadata describes usage of data associated with the virtual address. 3 . The method of claim 1 , wherein the metadata indicates an irregular memory access pattern, and the controlling comprises limiting storage of entries associated with the irregular memory access pattern in the translation lookaside buffer. 4 . The method of claim 1 , wherein the metadata indicates a dense memory access pattern, and the controlling comprises storing entries associated with the dense memory access pattern in the translation lookaside buffer. The method of claim 1 , wherein the metadata indicates a domain type, and the controlling is based on the domain type.

6. The method of claim 1, wherein the controlling comprises invalidating at least one of the entries in the translation lookaside buffer, replacing at least one of the entries in the translation lookaside buffer, or controlling when at least one of the entries is stored in the translation lookaside buffer.

7. The method of claim 1, wherein the controlling comprises pre-fetching at least one of the entries into the translation lookaside buffer based on the metadata.

8. The method of claim 1, wherein the controlling comprises controlling which entries are maintained in the translation lookaside buffer based on quality of service.

9. The method of claim 1, wherein the controlling comprises controlling page migration, page placement, page prefetching, or page removal.

10. The method of claim 1, wherein the virtual address is part of a unified virtual memory space in which a central processing unit and a parallel computing unit share physical memory as part of a heterogeneous system architecture.

11. A device, comprising: a translation lookaside buffer configured to store entries for translations between virtual addresses and physical addresses; and An address translation agent is configured to manage which entries are maintained in the translation lookaside buffer based on metadata received via a command packet, the metadata describing memory access characteristics.

12. The device of claim 11, further comprising a physical memory accessible via the physical address, and wherein the virtual address implements a unified virtual memory space in which a central processing unit and a parallel computing unit share the physical memory as part of a heterogeneous system architecture.

13. The device of claim 11, further comprising a command processor configured to receive the command packet having the metadata, instructions, and virtual addresses and to send the metadata to the address translation agent.

14. The apparatus of claim 13, wherein the command processor is configured as a programmable microcontroller to implement at least one policy that controls the sending of the metadata to the address translation agent.

15. The device of claim 11, wherein the address translation agent is configured to manage address translation requests.

Images