Address translation service management

Abstract

Address translation service management techniques are described that are based on metadata that can be used to provide hints as insight into memory accesses, and based on this, the use of translation lookaside buffers is optimized to control which entries are maintained in the queue and manage address translation requests.

Description

[Technical Field]

[0001] (Related Applications) This application claims priority to U.S. patent application Ser. No. 17 / 949,716, filed Sep. 21, 2022, entitled "Address Translation Service Management," the entire disclosure of which is incorporated herein by reference in its entirety. [Background technology]

[0002] Virtual memory is a technique for managing the use of shared physical memory, through which various functions are supported, including expanding the amount of storage made available to applications beyond the amount actually available in physical memory, supporting heterogeneous system architectures, offloading memory management from applications and operating systems, using a variety of different types of memory without application awareness, supporting memory optimization, and addressing memory fragmentation.

[0003] To do this, a memory management unit is used to manage the translation between virtual addresses and physical addresses. However, conventional techniques encounter challenges due to the ever-increasing capabilities of systems and devices that use virtual memory. Examples include the latency caused by systems with many virtual machines, which hinders the operation of devices that implement these systems.

[0004] The detailed description is made with reference to the accompanying drawings. [Brief explanation of the drawings]

[0005] [Figure 1] 1 is a block diagram of a non-limiting example system configured to use address translation service management techniques. [Figure 2]FIG. 2 is a block diagram of a non-limiting exemplary system configured to implement the memory management agent and translation lookaside buffer of FIG. 1 as part of a heterogeneous system architecture. [Figure 3] FIG. 3 is a non-limiting exemplary system block diagram illustrating the configuration of the parallel processing system of FIG. 2 as an accelerator. [Figure 4] FIG. 2 is a block diagram of a non-limiting example system illustrating use of reuse distance information for translation prefetching techniques by an address translation agent. [Figure 5] FIG. 1 is a block diagram of a non-limiting exemplary system illustrating irregular prefetching enabled through data structure analysis. [Figure 6] FIG. 1 illustrates a procedure in an exemplary embodiment for generating a command packet by a host to include metadata. [Figure 7] FIG. 1 illustrates a procedure in an exemplary embodiment for receiving a command packet that includes metadata and using the metadata to control the inclusion of entries in a translation lookaside buffer. DETAILED DESCRIPTION OF THE INVENTION

[0006] Virtual memory is a technology used to extend the capabilities made available by a device to manage data storage. To support this, a memory management unit is responsible for translating between virtual and physical addresses. In one example, this can be used to implement a heterogeneous system architecture in which a central processing unit and parallel processing units use virtual addresses as part of a unified virtual memory space to share physical memory. Other examples are also contemplated, such as extending the amount of storage made available to applications, offloading memory management, supporting the use of a variety of different types of memory, supporting memory optimization, and addressing memory fragmentation.

[0007] To support address translation, the memory management unit includes a translation lookaside buffer with entries that define the translation between virtual and physical addresses, i.e., the mapping between addresses. The translation lookaside buffer can be configured in a variety of ways, examples of which include an address translation cache (ATC), a content addressable memory (CAM) table, and a priority queue. However, conventional techniques have challenges in extending the functionality of devices that rely on address translation. An example of this involves the use of unified virtual memory and demand paging in heterogeneous system architectures, which encounters latency in translation and prevents the operation of translation lookaside buffers (TLBs) in real-world scenarios.

[0008] To solve these problems, address translation service (ATS) management techniques are described that are based on metadata that can be used to provide hints as insight into memory accesses, and based on this, optimize the use of ATS requests to control which address translations are maintained in a translation lookaside buffer (TLB) and which memory pages are stored in physical memory.

[0009] In one example, a host (e.g., an operating system or an application running on a central processing unit) generates a command packet. The command packet in this example includes an instruction (e.g., a command, a memory instruction to load or write data, a sequence of instructions, etc.), a virtual address, and metadata. The metadata describes memory access characteristics and can be used to control usage by a memory management agent. Once generated, the command packet is sent by the host to a command queue of the accelerator.

[0010] The command packets are received by the accelerator's command queue. For example, the accelerator's command processor processes the command packets using firmware to interpret fields containing metadata. The metadata is then utilized by a memory management agent (e.g., as part of the memory management unit's address translation agent) to control functions involving the ATS. This functionality includes managing entries in the translation lookaside buffer, ATS requests, translation prefetching, and memory page movements.

[0011] In one example, management is based on access patterns associated with memory accesses. The metadata can be configured to indicate, for example, an irregular memory access pattern with a high reuse distance. As a result, entries associated with the irregular memory access pattern are restricted from being stored in the TLB because the likelihood of the entry being reused is relatively low, correspondingly increasing the likelihood of replacing an entry in a queue with a higher reuse likelihood. Similarly, metadata indicating a high-density memory access pattern causes a memory management agent to store the corresponding entry in the TLB. Furthermore, this metadata supports prefetching of entries into the TLB, as described further below, such as by leveraging insight into the reuse distance in the high-density access pattern to obtain additional entries.

[0012] In another example, metadata is used to implement quality-of-service considerations by a memory management agent. This can be used to support, for example, priority information, deadlines for each task to meet corresponding service level agreements (SLAs), etc. In a further example, use of metadata by a memory management agent leverages prefetching techniques for memory translations, thereby reducing translation latency and supporting accurate and timely access to entries in the TLB, thereby reducing translation overhead. In a further example, use of metadata by a memory management agent leverages memory page prefetching techniques, thereby reducing page fault latency and supporting accurate and timely access to memory pages, thereby reducing page fault overhead.

[0013] In a further example, ATS requests are managed to improve operational efficiency. For example, in a scenario where a device sends requests to an I / O memory management unit (IOMMU) to retrieve requests, the requests are batched to avoid the overhead of sending individual requests, e.g., by sending a single request for multiple translations. Additionally, requests from multiple commands requesting access to the same page are batched in another scenario. This is done, for example, by detecting when a request matches an already-executing request and combining the requests. The response to the first request can also accommodate the second request. In another scenario, the first request is delayed for a period of time to detect whether a subsequent request requests access to the same memory, and if so, is combined before sending.

[0014] In embodiments, a command processor is also used for operation with the memory management agent. The command processor is programmable to implement policies that assist the memory management agent in managing TLB, ATS requests, and page movements. The policies are configurable to support domain-specific optimizations and device reuse (e.g., accelerator reuse) across applications.

[0015] In one such example, the memory management agent is implemented as part of a domain-specific accelerator, e.g., for machine learning. The domain-specific accelerator is configurable to the targeted functionality, thereby supporting intelligent address translation management. However, real-world scenarios encounter variations in virtual memory usage based on input data, individual kernel characteristics, and the like. Therefore, configuring a command processor to support programming in conjunction with the operation of the memory management agent enables the domain-specific firmware to tailor the address translation functionality to specific use cases via policies executed by the command processor. This also supports the device's ability to implement these features to evolve as application functionality evolves, thereby improving device operation. Various other examples are also contemplated, examples of which are described below and illustrated using corresponding figures.

[0016] In some aspects, the techniques described herein relate to a method that includes: a memory management agent receiving a command packet that includes a virtual address and metadata that describes memory access characteristics; and the memory management agent controlling, based on the metadata, which entries that support translation between the virtual address and a physical address of a physical memory are maintained in a translation lookaside buffer (TLB).

[0017] In some aspects, the technology described herein relates to how metadata describes the usage of data associated with a virtual address.

[0018] In some aspects, the techniques described herein relate to a method, wherein the metadata indicates an irregular memory access pattern, and wherein the controlling includes limiting storage of entries associated with the irregular memory access pattern in a translation lookaside buffer.

[0019] In some aspects, the techniques described herein relate to a method, wherein the metadata indicates a high-density memory access pattern, and wherein the controlling includes storing an entry associated with the high-density memory access pattern in a translation lookaside buffer.

[0020] In some aspects, the technology described herein relates to methods in which the metadata indicates a domain type and the control is based on the domain type.

[0021] In some aspects, the techniques described herein relate to a method, wherein controlling includes invalidating at least one such entry in the translation lookaside buffer, replacing at least one such entry in the translation lookaside buffer, or controlling when to store at least one such entry in the translation lookaside buffer.

[0022] In some aspects, the techniques described herein relate to a method, wherein controlling includes prefetching at least one such entry into a translation lookaside buffer based on the metadata.

[0023] In some aspects, the techniques described herein relate to a method, wherein controlling includes controlling which entries are maintained in the translation lookaside buffer based on a quality of service.

[0024] In some aspects, the techniques described herein relate to a method, wherein controlling includes controlling page movement, page placement, page prefetching, or page removal.

[0025] In some aspects, the technology described herein relates to a method in which the virtual address is part of a unified virtual memory space in which a central processing unit and parallel computing units share physical memory as part of a heterogeneous system architecture.

[0026] In some aspects, the techniques described herein relate to a device that includes: a translation lookaside buffer configured to store entries for translations between virtual addresses and physical addresses; and an address translation agent configured to manage which entries are maintained in the translation lookaside buffer based on metadata describing memory access characteristics received via command packets.

[0027] In some aspects, the technology described herein relates to a device further including physical memory accessible via physical addresses that implements a unified virtual memory space in which a central processing unit and parallel computing units share the physical memory as part of a heterogeneous system architecture.

[0028] In some aspects, the techniques described herein relate to a device configured to receive a command packet having metadata, instructions, and a virtual address, and further including a command processor that sends the metadata to an address translation agent.

[0029] In some aspects, the techniques described herein relate to a device in which a command processor is configured as a programmable microcontroller for implementing at least one policy that controls the transmission of metadata to an address translation agent.

[0030] In some aspects, the technology described herein relates to a device in which an address translation agent is configured to manage address translation requests.

[0031] In some aspects, the techniques described herein relate to a device, in which an address translation agent is configured to manage at least one address translation request by postponing transmission of the at least one address translation request.

[0032] In some aspects, the technology described herein relates to a device in which an address translation agent is configured to batch process multiple address translation requests as a single request.

[0033] In some aspects, the techniques described herein relate to a device in which multiple address translation requests correspond to multiple commands, respectively.

[0034] In some aspects, the techniques described herein relate to a method that includes: a host generating a command packet that includes a virtual address and metadata that describes memory access characteristics; and the host transmitting the command packet for receipt by a memory management agent, the metadata configured to control management of entries in a translation lookaside buffer by the memory management agent, the entries supporting translation of virtual and physical addresses.

[0035] In some aspects, the techniques described herein relate to a method in which a command packet includes instructions configured to load data from physical memory.

[0036] 1 is a block diagram of a non-limiting exemplary system 100 configured to employ a metadata-based translation lookaside buffer (TLB) management technique. System 100 includes a host 102, a memory management unit 104, and physical memory 106 (e.g., volatile or non-volatile memory). Host 102 in this example represents a function that issues command packets 108. Command packets 108 are configurable for use in a variety of scenarios, examples of which include use in a point-to-point bus with a serial link connecting corresponding devices to a root complex, as further described in connection with FIG. 3.

[0037] The command packet 108 includes instructions 110, such as memory instructions for loading data from or writing data to the physical memory 106 based on a corresponding virtual address 112. To do this, the memory management agent 114 includes entries 116, which are shown as being maintained in a translation lookaside buffer (TLB) 118. The entries 116 define a translation between the virtual addresses 112 and corresponding physical addresses 120 in the physical memory 106. In other words, the entries 116 define a mapping between the virtual addresses 112 and the corresponding physical addresses 120. This can be used to support a variety of functions, examples of which include heterogeneous system architectures, which are further described in connection with FIG. 2.

[0038] In the illustrated example, host 102 includes a metadata generation module 122. Metadata generation module 122 is configured to generate a portion of command packet 108 (e.g., metadata 124 for inclusion as the packet's payload). Metadata 124 describes memory access characteristics that can be used to describe characteristics of the data that is the subject of the command packet and to provide insight into potential subsequent memory accesses, among other things. In this manner, metadata 124 acts as a "hint" provided by host 102 regarding memory accesses. Memory management agent 114 is configured to leverage the insight provided by metadata 124 to manage which entries 116 are maintained in TLB 118.

[0039] Examples of functionality that can be used to manage access based on metadata 124 are shown as quality-of-service (QoS) management module 126, prefetch management module 128, pattern management module 130, and request management module 132. QoS management module 126 is configured to implement quality-of-service considerations to support, for example, priority information, task deadlines to meet corresponding service level agreements (SLAs), etc. Prefetch management module 128 supports prefetching of entries 116 into TLB 118. Pattern management module 130 can be configured to manage entries based on patterns of observed memory accesses and / or patterns of memory accesses that may be encountered as part of subsequent memory accesses.

[0040] The request management module 132 represents a function that manages address translation service requests used to initiate each translation operation. When a device sends requests to an I / O memory management unit (IOMMU) for retrieval, multiple such requests are batched by the request management module 132, e.g., batching multiple address translation requests into a single request to reduce the overhead encountered in sending individual requests. This can also be used, for example, to batch multiple address translation requests, each corresponding to multiple commands related to the same page. In one example, this can be accomplished by detecting when an address translation request matches an address translation request already “in progress” and combining the requests. In another example of request management, sending an address translation request is postponed to determine whether a subsequent request involves accessing the same memory, and then the requests are batched as described above.

[0041] As a result, metadata 124 provides a degree of control for host 102 to provide insight and improve address translation operations. Additionally, through the use of metadata 124, memory management agent 114 is configured to reduce translation latency, as further described in the examples below.

[0042] 2 is a block diagram of a non-limiting, exemplary system 200 configured to implement the memory management agent and TLB of FIG. 1 as part of a heterogeneous system architecture. System 200 includes a device 202 that can be configured in various manners. Examples of device 202 configurations include, but are not limited to, computing devices, servers, mobile devices (e.g., wearables, mobile phones, tablets, laptops), processors (e.g., graphics processing units, central processing units, accelerators), digital signal processors, disk array controllers, hard disk drive host adapters, memory cards, solid-state drives, wireless communication hardware connections, Ethernet hardware connections, switches, bridges, network interface controllers, and other equipment configurations. It should be understood that in various embodiments, device 202 and the devices implementing host 102, memory management unit 104, and physical memory 106 of FIG. 1 can be configured as any one or more of the devices described above and / or various other devices without departing from the spirit or scope of the described technology.

[0043] The device 202 is configured to implement a heterogeneous system architecture. In the illustrated example, this architecture includes a central processing unit 204, a parallel processing system 206, and physical memory 106 communicatively coupled to each other via the same bus system 208. The bus system 208 supports shared use of the physical memory 106 and tasks through the implementation of a unified virtual memory space 210 of virtual addresses 212. The bus system 208 can be configured, for example, as a point-to-point bus with serial links connecting corresponding devices to a root complex, an example of which includes a peripheral component interconnect express (PCIe).

[0044] The central processing unit 204 is configured to support the execution of an operating system 214 and applications 216 that act as the host 102 issuing command packets 108. The parallel processing system 206 can be configured in a variety of ways, examples of which include a graphics processing unit (GPU), an accelerator, a digital signal processor (DSP), an application specific integrated circuit (ASIC), etc. The parallel processing system 206 includes, for example, parallel computation units 218 that perform the logical operations of the parallel processing system 206.

[0045] The unified virtual memory space 210 is implemented in this example such that the central processing unit 204 and the parallel processing system 206 share page tables. This allows data exchange through the sharing of pointers, instead of copying data between devices as is involved in other architectures, in what is called a "zero-copy" operation, thus improving device operation. However, as mentioned above, conventional support for unified virtual memory and demand paging in some real-world scenarios results in slow translation and prevents translation lookaside buffer operations.

[0046] To solve this problem, device 202 incorporates memory management agent 114 as part of parallel processing system 206. Memory management agent 114 supports a flexible mechanism that is configurable to address considerations on a domain-specific basis, addressing long latency address translation in systems with multiple virtual machines targeting parallel processing system 206 (e.g., accelerators) within a heterogeneous system.

[0047] Figure 3 is a block diagram of a non-limiting exemplary system 300 illustrating the configuration of the parallel processing system 206 of Figure 2 as an accelerator. The parallel processing system 206 includes an address translation agent 302 implementing the memory management agent 114, an address translation queue 118, a command processor 304, a command queue 306, and an accelerator computing unit 308, which are communicatively coupled 310 to each other (e.g., via a bus).

[0048] In this example, metadata 124 regarding the use of the TLB 118 is embedded in the command packet 108. In an embodiment, the command packet 108 is a heterogenous system architecture (HSA) packet, although packets for use with any computational agent are contemplated.

[0049] The command processor 304 processes the command packet 108 using corresponding firmware and interprets the fields within the packet as metadata 124. The command processor 304 then sends information to the address translation agent 302 to help it make informed decisions regarding the provision of address translation services to the device 202.

[0050] Address translation agent 302 is configured to manage TLB 118, which includes controlling when to invalidate entries 116, how to replace entries 116, and / or when to cache entries 116. Because accelerators often execute kernels for a given domain, and therefore the programmer or intelligent runtime has insight into the intended virtual memory usage, metadata 124 is configured to provide hints that are passed to address translation agent 302 to communicate this insight about how to manage TLB 118.

[0051] For example, linear algebra accelerators can be used across a wide variety of domains (e.g., graph analysis, machine learning, scientific applications, etc.) that rely on matrix and vector multiplication. However, these domains have different degrees of regularity in memory access. Sparse linear algebra (SpLA) kernels, such as sparse matrix-vector multiplication (e.g., used in graph and machine learning applications), have highly irregular memory access patterns due to the inherent sparsity of matrices. However, dense linear algebra kernels are highly regular.

[0052] Thus, when dispatching an SpLA kernel, the command packet 108 embeds information as part of the metadata 124 that indicates to the address translation agent 302 that the workload has irregular memory accesses with a high reuse distance. The address translation agent 302 then dynamically determines, in this example, that translations for the SpLA are restricted from being cached because these entries are likely to replace other entries and corresponding translations with increased usefulness. On the other hand, entries 116 corresponding to translations for the dense kernel are cached because the entries 116 are likely to be reused.

[0053] Additionally, the address translation agent 302 can be configured to optimize based on input data in addition to kernel type. For example, in a dense linear algebra kernel, a reuse distance can be determined for pages in virtual memory based on the size and structure of the matrix. For example, for a relatively large matrix with a significant number of rows, an approximate row reuse distance can be determined. Thus, translation entries 116 corresponding to rows that are no longer in use are discarded from the TLB 118 to make room for translation entries 116 from the next row. In an embodiment, this technique is implemented in parallel with translation prefetching.

[0054] 4 is a block diagram of a non-limiting example system 400 illustrating the use of reuse distance information for translation prefetching techniques by address translation agent 302. In deep neural network (DNN) training, memory accesses follow different patterns based on whether a forward or backward path is encountered.

[0055] For example, in virtual memory page 402, pages in the first layer of forward pass 404 have a longer reuse distance than pages in later layers. Therefore, indications of this pattern and scenario in metadata 124 can be used by address translation agent 302 to discard entries 116 for pages in the early layers. Entries 116 for pages in later layers are maintained in address translation queue 118 by address translation agent 302. For reverse pass 406, entries 116 for these pages are completed by address translation agent 302, thereby optimizing address translation and device operation.

[0056] FIG. 5 is a block diagram of a non-limiting, exemplary system 500 illustrating irregular prefetching enabled through data structure analysis. The address translation agent 302 can be configured to make intelligent translation prefetch requests. Following the example above, dense linear algebra kernels typically use regular memory accesses and can be viewed as streaming consecutive memory pages when the accesses are performed on consecutive row and column data of the corresponding matrix. Therefore, the dispatch packet can be configured to include metadata 124 that informs the address translation agent 302 that the kernel is operating as a streaming kernel. In response, the address translation agent 302 initiates prefetching of adjacent pages as memory accesses from the computational accelerator near page boundaries.

[0057] Alternatively, SpLA memory accesses are typically irregular due to sparsity, making these accesses more difficult to predict. For example, operations on matrices stored in compressed sparse row (CSR) format are highly irregular due to the seemingly random access patterns to column vectors. However, the CSR data structure is typically static and known a priori. Therefore, in this example, the address translation agent 302 directly interprets the CSR column index array to prefetch translations in advance.

[0058] The illustrated example shows first, second, third, fourth, and fifth columns of data 502(1), 502(2), 502(3), 502(4), and 502(5). Entries are maintained in TLB 118 for first, third, and fourth columns 502(1), 502(3), and 502(4). Address translation agent 302 then determines that second and fifth columns 502(2), 502(5) are on page boundaries and, based on this, initiates prefetching 504 of these columns.

[0059] Similarly, in the DNN training example above, entries describing page translations used in later layers are kept in TLB 118, and entries with corresponding translations in earlier layers are discarded. Additionally, entries corresponding to page translations in earlier layers are prefetched in time on the reverse path 406.

[0060] In a virtualized accelerator environment, several processes / kernels may simultaneously execute on the accelerator. Therefore, managing TLB 118 requests for fairness and priority can be used to achieve performance and throughput benefits when quality of service is an issue. As an example, task priority information or deadlines for meeting respective service level agreements (SLAs) are provided to address translation agent 302 via metadata 124. QoS management module 126 of address translation agent 302 then uses this information from metadata 124 to schedule and batch entries 116 corresponding to translation requests.

[0061] For example, for tasks with high priority or short SLA deadlines, these requests are scheduled aggressively. Additionally, priority, along with other information as described above, can be used by memory management agent 114 to manage TLB 118. For example, entries 116 corresponding to higher priority requests are prioritized by address translation agent 302 for caching in TLB 118. If a task has a lower priority or a longer SLA deadline, address translation agent 302 batches entries 116 corresponding to these translation requests. For example, address translation agent 302 opens a window for translation requests, and as the SLA deadline approaches (e.g., based on a threshold), memory management agent 114 closes the window and sends the entries corresponding to each of the translation requests in a single request.

[0062] The functionality of memory management agent 114 is configurable using fixed-function hardware. As described above in connection with FIG. 3, the functionality of memory management agent 114 is configurable in part by including a programmable command processor 304, for example, using a RISC-V CPU running domain-specific firmware. Command processor 304 can be configured, for example, as a heterogeneous system architecture agent, where a primary heterogeneous system architecture agent sends service requests to command processor 304 using architected queuing language (AQL) agent dispatch packets.

[0063] While general-purpose accelerators and programmable accelerators (e.g., GPUs) can be configured to support a wide variety of use cases, domain-specific accelerators are configured for target functionality and therefore support intelligent address translation queue 118 management. However, slight variations in virtual memory usage are encountered in real-world scenarios based on input data or individual kernel characteristics. Therefore, the programmability of the command processor 304, working in conjunction with the memory management agent 114 of the address translation agent 302, supports the use of domain-specific firmware on the command processor 304 to tune the TLB 118 functionality to specific use cases. Additionally, as applications evolve, so does the programmability of the command processor 304.

[0064] Thus, metadata-based TLB management techniques support improved management of entries 116 in address translation queue 118 and reduced translation latency. These techniques also support increased precision and timeliness through the use of prefetching entries and corresponding translations, which reduces translation overhead. Furthermore, the ability to program policy 312 of command processor 304 to work in conjunction with memory management agent 114 supports domain-specific optimization and accelerator reuse across applications.

[0065] 6 illustrates a procedure 600 for an exemplary embodiment of a host generating a command packet to include metadata. A command packet is generated by the host. The command packet includes a virtual address and metadata describing memory access characteristics (block 602). The command packet can be configured to include an instruction, a sequence of instructions, a command, etc. As an example, a command packet 108 is received by the memory management unit 104 from the host 102. The command packet 108 includes metadata 124.

[0066] A command packet is sent by the host for receipt by the memory management agent. The metadata is configured to control management of entries in the TLB by the memory management agent. The entries support virtual and physical address translation (block 604). As an example, a command packet 108 is sent by the host 102 to the memory management unit 104. The command packet 108 includes metadata 124, as described above. The metadata 124 is usable by the memory management agent 114 of the memory management unit 104 to control which entries 116 are maintained in the TLB 118. Examples of this are described further below.

[0067] 7 illustrates a procedure 700 in an exemplary embodiment for receiving a command packet including metadata and using the metadata to control inclusion of an entry in a TLB. The command packet is received by a memory management agent. The command packet includes a virtual address and metadata describing memory access characteristics (block 702), and may also include an instruction, a command, a sequence of instructions, etc. By way of example, a command packet 108 with metadata 124 is received by memory management agent 114 of memory management unit 104, e.g., from host 102.

[0068] The memory management agent controls which entries are maintained in the translation lookaside buffer (TLB) based on the metadata. The entries support translations between virtual and physical addresses in physical memory (block 704). As an example, the memory management agent 114 controls which entries 116 are maintained in the TLB 118 based on the metadata 124. Exemplary functions for doing this are shown as the QoS management module 126, the prefetch management module 128, and the pattern management module 130. As another example, page movement is triggered based on the metadata. Page movement, page placement, and page prefetching are utilized to move actual memory pages corresponding to translations into device memory (e.g., physical memory) in anticipation of future use. Similarly, pages can be removed from device memory to create space for pages of increased utility, for example, if the removed pages have a high reuse distance.

[0069] It should be understood that many variations are possible based on the disclosure herein, and although features and elements are described above in particular combinations, each feature or element can be used alone without the other features and elements, or in various combinations with or without the other features and elements.

[0070] The various functional units illustrated in the figures and / or described herein (including, where appropriate, host 102 and device 202) may be implemented in any of a variety of different ways, such as hardware circuits, software or firmware running on a programmable processor, or any combination of two or more of hardware, software, and firmware. The provided methods may be performed in any of a variety of devices, such as a general-purpose computer, a processor, or a processor core. Suitable processors include, by way of example only, a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), a graphics processing unit (GPU), a parallel-accelerated processor, multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and / or a state machine.

[0071] In one or more embodiments, the methods and procedures provided herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-transitory computer-readable storage media include read only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

[0072] Although the systems and techniques have been described in language specific to structural features and / or methodological acts, it is to be understood that the systems and techniques defined in the appended claims are not necessarily limited to the particular features or acts described. Rather, the particular features and acts are disclosed as exemplary forms of implementing the claimed invention.

Claims

1. 1. A method comprising: receiving, by a memory management agent, a command packet including a virtual address and metadata describing memory access characteristics; the memory management agent controlling which entries are maintained in a translation lookaside buffer (TLB) based on the metadata, the entries supporting translation between the virtual addresses and physical addresses in physical memory. method.

2. the metadata describes usage of data associated with the virtual address; 10. The method of claim 1.

3. the metadata indicates irregular memory access patterns; the controlling includes limiting storage of entries associated with the irregular memory access pattern in the translation lookaside buffer.

10. The method of claim 1.

4. the metadata indicates a high-density memory access pattern; the controlling includes storing entries associated with high-density memory access patterns in the translation lookaside buffer.

10. The method of claim 1.

5. The metadata indicates a domain type; the controlling is based on the domain type.

10. The method of claim 1.

6. wherein the controlling includes invalidating at least one of the entries in the translation lookaside buffer, replacing at least one of the entries in the translation lookaside buffer, or controlling when to store at least one of the entries in the translation lookaside buffer.

10. The method of claim 1.

7. the controlling includes prefetching at least one of the entries into the translation lookaside buffer based on the metadata.

10. The method of claim 1.

8. controlling includes controlling which entries are maintained in the translation lookaside buffer based on a quality of service.

10. The method of claim 1.

9. The controlling includes controlling page movement, page placement, page prefetching, or page removal.

10. The method of claim 1.

10. the virtual address is part of a unified virtual memory space in which a central processing unit and parallel computing units share physical memory as part of a heterogeneous system architecture; 10. The method of claim 1.

11. A device, a translation lookaside buffer configured to store entries for translations between virtual addresses and physical addresses; an address translation agent configured to manage which entries are maintained in the translation lookaside buffer based on metadata describing memory access characteristics received via command packets; device.

12. further comprising a physical memory accessible via the physical address; the virtual addresses implement a unified virtual memory space in which a central processing unit and parallel computing units share the physical memory as part of a heterogeneous system architecture; The device of claim 11.

13. a command processor configured to receive the command packet including the metadata, instructions, and a virtual address, and to send the metadata to the address translation agent; The device of claim 11.

14. the command processor is configured as a programmable microcontroller that implements at least one policy that controls the transmission of the metadata to the address translation agent.

14. The device of claim 13.

15. the address translation agent is configured to manage address translation requests; The device of claim 11.

Images