Pre-fetch using direct memory access engine

Abstract

A processing system (100) includes one or more DMA engines (150, 160) that load data from a memory (140) or another cache location without storing the data after loading the data. As data propagates through caches ("intermediate caches") located between memory or other cache locations that store requested data, the data is selectively copied to these intermediate caches based on a cache replacement policy. Rather than the DMA engine manually stores the data into the intermediate caches, a cache replacement policy (512) of the intermediate caches determines whether the data is copied into each respective cache and a replacement priority of the data. By bypassing the stored data, the DMA engine enables prefetching of these intermediate caches without consuming unnecessary bandwidth or searching for memory locations to store data, thereby reducing latency and saving energy.

Description

Background Technology

[0001] A system direct memory access (DMA) engine is a hardware device that coordinates direct memory access data transfers between devices within a computer system (e.g., input / output interfaces and display controllers) and memory, or between different locations within memory. The DMA engine typically resides on a processor, such as a central processing unit (CPU) or accelerated processing unit, and receives commands from applications running on that processor. Based on these commands, the DMA engine reads data from a DMA source (e.g., a first memory buffer defined in memory) and writes the data to a DMA destination (e.g., a second buffer defined in memory). Summary of the Invention

[0002] In the embodiments described herein, techniques are provided for a DMA engine to load data from memory or another cache location without storing the data after loading. In one example embodiment, a method may include: loading data from memory in response to a request from a Direct Memory Access (DMA) engine associated with a processor; selectively copying the data to one or more caches of the processor located between the DMA engine and the memory; and bypassing storage of the data for the DMA engine.

[0003] In some implementations, the request is a prefetch request. In some implementations, the method further includes receiving a command at the DMA engine to send the request to load the data without storing it. The command may indicate the priority of the request. The memory may be either system memory or a second cache of the processor.

[0004] In some implementations, the data is selectively copied to one or more caches based on a cache replacement policy. Furthermore, the cache replacement policy determines which of the one or more caches the data is copied to. In some implementations, the cache replacement policy determines the replacement priority of the data.

[0005] In another example implementation, a processing system includes: a processor; memory; a direct memory access (DMA) engine configured to load data from the memory; and one or more cache controllers configured to selectively copy the data to one or more caches of the processor located between the DMA engine and the memory, wherein the DMA engine is further configured to bypass storing the data. The DMA engine may also be configured to bypass storing the data in response to receiving a command requesting to load the data without storing it.

[0006] In some implementations, the request is a prefetch request. In some implementations, the command indicates the priority of the command. The memory may be one of system memory or a second cache of the processor. Furthermore, in some implementations, data is selectively copied to one or more caches based on a cache replacement policy. The cache replacement policy determines at least one of the one or more caches in which the data is copied and the replacement priority of the data. Attached Figure Description

[0007] This disclosure can be better understood by referring to the accompanying drawings, and its many features and advantages will be apparent to those skilled in the art. The same reference numerals are used in different drawings to denote similar or identical items.

[0008] Figure 1 It is a block diagram of a processing system according to some implementation schemes, including a direct memory access (DMA) engine configured to prefetch data into one or more caches.

[0009] Figure 2 This is a block diagram of a processing system based on some implementation schemes, illustrating how the DMA engine sends a prefetch request for data to the system memory, and how the cache between the system memory and the DMA engine selectively copies data based on a cache replacement strategy.

[0010] Figure 3 This is a block diagram illustrating how a computing unit sends a prefetch command to a DMA engine according to some implementation schemes, and how the DMA engine sends a prefetch request based on the command.

[0011] Figure 4 This is a block diagram of a processing system according to some implementation schemes, illustrating how a prefetch request from a DMA engine is propagated via a PCIe bus to the system memory and intermediate cache controller, which selectively copies the requested data based on a cache replacement policy.

[0012] Figure 5 An example of a cache controller according to some implementations is shown, which selects where in the cache replacement chain to insert data prefetched by the DMA engine based on a cache replacement policy.

[0013] Figure 6 This is a flowchart illustrating a method for prefetching data using a DMA engine according to some implementation schemes. Detailed Implementation

[0014] A typical processor includes one or more DMA engines to read and write blocks of data stored in system memory. DMA engines alleviate the burden on the processor core to manage transfers. In response to a data transfer request from the processor core, the DMA engine provides the necessary control information to the corresponding source and destination, enabling data transfer operations to be performed without delaying computational code, thus allowing communication and computation to overlap in time. Because the DMA engine handles the formation and communication of control information asynchronously, the processor core is freed to perform other tasks while awaiting the fulfillment of data transfer requests.

[0015] Typically, a DMA engine copies data from one location to another by performing load / store operations. This involves loading data from system memory via, for example, a high-speed peripheral component interconnect (PCIe) bus and storing the data in another memory component, such as dynamic random access memory (DRAM). For instance, a DMA engine manually stores data in a cache by loading it into notepad memory and then copying it to a cache. However, storing the data used by the DMA engine consumes bandwidth and requires software to locate the memory location where the data is stored. Furthermore, the DMA engine typically indicates to the requesting entity that the copy has been completed. However, because prefetching has semantic effects on applications, the requester of a prefetch does not need to wait for the prefetch to complete.

[0016] To facilitate asynchronous DMA prefetching of data to one or more caches in the processing system Figures 1 to 6 This illustrates a technique used by a DMA engine to load data from memory or another cache location without storing the data after loading. As data propagates through caches (referred to herein as "intermediate caches") between memory or other cache locations storing the requested data, the data is selectively copied to the intermediate cache based on a cache replacement policy. Instead of the DMA engine manually storing the data into the intermediate cache, the intermediate cache's cache replacement policy determines whether the data is copied to each appropriate cache and where in the cache's replacement chain the data is placed.

[0017] By bypassing data storage, the DMA engine prefetches data from intermediate caches without consuming unnecessary bandwidth or searching memory locations to store the data, thus reducing latency and saving energy. Specifically, by skipping data storage, the DMA engine eliminates the need to allocate storage locations, prevents cache contamination and unnecessary evictions, and allows greater flexibility in prioritizing data loading (e.g., by allowing speculative prefetching and ignoring page faults during loading).

[0018] In some implementations, the processor (e.g., a compute unit) issues a command to the DMA engine to send a request to load data, without storing the data on behalf of the DMA engine. In some implementations, the command is an explicit prefetch command. In some implementations, the command itself indicates the priority of the request or specifies the cache where the requested data will be copied. The request propagates from the DMA engine to system memory through the processing system. The request is perceived as a memory access request by cache controllers along the propagation path, such that if the requested data is stored in an intermediate cache between the DMA engine and system memory, the requested data is returned from the location where the data is stored to the DMA engine. Otherwise, the requested data is returned from system memory. Based on the priority of the request and the cache replacement policy, each cache exposed to the request determines whether to copy the requested data when returning data in response to the request. Therefore, the cache controller for each cache selectively copies data to its corresponding cache based on the cache replacement policy and the priority of the request.

[0019] In some implementations, the DMA engine's firmware is configured to enable the DMA engine to perform only load operations without performing storage operations. In other implementations, the DMA engine's hardware state machine is adapted to receive commands from the processor instructing the DMA engine to perform only load operations without performing storage operations.

[0020] Figure 1 This is a block diagram of a processing system 100, according to some embodiments, including a direct memory access (DMA) engine 150 configured to prefetch data into one or more caches. According to some embodiments, the processing system 100 implements a multi-die processing unit 101. In various embodiments, the multi-die processing unit 101 is a parallel processor comprising any cooperative set of hardware and / or software that performs functions and computations associated with accelerated graphics processing tasks, data-parallel tasks, and nested data-parallel tasks in an accelerated manner relative to resources such as a conventional central processing unit (CPU), a conventional graphics processing unit (GPU), and combinations thereof.

[0021] The processing system 100 includes one or more CPUs 170. Although Figure 1A CPU 170 is shown, but some embodiments of the processing system 100 include more CPUs. Bus 176 supports data flow between endpoints within the processing system 100, including the multi-die processing unit 101 and the CPU 170, and other endpoints. In various embodiments, the CPU 170 is connected to system memory 140 via bus 176, which is implemented as dynamic random access memory (DRAM) in some embodiments. In various embodiments, system memory 140 may also be implemented using other types of memory, including static random access memory (SRAM), non-volatile RAM, etc. In the illustrated embodiment, the CPU 170 communicates with system memory 140 and the multi-die processing unit 101 via bus 176. However, some embodiments of the processing system 100 include a multi-die processing unit 101 that communicates with the CPU 170 via direct connection or via a dedicated bus, bridge, switch, router, etc.

[0022] As shown, CPU 170 includes multiple processes, such as executing one or more application programs 172 to generate graphics commands or other parallel processing commands. In various embodiments, one or more application programs 172 include applications that utilize the functionality of the multi-die processing unit 101, such as applications that generate work within the processing system 100 or operating system (OS). In some specific embodiments, application program 172 includes one or more graphics instructions instructing the multi-die processing unit 101 to render a graphical user interface (GUI) and / or a graphical scene. For example, in some specific embodiments, the graphics instructions include instructions defining a set of one or more graphical primitives to be rendered by the multi-die processing unit 101.

[0023] In some implementations, application 172 utilizes a graphics application programming interface (API) 174 to invoke a user-mode driver (not shown) (or a similar GPU driver). The user-mode driver issues one or more commands to the multi-die processing unit 101 for rendering one or more graphics primitives into a displayable graphical image. Based on the graphics instructions issued by application 172 to the user-mode driver, the user-mode driver formulates one or more graphics commands that specify one or more operations to be performed by the multi-die graphics primitive 101 for rendering graphics. In some implementations, the user-mode driver is part of application 172 running on CPU 170. For example, in some implementations, the user-mode driver is part of a game application running on CPU 170. Similarly, in some implementations, a kernel-mode driver (not shown) formulates the one or more graphics commands, either alone or in combination with the user-mode driver, as part of an operating system running on CPU 170.

[0024] Figure 1 The multi-die processing unit 101 illustrated includes two chiplets 102 and 112. Each chiplet 102 and 112 includes a configurable number of computing units 104 and 114. Each computing unit 104 and 114 includes one or more Single Instruction Multiple Data (SIMD) units that execute operations in parallel according to the SIMD paradigm. The SIMD paradigm is a paradigm in which multiple processing elements share a single program control flow unit and program counter, and thus execute the same program, but can execute the program with different data. Although Figure 1 Two small chipsets 102 and 112 are illustrated, but some embodiments of the multi-die processing unit 101 include more or fewer chipsets. Each CU 104, 114 has an associated L1 cache 106 to store data directly accessed by the corresponding CU 104, 114 and the corresponding L1 cache controller (not shown). Additionally, each set of CUs 104, 114 in the small chipsets 102, 112 is connected to an L2 cache die (e.g., L2 cache 108, L2 cache 118) that stores frequently used data and instructions accessible to the CUs 104, 114 and the corresponding L2 cache controller (not shown). In some embodiments, each L2 cache 108, 118 is connected to one or more L1 caches 106, 116 implemented in the corresponding small chipsets 102, 112, and one or more L3 caches (or other last-level caches), such as L3 cache 122, implemented in the processing system 100. The caches together form a cache hierarchy. Each small chip set 102, 112 is further connected to the corresponding off-die memory, such as system memory 140, via a corresponding local memory channel (not shown).

[0025] exist Figure 1 In the depicted implementation, the consistent data texture 120 communicatively couples L2 caches 108, 118 to each other. The consistent data texture 120 includes levers for routing memory access requests and responses between any memory accesses. In some implementations, the consistent data texture 120 also includes a system memory map defined by the Basic Input / Output System (BIOS) for determining the destination of memory accesses based on system configuration, and buffers for each virtual connection.

[0026] Each of the small chipsets 102, 112 also includes one or more DMA engines, such as DMA engines 150, 160. In various embodiments, one or more DMA engines 150 coordinate DMA data transfers between devices and memory (or between different locations within memory) within system 100. In various embodiments, one or more DMA engines 150 coordinate data movement between multiple caches and system memory 140 while performing computations on other data, such as at CUs 104, 114. In various embodiments, one or more DMA engines 150 are, in some embodiments, part of a DMA controller (not shown), but the terms DMA engine and DMA controller are used interchangeably herein. In response to commands, DMA engines 150 operate to move data in and out of, for example, one or more caches without involving computation units 104, 114. Similarly, in some embodiments, DMA engines 150 perform in-chip data transfers. As will be understood, DMA engines 150 alleviate the burden on the processor core to manage data transfers and, in various embodiments, are used to prefetch data into one or more caches based on cache replacement strategies.

[0027] To facilitate data prefetching into the cache hierarchy, DMA engine 150 is configured to load requested data without storing it. As requested data propagates from the cache or system memory 140 where it resides toward the requesting DMA engine 150, intermediate cache controllers selectively copy the data to their respective caches based on the cache replacement policy of the corresponding cache and the priority of the requested data. In some embodiments, DMA engine 150 receives commands in software packet form or via hardware wiring from application 172 or from associated CU 104, instructing DMA engine 150 to prefetch a certain amount of data (e.g., multiple bytes) from a specified address. In some embodiments, the command includes a step size or access pattern for prefetching non-linear locations. In some embodiments, the command also indicates a priority. Based on the priority indicated by the command, DMA engine 150 selectively ignores commands or interrupts prefetching to support higher-priority operations (e.g., in response to low priority).

[0028] Figure 2 This is a block diagram 200 illustrating a processing system 100 according to some implementations, where a DMA engine 150 sends a prefetch request 202 for data to a system memory 140 and caches the data 204 returned in response to the prefetch request 202 based on a cache replacement policy.

[0029] In the illustrated example, CU 104 issues a command (not shown) to DMA engine 150 to prefetch data. In response to this command, DMA engine 150 issues a prefetch request 202 that traverses the cache hierarchy from L1 cache 106 to L2 cache 108, through data texture 120 to L3 cache 122. At each of L1 cache 106, L2 cache 108, and L3 cache 122, prefetch request 202 prompts a cache check to determine if the data in response to prefetch request 202 is stored in the cache. For example, if prefetch request 202 misses at L1 cache 106 but hits at L2 cache 108, data 204 is returned from L2 cache 108 to DMA engine 150. When data 202 flows through L1 cache 106 to DMA engine 150, the L1 cache controller (not shown) selectively copies data 204 into L1 cache 106 based on the cache replacement policy of L1 cache 106, and in some embodiments, additionally based on the priority indicated by prefetch request 202.

[0030] In another example, if prefetch request 202 misses at L1 cache 106, L2 cache 108, and L3 cache 122, then prefetch request 202 retrieves data 204 from system memory 140. Data 204 is propagated back to DMA engine 150 via processing system 100, and as data 204 passes through each of L3 cache 122, L2 cache 108, and L1 cache 106, the cache controllers for each of L3 cache 122, L2 cache 108, and L1 cache 106 selectively copy data 204 into their respective caches based on their cache replacement policies and / or the priorities indicated by prefetch request 202.

[0031] In some implementations, the cache replacement policy and / or priority indicated by prefetch request 202 causes prefetch request 202 from DMA engine 150 to not only skip data return to memory, but also data return to cache hierarchies higher than the target cache level. For example, if the priority instructs DMA engine 150 to prefetch data into L3 cache 122 for use by other processors, prefetch request 202 propagates through L1 cache 106 and L2 cache 108 to L3 cache 122, and if prefetch request 202 misses at these cache levels, it propagates to system memory 140. However, based on the priority indicated by prefetch request 202, instead of returning data 204 from L3 cache 122 to L2 cache 108, and then to L1 cache 106 and DMA engine 150, prefetch request 202 does not return data 204 to L2 cache 108, L1 cache 106, or DMA engine 150 after data 204 has been cached in L3 cache 122. Therefore, in this example, data 204 is selectively copied only to L3 cache 122. By not copying data 204 to other levels of the cache hierarchy or DMA engine 150, prefetch request 202 saves bandwidth and power between L3 cache 122, L2 cache 108, L1 cache 106, and DMA engine 150, as well as memory outages. In some implementations, an acknowledgment (not shown) is propagated to DMA engine 150 to notify DMA engine 150 that prefetching of L3 cache 122 has been completed, even if data 204 has not been propagated through L3 cache 122. In other implementations, if prefetch request 202 is discarded due to being a low-priority case in a contention-based system, the return (and acknowledgment) is discarded entirely.

[0032] Using the DMA engine 150 to prefetch data without storing it enables cache prefetching, avoiding the overhead of storage, the additional computational resources of the prefetch kernel, and the inaccuracies of hardware prefetchers. For example, if the host application programmer knows that a first kernel may need to access tens of megabytes of data (data X) and a second kernel may subsequently need to access different tens of megabytes of data (data Y), then after the first kernel is booted into the small chipset's compute unit, the application submits commands to the small chipset's DMA engine to prefetch data Y. The prefetching of data Y occurs while the first kernel is executing and loading data Y into one or more caches in preparation for the execution of the second kernel.

[0033] If this type of prefetch is performed by a regular DMA engine copy command that causes the DMA engine to asynchronously copy data Y to a specific location, that location must be assigned, and storage at that location will incur bandwidth overhead. If the prefetch is performed by the prefetch kernel, the prefetch kernel will compete for computational resources with the first kernel. Finally, if this type of prefetch is performed by a hardware prefetcher, the hardware prefetcher may fail to prefetch data Y if it is not associated with the access patterns of the first kernel, because hardware prefetchers typically perform prefetching based on heuristics that predict future memory accesses based on recent memory accesses.

[0034] Figure 3 This is a block diagram 300 illustrating, according to some embodiments, CU 104 sending a prefetch command 302 to DMA engine 150 and DMA engine 150 sending a prefetch request 202 based on command 302. Queue 306 is associated with DMA engine 150. Queue 306 holds one or more commands, such as command 302 for DMA engine 150. In the illustrated embodiment, command 302 includes an indication 304 of the priority of command 302. If indication 304 indicates that command 302 has a low priority, in some embodiments, DMA engine 150 ignores command 302 to support higher priority work, and executes command 302 only if DMA engine 150 would otherwise be idle.

[0035] The scheduler (not shown) schedules a command buffer from the head entry of queue 306 for execution at DMA engine 150. In some cases, CU 104 operates in user mode, enabling the scheduler to generate and schedule commands other than those received from a user-mode driver (not shown) in CPU 170. The scheduler can schedule commands for execution at DMA engine 150 without notifying CU 104. The scheduler provides the commands to queue 306. For example, in some embodiments, the scheduler schedules one or more commands to program DMA engine 150 to prefetch a block of data starting at address X.

[0036] During operation, in response to a notification (e.g., a doorbell ring), DMA engine 150 reads from DMA queue 306 and executes DMA prefetch command 302 (and its associated parameters and priority indication 304) to perform a data prefetch operation. In various implementations, the operation parameters (e.g., DMA command parameters) are typically the base address, stride, element size, and number of elements to be communicated for both the sender and receiver.

[0037] Based on command 302, DMA engine 150 generates a prefetch request, such as request 202, to be provided to queue 308. In the illustrated embodiment, prefetch request 202 includes an indication 314 of the priority of prefetch request 202. As prefetch request 202 propagates through processing system 100, a cache controller for caches along the propagation path reads indication 314 and determines, based on cache replacement policy and indication 314, whether to copy and store the data returned in response to the request at its corresponding cache location and where to store the data in the replacement chain.

[0038] Figure 4 This is a block diagram of a processing system 100 400 according to some implementation schemes, illustrating a prefetch request 202 from a DMA engine 150 propagated to system memory 140 via a PCIe bus 420, and intermediate cache controllers 406, 408, 410 selectively copying the requested data 204 based on a cache replacement policy.

[0039] In the illustrated example, DMA engine 150 issues a prefetch request 202 via PCIe bus 420. Prefetch request 202 first propagates to L1 cache controller 406. L1 cache controller 406 determines whether the data 204 requested in prefetch request 202 is stored in L1 cache 106. If the data is stored in L1 cache 106, request 202 does not propagate further along PCIe bus 420. Data 204 is returned to DMA engine 150, which does not store the data (i.e., DMA engine 150 discards data 204).

[0040] If data 204 is not stored in L1 cache 106, request 202 is then propagated to L2 cache controller 408, which determines whether the data 204 requested in prefetch request 202 is stored in L2 cache 108. If data 204 is stored in L2 cache 108, the data is returned to DMA engine 150 via PCIe bus 420. As data 204 propagates through L1 cache controller 406, L1 cache controller 406 determines whether to copy data 204 to L1 cache 106 based on L1 cache 106's cache replacement policy and data 204's priority indication 314.

[0041] If data 204 is not stored in L2 cache 106, prefetch request 202 is then propagated to L3 cache controller 410, which determines whether the data 204 requested in prefetch request 202 is stored in L3 cache 122. If data 204 is stored in L3 cache 122, data 204 is returned to DMA engine 150 via PCIe bus 420. As data 204 propagates through L2 cache controller 408 and L1 cache controller 406, L2 cache controller 408 and L1 cache controller 406 determine, based on the cache replacement policies of L2 cache 108 and L1 cache 106 and instruction 314, whether to copy data 204 to L2 cache 108 and L1 cache 106 respectively.

[0042] If data 204 is found in any of the L1 cache 106, L2 cache 108, or L3 cache 122, then satisfying the prefetch request 202 at any of the caches results in a cache hit. This affects the priority of data 204 according to the cache replacement policy of the respective cache. For example, if data 204 is found in L2 cache 108 and L2 cache 108 uses a Least Recently Used (LRU) cache replacement policy, then data 204 will be considered the most recently used and therefore unlikely to be evicted from L2 cache 108.

[0043] If data 204 is not stored in L3 cache 122, a prefetch request is propagated to memory controller 130. Memory controller 130 retrieves the requested data 204 from system memory 140. As the requested data 204 propagates along PCIe bus 420 toward DMA engine 150, each corresponding cache controller determines whether to copy data 204 to its corresponding cache and where in the replacement chain to place data 204 within the cache, based on a cache replacement policy for its corresponding cache and an indication 314 of the priority of data 204. Thus, for example, L3 cache controller 410 determines whether to copy data 204 to L3 cache 122 based on the cache replacement policy of L3 cache 122. Additionally, in some embodiments, L3 cache controller 410 determines where to place data 204 in the priority chain of L3 cache 122.

[0044] Similarly, as data 204 propagates further along the PCIe bus 420 toward the DMA engine 150, the L2 cache controller 408 determines, based on the cache replacement policy and indication 314 of the L2 cache 108, whether to copy data 204 to the L2 cache 108 and where to place data 204 in the priority chain of the L2 cache 108. The L1 cache controller 406 also determines, based on the cache replacement policy and indication 314 of the L1 cache 106, whether to copy data 204 to the L1 cache 106 and where to place data 204 in the priority chain of the L1 cache 106.

[0045] Figure 5 An example of a cache controller 510 according to some embodiments is illustrated, which selects where in the reference chain 500 of cache 502 data prefetched by the DMA engine to be inserted based on a cache replacement policy 512. According to some embodiments, data is assigned a replacement priority based on the cache replacement policy 512. In some embodiments, the data is further prioritized based on an indication 314 of the priority of the requested data. If a non-random replacement policy is used, cache 502 effectively maintains an ordered list of eviction candidates, which may be referred to as a reference chain of entries in the case of LRU or Most Recently Used (MRU) type policies, where accessed entries are promoted to the head 504 of the chain according to the replacement policy. Under the LRU policy, entries are evicted from the tail 506 of the chain, while under the MRU policy, entries are evicted from the head 504 of the chain.

[0046] In the illustrated example, cache 502 includes a reference chain 500 for cache entries. Under a cache replacement policy that neither supports nor supports data returned in response to a prefetch request from the DMA engine, data fetched into cache 502 is initially placed in the middle of the reference chain (i.e., the data is assigned a neutral replacement priority). If data is subsequently requested, the data moves toward the head 504 of the reference chain. When additional data is fetched into cache 502, data initially stored in the middle of the reference chain, if not requested, moves toward the bottom of the reference chain, referred to as the tail 506. When data is evicted from cache 502, it is replaced from the tail 506 of the reference chain. Under cache replacement policy 512, cache controller 510 assigns all data 508 (including data copied to cache 502 in response to DMA prefetch request 202) to the same priority within the reference chain when it is first fetched into cache 502.

[0047] If cache replacement policy 512 prioritizes data 204 copied to cache 502 in response to DMA prefetch request 202, cache controller 510 assigns data 204 a priority to place it at the head 504 of reference chain 500. Therefore, other data is more likely to arrive at the tail 506 of reference chain 500 before DMA prefetch data 204, and thus faces a higher probability of being evicted from cache 502 than prefetched data 204.

[0048] Figure 6 This is a flowchart illustrating a method 600 for prefetching data using a DMA engine according to some implementations. In some implementations, method 600 is implemented at a processing system (such as processing system 100).

[0049] At block 602, DMA engine 150 receives command 302 to load data without storing the data. In some embodiments, command 302 is issued by CU 104 associated with DMA engine 150, and in other embodiments, command 302 is issued by application 172 executing at CPU 170. In some embodiments, command 302 is stored in queue 306 and includes an indication 304 of the priority of command 302. If indication 304 indicates that command 302 has a low priority, in some embodiments, DMA engine 150 ignores command 302 to support higher priority work.

[0050] At block 604, DMA engine 150 sends a request for data indicated by command 302. In some embodiments, the request is a prefetch request 202, and in other embodiments, the request is a request to load data without storing the data. DMA engine 150 propagates request 202 to system memory 140 along PCIe bus 420 or other communication medium employed by processing system 100. As request 202 propagates through the cache hierarchy of processing system 100, at block 606, each cache controller of each cache in the cache hierarchy checks whether the requested data is stored at its corresponding cache location.

[0051] At box 608, if the requested data 204 is found in the intermediate cache, the data is returned to the DMA engine 150. If the requested data 204 is not found in any cache in the cache hierarchy, the memory controller 130 returns the data 204 from the system memory 140.

[0052] At box 610, each cache controller between the cache where the data is located and the DMA engine 150 selectively copies the data 204 to its corresponding cache based on the cache replacement policy and the priority of the data 204, indicated by instruction 314.

[0053] At block 612, after data 204 has been selectively copied to an intermediate cache based on its corresponding cache replacement policy and the indication 314 of data 204's priority, DMA engine 150 bypasses storing data 204. Therefore, DMA engine 150 does not allocate a location for storing data 204, and in some embodiments, DMA engine 150 discards data 204. DMA engine 150 thus prefetches data 204 into one or more caches in the cache hierarchy without performing storage of data 204.

[0054] In some implementations, the above-described apparatus and techniques are implemented in systems including one or more integrated circuit (IC) devices (also referred to as integrated circuit packages or microchips), such as those described above. Figures 1 to 6 The described processing system. Electronic design automation (EDA) and computer-aided design (CAD) software tools can be used in the design and manufacture of these IC devices. These design tools are typically represented as one or more software programs. One or more software programs include code executable by a computer system to manipulate the computer system to operate on code representing a circuit of one or more IC devices to perform at least a portion of a process for designing or adapting a manufacturing system to manufacture the circuit. The code may include instructions, data, or a combination of instructions and data. Software instructions representing design or manufacturing tools are typically stored in a computer-readable storage medium accessible to the computing system. Similarly, code representing one or more stages of the design or manufacture of an IC device may be stored in or accessed from the same computer-readable storage medium or different computer-readable storage media.

[0055] One or more of the aforementioned elements are circuits designed and configured to perform the corresponding operations described above. In at least some embodiments, this circuit is any one or a combination of the following: hard-coded circuitry (e.g., a corresponding portion of an application-specific integrated circuit (ASIC) or a set of logic gates, memory elements, and other components selected and arranged to perform the described operations), programmable circuitry (e.g., a corresponding portion of a field-programmable gate array (FPGA) or a programmable logic device (PLD), or one or more processors executing software instructions that cause one or more processors to perform the described actions. In some embodiments, the circuitry of a particular element is selected, arranged, and configured by one or more computer-implemented design tools. For example, in some embodiments, the sequence of operations for a particular element is defined in a specified computer language (such as register-transfer language), and the computer-implemented design tools select, configure, and arrange the circuitry based on that defined sequence of operations.

[0056] Within this disclosure, in some instances, different entities (which are referred to differently as “components,” “units,” “devices,” “circuits,” etc.) are described or claimed to be “configured” to perform one or more tasks or operations. This expression (i.e., an [entity] configured to [perform one or more tasks]) is used herein to refer to a structure (i.e., a physical structure, such as an electronic circuit). More specifically, this expression is used to indicate that the physical structure is arranged to perform the one or more tasks during operation. A structure may be referred to as being “configured” to perform a task even if the structure is not currently operating. “Memory device configured to store data” is intended to cover, for example, an integrated circuit having circuitry for storing data during operation, even if the integrated circuit in question is not currently in use (e.g., a power supply is not connected to the integrated circuit). Therefore, an entity described or stated as being “configured” to perform a task refers to a physical structure, such as a device, circuit, memory storing program instructions that can be executed to perform the task, etc. This phrase is not used herein to refer to an intangible structure.

[0057] Furthermore, the term "configured as" is not intended to mean "configurable as". For example, an unprogrammed field-programmable gate array would not be considered "configured as" to perform a particular function, but it could be "configurable as" to perform that function after programming. Additionally, the statement in the appended claims that a structure is "configured as" to perform one or more tasks is not expressly intended to be interpreted as having components plus functional elements.

[0058] Computer-readable storage media may include any non-transitory storage medium or a combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., optical discs (CDs), digital versatile optical discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tapes, or magnetic hard disks), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS) based storage media. Computer-readable storage media may be embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard disk drive), removably attached to a computing system (e.g., an optical disc or a USB-based flash memory), or coupled to a computer system via a wired or wireless network (e.g., a network-accessible storage device (NAS)).

[0059] In some implementations, certain aspects of the above-described techniques may be implemented by one or more processors of a processing system executing the software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer-readable storage medium. The software may include instructions and certain data that, when executed by one or more processors, manipulate one or more processors to perform one or more aspects of the above-described techniques. The non-transitory computer-readable storage medium may include, for example, disk or optical disk storage devices, solid-state storage devices such as flash memory, cache, random access memory (RAM), or one or more other non-volatile memory devices. The executable instructions stored on the non-transitory computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats that are interpreted or otherwise executed by one or more processors.

[0060] It should be noted that not all activities or elements described above in the general description are essential. A particular activity or part of the apparatus may not be essential, and one or more additional activities may be performed, or elements may be included in addition to those described. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Additionally, these concepts have been described with reference to specific embodiments. However, those skilled in the art will understand that various modifications and changes can be made without departing from the scope of this disclosure as set forth in the following claims. Therefore, the specification and drawings are to be considered illustrative rather than restrictive, and all such modifications are intended to be included within the scope of this disclosure.

[0061] The benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and any features that may lead to or make any benefit, advantage, or solution appear or become more significant should not be construed as key, essential, or fundamental features of any or all claims. Furthermore, the specific embodiments disclosed above are merely illustrative, as the disclosed subject matter can be modified and practiced in different but equivalent ways that will be apparent to those skilled in the art who benefit from the teachings herein. No limitation is intended on the details of the constructions or designs shown herein, except as described in the following claims. Therefore, it will be apparent that the specific embodiments disclosed above can be changed or modified, and all such changes are considered to be within the scope of the disclosed subject matter. Therefore, the protection sought herein is as set forth in the following claims.

Claims

1. A method, the method comprising: In response to a request from the Direct Memory Access (DMA) engine associated with the processor, load data from memory; The data is selectively copied to one or more caches of the processor located between the DMA engine and the memory; as well as Bypass the storage of the data used by the DMA engine.

2. The method of claim 1, wherein the request is a prefetch request.

3. The method according to claim 1 or claim 2, further comprising: The DMA engine receives a command to send the request to load the data without storing it.

4. The method of claim 3, wherein the command indicates the priority of the request.

5. The method according to any one of claims 1 to 4, wherein the memory is one of system memory or a second cache of the processor.

6. The method according to any one of claims 1 to 5, wherein the data is selectively copied to the one or more caches based on a cache replacement strategy.

7. The method of claim 6, wherein the cache replacement policy determines that the data is copied to one or more caches therein.

8. The method of claim 6, wherein the cache replacement strategy determines the replacement priority of the data.

9. A processing system, the processing system comprising: processor; Memory; A direct memory access (DMA) engine, which is configured to load data from the memory; and One or more cache controllers are configured to selectively copy the data to one or more caches of the processor located between the DMA engine and the memory, wherein the DMA engine is further configured to bypass storing the data.

10. The processing system of claim 9, wherein the DMA engine is further configured to bypass storing the data in response to receiving a command to send a request to load the data without storing the data.

11. The processing system of claim 10, wherein the request is a prefetch request.

12. The processing system of claim 10 or claim 11, wherein the command indicates the priority of the command.

13. The processing system according to any one of claims 9 to 12, wherein the memory is one of system memory or a second cache of the processor.

14. The processing system according to any one of claims 9 to 13, wherein the data is selectively copied to the one or more caches based on a cache replacement strategy.

15. The processing system of claim 14, wherein the cache replacement policy determines at least one of the one or more caches in which the data is copied and the replacement priority of the data.

Images