Prefetch disable of memory requests targeting data that lacks locality

Abstract

Systems and methods are described for efficiently processing memory requests. A processing unit includes at least a processor core, a cache, and a non-cache storage buffer capable of storing data that is blocked from being stored in the cache. In processing a memory request targeted to the non-cache storage buffer, the processor core checks a flag stored in a tag of the memory request. Based on determining that the flag specifies that data is to be blocked from being prefetched into one or more of the target address of the memory request, the non-cache storage buffer, and the cache during processing of the instance of the memory request, the processor core blocks data from being prefetched into one or more of the non-cache storage buffer and the cache. In processing a prefetch hint instruction, the processor core determines whether to block prefetching based on the tag.

Description

Background Technology

[0001] Description of related technologies

[0002] With advancements in semiconductor manufacturing processes and reductions in die geometry, semiconductor chips comprising one or more processing units offer greater functionality and performance. For example, a semiconductor chip may include one or more processing units. A processing unit can represent one of a variety of data processing integrated circuits. Examples of processing units include general-purpose central processing units (CPUs), multimedia engines for audio / video (A / V) data processing, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and parallel data processing engines such as graphics processing units (GPUs), digital signal processors (DSPs), etc. Processing units handle instructions from a general-purpose instruction set architecture (ISA), digital signals, analog signals, mixed-signal signals, and radio frequency (RF) functions.

[0003] However, modern technology still presents design challenges in processing and integrated circuit design that can limit potential benefits. One issue is the ever-increasing interconnect latency per unit length, the higher impedance between individual chips also increasing latency, and the fact that most software applications accessing large amounts of data are typically memory-limited, as computation time is often determined by memory bandwidth. The performance of one or more computing systems depends on rapid access to stored data. Memory access operations include read operations, write operations, memory-to-memory copy operations, and so on.

[0004] To address many of the aforementioned problems, one or more processing units incorporate one or more levels of a cache hierarchy as part of a memory hierarchy to reduce the latency of memory access requests that access copies of data in system memory for read or write operations. Representative memory hierarchies range from small, relatively fast volatile static memory (such as registers located on the semiconductor die of the processor core and caches located on or connected to the die) to larger, volatile dynamic off-chip memory, and then to relatively slower non-volatile memory. Generally, a cache stores one or more blocks, each of which is a copy of the data stored at the corresponding address in system memory.

[0005] One or more caches are populated by both on-demand memory requests generated by the instruction being processed and prefetch memory requests generated by the prefetch engine. The prefetch engine attempts to hide off-chip memory latency by detecting patterns of memory accesses to off-chip memory that could cause the processor to pause and initiate a memory access to off-chip memory before the corresponding request instruction. The simplest pattern is a sequence of memory accesses referencing a set of consecutive cache lines (blocks) in a monotonically increasing or decreasing manner. In response to the detection of this memory access sequence, the prefetch unit begins prefetching a specific number of cache lines preceding the currently requested cache line. However, since caches have a finite size, the total number of cache blocks is inherently limited. Additionally, there are limitations on the number of blocks mapped to a given set in a set-associative cache. Sometimes, there are beneficial situations where a finer limit is placed on the number of cache blocks associated with a particular instruction type, compared to the limitations provided by cache capacity or cache associativity.

[0006] In view of the above, there is a need for efficient methods and mechanisms to handle memory requests efficiently. Attached Figure Description

[0007] Figure 1 It is a schematic diagram of one implementation scheme of the computing system.

[0008] Figure 2 It is a schematic diagram of one implementation of the processor core.

[0009] Figure 3 This is a schematic diagram of another implementation of a method for handling memory requests lacking locality.

[0010] Figure 4 This is a schematic diagram of another implementation of a method for efficiently processing memory requests.

[0011] Figure 5 This is a schematic diagram of another implementation of a method for efficiently processing memory requests.

[0012] While the invention may have various modifications and alternatives, specific embodiments are shown by way of example in the accompanying drawings and are described in detail herein. However, it should be understood that the drawings and the detailed description thereof are not intended to limit the invention to the specific forms disclosed, but rather, the invention covers all modifications, equivalents, and alternatives falling within the scope of the invention as defined by the appended claims. Detailed Implementation

[0013] In the following description, numerous specific details are set forth to provide a thorough understanding of the invention. However, those skilled in the art will recognize that the invention can be practiced without these specific details. In some instances, well-known circuits, structures, and techniques have not been shown in detail to avoid obscuring the invention. Furthermore, it should be understood that, for the sake of simplicity and clarity, the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some elements are enlarged relative to other elements.

[0014] Imagine systems and methods for efficiently processing memory requests. A semiconductor chip may include one or more processing units. A processing unit can represent one of various data processing integrated circuits. Examples of processing units are general-purpose central processing units (CPUs), multimedia engines for audio / video (A / V) data processing, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and parallel data processing engines such as graphics processing units (GPUs), digital signal processors (DSPs), etc. A semiconductor chip's processing unit includes at least one processor core and access to at least one cache (on-die or off-die), such as a level 1 (L1) cache.

[0015] The processor core also uses a non-cacheable memory buffer capable of storing data that is prevented from being stored in the cache. In various implementations, the processor core prevents specific data from being stored in the cache based on an indication in the opcode of a memory access instruction that the requested data lacks one or more of temporal and spatial locality. The processor core prevents the requested data from being stored in the cache to reduce the penalty of cache pollution. Examples of these penalties are performance degradation, increased power consumption, and increased memory bus utilization. If requested data lacking one or more of temporal and spatial locality is stored in the cache, the number of cache misses increases. For example, when a particular set in a set-associative cache organization is full, data that is localized and already stored in that particular set is evicted to create a storage location for that data. This is an example of cache pollution. Similarly, due to the recent placement of data in the cache set, other cacheable data in the cache set may now be candidates for earlier evicting before that data. This is another example of cache pollution.

[0016] Preventing data lacking one or more of temporal and spatial locality from being stored in the cache reduces the penalty of cache pollution. In various implementations, one or more of the software designer, compiler, or other software determines which memory access instructions target data lacking one or more of temporal and spatial locality. In one example, the compiler samples an access counter to make this determination and then modifies the opcodes of these memory access instructions to include opcodes specifying that the requested data for the memory access instruction lacks one or more of temporal and spatial locality.

[0017] The processor core's Load / Store Unit (LSU) issues memory access instructions that request data lacking one or more of the temporal and spatial locality properties to a non-cached memory buffer that stores data that is prevented from being stored in the cache, rather than issuing these instructions to the cache. As used herein, "memory request" is also referred to as "memory access request" and "access request." "Memory access request" includes "read access request," "read request," "load instruction," "write access request," "write request," "store instruction," and "snoop request."

[0018] As used herein, "data prefetching" or "prefetching" refers to the operation of prefetching data from lower-level memory into one or more cached and non-cached memory buffers. The prefetched data includes any of the application's instructions, application source data, intermediate data, and result data. In various embodiments, processor core circuitry examines an indication stored in a tag of a memory request that specifies whether data prefetching should be blocked or allowed during processing of that instance of the received memory request. In some embodiments, the tag of the received memory request includes this indication. In various embodiments, when a tag indicates that data prefetching should be blocked during processing of that instance of the received memory request, other memory requests with the same target address (such as newer, program-sequential memory requests and other unprocessed instances of that memory request) may be allowed to be prefetched for that target address. Each of the tags of those memory requests also includes a flag indicating whether data prefetching should be allowed or blocked during processing of those instances of the memory requests. Therefore, control of data prefetching is at the granularity of the instance of the memory request being processed, rather than at the granularity of the target address.

[0019] A prefetch engine is capable of performing operations such as prefetching data or data prefetching. As used herein, "prefetch engine" is also referred to as "prefetcher". Prefetch operations include performing one or more of prefetch training and generating prefetch requests. For example, a prefetch engine can generate one or more memory access requests (prefetch requests) for data before the processor issues an on-demand memory access request targeting the prefetched data. Additionally, a prefetch engine can perform prefetch training to determine whether to generate one or more prefetch requests based on the received target address.

[0020] During training, the prefetch engine identifies one or more of the following: memory access sequences referencing a set of contiguous cache lines and memory access sequences referencing a set of cache lines separated by strides. It is possible and expected that the prefetch engine will be able to identify other types of memory access patterns. When the prefetch engine identifies a memory access pattern, it generates one or more prefetch requests based on the identified pattern. In some implementations, the prefetch engine stops generating prefetch requests when it determines that on-demand memory access provided by the processor core includes a request address that does not match the identified memory access pattern.

[0021] The LSU also executes prefetch hints. As used herein, a "prefetch hint" is a memory access instruction among various types of memory access instructions inserted into an application to prefetch data into one or more cached and non-cached data storage devices before a newer (in program order) on-demand memory access instruction requests that specific data. Examples of such prefetch hints are the AMD 64-bit instruction PREFETCHNTA for data lacking temporal locality, PREFETCH1 for prefetching data into the L1 cache, etc. However, the processor core processes a prefetch hint instruction differently from other types of prefetch hint instructions due to a non-temporal indication in a particular prefetch hint instruction, such as the AMD 64-bit instruction PREFETCHNTA for data lacking temporal locality. In various implementations, the processor core's circuitry examines an indication stored in the label of the prefetch hint instruction that specifies whether to block or allow data prefetching during processing of that instance of the prefetch hint instruction. The indication stored in the tag of the prefetch cue instruction is processed in a manner similar to that described above for memory requests targeting data that lack one or more of the characteristics of temporal locality and spatial locality.

[0022] refer to Figure 1 A schematic block diagram of one embodiment of a computing system 100 is shown. The computing system 100 includes processing units 110A to 110B, interconnect unit 118, shared cache memory subsystem 120, and memory controller 130 capable of communicating with memory 140. For ease of illustration, Figure 1 Clock sources such as phase-locked loops (PLLs), interrupt controllers, power managers, input / output (I / O) interfaces, and devices are not shown. Note also the number of components in computing system 100 and... Figure 1 The number of sub-components of those components shown (such as within each of processing units 110A to 110B) may vary depending on the implementation. The number of each component / sub-component may be more or less than the number shown in computing system 100.

[0023] In one embodiment, the illustrated functionality of computing system 100 is integrated onto a single integrated circuit. For example, computing system 100 is a system-on-a-chip (SoC) that includes multiple types of integrated circuits on a single semiconductor die. These multiple types of integrated circuits provide individual functions. In other embodiments, the multiple integrated components are individual dies packaged within a package such as a system-in-package (SiP), multi-chip module (MCM), or chipset. In other embodiments, the multiple components are individual dies or chips on a printed circuit board.

[0024] As shown in the figure, processing units 110A to 110B include one or more processor cores 112A to 112B and corresponding cache memory subsystems 114A to 114B. Processor cores 112A to 112B include circuitry for processing instructions. Processing units 110A to 110B represent one of various data processing integrated circuits. Examples of processing units are general-purpose central processing units (CPUs), multimedia engines for audio / video (A / V) data processing, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and parallel data processing engines such as graphics processing units (GPUs), digital signal processors (DSPs), etc. Each processing unit in processing units 110A to 110B processes instructions of a general-purpose instruction set architecture (ISA), digital signals, analog signals, mixed-signal signals, and radio frequency (RF) functions.

[0025] Processor cores 112A to 112B support simultaneous multithreading. Multiple threads executed by processor cores 112A to 112B share at least a shared cache memory subsystem 120, other processing units (not shown) of different processing types, and I / O devices (not shown). Interconnect unit 118 includes one or more of a router, switch, bus, point-to-point connection, queue, and arbitration circuitry for routing packets between components. In some embodiments, interconnect unit 118 is a communication structure. Packets include various types such as memory access requests, responses, commands, messages, and listen requests. Interfaces 116A to 116B support communication protocols for routing data through interconnect unit 118 to the shared cache memory subsystem 120, memory controller 130, I / O devices (not shown), power manager (not shown), and other components.

[0026] The address space of computing system 100 is partitioned among multiple memories. Memory mapping is used to determine which addresses are mapped to which memories. In one embodiment, the coherency point of an address is a memory controller 130 connected to memory 140 that stores data corresponding to that address. Memory controller 130 includes circuitry for interfacing to memory 140 and a request queue for queuing memory requests and responses. Although a single memory controller 130 and memory 140 are shown, in other embodiments, computing system 100 uses a different number of memory controllers and memories. In various embodiments, memory controller 130 receives memory requests from processing units 110A to 110B, schedules the memory requests, and issues the scheduled memory requests to memory 140.

[0027] Memory 140 is used as system memory in computing system 100. Memory 140 stores an operating system, which includes a scheduler for allocating software threads to hardware in processing units 110A to 110B. Memory 140 also includes one or more of the following: a hypervisor, basic input / output software (BIOS) control functions, one or more applications utilizing an application programmer interface (API), page tables storing virtual-to-physical address mappings and access permissions, application source data, and result data. Memory 140 uses any of a variety of memory devices.

[0028] A copy of a portion of the data stored in memory 140 is stored in one or more caches (114A to 114B, 120). The memory hierarchy transitions from relatively fast volatile memory (such as registers on the processor die and caches located on or connected to the processor die) to non-volatile and relatively slow memory. In some embodiments, the faster volatile memory is considered to be at the top or highest level of the memory hierarchy, while the slower non-volatile memory is considered to be at the bottom or lowest level of the memory hierarchy. In these embodiments, a first level in the memory hierarchy where the faster volatile memory is located closer to the hierarchy than a second level in the memory hierarchy is considered to be at a level "higher" than the second level. In other embodiments, the slower non-volatile memory is considered to be at the top or highest level of the memory hierarchy. Although both ways of describing the memory hierarchy are possible and contemplated, in the following description, the faster volatile memory is considered to be at the top or highest level of the memory hierarchy. Therefore, higher levels of the memory hierarchy include faster, volatile memory, such as processor registers and L1 local cache, while lower levels of the memory hierarchy include slower, non-volatile memory, such as hard disk drives (HDDs) or solid-state drives (SSDs).

[0029] Cache memory subsystems 114A to 114B and 120 use high-speed cache memory to store data blocks. In some embodiments, cache memory subsystems 114A to 114B are integrated within the corresponding processor cores 112A to 112B. Alternatively, cache memory subsystems 114A to 114B are connected to processor cores 112A to 112B in a back-end cache configuration or an inline configuration as needed. In various embodiments, cache memory subsystems 114A to 114B are implemented as a cache hierarchy. One or more tiers of the cache hierarchy include a translated back cover buffer (TLB) for storing virtual-to-physical address mappings and access permissions, a tag array, a data array, and a cache controller. One or more tiers of the cache hierarchy also use a prefetch engine capable of generating prefetch requests to populate the cache with data from lower-level memory. If necessary, the cache (within the hierarchy) closer to processor cores 112A to 112B is integrated into processor core 112. In one implementation, cache memory subsystems 114A to 114B each represent L1 and L2 cache structures, and shared cache subsystem 120 represents a shared L3 cache structure. Other cache arrangements are possible and anticipated.

[0030] The circuitry of processor cores 112A to 112B implements non-cached memory buffers 113A to 113B using one or more registers, one type of flip-flop circuit, one type of random access memory (RAM), and content-addressable memory (CAM). Processor cores 112A to 112B use non-cached memory buffers 113A to 113B capable of storing data that is blocked from being stored in the cache. Memory 140 stores one or more of the irregular memory bandwidth limitations required for high-performance computing (HPC) and data center applications, such as data with irregular memory access (e.g., data lacking one or more of temporal and spatial locality). HPC applications are used in computational fluid dynamics, dense linear algebra libraries, climate and atmospheric modeling using high-order method modeling environments (HOMME), etc. Examples of data center applications are social network crawlers, cloud flow analysis applications, and internet mapping applications.

[0031] Processor cores 112A to 112B issue memory requests to non-cache memory buffers 113A to 113B that target data lacking one or more of temporal and spatial locality, rather than issuing these types of memory requests to caches (such as the L1 cache of processor cores 112A to 112B). In various embodiments, non-cache memory buffers 113A to 113B use prefetchers. The prefetchers of non-cache memory buffers 113A to 113B are capable of generating prefetch requests to fill data from lower-level memory into non-cache memory buffers 113A to 113B. Similarly, the L1 cache of cache memory subsystems 114A to 114B uses prefetchers 115A to 115B capable of generating prefetch requests to fill data from lower-level memory into the L1 cache. The prefetchers of computing system 100 are capable of using various prefetching methods known to those skilled in the art.

[0032] Circuits in processor cores 112A to 112B examine indications (such as flags) stored in tags of memory requests targeting data lacking one or more of temporal and spatial locality. These flags indicate whether data prefetching based on the target address of the memory request is permitted or prohibited during processing of these instances of the memory request. The flag is a field of one or more bits specifying whether data prefetching is permitted or prohibited. If the flag specifies that data prefetching is prohibited during processing of that instance of the received memory request, then data prefetching is prohibited during processing of that instance of the received memory request. For example, circuits in processor cores 112A to 112B prevent data prefetching based on the target address of the received memory request by one or more of the prefetchers 113A to 113B and prefetch engines 115A to 115B during processing of that instance of the received memory request. Similarly, circuits in processor cores 112A to 112B examine flags stored in tags of prefetch hint instructions and perform data prefetching in a similar manner based on the values ​​stored in those flags.

[0033] refer to Figure 2 This illustrates one implementation of a general-purpose processor core 200 that performs out-of-order execution. In one implementation, the processor core 200 performs processing in units such as ( Figure 1 Two or more threads may be processed simultaneously in one of the processing units 112A to 112B. The functionality of the processor core 200 is implemented by hardware such as circuitry. The instruction cache (i-cache) of block 202 stores instructions for software applications, and the corresponding instruction translation lookup buffer (TLB) of block 202 stores the virtual-to-physical address mapping required to access the instructions. In some embodiments, the instruction TLB (i-TLB) also stores the access permission corresponding to the address mapping.

[0034] If no miss is found in the instruction cache or instruction TLB of block 202, the instruction fetch unit (IFU) 204 fetches multiple instructions from the instruction cache 202 per clock cycle. IFU 204 includes a program counter that holds a pointer to the address of the next instruction to be fetched from the instruction cache 202, which is compared with the address mapping in the instruction TLB. IFU 204 also includes a branch prediction unit (not shown) to predict the result of a conditional instruction before the execution unit determines the actual result in a later pipeline stage.

[0035] Decoder unit 206 decodes the opcodes of multiple fetched instructions and allocates entries in an ordered deregistration queue (such as reordering buffer 218), in reservation station 208, and in load / store unit (LSU) 220. In some embodiments, decoder unit 206 also performs register renaming. In other embodiments, reordering buffer 218 performs register renaming. In some embodiments, decoder 206 generates multiple micro-ops from a single fetched instruction. In various embodiments, at least a memory request targeting data lacking one or more of temporal and spatial locality includes an indication (such as a flag within a tag) specifying whether data prefetching based on the target address of the memory request is permitted or prohibited during processing of those instances of the memory request. In some embodiments, decoder 206 inserts this flag into a tag of the decoded memory request. In various embodiments, decoder 206 determines the value of the flag based on the opcode or other fields of the received memory request. In one implementation, when the flag is asserted, prefetching based on the target address of the received memory request is performed in a later pipeline stage during processing of that instance of the received memory request; however, when the flag is negated, prefetching based on the target address of the received memory request is prevented in a later pipeline stage during processing of that instance of the received memory request. It should be noted that in some designs, a binary logic high value is used as the assertion value and a binary logic low value is used as the negation value, while in other designs, a binary logic low value is used as the assertion value and a binary logic high value is used as the negation value. Other values ​​indicating the assertion and negation flags are possible and anticipated. For example, in some implementations, the flag includes two or more bits, and the flag is capable of indicating that prefetching is allowed for one type of memory device (such as a non-cached memory buffer) but prevented for another type of memory device (such as a cache). Other combinations of indications are possible and anticipated.

[0036] Reservation station 208 acts as an instruction queue, where instructions wait until their operands become available. When operands are available and hardware resources are also available, the circuitry of reservation station 208 issues instructions out of order to integer and floating-point function units 210 or load / store unit 220. Function unit 210 includes an arithmetic logic unit (ALU) for performing operations such as addition, subtraction, multiplication, division, and square root. Additionally, the circuitry in function unit 210 determines the result of conditional instructions such as branch instructions.

[0037] Load / store unit (LSU) 220 receives memory requests, such as load and store operations, from one or more of decode unit 206 and retention station 208. Load / store unit 220 includes queues and circuitry to execute memory requests. In one embodiment, load / store unit 220 includes verification circuitry to ensure that a load instruction receives forwarded data from the correct, latest store instruction. In various embodiments, load / store unit 220 uses a non-cached memory buffer 222, which has a... Figure 1 The non-cacheable memory buffers 113A to 113B have equivalent functionality. In various embodiments, the circuitry of the load / store unit 220 implements the non-cacheable memory buffer 222 using registers, one of various flip-flop circuits, one of various types of random access memory (RAM), or content addressable memory (CAM).

[0038] Non-cached storage buffer 222 is capable of storing data that is prevented from being stored in a cache such as block 230, such as a Level 1 (L1) data cache (d cache). In one embodiment, non-cached storage buffer 222 stores data lacking one or more of temporal and spatial locality. Prefetcher 224 is capable of performing data prefetching operations as described above. In response to a flag in a determined memory request label specifying that prefetching based on the target address of the memory request is prevented during processing of that instance of the memory request, one or more of LSU 220 and prefetcher 224 prevent data prefetching based on the target address of the memory request during processing of that instance of the memory request. Conversely, in response to a flag specifying that prefetching based on the target address of the memory request is permitted during processing of that instance of the memory request, prefetcher 224 performs data prefetching based on the target address of the memory request during processing of that instance of the memory request.

[0039] Load / store unit 220 issues memory requests to the Level 1 (L1) data cache (d cache) of block 230. The L1 data cache of block 230 uses a TLB, tag array, data array, cache controller, and prefetch engine 232. The cache controller uses various circuitry and queues, such as read / write request queues, miss queues, read / write response queues, read / write schedulers, and fill buffers. Similar to prefetcher 224, prefetch engine 232 is capable of performing data prefetching operations as previously described. In some embodiments, prefetch engine 232 is capable of generating prefetch requests to fill data from lower-level memory into one or more of the L1 data cache and non-cacheable memory buffers 222 of block 230. In one embodiment, prefetch engine 232 generates a prefetch request after monitoring multiple on-demand memory requests within an address range. The circuitry of block 230 receives on-demand memory requests from LSU 220.

[0040] Although no data is stored in the L1 data cache of block 230 for memory requests targeting non-cached memory buffer 222, in some embodiments, load / store unit 220 still sends an indication of a memory request to the L1 data cache of block 230. To prevent poor performance of the prefetch engine 232 due to training based on the target addresses of these types of memory requests, a flag is inserted into the tag of the memory request. As described earlier, this flag specifies whether data prefetching based on the target address of the memory request is allowed or blocked during the processing of those instances of the memory request.

[0041] In response to the determination flag specifying that prefetching based on the target address of the memory request is prevented during processing of the instance of the memory request, one or more of the circuitry in block 230 and the prefetch engine 232 prevent one or more steps of performing data prefetching based on the target address of the instance of the memory request. In other words, one or more of the circuitry in block 230 and the prefetch engine 232 prevent the prefetch engine 232 from generating a prefetch request and performing prefetch training based on the target address of the instance of the memory request. For prefetch cue instructions, the use of this flag is similar to using the flag for memory requests targeting data lacking one or more of temporal and spatial locality.

[0042] In some implementations, processor core 200 also includes a Level 2 (L2) cache 240, which serves memory requests from L1 data cache 230 and L1 instruction cache 202. The TLB of block 240 serves address mapping requests from the instruction TLB of block 202 and the data TLB of block 230. If the requested memory line is not found in the L1 data cache of block 230 or in the instruction cache of block 202, the corresponding cache controller sends a miss request to the L2 cache of block 240. When the requested memory line is not found in L2 cache 240, the L2 cache controller sends a miss request to access lower-level memory such as memory in the Level 3 (L3) cache or system memory.

[0043] Functional unit 210 and load / store unit 220 present results on common data bus 212. Reorder buffer 218 receives results from common data bus 212. In one implementation, reorder buffer 218 is a first-in-first-out (FIFO) queue that ensures instructions are de-escalated in program order. Here, the instruction whose result is received is marked as de-escalated. If the instruction is at the head of the queue, the circuitry of reorder buffer 218 sends its result to register file 214. Register file 214 maintains the architectural state of the general-purpose registers of processor core 200. Instructions in reorder buffer 218 are then de-escalated in order, and their queue head pointer is logically updated to point to subsequent instructions in program order. Results on common data bus 212 are also sent to reservation station 208 to forward values ​​to operands of instructions waiting for results. Multiple threads share multiple resources within core 200. For example, these multiple threads share... Figure 2 Each of blocks 202 to 240 shown.

[0044] It should be noted that in some implementations, processor core 200 uses separate buffers to implement non-cache memory buffer 222. For example, in one implementation, processor core 200 uses a “stream load buffer” capable of storing data targeted by load instructions that lack one or more of spatial and temporal locality, and a separate write combination buffer capable of storing data targeted by store instructions that lack one or more of spatial and temporal locality. These types of load instructions are also referred to as “stream load instructions.” Similarly, these types of store instructions are also referred to as “stream store instructions.”

[0045] Now for reference Figure 3 An embodiment of a method 300 for handling memory requests lacking locality is shown. For the purposes of discussion, this embodiment (and) is shown in sequence. Figures 4 to 5 The steps in the diagram are shown. However, in other embodiments, some steps occur in a different order than those shown, some steps are performed simultaneously, some steps are combined with other steps, and some steps are not present.

[0046] In various embodiments, the processing unit includes at least a processor core, a cache, and a non-cache memory buffer capable of storing data that is blocked from being stored in the cache. When processing a memory request targeting a non-cache memory buffer, the processor core sends the memory request to the circuitry of the non-cache memory buffer. The circuitry for the non-cache memory buffer receives the issued memory request targeting the non-cache memory buffer (box 302). In some embodiments, the non-cache memory buffer stores data that is not stored in the cache due to being targeted by a memory request lacking one or more of temporal and spatial locality. The access circuitry accesses the non-cache memory buffer according to the indication of the memory request (box 304). For example, the memory request indicates whether the memory access is a read access or a write access. The circuitry of the non-cache memory buffer examines a tag on the received memory request to determine whether data prefetching using the target address of the memory request is blocked or allowed during processing of that instance of the memory request (box 306). For example, in one embodiment, the tag includes a flag specifying whether data prefetching is blocked or allowed during processing of that instance of the received memory request.

[0047] If the label indicates that data prefetching is permitted during processing of that instance of the received memory request (the "Allow" branch of condition box 308), the prefetch engine of the non-cached memory buffer generates one or more prefetch requests based on the destination address of the received memory request to prefetch data into the non-cached memory buffer (box 310). In some embodiments, the destination address of the memory request is sent to the prefetch engine, which initiates sequential prefetching after multiple on-demand memory accesses within a monitored address range. The prefetch engine stops this type of sequential prefetching if a match with a sequential memory access operation forwarded to the prefetch engine fails. In other embodiments, the prefetch engine automatically prefetches multiple cache lines based on the destination address.

[0048] In some implementations, one or more of the circuitry for the load / store unit and the non-cached memory buffer send an indication to the cache of a memory access to the non-cached memory buffer (box 312). The cache receives a flag indicating whether to block or allow data prefetching during processing of that instance of the received memory request. The cache does not store the data requested by the received memory request, but the cache's prefetch engine is capable of performing data prefetching or prefetch training based on the target address (request address) of the received memory request. In various implementations, when the flag indicates that prefetching is allowed during processing of that instance of the received memory request, the cache's prefetch engine uses the target address to perform one or more of generating a prefetch request and training based on the target address. For example, the prefetch engine performs monitoring of multiple on-demand memory accesses within an address range.

[0049] If a tag indicates that data prefetching should be blocked during processing of that instance of the received memory request (the “Block” branch of condition block 308), then data prefetching into the non-cached memory buffer and the cache should be blocked during processing of that instance of the received memory request (block 314). Other memory requests with the same destination address (such as newer, program-order memory requests and other instances of that memory request that have not yet been processed) may allow data prefetching for that destination address. Each of the tags for those memory requests includes a flag indicating whether data prefetching based on the destination address of the memory request should be allowed or blocked during processing of those instances of the memory request. However, during processing of that instance of the received memory request, data prefetching is blocked due to the indication specified by the flag stored in the tag of the received memory request. In some embodiments, the circuitry of the non-cached memory buffer does not send the destination address of the received memory request to either the prefetching engine of the non-cached memory buffer or the prefetching engine of the cache. In one embodiment, the circuitry of the non-cached memory buffer does not send any information about the received memory request to the cache. In other implementations, the non-cached memory buffer sends information about the received memory request to the cache, including a flag specifying whether prefetching is allowed or blocked during processing of that instance of the received memory request. In such implementations, circuitry of one or more of the cache controller and the cache's prefetch engine determines whether to allow or block generating prefetch requests based on the target address and performing prefetch training based on the received flag.

[0050] Now for reference Figure 4This illustrates one embodiment of a method 400 for handling memory requests lacking locality. The cache receives an indication of a memory access to a non-cached memory buffer (box 402). In various embodiments, the memory access corresponds to a memory request targeting a non-cached memory buffer. The cache bypasses read or cache write processing of the received memory access (box 404). The cache circuitry examines the tag of the received memory access to determine whether to block or allow data prefetching using the target address of the received memory access during processing of that instance of the memory access (box 406).

[0051] If the label indicates that data prefetching is allowed during the processing of the instance of the received memory access (the "Allow" branch of condition box 408), the cache prefetch engine performs one or more prefetch requests based on the target address of the received memory access and performs one or more of the prefetch training (box 410). However, if the label indicates that data prefetching is blocked during the processing of the instance of the received memory access (the "Block" branch of condition box 408), the cache prefetch engine blocks one or more prefetch requests based on the target address of the received memory access and performs one or more of the prefetch training (box 412). Other memory accesses with the same target address (such as memory accesses corresponding to newer program-order memory requests and other instances of that memory access that have not yet been processed) may be allowed to prefetch data for that target address. However, during the processing of the instance of the received memory access, data prefetching is blocked due to an indication specified by a flag stored in the label of the received memory access.

[0052] It should be noted that in some implementations, the circuitry supporting the non-cached memory buffer is external to the cache. Therefore, in one implementation, when a received memory request specifies an indication to block data prefetching during processing of that instance of the received memory request, the non-cached memory buffer prevents the destination address (request address) from being sent to the cache. In other implementations, when a flag indicates to block data prefetching during processing of that instance of the received memory access, the cache's prefetch engine still generates one or more prefetch requests based on the request address of the received memory access, but the cache controller provides a value for the Least Recently Used (LRU) value used in the cache line replacement policy, which limits the amount of time the prefetched data resides in the cache. Alternatively, the cache controller limits the placement of prefetched data in a manner specific to multiplexed cache organization.

[0053] Now go to Figure 5This illustrates one embodiment of a method 500 for handling memory requests lacking locality. Circuitry of a processor unit (such as a load / store unit (LSU) or other units of the processor) receives a prefetch cue instruction (block 502) requesting data lacking locality. An example of a prefetch cue instruction is the AMD 64-bit instruction PREFETCHNTA for data lacking temporal locality. Other examples are possible and contemplated. Because the prefetch cue instruction specifies an indication that the requested data lacks locality, the processor retrieves the data for the prefetch cue instruction in a manner different from other types of load instructions. In some embodiments, the processor retrieves the data from system memory and stores the retrieved data in a specific level of the cache memory subsystem. For example, a copy of the retrieved data is stored in the Level 2 (L2) cache, while the Level 1 (L1) and Level 3 (L3) caches (if used) are bypassed. The processor also prefetches a specific amount of data based on the target address of the prefetch cue instruction. For example, in some cases, the processor fetches one or more cache lines based on the target address of the prefetch cue instruction. When retrieved or prefetched data is later evicted from any cache, the processor does not store copies of these types of data at any level of the cache memory subsystem. Other types of processing data for this prefetch hint instruction are possible and anticipated. Data prefetching as described above can be prevented by using a flag in the label of the prefetch hint instruction.

[0054] The processor executes a read access indicated by the received prefetch hint instruction (box 504). The processor retrieves data stored in the memory location pointed to by the target address of the prefetch hint instruction. The processor stores a copy of the retrieved data in one or more cache and non-cache memory buffers (box 506). The specific cache level is based on design requirements; therefore, in some examples, the L1 cache, located closest to the processor core, does not store a copy of the retrieved data, while the L2 cache stores a copy. Other memory arrangements are possible and anticipated. In some implementations, when prefetched data is stored in the cache, the corresponding cache controller sets limits on the storage time and / or storage location of the data targeted by the prefetch hint instruction in the cache data array. For example, the cache controller provides a value for the Least Recently Used (LRU) value used in a cache line replacement policy, which limits the amount of time the prefetched data resides in the cache. The cache controller is also able to limit the placement of prefetched data in a specific manner of multiplexed set-associative cache organization.

[0055] The processor examines the label of the prefetch hint instruction to determine whether to block or allow data prefetching using the target address of the prefetch hint instruction during processing of that instance of the prefetch hint instruction (box 508). If the label indicates that data prefetching is allowed during processing of that instance of the prefetch hint instruction (the "Allow" branch of condition box 510), the prefetch engine of one or more of the cache and non-cache memory buffers generates one or more prefetch requests based on the target address of the prefetch hint instruction (box 512). However, if the label indicates that data prefetching is blocked during processing of that instance of the prefetch hint instruction (the "Block" branch of condition box 510), the processor prevents the prefetch engine of one or more of the cache and non-cache memory buffers from generating prefetch requests based on the target address of the prefetch hint instruction (box 514). In some embodiments, the processor blocks prefetching by preventing any information about the prefetch hint instruction from being sent to the prefetch engine of one or more of the cache and non-cache memory buffers.

[0056] It should be noted that one or more of the above embodiments include software. In such embodiments, program instructions for implementing the methods and / or mechanisms are delivered or stored on a computer-readable medium. Many types of media configured to store program instructions are available and include hard disks, floppy disks, CD-ROMs, DVDs, flash memory, programmable ROMs (PROMs), random access memory (RAM), and various other forms of volatile or non-volatile storage devices. Generally, computer-accessible storage media include any storage medium that can be accessed by a computer during use to provide instructions and / or data to the computer. For example, computer-accessible storage media include storage media such as magnetic or optical media, such as magnetic disks (fixed or removable), magnetic tape, CD-ROMs or DVD-ROMs, CD-Rs, CD-RWs, DVD-Rs, DVD-RWs, or Blu-ray discs. Storage media also include volatile or non-volatile storage media, such as RAM (e.g., Synchronous Dynamic RAM (SDRAM), Dual Data Rate (DDR, DDR2, DDR3, etc.) SDRAM, Low Power DDR (LPDDR2, etc.) SDRAM, Rambus DRAM (RDRAM), Static RAM (SRAM), etc.), ROM accessible via a peripheral device interface (such as a Universal Serial Bus (USB) interface), flash memory, and non-volatile memory (e.g., flash memory). Storage media include microelectromechanical systems (MEMS), and storage media accessible via communication media such as networks and / or wireless links.

[0057] Additionally, in various implementations, program instructions include behavioral-level or register-transfer-level (RTL) descriptions of hardware functionality in a high-level programming language (such as C) or design language (HDL) (such as Verilog, VHDL, or database formats such as GDS II streaming format (GDSII)). In some cases, the description is read by a synthesis tool that synthesizes the description to produce a netlist comprising a list of gates from a synthesis library. The netlist includes gate sets, which also represent the functionality of the hardware comprising the system. The netlist is then placed and routed to produce a dataset describing the geometry to be applied to a mask. The mask is then used in various semiconductor manufacturing steps to produce semiconductor circuits or circuits corresponding to the system. Alternatively, instructions on a computer-accessible storage medium are, as desired, netlists (with or without synthesis libraries) or datasets. Furthermore, the instructions are used by, for example... and Mentor The purpose of this type of supplier's hardware-based type simulator is to perform simulation.

[0058] Although the above embodiments have been described in considerable detail, many variations and modifications will become apparent to those skilled in the art once the foregoing disclosure is fully understood. The following claims are intended to be interpreted as covering all such variations and modifications.

Claims

1. An apparatus comprising: Non-cache storage buffer; and The circuit is configured to: Receive a first memory request targeting the non-cache storage buffer; At least in part based on the first memory request, prevent data from being prefetched into the non-cache memory buffer; Receive a second memory request targeting the non-cache storage buffer; Determining the second memory request includes an indication that allows data prefetching using the target address of the first memory request; as well as The data can be prefetched into the non-cache memory buffer, at least in part based on the second memory request.

2. The device of claim 1, wherein the circuitry is further configured to, in response to a first memory request including a first indication specifying a blocking of data prefetching using the target address of the first memory request, block data prefetching into the non-cache memory buffer at least in part based on the first memory request.

3. The device according to claim 2, wherein, Based at least in part on the determination that the first memory request includes the first indication, the circuitry is further configured to prevent data from being prefetched into the cache using the target address of the first memory request.

4. The device of claim 1, wherein the first memory request includes an indication that the indication is a memory request lacking one or more of temporal locality and spatial locality.

5. The device of claim 1, wherein the circuit is further configured to: Receive a third memory request; and The cache sets a limit on one or more of the storage time and storage location of data retrieved from lower-level memory during the processing of the third memory request.

6. The device of claim 5, wherein the circuitry is further configured to determine that the third memory request is a prefetch cue instruction that includes a third indication specifying the setting of the limitation.

7. A method comprising: The circuit receives a first memory request targeting a non-cache memory buffer. The circuitry prevents data from being prefetched into the non-cache memory buffer, at least in part, based on the first memory request. Receive a second memory request targeting the non-cache storage buffer; Determining the second memory request includes an indication that allows data prefetching using the target address of the first memory request; as well as The data can be prefetched into the non-cache memory buffer, at least in part based on the second memory request.

8. The method of claim 7, further comprising, in response to the first memory request including a first indication specifying to prevent data prefetching using the target address of the first memory request, the circuitry at least partially based on the first memory request to prevent data prefetching into the non-cache memory buffer.

9. The method of claim 8, wherein in response to determining that the first memory request includes the first indication, the method further includes preventing data from being prefetched into a cache using the target address of the first memory request.

10. The method of claim 7, further comprising: Receive a request for third memory; as well as The cache sets a limit on one or more of the storage time and storage location of data retrieved from lower-level memory during the processing of the third memory request.

11. The method of claim 10, further comprising determining that the third memory request is a prefetch cue instruction including a third indication specifying the setting of the limitation.

12. A processing unit, comprising: An interface configured to communicate with lower-level memory; Cache; A processor core, the processor core comprising the device according to any one of claims 1-6.

Images