System, device and method for overriding non-locality-based command handling

Abstract

Processor (202) comprising: a core (104) including a decode unit (230) to decode a memory access instruction (214) having a non-locality hint to indicate that data associated with the memory access instruction (214) has at least one of non-spatial locality and non-temporal locality, wherein the non-spatial locality implies that when the data is accessed from an address in a memory (118), adjacent data is not likely to be accessed within a time period, and wherein the non-temporal locality implies that when the data is accessed from the address in the memory (118), the same data is not likely to be accessed again within the time period; and a memory controller (246) to issue requests to the memory (118), the memory controller (246) including a locality controller (347) to determine whether to override the non-locality indication based at least in part on one or more performance monitoring values.

Description

Technical area

[0001] Embodiments relate to efficient handling of memory accesses in a computing system. background

[0002] Workloads that process small amounts of data, such as graph analysis, perform irregular memory accesses. In some cases, these accesses are to large data structures and are pseudorandom. In modern processors, entire cache lines are fetched from memory and inserted into a cache. However, in many cases, these lines are flushed before any reuse (either due to spatial or temporal locality). This leads to both cache pollution and waste of external memory bandwidth. Newer memory interfaces provide fine-grained memory access capabilities, namely memory access to less than a given memory line or cache line width. However, memory accesses below the cache line result in cache sublines in a processor cache hierarchy, which can complicate cache construction.

[0003] US 2005 / 0080997 A1 relates to a microprocessor with a prefetch instruction (PREFETCH) that prefetches a specific number of cache lines. According to one example, the microprocessor may execute a repeat prefetch instruction (REP PREFETCH). The REP PREFETCH prefetches multiple cache lines, where the number of cache lines can be specified in the instruction. With REP PREFETCH, a line is prefetched only if the number of free response buffers is above a programmable threshold. The REP PREFETCH is performed with a lower priority than other activities that require access to the cache. Summary of the invention

[0004] The inventive solution is defined in the processor according to the main claim 1, the method according to the independent claim 12, the computer-readable medium with computer-readable instructions according to the independent claim 18, the device according to the independent claim 19, the system according to the independent claim 20 and the device according to the independent claim 23. The dependent claims form embodiments or further developments of the inventive solution according to the main claim and the independent claims. Brief description of the drawings Fig. 1 is a block diagram of a system in accordance with an embodiment of the present invention. Fig. 2 is a block diagram of a processor in accordance with an embodiment of the present invention. Fig.3 is a block diagram of a system in accordance with an embodiment of the present invention. Fig. 4 is a flow diagram of a method in accordance with an embodiment of the present invention. Fig. 5 is a flow diagram of a method in accordance with another embodiment of the present invention. Fig. 6 is a block diagram of an infrequent access buffer in accordance with an embodiment of the present invention. Fig. 7A is a block diagram of an exemplary in-order pipeline and an exemplary register renaming, out-of-order issue / execution pipeline to be included in a processor, according to embodiments of the present invention. Fig.7A is a block diagram illustrating both an exemplary embodiment of an in-order architecture core and an exemplary register renaming, out-of-order issue / execution architecture core to be included in a processor according to embodiments of the present invention. Fig. 8 is a block diagram of a single-core processor and a multi-core processor with integrated memory controller and graphics according to embodiments of the present invention. Fig. 9 illustrates a block diagram of a system in accordance with an embodiment of the present invention. Fig. 10 illustrates a block diagram of a system on chip (SoC) in accordance with an embodiment of the present invention. Fig.11 illustrates a block diagram highlighting the use of a software instruction converter to convert binary instructions in a source instruction set to binary instructions in a target instruction set, according to embodiments of the present invention. Fig. 12 is a flow diagram of a method in accordance with an embodiment of the present invention. Fig. 13 is a flow diagram of a method in accordance with another embodiment of the present invention. Detailed description

[0005] In various embodiments, a processor is configured to optimize the handling of data exhibiting nonlocality (referred to herein as nonlocality data) by accessing such data in a potentially fine-grained manner and potentially bypassing storage in a cache. Some embodiments may leverage user-level nonlocality memory access instructions, which may be used to read and write data in a manner that bypasses the cache hierarchy. Such instructions may also be used to perform fine-grained memory accesses as described herein.Although the scope of the present invention is not limited in this respect, such instructions include streaming load and store instructions, in response to which a core may issue different loads or stores for individual accessed data elements, carrying an indication that the memory accesses should bypass storing in caches, and may be handled in a limited manner in a memory controller.

[0006] By selectively enabling full-width memory accesses in response to specific nonlocality requirements described here, locality can be exploited when an access pattern is truly cache-friendly, e.g., if a data structure accessed in a pseudorandom manner fits into on-chip caches, or if the data structure is unexpectedly accessed in a regular pattern with significant spatial locality. In this way, embodiments can exploit the fine-grained memory access capability provided by new memory technologies without major changes to the design of a cache hierarchy, while preserving locality in the accesses, where it exists.

[0007] In embodiments, a processor includes microarchitectural support for non-locality (NL) loads. Unlike non-temporal loads, NL loads imply neither temporal nor spatial locality. Such NL loads may cause a processor to query the caches and return the data if a cache hit occurs, as with a conventional load. In contrast, if a miss occurs and the NL request is received at the memory controller, the memory controller causes a fine-grained memory access to take place to retrieve a cache subline from memory. Note that this cache subline is not inserted into caches, but instead bypasses the caches and is delivered directly to the core.

[0008] Fig.1 is a block diagram of one embodiment of a system 100 including a processor 102 having a cache hierarchy 103 and a memory 118. The processor 102 and the memory 118 are coupled together by one or more interconnects. The processor 102 includes one or more cores 104. In the illustrated example, the processor 102 includes a first core 104-1, optionally up to an Nth core 104-N. The processor may include any desired number of cores (e.g., often in the range of one to the order of hundreds). The core 104-1 includes one or more execution units 106-1, and the core 104-N includes one or more execution units 106-N.

[0009] Typically, processor 102 may also include one or more caches 108, 110. The caches may include relatively smaller and faster memory types than memory 118. The caches may also be closer to the cores and / or execution units than memory 118. The caches may be used to buffer or store data brought into the processor from memory 118 (e.g., in response to a given memory access instruction 112, such as a load instruction) to provide faster subsequent access to the data. When the processor wishes to read or write data from / to memory 118, it may first check to see if a copy of the data is stored in the caches. If the data is found in a cache, the processor may access the data from the cache faster than if the data 118 were accessed from memory 118.As a result, including caches can help reduce the mean time required to access data, which can improve the performance and / or throughput of the processor.

[0010] Still referring to Fig.1, a cache hierarchy 103 includes multiple levels of caches that differ in their relative proximity to the cores and / or to the execution units of the processor. Core 104-1 includes a first-level, or Level 1, cache (L1 cache) 108-1. Similarly, core 104-N includes an L1 cache 108-N. Each of the L1 caches may be dedicated to the corresponding core in which it is included. The L1 caches represent the cache level closest to the cores. The processor also includes a second-level, or Level 2, cache (L2 cache) 110. The L2 cache represents the second-closest cache level to the cores. In some implementations, the L2 cache may be shared among the cores. Although not shown, one or more additional cache levels may optionally be present that are even further away from the cores (e.g., a level 3 cache (L3 cache)).

[0011] One reason for including caches in processors is that memory references often exhibit a locality attribute. For example, references to data in memory often exhibit temporal and / or spatial locality. Temporal locality implies that when data is accessed from an address in memory, it is likely that the same data will be accessed again within a short period of time. For example, this may be the case when the same value is to be reused in a loop, used repeatedly in a group of computations, or for various other reasons. In such cases, it may be useful to store the data in a cache after accessing the data from memory so that subsequent accesses to the data can be performed faster from the cache rather than slowly from memory.

[0012] Spatial locality implies that if a given piece of data is accessed from an address in memory, adjacent data is also likely to be accessed within a short period of time. For example, both pieces of data may be part of the same content (e.g., an image, a table, a data structure, a video, etc.) and may be processed at approximately the same time. Spatial locality can also arise for various other reasons. Caches exploit spatial locality by storing not only the initially required data but also adjacent data from adjacent addresses.

[0013] Typically, the smallest amount of data accessed from memory and stored in the cache is a full-width cache line, even if only a much smaller amount of data may be needed initially. For example, an entire 512-bit cache line can typically be accessed from memory and stored in the cache, even if only a single 8-bit, 16-bit, 32-bit, 64-bit, or 128-bit data item is initially needed. If spatial locality exists, this will be useful, as the additional data being injected into the cache is likely to be needed in the near future.

[0014] During operation, data in the caches is continuously modified by flushing data that is unlikely to be needed in the near future to make room for data that is likely to be needed in the near future. Various replacement algorithms and strategies are used, many of which are often heavily based on the age of the data (e.g., a least recently used indication) due to temporal locality.

[0015] Memory access instructions 112 exploit temporal or spatial locality by storing data items acquired from memory 118 in cache hierarchy 103. Many applications and data types exhibit significant spatial and / or temporal locality in their access stream and thus benefit from accessing and storing an entire cache line's worth of data in the cache for each acquired data item. However, not all applications and / or data types exhibit sufficient temporal and / or spatial locality to justify accessing and storing entire cache lines in the caches for load and / or store instructions. Specific data items may be needed once, but may be unlikely to be needed again in the near future.For example, this may be the case in specialized data streaming applications, high-performance computing applications, applications that involve a stream of very infrequent memory accesses, and various other applications. Furthermore, in many cases, a programmer and / or the software (e.g., an operating system) may be able to identify locality.

[0016] Still with reference to Fig.1, one or more cores 104 may receive and execute a memory access instruction 114 with a nonlocality hint (e.g., a load instruction with a nonlocality hint, such as a nonlocality hint load or acquire instruction, and / or a nonlocality hint store instruction, such as a nonlocality hint store or scatter instruction). The hint may indicate that the data to be accessed (e.g., acquired or scattered) has insufficient spatial and / or temporal locality. In some embodiments, the nonlocality hint may be an indication of no temporal locality. In other embodiments, the nonlocality hint may be an indication of no spatial locality. In still other embodiments, the nonlocality hint may be an indication of no temporal and no spatial locality.In the case of a non-temporal locality indication, in some embodiments, acquired data items may bypass the cache hierarchy 103 and / or not be stored in the cache hierarchy 103, which may help reduce cache pollution. In the case of a non-spatial locality indication, in some embodiments, accesses to data items may be performed with only data amounts below a cache line (e.g., data amounts of half or a quarter of a cache line), or in some cases, data amounts of a single item, which may help reduce waste of interconnect bandwidth and / or reduce power consumption. Using non-locality-indicated memory access instructions may help improve performance and / or reduce power consumption at times when insufficient spatial and / or temporal locality exists.

[0017] Fig.2 is a block diagram of one embodiment of a processor 202 operative to execute one embodiment of a non-locality hint memory access instruction 214. The non-locality hint memory access instruction may also be referred to herein as a non-locality hint load instruction. In some embodiments, the processor 200 may be a general-purpose processor (e.g., a general-purpose microprocessor or a central processing unit (CPU) of the type used in a desktop, laptop, or other computer). Alternatively, the processor may be a special-purpose processor. Examples of suitable special-purpose processors include, but are not limited to, graphics processors, network processors, communications processors, cryptographic processors, coprocessors, embedded processors, digital signal processors (DSPs), and control units (e.g., microcontrollers).The processor 202 may be any of various complex instruction set processors (CISC processors), reduced instruction set processors (RISC processors), very long instruction word processors (VLIW processors), hybrids thereof, other types of processors, or may include a combination of different processors (e.g., in different cores).

[0018] During operation, processor 202 may receive a non-local hint memory access instruction 214. For example, the non-local hint memory access instruction may be received from an instruction fetch unit, an instruction queue, or the like. The non-local hint memory access instruction may be a macroinstruction, an assembly language instruction, a machine code instruction, or another instruction or control signal of an instruction set of processor 202.

[0019] Still with reference to Fig.2, processor 202 includes a decoding unit or decoder 230. Decoding unit 230 may receive and decode a non-locality hint memory access instruction 214. Decoding unit 230 may output one or more microinstructions, micro-operations, microcode entry points, decoded instructions, or control signals, or other relatively lower-level instructions or control signals that reflect, represent, and / or are derived from the non-locality hint memory access instruction. The one or more lower-level instructions or control signals may implement the higher-level non-locality hint memory access instruction through one or more lower-level (e.g., circuit-level or hardware-level) operations.The decode unit 230 may be implemented using various different mechanisms, including, but not limited to, microcode read-only memories (ROMs), lookup tables, hardware implementations, programmable logic arrays (PLAs), and other mechanisms used to implement decode units known in the art.

[0020] Again with reference to Fig.2, processor 202 also includes a group of general-purpose registers 242 and a group of packed data registers 232. Each of the packed data registers may represent an on-chip memory location operable to store packed data, vector data, or single-instruction multiple data (SIMD) data. The registers may represent architecturally visible registers (e.g., an architectural register file) visible to software and / or a programmer and / or may be the registers specified by instructions of an instruction set to identify operands. Execution unit 206 is coupled to decode unit 230 and registers 232, 242. Execution unit 206 may receive the one or more decoded or otherwise implemented instructions or control signals representing and / or derived from the nonlocality hint load instruction.

[0021] Execution unit 206 operates in response to and / or as a result of the nonlocality hint load instruction (e.g., in response to one or more instructions or control signals decoded directly or indirectly (e.g., via emulation) from the instruction) to access a location in memory. Execution unit 206 may provide nonlocality hint load operations 248 to one or more cache control units 244. In some embodiments, a single cache level and a single control unit (e.g., an L1 cache control unit) may be present. In other embodiments, two or more cache control units (e.g., an L1 cache control unit, an L2 cache control unit, and optionally an L3 cache control unit) may be present.

[0022] In some embodiments, if the nonlocality hint load instruction 248 finds a hit in the cache 208, then the associated cache controller 244 may optionally return an amount of data below a cache line from the cache 208 (where the cache supports accesses below the cache line, such as a sectored cache). In various embodiments, the amount of data below a cache line may be as little as half a cache line (e.g., as little as 256 bits of a 512-bit cache line), as little as a quarter of a cache line (e.g., as little as 128 bits), as little as an eighth of a cache line (e.g., as little as 64 bits), or as little as a single data element (e.g., one 128-bit, 64-bit, 32-bit, 16-bit, or 8-bit data element). In other cases, a cache hit may return a complete cache line.

[0023] Conversely, if the nonlocality hint load operation 248 fails in all cache(s) 208, in response to the request 249, the request for data may then be sent to a memory controller 246. In some embodiments, the memory controller 246 may perform a data access below a cache line and return 250 from memory (e.g., external memory). Note that the associated cache controller 244 does not need to allocate space in the cache for the requested data, as it normally would for a regular load instruction (i.e., without a nonlocality hint). In various embodiments, the access below a cache line and the return 250 may only access half a cache line (e.g., only 250 bits), only a quarter of a cache line (e.g., only 128 bits), only an eighth of a cache line (e.g., only 64 bits), or only a single data element (e.g.,a 64-bit, 32-bit, 16-bit, or 8-bit data item). That is, the memory controller 246 can load and return data from memory with a smaller size access and return data that would normally be used for a load operation without a nonlocality hint (e.g., a conventional load operation). As a specific example, only one of a pair of 256-bit bus signals normally used to access an entire amount of data of a 512-bit cache line may be sent from the memory controller 246 to memory, with the one sent being the one containing the desired data item. In some embodiments, the smallest size access and return of data sufficient to contain the desired data item may optionally be used. The memory controller 246 may provide a data return 250 below a cache line to the cache controller(s) 244.The cache control units may provide a corresponding amount of data 252 below a cache line to the execution unit 206. Thus, in various embodiments, nonlocality hint load instructions may cause both data access below a cache line and direct return to a core, bypassing the cache hierarchy.

[0024] In embodiments, processor 202 includes a so-called rare access buffer (SAB) 245, which is a memory structure included in or associated with memory controller 246. Memory controller 246 stores addresses of fine-grained accesses in this buffer. When an NL load reaches the memory controller, the controller looks up the buffer. A hit on a corresponding entry in SAB 245 implies locality (such as when data previously bypassed the caches, but the line is now re-referenced). In one example, in response to this situation, the NL load request is selectively treated as a regular load, so that a full cache line is returned from memory as a selective full-width return 251 and inserted into caches to exploit locality for future accesses.In this way, information in the SAB 245 can be used by the memory controller 246 to selectively filter accesses with locality potential and insert their target data into caches as complete cache lines, while redirecting accesses without locality.

[0025] Embodiments can be used to handle memory accesses for cache sublines without using a sectored cache, which can simplify cache design. Furthermore, one embodiment can exploit the potential for locality, where present, across caches. NL loads with the potential for locality include workloads for processing less data that can exhibit locality potential based on the input data sets (e.g., size, type, topology, etc.).

[0026] In one embodiment, the SAB structure is included in the memory controller to track locality behavior on the memory side. With this arrangement, the SAB primarily observes low-locality access traffic, since high-locality accesses are filtered by caches. In contrast, core-side tracking would receive an access stream with both high- and low-locality accesses. Thus, a memory-side configured SAB can provide much higher coverage for tracking low-locality accesses for a given number of entries. As an example, even a SAB with as few as 32 entries can provide reasonable performance, whereas core-side tracking may consume a much larger number of entries, e.g., 256k entries.

[0027] Sparse data structures are very common in big data analysis, machine learning, and high-performance computing. These workloads sometimes have very large data structures that cannot be accommodated in caches. These large data structures are often indexed via sparse connectivity matrices, leading to irregular accesses with poor spatial locality and reuse (e.g., a loop over A[B[i]], where A is large and B[i] may be completely random). Using fine-grained memory accesses via partial-width loads improves memory system performance by reducing bandwidth waste and cache pollution.

[0028] In one embodiment, instruction set architecture (ISA) support for nonlocal loads may include special scalar loads and single-instruction multiple data acquisition (SIMD acquisition), which do not imply spatial or temporal locality. As a baseline framework, the following semantics for NL loads may be assumed. NL loads are permitted on cacheable write-back memory with weak ordering semantics. This allows hardware to read data speculatively and insert not-yet (potentially partial) cache lines into the cache hierarchy for data snooping.

[0029] Embodiments perform dynamic decision making to determine whether to follow an NL hint or ignore the hint to store received data (which is potentially full-width) in a cache hierarchy. Simply bypassing caches for every NL load access can result in performance degradation based on a few factors: (i) sparsely indexed data structures may be small enough to fit in caches, since programmers often do not know the input sizes in advance; and (ii) sparse input data may exhibit different access distributions unknown to the compiler or programmer. For example, the distribution may be uniformly random or, in the case of many social network graphs, exhibit an exponential distribution, with a few hub nodes that see many backreferences but many leaf nodes with no reuse.Selectively inserting hub nodes into caches and redirecting leaf nodes would improve overall performance. As a further factor, a programmer can misuse NL loads. A mechanism to detect locality potential for NL loads can override NL behavior in these cases. As such, a dynamic remediation mechanism, in accordance with one embodiment, allows programmers to use the NL load feature more freely.

[0030] In operation, an NL load begins like a regular load. If it finds a hit in a cache, the hardware returns the cache line as usual, inserts the cache line into cache(s) closer to the core, and updates usage bits (although the insertion and replacement may be optimized based on the NL hint). In some embodiments, if an already pending miss exists for the same address, the NL load miss is merged with the pending load. If no pending miss exists for the NL load's address, it allocates a fill buffer, such as a miss status holding register (MSHR) entry, with an NL flag or tag.If an incoming miss occurs at an address that already has an outstanding NL load miss, the incoming miss will not rely on the outstanding miss with the NL flag to bring the cache line. The new miss may be blocked (because the MHRS is full), or the outstanding miss with the NL flag may be converted to a regular miss to be inserted into the cache.

[0031] If an NL load fails for all caches and requires memory access, the memory controller dynamically determines whether to perform this request as a regular memory access for a full cache line (via a selective override of the nonlocality hint) or as a fine-grained (below the cache line width) memory access (according to the nonlocality hint). If a full cache line access occurs, the load completes as usual, the cache line is returned, and the line is inserted into the cache(s) and the associated MSHR entries are updated. However, if the decision is to perform a fine-grained memory access, the memory controller returns the data below a cache line directly to the core without inserting the data into any caches.After the completion of this failure, corresponding MSHR entries with the NL flag are deleted.

[0032] Now referring to Fig.Figure 3 shows a block diagram of a system in accordance with an embodiment of the present invention. In particular, system 300 includes a processor 302, a multi-core processor. As can be seen, processor 302 includes multiple cores 304-1-304-N with respective dedicated caches (namely, L1 caches) 306-1-306-N. These caches, in turn, are coupled to a shared cache 310, which in one embodiment may be an L2 cache (and which in some cases may be a last-level cache (LLC)). Cache 310, in turn, is coupled to a memory controller 346. In the embodiment shown, memory controller 346 is coupled to an infrequent access buffer 345.Although shown as separate components for ease of illustration, it should be understood that in specific embodiments, infrequent access buffer 345 may be implemented within memory controller 346. In any event, infrequent access buffer 345 is separated from cores 304 as a non-core circuit component and thus is not part of a coherent domain. Memory controller 346, in turn, interfaces with system memory 360, which, in one embodiment, may be any type of memory structure capable of handling fine-grained accesses.

[0033] It should be understood that the memory controller 346 may further include a locality control circuit (locality control unit) 347 configured, in response to receiving the NL load request, to dynamically determine whether such a request should be treated as an NL load with NL load processing or whether the request should be treated with full-width load processing. As described herein, this determination may be based at least in part on information within the rare access buffer 345.

[0034] Still with reference to Fig.3 shows flows for regular load handling and NL landing handling. Specifically, for a regular load operation, a load request is issued by a given core 304 and is presented to the cache hierarchy to determine if the request finds a match in the cache hierarchy. If so, the data is returned. If a cache miss occurs, the request is passed to the memory controller 346, which issues a corresponding request to the system memory 360 to obtain a full width of data (namely, a full cache line). Following this data return, the memory controller 346 presents the data for insertion into the cache hierarchy and presentation to the requesting core 304.

[0035] In contrast, should an NL load request fail in the cache hierarchy, the memory controller 346 accesses the infrequent access buffer 345 to determine whether the request should be treated with NL processing or full-width processing. If an address of the request within the infrequent access buffer 345 hits and an associated confidence value meets a threshold, the memory controller 346 may treat the request as a full-width load, with treatment as discussed above. Otherwise, the memory controller 346 requests a partial width of the data from the system memory 360 and provides the returned data directly to the requesting core 304, bypassing the cache hierarchy. It should be understood that many variations and alternatives are possible, although in the embodiment of Fig.1 is shown at this high level.

[0036] Now referring to Fig.4 shows a flow diagram of a method in accordance with an embodiment of the present invention. In particular, method 400 may be executed by hardware circuitry, software, firmware, and / or combinations thereof. In particular, method 400 may be executed by various logic and circuitry within a processor to handle a memory access instruction including a non-locality indication. As illustrated, method 400 begins by receiving a memory access instruction including a non-locality indication (block 410). This instruction may be received within a decode unit of the processor that decodes the instruction. Control, in turn, transfers to diamond 420, where it is determined whether an address associated with this instruction finds a hit within the cache hierarchy.If so, control transfers to block 430, where the data is returned to the requester. It should be understood that additional processing may occur, such as updating the freshness information of the hit cache line, and so on.

[0037] Still with reference to Fig.4, if the request fails within the cache hierarchy, control transfers to block 440, where the type of memory access treatment may be determined. In particular, at block 440, which may be executed in a locality control unit of a memory controller, information from an infrequent access buffer may be used to determine appropriate treatment, as discussed above. If nonlocality treatment is to be performed, control transfers to block 450, where partial-width data is obtained and returned directly to the requestor. This direct return of partial-width data to the requestor (e.g., the core) bypasses the cache hierarchy. Further still, information within the infrequent access buffer may be updated. For example, if an entry already exists for an address associated with this memory access command, a confidence value may be updated (e.g.,incremented). Or, if no entry previously existed in the infrequent access buffer, a new entry can be inserted for that address. Depending on the implementation of the infrequent access buffer, the partial-width data itself can also be stored.

[0038] Still with reference to Fig. 4, if it is instead determined that full-width selective handling should be performed for this memory access instruction, control transfers to block 460 where full-width data may be obtained. This data may be provided to the requestor via the cache hierarchy. It should also be noted that if an entry in the rare access buffer exists for this address, it may be removed. It should be understood that many variations and alternatives are possible, although in the embodiment of Fig. 4 is shown at this high level.

[0039] Thus, in embodiments, the SAB can be used to dynamically detect potential locality for NL loads that miss across all caches, e.g., using a fine-grained / full access decision. In response to a fine-grained memory access, an entry is allocated in the SAB. A new NL load received at the memory controller and finding a hit in the SAB indicates that this address was recently accessed but bypassed the caches and is now being referenced again. In this case, a confidence flag (e.g., one or more so-called confidence bits) to identify potential locality is incremented for this particular address.If the trust level is high enough when the new NL load is received in the memory controller, then a regular memory access is performed for a full cache line, and the data is inserted into caches to preserve locality for the next requests.

[0040] Now referring to Fig. 5 shows a flow diagram of a method in accordance with another embodiment of the present invention. In particular, method 500 sets forth the handling of a load instruction including a nonlocality hint within a processor. As such, method 500 may be executed by hardware circuitry, software, firmware, and / or combinations thereof.

[0041] As illustrated, method 500 begins by determining whether an address of the NL load instruction hits any level of the cache hierarchy (diamond 510). If so, control transfers to block 520, where data is returned to the core. Further still, depending on the processor implementation, the hit data may also be inserted into one or more cache levels (namely, one or more cache levels closer to the core). Instead, if it is determined that the address of the NL load does not hit within any cache in the processor, control transfers to diamond 530 to determine whether the address hits within a SAB.

[0042] If not, handling this load request may occur with different procedures depending on whether the SAB is implemented as a redirected address filter (BAF) or a read combining buffer (RCB). In the case of a BAF, control transfers to block 540, where the memory controller may issue a narrow data request (namely, for data less than a cache line wide) and, upon receiving the requested data from memory, return the data directly to the core, bypassing the cache hierarchy. As such, this data is not inserted into any cache memory in the cache hierarchy. Further, in the event of a miss within the SAB, a new SAB entry may be allocated for the address associated with this NL load.In this case of a miss within the SAB for an address associated with the NL load, in the context of an RCB implementation, control instead transfers to block 545. There, data of a width below a cache line goes directly to the core, bypassing the cache hierarchy. Further, in this case of a miss within the SAB for the address associated with this NL load, a new SAB entry may be allocated, and the partial-width data itself may be stored within the allocated entry.

[0043] Still with reference to Fig.5, if the NL load address within the SAB finds a hit, control transfers to diamond 550 to determine whether a confidence value of the hit entry exceeds a confidence threshold. In one embodiment, the confidence value may simply be a single bit set to one upon the first hit to the SAB entry. In other cases, the confidence value may be implemented as multi-bit count information, such that for each hit to that entry, the confidence value may be updated, e.g., by one.

[0044] In any case, if the confidence value does not exceed the confidence threshold, control proceeds to one of blocks 560 and 565, depending on whether the SAB is implemented as a BAF (block 560) or an RCB (block 565). As illustrated, for a BAF implementation, a partial width of the data is returned directly to the core, bypassing the cache hierarchy and without insertion into any cache from the cache hierarchy. Furthermore, the confidence value of the entry is incremented. Instead, for an RCB implementation, the partial width of the data is returned directly to the core from the SAB itself, bypassing the cache hierarchy. Furthermore, the confidence value of the entry is incremented.

[0045] Still with reference to Fig.5, if it is instead determined at diamond 550 that the confidence value of the hit entry exceeds the confidence threshold, control transfers to one of blocks 570 and 575, depending on the SAB implementation. In the case of a BAF implementation, at block 570, the NL hint of this NL load is overridden or ignored, so that the memory controller requests and receives the full data width and returns that full data width to the core. In one embodiment, this return includes inserting into one or more caches of the cache hierarchy, since with this override, a locality determination has been made such that the data is likely to be reused with one or more of a spatial and / or temporal locality. Furthermore, at block 570, the memory controller may invalidate the SAB entry associated with the address of the NL load.In this way, additional space in the SAB can be used by another NL load for a different address. Instead, for an RCB implementation, control transfers to block 575, where the NL hint of that NL load is overridden or ignored, so that the memory controller requests and receives the remaining data portion from memory and combines it with the portion contained in the RCB to obtain a full width of data, which is returned to the core (and inserted into one or more caches of the cache hierarchy). Additionally, at block 575, the memory controller may invalidate the SAB entry associated with the address of the NL load. It should be understood that many variations and alternatives are possible, although it is discussed in more detail below. Fig. 5 is shown at this high level.

[0046] Various types of implementations for the rare access buffer are possible. Although the scope of the present invention is not limited in this regard, two possible implementations include a read combining buffer (RCB) and a redirected address filter (BAF).

[0047] In one implementation, an RCB is a small memory-side buffer contained within or closely associated with the memory controller. Such an RCB may contain multiple entries to store redirected addresses, locality confidence values, and fine-grained accessed data. This buffer is outside the coherence domain of processor caches, and it stores only the amount of data below a cache line from its memory channel that is not inserted into caches. The data in the RCB is accessed only if the corresponding address is not found in the cache hierarchy. Regular writes received at the memory controller (e.g., dirty cache flushes) also check the RCB and update the contents if the corresponding address in the RCB is valid.

[0048] When an NL load encounters a hit in the RCB, locality confidence values ​​are incremented, and the load is served from the RCB to the core without any external memory access. If the confidence value of the corresponding entry is high enough, the remaining portion of the line is fetched from memory and combined with the portion already existing in the RCB to form a cache line. This line is then inserted into caches to capture its potential reuse for later accesses. Once the line is inserted into the caches, it becomes part of the cache coherence domain, and thus the corresponding entry is invalidated from the RCB, which is strictly exclusive to the CPU caches. Invalidating entries after insertion into the cache hierarchy frees up space in the RCB to allocate new entries, minimizing flushes from the RCB and increasing the effective coverage for redirected addresses.In one embodiment, the RCB may be organized as a set-associative cache to reduce the likelihood of conflict misses. Note that the RCB may be space-intensive, as in some embodiments it may hold full addresses, fine-grained data, and metadata.

[0049] In another implementation, the SAB can be implemented as a redirected address filter (BAF), which can be configured as a small buffer contained within or closely associated with the memory controller. This buffer holds the set of addresses that have recently bypassed the cache hierarchy and can be used to make fine-grained memory access decisions. Unlike the RCB, the BAF holds only addresses and confidence flags, which can reduce range costs. When an NL load encounters a hit in the BAF, locality confidence flags are incremented, and the data is fetched from memory via a fine-grained memory access if the locality confidence value does not meet a threshold.If the confidence value meets the threshold, the request is served as a regular memory access to bring a full cache line, which is inserted into the cache hierarchy to ensure reuse for future references. When a line is inserted into the cache, the corresponding entry from the BAF is invalidated, freeing up space to allocate new entries, minimizing BAF flushes and increasing the effective coverage of redirected addresses.

[0050] Note that with a BAF implementation, a memory access always occurs to fetch the fine-grained data. Furthermore, to promote accesses to full cache lines, the BAF fetches the entire cache line from memory. An RCB implementation can conserve memory bandwidth by servicing hits directly from the RCB, and promoting full cache lines only retrieves the remaining portion of the cache line from memory. However, the memory footprint of a BAF implementation can be much smaller than that of the RCB because it does not hold data below the cache.

[0051] Additionally, the BAF can maintain proximity information to reduce costs without sacrificing accuracy. Since the BAF only influences fine-grained memory access decisions, a failure will only result in reduced efficiency. Proximity information can, for example, be in the form of storing partial addresses in the BAF. This reduces the capacity requirements for the BAF or can be used to increase bypass address coverage with the same capacity. In the case of partial addresses, the BAF can report false positives (false hits). In other words, a reference address can incorrectly match a partial address in the BAF. Such false positives can falsely increment the confidence values, and thus can create a tendency for access to full cache lines to occur more frequently. This can be addressed by increasing the confidence threshold.Nevertheless, these false positives will not cause any correctness issues. It should be noted that holding partial addresses is not possible with an RCB implementation, as false positives could result in incorrect data being returned to the core.

[0052] If a complete address (e.g., cache line address and block offset) is provided to the memory controller, then the SAB can be used to track both spatial and temporal locality. For example, assuming 8 bytes (B) tracking granularity, for each 64B cache line, an SAB entry can contain 8 confidence counters (note that a confidence counter can be as low as 1 bit). For each cache line, the sum of all counters can be combined to yield the temporal locality confidence, and neighboring counters with non-zero confidence values ​​can indicate spatial locality confidence for the redirected addresses. The threshold for promoting an access to a regular full memory access is determined using these temporal / spatial confidence counters.In the simplest version, the tracking granularity is matched to the cache line size, and a single one-bit counter is used. Thus, in the simplest configuration, an access is promoted to a full access after the first hit in the SAB (either a temporal hit to the same data block or a spatial hit to a different data block in the same cache line).

[0053] Now referring to Fig. Figure 6 is a block diagram of a rare access buffer in accordance with an embodiment of the present invention. In particular, Fig. 6 illustrates an embodiment of a SAB 600 in the form of a redirected address filter, in which the SAB 600 does not include a partial width data memory. Thus, information pertaining to more addresses can be stored with a buffer of the same size.

[0054] As shown, the SAB 600 contains several entries 6100-610 n As can be seen, each entry 610 includes several fields, including a validity field 612, a tag field 614, and a trust field 616. In one embodiment, the validity field 612 may be a single bit to indicate whether the given entry is valid or not (e.g., set to a value of logical 1 to indicate that the entry is valid, and vice versa). The tag field 614 may store at least a portion of an address of a given NL memory access instruction. As discussed above, this tag portion may store only a partial address in favor of providing broader coverage at the expense of possible false positives. The trust field 616, in turn, may store one or more bits to provide trust information. In the Fig.In the embodiment shown in Figure 6, the trust field may be a multi-counter field. For example, in a given embodiment, for an entry 610 associated with a 64-bit cache line, eight counters may be provided, each of which may be one or more bits. With this type of information, it is possible to determine temporal locality based on a sum of the count values ​​of each of these counters. Spatial locality, in turn, may be determined by reference to neighboring counters that have non-zero values.

[0055] Different analyses of the values ​​of these multiple counters are contemplated in different embodiments. For example, in one embodiment, a memory controller (and in particular, a locality controller of such a memory controller) may simply consider the sum of the confidence values ​​when determining whether a specific memory access instruction containing a non-locality hint should be treated instead of a memory access instruction containing locality (namely, with a full data return and storing in a cache hierarchy). In such cases, this sum of the confidence values ​​may be compared to a given threshold, and if the sum exceeds the threshold, locality-based treatment may be selectively enabled.

[0056] In another embodiment, the memory controller (and in particular, the locality controller) may alternatively determine to selectively enable locality-based treatment where multiple neighbors of these counters have non-zero values, indicating spatial locality. In yet other embodiments, combinations of a counter sum technique and a neighbor counter value technique may be used. As such, embodiments may use only spatial, only temporal, and / or combinations of temporal / spatial locality information to make full / partial access decisions.

[0057] Without one embodiment, simply bypassing all NL accesses can lead to increased memory traffic and reduced performance due to lost locality. By providing a SAB in accordance with one embodiment, high-locality data can be selectively inserted into caches so that rereferences from caches are serviced. Thus, a SAB-based dynamic strategy, which can be executed independently on a given cache line basis for each NL load, can provide significant memory traffic reduction and performance improvement compared to always bypassing fine-grained accesses.

[0058] It should be noted that although embodiments described herein primarily relate to load operations, the scope of the present invention is not limited in this regard. This is because embodiments can also be used to handle write operations that include nonlocality hints with the same selective activation operations. For conventional write operations, a full-width cache line is received at the processor, and then at least a portion of the cache line is overwritten with new data. Thereafter, the full cache line width is written back to memory. In contrast, for a write operation that includes a nonlocality hint, a limited write request can be used to write only a portion of a cache line width to memory, which may include the sub-cache line processing discussed above.Selectively enabling full-width treatment of such nonlocality hint write operations can apply in the same way.

[0059] Embodiments for dynamically performing fine-grained memory access in various ways can have many potential applications in HPC, big data analytics, machine learning, etc. Although the scope of the present invention is not limited in this respect, embodiments may be specifically adapted for use with a particular workload, such as page-ranking algorithms, sparse matrix vector multiplication workloads, or stochastic gradient descent algorithms. This is because irregular accesses to large data structures are widespread in HPC, graph analytics, machine learning, etc. Therefore, memory performance can become a bottleneck in these applications. Embodiments enable fetching data from memory at smaller granularities and bypassing caches when locality is lacking.This operation improves cache utilization and reduces bandwidth waste, significantly improving memory performance for such high-value applications while efficiently utilizing the fine-grained access capability provided by these memories. Dynamic execution can detect locality potential and convert fine-grained accesses into regular accesses to full cache lines. Thus, any misuse of the fine-grained feature is dynamically corrected. This arrangement provides programmers with more freedom to use these advanced features without having to worry too much about negative performance impacts.

[0060] Fig.7A is a block diagram illustrating both an exemplary in-order pipeline and an exemplary register renaming, out-of-order issue / execution pipeline to be included in a processor according to embodiments of the present invention. Fig. Figure 7B is a block diagram illustrating both an exemplary embodiment of an in-order architecture core and an exemplary register renaming, out-of-order issue / execution architecture core to be incorporated into a processor, according to embodiments of the present invention. The boxes bounded by solid lines in the Fig.Figures 7A-B represent the in-order pipeline and core, while the optional addition of the dashed-line boxes represents the register renaming and out-of-order issue / execution pipeline and core. Assuming that the in-order aspect is a subset of the out-of-order aspect, the out-of-order aspect is described.

[0061] In Fig. 7A, a processor pipeline 700 includes a fetch stage 702, a length decode stage 704, a decode stage 706, an allocation stage 708, a rename stage 710, a scheduling stage 712 (also referred to as a dispatch or issue stage), a register read / memory read stage 718, an execution stage 716, a write-back / memory write stage 718, an exception handling stage 722, and a store stage 724.

[0062] Fig.7B shows the processor core 790, which includes a front-end unit 730 coupled to an execution engine unit 750, both of which are coupled to a memory unit 770. The core 790 may be a reduced instruction set (RISC) core, a complex instruction set (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As yet another option, the core 790 may be a special-purpose core, such as a network or communications core, a compression engine, a coprocessor core, a general-purpose computing graphics processing unit (GPGPU) core, a graphics core, or the like.

[0063] The front-end unit 730 includes a branch prediction unit 732 coupled to an instruction cache unit 734, which is coupled to an instruction translation lookaside buffer (instruction TLB) 736, which is coupled to an instruction fetch unit 738, which is coupled to a decode unit 740. The decode unit 740 (or decoder) can decode instructions and generates as an output one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals that are decoded from, or that otherwise reflect, or are derived from, the original instructions. The decode unit 740 can be implemented using various different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memories (microcode ROMs), etc.In one embodiment, core 790 includes a microcode ROM or other medium that stores microcode for specific macroinstructions (e.g., in decode unit 740 or otherwise within front-end unit 730). Decode unit 740 is coupled to a rename / allocator unit 752 in execution engine unit 750.

[0064] The execution engine unit 750 includes the rename / allocator unit 752, which is coupled to a retirement unit 754 and a group of one or more scheduler units 756. The scheduler unit(s) 756 represent any number of different schedulers, including reservation stations, a central instruction window, etc. The scheduler unit(s) 756 are coupled to the physical register file(s) unit(s) 758. Each of the physical register file(s) unit(s) 758 represents one or more physical register files, different ones of which store one or more different data types, such as scalar integer, scalar floating-point number, packed integer, packed floating-point number, vector integer, vector floating-point number, status (e.g., an instruction pointer that is the address of the next instruction to be executed), etc.In one embodiment, the physical register file(s) unit 758 includes a vector register unit, a write mask register unit, and a scalar register unit. These register units may provide architectural vector registers, vector mask registers, and general-purpose registers. The physical register file(s) unit(s) 758 are overlapped by the retirement unit 754 to illustrate various ways in which register renaming and out-of-order execution may be implemented (e.g., using reorder buffer(s) and retirement register file(s); using future file(s), history buffer(s), and retirement register file(s); using a register map and a pool of registers; etc.). The retirement unit 754 and the physical register file(s) unit(s) 758 are coupled to the execution cluster(s) 760.The execution cluster(s) 760 includes a group of one or more execution units 762 and a group of one or more memory access units 764. The execution units 762 perform various operations (e.g., shifts, addition, subtraction, multiplication) and on various data types (e.g., scalar floating-point, packed integer, packed floating-point, vector integer, vector floating-point). Although some embodiments may include a number of execution units dedicated to specific functions or groups of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions.The scheduler unit(s) 756, physical register file(s) unit(s), and execution cluster(s) 760 are shown as possibly multiple because particular embodiments create separate pipelines for particular types of data / operations (e.g., a scalar integer pipeline, a scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipeline, and / or a memory access pipeline, each having its own scheduler unit, physical register file(s) unit, and / or execution cluster—and in the case of a separate memory access pipeline, particular embodiments are implemented in which only the execution cluster of that pipeline includes the memory access unit(s) 764). It should also be understood that where separate pipelines are used, one or more of those pipelines may be out-of-order output / execution and the rest may be in-order.

[0065] The group of memory access units 764 is coupled to the memory unit 770, which includes a data TLB unit 772 coupled to a data cache unit 774 coupled to a level 2 (L2) cache unit 776. The instruction cache unit 734 and the data cache unit 774 may collectively be considered a distributed L1 cache. In an exemplary embodiment, the memory access units 764 may include a load unit, a store address unit, and a store data unit, each of which is coupled to the data TLB unit 772 in the memory unit 770. The instruction cache unit 734 is further coupled to a level 2 (L2) cache unit 776 in the memory unit 770. The L2 cache unit 776 may be coupled to one or more levels of cache and ultimately to main memory.

[0066] As an example, the exemplary register renaming, out-of-order issue / execution core architecture may implement pipeline 700 as follows: 1) instruction fetch unit 738 executes fetch and length decode stages 702 and 704; 2) decode unit 740 executes decode stage 706; 3) rename / allocator unit 752 executes allocation stage 708 and rename stage 710; 4) scheduler unit(s) 756 execute scheduling stage 712; 5) physical register file unit(s) 758 and memory unit 770 execute register read / memory read stage 714; execution cluster 760 executes execution stage 716; 6) the memory unit 770 and the physical register file(s) unit(s) 758 execute the write-back / memory write stage 718; 7) various units may participate in the exception handling stage 722;and 8) the retire unit 754 and the physical register file(s) unit(s) 758 execute the store stage 724.;

[0067] The core 790 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set developed by MIPS Technologies in Sunnyvale, CA; the ARM instruction set (with optional additional extensions such as NEON) from ARM Holdings in Sunnyvale, CA) that include the instruction(s) described herein. In one embodiment, the core 790 includes logic to support a packed data instruction set extension (e.g., AVX1, AVX2, and / or some form of the generic vector-friendly instruction format (U=0 and / or U=1), thereby enabling the operations used by many multimedia applications to be performed using packed data.

[0068] It should be understood that the 790 core can support multithreading (executing two or more parallel groups of operations or threads) and can do so in a variety of ways, including time-sliced ​​multithreading, concurrent multithreading (where a single physical core provides a logical core for each of the threads that the physical core is concurrently multithreading), or a combination thereof (e.g., time-sliced ​​fetching and decoding, and concurrent multithreading thereafter, such as in Intel® Hyperthreading Technology).

[0069] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming may be used in an in-order architecture. Although the illustrated embodiment of the processor also includes separate instruction and data cache units 734 / 774 and a shared L2 cache unit 776, alternative embodiments may include a single internal cache for both instructions and data, such as an internal L1 cache, or multiple levels of internal cache. In some embodiments, the system may include a combination of an internal cache and an external cache that is external to the core and / or the processor. Alternatively, all caches may be external to the core and / or the processor.

[0070] It is in Fig.7B further illustrates that the processor may also include a memory control unit 780 that may perform the dynamic determination described herein to potentially selectively override a non-locality hint of a memory access instruction and execute the memory access instruction as if it exhibited locality (e.g., a full-width load and return to core 790 via a cache hierarchy such as memory unit 770). Instead, if memory control unit 780 determines to treat a memory access according to the non-locality hint, partial-width data may be obtained and returned directly to core 790, bypassing the cache hierarchy including memory unit 770. Note that in the Fig. 7B, the memory control unit 780 includes an SAB 785.

[0071] Fig.Figure 8 is a block diagram of a processor 800 that may have more than one core, an integrated memory controller, and integrated graphics, according to embodiments of the invention. Boxes bounded by solid lines in Fig.8 illustrate a processor 800 with a single core 802A, a system agent circuit 810, and a group of one or more bus controller circuits 816, while the optional addition of the boxes bounded by dashed lines illustrates an alternative processor 800 with multiple cores 802A-N and a group of one or more integrated memory controller circuits 814 in the system agent circuit 810. The integrated memory controller circuits 814 may include one or more SABs 815 as described herein to enable the memory controller to dynamically determine whether to selectively enable locality-based treatment for incoming memory access requests that include non-locality indications. As further described in Fig. 8, the processor 800 also includes a graphics circuit 808 for dividing graphics, integrated within the processor 800N.

[0072] Processor 800 may be a general-purpose processor, a coprocessor, or a special-purpose processor, such as a network or communications processor, a compression engine, a graphics processor, a GPGPU (general-purpose graphics processing unit), a high-throughput multi-core coprocessor (MIC coprocessor) (containing 30 or more cores), an embedded processor, an accelerator device, or the like. The processor may be implemented in one or more chips. Processor 800 may be a part of one or more substrates and / or implemented thereon using a number of process technologies, such as BiCMOS, CMOS, or NMOS.

[0073] The memory hierarchy includes one or more levels of cache circuits 804A-N (including L1 cache) within the cores 802A-N, a group of one or more shared cache circuits 1006, and external memory (not shown) coupled to the group of integrated memory controller circuits 814. The group of shared cache circuits 806 may include one or more mid-level caches, such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other cache levels, a last-level cache (LLC), and / or combinations thereof. Although in one embodiment, a ring-based interconnect circuit 812 interconnects the graphics circuit 808, the group of shared cache units 806, and the system agent circuit 810 / integrated memory controller circuit(s) 814, alternative embodiments may use any number of known techniques for interconnecting such circuits.In one embodiment, coherency is maintained between one or more cache circuits 806 and cores 802A-N.

[0074] In some embodiments, one or more of cores 802A-N are capable of multithreading. System agent circuitry 810 includes the components that coordinate and operate cores 802A-N. System agent circuitry 810 may include, for example, a power control unit (PCU) and a display unit. The PCU may be or include logic or components for regulating the power state of cores 802A-N and / or graphics circuitry 808. The display unit may be for driving one or more externally connected display devices.

[0075] Cores 802A-N may be homogeneous or heterogeneous in terms of architectural instruction set; that is, two or more of cores 802A-N may be capable of executing the same instruction set, while others may be capable of executing only a subset of that instruction set or a different instruction set. In one embodiment, cores 802A-N are heterogeneous and include both the "small" cores and the "large" cores described below.

[0076] Fig.9 is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the art for laptops, desktops, tablets, portable PCs, personal digital assistants, technical workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, mobile phones, smartphones, portable media players, wearable devices, and various other electronic devices are also suitable. In general, a wide variety of systems or electronic devices capable of integrating a processor and / or other execution logic as disclosed herein are generally suitable.

[0077] Now referring to Fig.9 is a block diagram of an exemplary system 900 in accordance with an embodiment of the present invention. As shown in Fig. 9, the multiprocessor system 900 is a point-to-point interconnect system and includes a first processor 970 and a second processor 980 coupled via a point-to-point interconnect 950. Each of the processors 970 and 980 may be a version of the processor 800 of Fig. be 8.

[0078] Processors 970 and 980 are shown as including integrated memory control unit (IMC) units 972 and 982, respectively. Processor 970 also includes, as part of its bus control units, point-to-point (PP) interfaces 976 and 978; similarly, the second processor 980 includes PP interfaces 986 and 988. As can be seen, processors 970, 980 further include SABs 975, 985, as described herein. Processors 970, 980 may exchange information via a point-to-point (PP) interface 950 using PP interface circuits 978, 988. As shown in Fig. 9, the IMCs 972 and 982 couple the processors to respective memories, namely a memory 932 and a memory 934, which may be portions of the main memory locally attached to the respective processors.

[0079] Processors 970, 980 may each exchange information with a chipset 990 via individual PP interfaces 952, 954 using point-to-point interface circuits 976, 994, 986, 998. Chipset 990 may optionally exchange information with coprocessor 938 via a high-performance interface 939 using point-to-point interface circuit 992. In one embodiment, coprocessor 938 is a special-purpose processor such as a high-throughput MIC processor, a network or communications processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, or the like.

[0080] A shared cache (not shown) may be provided either in one processor or external to both processors, but connected to the processors via a PP interconnect so that the local cache information of one or both processors can be stored in the shared cache if one processor is placed in a low-power mode.

[0081] Chipset 990 may be coupled to a first bus 916 via an interface 996. In one embodiment, first bus 916 may be a peripheral interconnect bus (PCI bus) or a bus such as a PCI Express bus or other third-generation I / O interconnect bus, although the scope of the present invention is not so limited.

[0082] As in Fig.9, various I / O devices 914 may be coupled to the first bus 916, along with a bus bridge 918 that couples the first bus 916 to a second bus 920. In one embodiment, one or more additional processors 915, such as coprocessors, high-throughput MIC processors, GPGPUs, accelerators (such as graphics accelerators or digital signal processing (DSP) units), field-programmable gate arrays, or any other processor, are coupled to the first bus 916. In one embodiment, the second bus 920 may be a low pin count (LPC) bus. Various devices may be coupled to a second bus 920, including, for example, a keyboard and / or mouse 922, communication devices 927, and a memory unit 928, such as a flash memory (RAM). B. a disk drive or other mass storage device that may contain instructions / code or data 930, in one embodiment.Furthermore, an audio I / O 924 may be coupled to the second bus 920. It should be noted that other architectures are possible. For example, instead of the point-to-point architecture of . Fig. 9 a system may implement a multi-drop bus or other such architecture.

[0083] Now referring to Fig. Figure 10 shows a block diagram of an SoC 1000 in accordance with an embodiment of the present invention. Additionally, dashed-line boxes are optional features on more advanced SoCs. Fig.10 is an interconnect circuit(s) 1012 coupled to: an application processor 1010 including a group of one or more cores 1002A-N having cache circuit(s) 1004A-1004N, and shared cache circuit(s) 1006; a system agent circuit 1010; a bus controller circuit(s) 1016; an integrated memory controller circuit(s) 1014, which may include an SAB 1015 as described herein; a group of one or more coprocessors 1020, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) circuit 1030; a direct memory access (DMA) circuit 1032; and a display circuit 1040 for coupling to one or more external display devices.In one embodiment, coprocessor(s) 1020 include a special-purpose processor, such as a network or communications processor, a compression engine, a GPGPU, a high-throughput MIC processor, an embedded processor, or the like. For the purposes of this application, a processing system includes any system that includes a processor, such as a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0084] The program code can be implemented in a high-level procedural or object-oriented programming language to communicate with a processing system. The program code can also be implemented in assembly or machine language, if desired. Indeed, the mechanisms described here are not limited in scope to any specific programming language. In any case, the language can be a compiled or interpreted language.

[0085] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a non-transitory machine-readable medium that represent various logic within the processor that, when read by a machine, cause the machine to manufacture logic to perform the techniques described herein. Such representations, referred to as "IP cores," may be stored on a tangible non-transitory machine-readable medium and provided to various customers or manufacturing facilities to be loaded into the production machines that actually manufacture the logic or processor. Accordingly, embodiments of the invention also include non-transitory tangible machine-readable media that contain instructions or contain design data, such asHardware Description Language (HDL) that defines structures, circuits, devices, processors, and / or system features described herein. Such embodiments may also be referred to as program products.

[0086] In some cases, an instruction translator can be used to translate an instruction from a source instruction set to a target instruction set. For example, the instruction translator can translate (e.g., using static binary translation, dynamic binary translation that includes dynamic compilation), convert, emulate, or otherwise translate an instruction into one or more instructions to be executed by the core. The instruction translator can be implemented in software, hardware, firmware, or a combination thereof. The instruction translator can be internal to the processor, external to the processor, or partially internal and external to the processor.

[0087] Fig. Figure 11 is a block diagram highlighting the use of a software instruction converter to convert binary instructions in a source instruction set to binary instructions in a target instruction set, according to embodiments of the present invention. In the illustrated embodiment, the instruction converter is a software instruction converter, although alternatively, the instruction converter may be implemented in software, firmware, hardware, or various combinations thereof. Fig.11 shows a program in a high-level language 1102 that can be compiled using an x86 compiler 1104 to produce x86 binary code 1106 that can be natively executed by a processor having at least one x86 instruction set core 1116. The processor with at least one x86 instruction set core 1116 represents any processor that can perform substantially the same functions as an Intel processor with at least one x86 instruction set core by compatibly executing or otherwise processing (1) a substantial portion of the instruction set of the Intel® x86 instruction set core or (2) object code versions of applications or other software designed to run on an Intel processor with at least one x86 instruction set core to achieve substantially the same result as an Intel® processor with at least one x86 instruction set core.The x86 compiler 1104 represents a compiler that operates to generate x86 binary code 1106 (e.g., object code) that can be executed with or without additional linkage processing on the processor having at least one x86 instruction set core 1116. Similarly, FIG. Fig.11, the program in the high-level language 1302 can be compiled using an alternative instruction set compiler 1108 to produce binary code 1110 of an alternative instruction set that can be natively executed by a processor without at least one x86 instruction set core 1114 (e.g., a processor with cores executing the MIPS instruction set from Technologies of Sunnyvale, CA, and / or executing the ARM instruction set from ARM Holdings of Sunnyvale, CA). The instruction converter 1112 is used to convert the x86 binary code 1106 into code that can be natively executed by the processor without an x86 instruction set core 1114. It is not likely that this translated code will be the same as the binary code 1110 of the alternative instruction set, because an instruction translator capable of doing so is difficult to manufacture; however, the translated code will achieve general operation and consist of instructions from the alternative instruction set.Thus, the instruction converter 1112 represents software, firmware, hardware, or a combination thereof that, through emulation, simulation, or any other process, enables a processor or other electronic device that does not have a processor or core with the x86 instruction set to execute the x86 binary code 1106.

[0088] As described above, embodiments may dynamically schedule nonlocality load instructions (e.g., streaming load instructions). In the above embodiments, a dynamic prediction is performed for each NL memory access instruction to determine whether the associated data has sufficient locality. If not, the hardware performs a limited memory access, bypassing the caches in turn. If sufficient locality is detected, the hardware treats the access as a regular load and fetches a full cache line from memory and inserts it into the caches. In the above embodiments, this prediction technique uses a special access history to predict locality for each line independently.

[0089] Selectively translating fine-grained accesses into full memory accesses to exploit locality can work well for large input data sets with special locality distributions, as described here. However, these techniques incur the overhead of learning the locality of each line separately, with hardware initially bypassing the cache and later accessing the requested address as a memory access to a full cache line. Thus, learning locality patterns can be a slow process when cross-address information would help. For example, it may take multiple accesses to the same line to install that line in the cache if: (i) the working set or even the entire input data set unexpectedly fits into on-chip caches; or (ii) a misused nonlocality (or streaming) load instruction touches data with locality very frequently.

[0090] To reduce this penalty, some embodiments further provide a memory-side performance counter-tracking technique to predict whether cache bypassing for nonlocality loads is effective. If it is dynamically determined to be ineffective, the processor is configured to dynamically override the nonlocality behavior at the time of core-side instruction issue. As such, a processor in accordance with one embodiment can learn specific locality patterns much more quickly and recover lost performance by falling back to a baseline behavior (with full data access and caching).

[0091] It should be noted that NL loads are only useful when cache utilization is low (high miss rate) and memory bandwidth utilization is high. Embodiments may use hardware counters, including, for example, a miss rate counter (specifically, misses per kilo instruction, MPKI) for the last-level cache (LLC) and optionally bandwidth utilization, to determine whether the processor is operating in an operating state that can benefit from nonlocality loads. Of course, embodiments are not limited to these specific performance counters, and other embodiments may use information from additional or different performance monitors.If cache utilization is higher than a given threshold (and bandwidth utilization is lower than another threshold), the processor can override the nonlocality behavior, so that all NL loads are treated as regular loads. However, if cache utilization is lower than the threshold (and bandwidth utilization is higher than the other threshold), then the processor can potentially benefit from the NL load operation.

[0092] These metrics alone may not be sufficient to deduce that NL loads are actually useful (e.g., NL loads may still be misused in a bandwidth-intensive application). As such, embodiments may also leverage the SAB to track addresses of NL loads that have recently bypassed caches. If the overall hit rate in the SAB is high, then the application is consistently using NL loads for data with locality. In this case, NL behavior may be overridden even if cache / bandwidth utilization suggests that NL loads might help. The SAB and performance monitors may reside in or be associated with the memory controller. However, the override decision may be sent to and stored in the core. The core may retain this override decision for a given time window (e.g.,10K cycles) to dynamically translate future NL requests within the time window into regular loads. The override decision can be made for all cores, a subset of cores known to be running threads for the same process, only the core that executed the most recent NL load, only the thread that executed the most recent NL load, and / or only the static instruction that initiated the most recent NL load, in different embodiments.

[0093] Selective activation (SE) based on the rare access buffer, as discussed above, works well for memory accesses generated over power-law-like locality distributions, where there are a few data elements exhibiting locality but many data elements without reuse (e.g., processing social network graphs). However, with the initial bypass and subsequent hit(s) for the requested address to be translated into a memory access to a full cache line, this technique cannot fully recover lost locality. Embodiments that provide overriding (OR) of NL behavior can mitigate lost locality for such cases.If the processor dynamically predicts that NL loads are ineffective and enables the override feature, NL loads can be converted to regular loads at issue time in the core without checking any entries in the SAB. Thus, in override mode, converting NL loads to regular loads does not incur an initial bypass and (at least one) subsequent hit penalty.

[0094] Furthermore, with the SE technique, the dynamic decision based on SAB tracking information occurs after the request has been generated and received in the SAB. Thus, a processor includes a microarchitectural infrastructure to handle both bypassing fine-grained responses and inserting full access responses for pending requests. However, in some embodiments, the override technique may not implement this infrastructure to dynamically modify pending requests based on the memory controller's determination. This means, as with the OR mechanism, that requests are made as regular or nonlocal at issue time in the core and completed as intended, simplifying microarchitectural support.

[0095] In one embodiment, the SAB structure may be simplified because the total hit rate in the SAB can be used as a statistic to identify locality. As such, embodiments may dispense with individual trust labels per fine-grained block. In other embodiments, an SE technique may complement an OR technique so that when no override is detected (e.g., nonlocality behavior is useful), SE can still selectively enable full access for specific data exhibiting locality.

[0096] In embodiments, epoch-based monitoring may be performed, where time is divided into epochs. A new override (or not) decision is made at the beginning of an epoch using statistics collected from the just-completed epoch (and possibly additional prior epochs) as to whether to override NL behavior. This determination may be based on one or more performance monitors, including those described above. At the beginning of each epoch, override logic (which may be present in a locality controller of a memory controller) may determine whether the statistics from the prior epoch indicate an execution operating state that may benefit from NL loads. For example, the override logic may determine whether the LLC MPKI and memory bandwidth counters are higher than predetermined thresholds.Since fine-grained memory accesses provide an advantage when the LLC miss rate is high and there is a high demand for memory bandwidth, NL loads are allowed to proceed as fine-grained accesses if the thresholds are exceeded. Otherwise, the NL loads are converted to regular loads. In some embodiments, SAB statistics can also be analyzed at epoch boundaries. Such SAB statistics are collected only when NL loads are enabled. In some embodiments, the epochs for collecting SAB statistics are different from the epochs for collecting LLC / MC statistics.

[0097] Now referring to Fig. 12 is a flow diagram of a method in accordance with an embodiment of the present invention. In particular, the Fig.12 may be executed by hardware, software, firmware, and / or combinations thereof, such as control logic, to determine whether to override NL indications of one or more memory access instructions within an epoch based on performance monitoring information, as described herein. As can be seen, the method 1200 begins by receiving performance monitoring information for an epoch (block 1210). Although the scope of the present invention is not limited in this regard, in one embodiment, this performance monitoring information may include cache miss rate information, such as LLC miss rates, and memory bandwidth information, such as a number of memory accesses per second, and so forth.Control then transfers to diamond 1220 to determine whether one or more of these performance monitoring values ​​of the performance monitoring information exceed a corresponding threshold. In some examples, a single performance monitoring value less than a threshold may be sufficient to cause control to next transfer to block 1240 for handling the override of nonlocality indications. In other cases, more than one, and possibly all, of the performance monitoring values ​​may be determined to be less than their corresponding threshold before this override handling is to occur.

[0098] Based on the determination at diamond 1220, if control transfers to block 1240, nonlocality hints in all memory access instructions may be overridden for the next epoch. As such, those memory access instructions that have nonlocality hints may, for example, be converted to normal load operations and treated as such, e.g., so that the full width of data can be returned and insertion into a cache hierarchy can occur.

[0099] Still with reference to Fig.12, control instead transfers to block 1230 if it is determined at block 1220 that one or more performance monitoring values ​​exceed the corresponding threshold. There, nonlocality treatment may be enabled for the next epoch. As such, memory access instructions that have nonlocality hints may be treated accordingly, namely, partial widths of data are obtained and returned directly to a core without insertion into a cache hierarchy. It should be noted that in this case, it is possible for individual memory access instructions that have nonlocality hints to still be selectively enabled for normal load treatment based on SAB information for such instructions (in cases where an SAB remains active during the epoch). It should be understood that many variations and alternatives are possible, although in the embodiment of Fig. 12 is shown at this high level.

[0100] Now referring to Fig.Figure 13 shows a flow diagram of a method in accordance with another embodiment of the present invention. In particular, method 1300 describes further details of a dynamic process for deciding whether to override nonlocality. In a particular embodiment, method 1300 may be executed by a memory controller, and in particular, by a locality controller of such a memory controller. As can be seen, method 1300 begins by receiving a plurality of performance monitoring values ​​for a given epoch (namely, a just-completed epoch) (block 1305) and comparing these values ​​to corresponding thresholds (diamond 1310).In one embodiment, these performance monitoring values ​​may be obtained from LLC-MPKI and memory bandwidth counters to determine whether the processor is in a mode of operation that can benefit from the NL load feature. If not, the NL load behavior is overridden, and the issue logic issues incoming NL loads as regular loads. In particular, in this case (e.g., when the cache miss rate and / or memory bandwidth are relatively low), control transfers from diamond 1310 to block 1320, where the override of the nonlocality load behavior is enabled. To this end, the locality control unit may send a control signal to a core (e.g., to the issue logic, the decode logic, and so on) to cause, within the core itself, incoming memory access instructions that have nonlocality hints to be converted into regular memory access instructions, e.g.,regular charging requirements.

[0101] Still referring to Fig.13, if at least one of the performance counters exceeds a given threshold, control instead transfers to diamond 1330 to determine whether an override decision is currently active. If no previously activated override decision exists (and no SAB statistics from the previous epoch are available (as determined at diamond 1340)), the NL load handler is activated, and SAB monitoring is initiated (block 1350). Such NL loads are activated until SAB statistics for at least one epoch are collected. If sufficient SAB statistics are collected (e.g., sufficient statistics from the previous epoch are available), the SAB hit rate may be compared to a threshold (diamond 1360).Although the scope of the present invention is not limited in this respect, in one embodiment, this SAB hit rate may be set at a relatively low level, e.g., approximately 20% or less. In a specific embodiment, an SAB hit rate of approximately 11% may be used as the threshold.

[0102] Based on the determination at diamond 1360, control transfers to either block 1370 or block 1380. If it is determined that the SAB hit rate from the previous epoch is higher than this given threshold, there is a need for high bandwidth, but the NL loads are exhibiting locality. This may be due to a misused NL load instruction in a high-bandwidth application. If it is determined that the SAB hit rate is high, NL loads are overridden at block 1370. Otherwise, if it is determined that the SAB hit rate from the previous epoch is below the given threshold, control transfers from diamond 1360 to block 1380, where NL loads are enabled for the next epoch.

[0103] Note that if the decision is to proceed to block 1370 so that all NL loads are deactivated (i.e., overridden) for the next epoch, it is not possible to use SAB monitoring to check whether program behavior has changed. Here, NL loads may be enabled periodically, but sporadically, for an epoch to detect changes in program behavior, provided the performance monitoring values ​​are high. To this end, in the case where NL loads are overridden, control may transfer from diamond 1330 to diamond 1390 to determine whether NL load handling should be periodically reactivated for a next epoch. If so, control transfers to block 1350, described above. Otherwise, control transfers to block 1395, where the override of NL loading may be maintained in its enabled state.It should be noted that in the case where periodic reactivation for a next epoch occurs, the SAB hit rate can still be monitored to detect a change in program behavior. It should be understood that many variations and alternatives are possible, although in the embodiment of [ ]. Fig. 13 is shown at this high level.

[0104] It is pointed out that Fig.13 assumes a simplest option in which NL loads are globally disabled. However, an override decision can be made for individual cores, threads, or instructions based on the availability of a core / thread ID and an instruction pointer (IP) of the load in the memory control unit. In such a case, redirected addresses are further labeled with the core ID, thread ID, and / or IP, and the SAB hit / miss rate can be tracked per core, per thread, or per IP. The override decision flow follows the same path described in Fig.13. If the performance counters indicate that NL loads cannot be effective, the NL behavior is globally overridden. Otherwise, if SAB statistics were collected in the previous epoch, the SAB hit rate per core / thread / IP is checked. The override decision is then delivered to cores with core / thread / IP information. Overriding NL behavior for one core / thread / IP does not block NL behavior for other cores / threads / IPs. Therefore, correctly used NL loads can be enabled, while misused ones are overridden at the core / thread / IP granularity.

[0105] In some cases, consistently performing fine-grained accesses and bypassing caches can result in significant locality and performance losses for specific workloads. An SE technique dynamically selects accesses with locality and translates them into regular memory accesses to recover lost locality and improve performance. However, for cases where NL loads are almost always counterproductive, SE cannot fully recover lost locality. Embodiments with an override operation can detect such cases and override NL loads, allowing performance to fall back to baseline. However, an OR technique cannot fully exploit selective bypassing when NL loads are actually beneficial (when some rows exhibit locality and others do not). A combination of OR and SE techniques can provide the benefits of both approaches.In particular, combining these techniques can minimize outliers while providing a significant performance boost for workloads that benefit.

[0106] Referring now to Table 1, examples of various workloads and their operation are shown using: (1) selectively enabling locality-based treatment for individual instructions that have nonlocality hints (NL enabled equals "TRUE" in column 3); and (2) globally overriding nonlocality treatment (NL enabled equals "FALSE" in column 3). Specifically, Table 1 presents in column 1 whether, for a given workload, a first performance monitor value (namely, LLC-MPKI) exceeds a corresponding threshold, and in column 2 whether a performance metric of a rare-access buffer (namely, the SAB hit rate) is below a corresponding threshold. As discussed above, column 3 indicates an override decision, where NL enabled (TRUE) means that the global override features are disabled.Column 4, in turn, identifies a performance impact associated with selectively enabling locality-based cues (compared to a baseline implementation where all NL cues are followed). Thus, if NL is enabled, the resulting performance will be the corresponding value from this column. Conversely, if NL is disabled (override is enabled), the resulting performance will be the baseline performance.

[0107] Note that in Table 1, override is enabled and thus "turned on" for 7 workloads (i.e., NL enabled is FALSE). It should also be noted that the performance impact for all of these cases is below 1. Thus, if NL loads were enabled, they would have resulted in performance degradation. As such, using an override technique as described here can successfully predict these cases and disable NL behavior accordingly. Furthermore, note that for workloads where the override operation is disabled (i.e., NL enabled is TRUE), most of the performance improvements are above 1. However, there are two outliers where NL treatment is predicted to be beneficial and thus disable override, but results in a performance degradation.However, as can be seen in column 5, providing an override technique as described here is mostly successful in predicting both positive and negative outcomes of NL loading execution using performance monitoring information. Table 1 LLC-MPKI >THS SAB HIT< THS NL activated? Performance improved for SE OR decision correct? INCORRECT N / A INCORRECT 0,87 TRUE TRUE TRUE TRUE 1,31 TRUE INCORRECT N / A INCORRECT 0,93 TRUE INCORRECT N / A INCORRECT 0,83 TRUE TRUE TRUE TRUE 1,05 TRUE TRUE TRUE TRUE 1,05 TRUE TRUE INCORRECT INCORRECT 0,95 TRUE TRUE TRUE TRUE 1,05 TRUE TRUE TRUE TRUE 0,99 INCORRECT TRUE TRUE TRUE 1,09 TRUE TRUE TRUE TRUE 1,08 TRUE TRUE TRUE TRUE 1,13 TRUE INCORRECT N / A INCORRECT 0,77 TRUE TRUE TRUE TRUE 0,79 INCORRECT INCORRECT N / A INCORRECT 0,82 TRUE INCORRECT N / A INCORRECT 0,94 TRUE TRUE TRUE TRUE 1,04 TRUE TRUE TRUE TRUE 1,08 TRUE TRUE TRUE TRUE 1,14 TRUE TRUE TRUE TRUE 1,08 TRUE TRUE TRUE TRUE 1,08 TRUE TRUE TRUE TRUE 1,19 TRUE

[0108] Embodiments can globally enable / disable both the selective enabling of NL handling and the override operation via a control register and / or basic input / output system (BIOS) settings. Dynamic execution, as described here, can detect locality potential and translate fine-grained accesses into regular accesses to full cache lines. Furthermore, it can override such accesses in the core if no benefit is predicted. Thus, misuse of the fine-grained feature is dynamically corrected. This gives programmers more freedom to use these advanced features without having to worry too much about negative performance impacts.

[0109] The following examples are further embodiments.

[0110] In one example, a processor includes: a core including a decode unit to decode a memory access instruction having a non-locality indication to indicate that data associated with the memory access instruction has at least one of non-spatial locality and non-temporal locality; and a memory controller to issue requests to a memory, the memory controller including a locality controller to determine whether to override the non-locality indication based at least in part on one or more performance monitor values.

[0111] In one example, the one or more performance monitoring values ​​include a cache memory miss rate and a memory bandwidth.

[0112] In one example, the locality controller is to override the non-locality indication in response to a comparison of at least one of the cache miss rate and the memory bandwidth with a corresponding threshold.

[0113] In one example, the processor further includes a rare access buffer having a plurality of entries, each of which is used to store address information and count information for a memory access instruction to a particular address.

[0114] In one example, the locality controller is to further override the non-locality indication when a hit rate of the infrequent access buffer exceeds a second threshold.

[0115] In one example, the locality controller is to obtain the one or more performance monitoring values ​​for a first time period and to override the non-locality indication for a second time period, including overriding multiple non-locality indications from multiple memory access instructions during the second time period.

[0116] In one example, the processor further comprises an infrequent access buffer having a plurality of entries, each of which is operable to store address information and count information for a memory access instruction to a particular address, wherein the locality control unit is operable to disable the infrequent access buffer during the second time period.

[0117] In one example, the locality control unit is to issue an override signal to the core to cause the core to translate one or more memory access instructions having the non-locality hint into one or more memory access instructions without non-locality hints.

[0118] In one example, the locality control unit is to override the nonlocality hint for a first memory access instruction of a first thread and not override the nonlocality hint for a second memory access instruction of a second thread.

[0119] In one example, the non-locality hint is to cause data to be returned to the core bypassing a cache of the processor, and in response to overriding the non-locality hint, the locality controller is to cause a full-width portion of data to be obtained from memory and provided to the core.

[0120] In one example, the memory controller is to, in response to the nonlocality hint, cause a portion of data below a cache line to be retrieved from memory and provided to the core.

[0121] In another example, a method comprises: receiving, at a control unit of a processor, performance monitoring information for a first period of time; comparing one or more performance monitoring values ​​from the performance monitoring information to a corresponding threshold; and in response to at least one of the one or more performance monitoring values ​​not exceeding the corresponding threshold, causing the processor to override nonlocality indications of one or more memory access instructions during a second period of time.

[0122] In one example, causing the processor to override the non-locality hints of the one or more memory access instructions during the second time period includes sending a signal to dispatch logic of the processor to cause the dispatch logic to translate one or more memory access instructions having the non-locality hint into one or more memory access instructions without the non-locality hint during the second time period.

[0123] In one example, the method further comprises, in response to the one or more performance monitoring values ​​exceeding the corresponding threshold, enabling non-locality memory access command handling for the second time period.

[0124] In one example, nonlocality memory access instruction handling for a first memory access instruction having the nonlocality hint includes one or more of the following: returning data to the core bypassing a cache of the processor; and returning a portion of data below a cache line to the core.

[0125] In one example, the method further comprises, after a plurality of time periods in which the nonlocality hints are disabled, enabling nonlocality memory access instruction handling for a next time period.

[0126] In one example, the method further comprises: obtaining at least one performance metric for an infrequent access buffer for the next time period; and enabling nonlocality memory access command handling for a time period following the next time period based at least in part on the at least one performance metric for the infrequent access buffer.

[0127] In another example, a computer-readable medium containing instructions is used to carry out the method of any of the preceding examples.

[0128] In another example, a computer-readable medium containing data is to be used by at least one machine to manufacture at least one integrated circuit to perform the method of any of the preceding examples.

[0129] In a further example, a device comprises means for carrying out the method according to any one of the preceding examples.

[0130] In yet another embodiment, a system comprises: a processor having a core to process memory access instructions that include nonlocality hints to indicate nonlocality of data associated with the memory access instructions, the core having one or more performance monitors; a memory controller to issue requests to a system memory, the memory controller including a locality controller to receive the memory access instructions that include the nonlocality hints and to override the nonlocality hints of the memory access instructions that include the nonlocality hints for a second period of time based at least in part on a performance metric for a first period of time obtained from the one or more performance monitors; and the system memory coupled to the memory controller.

[0131] In one example, in the second time period and in response to overriding the non-locality hints of the memory access instructions, the memory controller is to return multiple full-width cache lines to the core via a cache hierarchy, and in a third time period in which non-locality hints are not to be overridden, the memory controller is to bypass caching and return data less than a cache line wide to the core for one or more memory access instructions that have non-locality hints.

[0132] In one example, the memory controller is to, in response to a determination to override the nonlocality hints of the memory access instructions having the nonlocality hints for a second period of time, issue an override signal to the core to cause the core to translate the memory access instructions having the nonlocality hints into memory access instructions without nonlocality hints.

[0133] In another example, an apparatus comprises: means for receiving performance monitoring information of a processor for a first period of time; means for comparing one or more performance monitoring values ​​from the performance monitoring information to a corresponding threshold; and means for causing the processor to override nonlocality indications of one or more memory access instructions during a second period of time based at least in part on a first result of comparing the one or more performance monitoring values ​​of the performance monitoring information to the corresponding threshold.

[0134] In one example, the device further comprises means for sending a signal to an output means for causing the output means to convert one or more memory access instructions having the non-locality indication into one or more memory access instructions without the non-locality indication during the second time period.

[0135] In one example, the device further comprises means for enabling non-locality memory access command handling for the second time period based at least in part on a second result of comparing the one or more performance monitoring values ​​of the performance monitoring information to the corresponding threshold.

[0136] In one example, the apparatus further comprises means for returning data to a core means of the processor bypassing a cache means of the processor.

[0137] In one example, the device further comprises means for returning a data portion below a cache line to a core means of the processor.

[0138] It should be understood that various combinations of the above examples are possible.

[0139] It should be noted that the terms "circuit" and "circuitry" are used interchangeably herein. As used herein, these terms and the term "logic" are used, alone or in combination, to refer to analog circuitry, digital circuitry, hard-wired circuitry, programmable circuitry, processor circuitry, microcontroller circuitry, hardware logic circuitry, state machine circuitry, and / or any other type of physical hardware component. Embodiments may be used in many different types of systems. For example, in one embodiment, a communications device may be configured to perform the various methods and techniques described herein.Of course, the scope of the present invention is not limited to a communications device, and instead, other embodiments may be directed to other types of devices for processing instructions or one or more machine-readable media containing instructions that, in response to being executed on a computing device, cause the device to perform one or more of the methods and techniques described herein.

[0140] Embodiments may be implemented in code and may be stored on a non-transitory storage medium having instructions stored thereon that can be used to program a system to execute the instructions. Embodiments may be implemented in data and may be stored on a non-transitory storage medium that, when used by at least one machine, causes the at least one machine to manufacture at least one integrated circuit to perform one or more operations. Still further embodiments may be implemented in a computer-readable storage medium that contains information that, when manufactured into an SoC or other processor, is used to configure the SoC or other processor to perform one or more operations.The storage medium may include any type of disk including, but not limited to, floppy disks, optical disks, solid-state drives (SSDs), compact disk read-only memories (CD-ROMs), rewritable compact disks (CD-RWs), and magneto-optical disks, semiconductor devices such as read-only memories (ROMs), random access memories (RAMs) such as dynamic random access memories (DRAMs), static random access memories (SRAMs), erasable programmable read-only memories (EPROMs), flash memory, electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, and any other type of media suitable for storing electronic instructions.

[0141] Although the present invention has been described with respect to a limited number of embodiments, those skilled in the art will recognize numerous modifications and variations therefrom. It is intended that the appended claims cover all such modifications and variations as falling within the true spirit and scope of this present invention.

Claims

[1] Processor (202) comprising: a core (104) including a decode unit (230) to decode a memory access instruction (214) having a non-locality hint to indicate that data associated with the memory access instruction (214) has at least one of non-spatial locality and non-temporal locality, wherein the non-spatial locality implies that when the data is accessed from an address in a memory (118), adjacent data is not likely to be accessed within a time period, and wherein the non-temporal locality implies that when the data is accessed from the address in the memory (118), the same data is not likely to be accessed again within the time period; and a memory controller (246) to issue requests to the memory (118), the memory controller (246) including a locality controller (347) to determine whether to override the non-locality indication based at least in part on one or more performance monitoring values. [2] The processor (202) of claim 1, wherein the one or more performance monitoring values ​​comprise a cache memory miss rate and a memory bandwidth. [3] The processor (202) of claim 2, wherein the locality control unit (347) is to override the non-locality indication in response to a comparison of at least one of the cache miss rate and the memory bandwidth with a corresponding threshold. [4] The processor (202) of claim 1, further comprising a rare access buffer (245) having a plurality of entries, each of which is for storing address information and count information for a memory access instruction (112) to a particular address. [5] The processor (202) of claim 4, wherein the locality control unit (347) is to further override the non-locality indication when a hit rate of the infrequent access buffer (245) exceeds a second threshold. [6] The processor (202) of claim 1, wherein the locality control unit (347) is to obtain the one or more performance monitoring values ​​for a first time period and to override the non-locality indication for a second time period, including overriding a plurality of non-locality indications from a plurality of memory access instructions (112) during the second time period. [7] The processor (202) of claim 6, wherein the processor (202) further comprises an infrequent access buffer (245) having a plurality of entries, each of which is for storing address information and count information for a memory access instruction (112) to a particular address, wherein the locality control unit (347) is for disabling the infrequent access buffer (245) during the second time period. [8] The processor (202) of claim 1, wherein the locality control unit (347) is to issue an override signal to the core (104) to cause the core (104) to translate one or more memory access instructions (214) having the non-locality hint into one or more memory access instructions without non-locality hints. [9] The processor (202) of claim 1, wherein the locality control unit (347) is to override the non-locality indication for a first memory access instruction of a first thread and not to override the non-locality indication for a second memory access instruction of a second thread. [10] The processor (202) of claim 1, wherein the non-locality hint is to cause data to be returned to the core (104) bypassing a cache memory (208) of the processor (202), and the memory controller (246) is to cause, in response to the non-locality hint being overridden, a full-width portion of data to be obtained from the memory (118) and provided to the core (104). [11] The processor (202) of claim 1, wherein the memory controller (246) is responsive to the non-locality indication to cause a portion of data below a cache line to be obtained from the memory (118) and provided to the core (104). [12] Procedure (1200) comprising: Receiving performance monitoring information for a first period of time in a control unit of a processor (1210); Comparing one or more performance monitoring values ​​from the performance monitoring information with a corresponding threshold value (1220); and in response to at least one of the one or more performance monitoring values ​​not exceeding the corresponding threshold, causing the processor to override (1240) non-locality indications of one or more memory access instructions during a second period of time, wherein the non-locality indication indicates that data associated with the one or more memory access instructions has at least one of non-spatial locality and non-temporal locality, wherein the non-spatial locality implies that when the data is accessed from an address in memory, adjacent data is not likely to be accessed within a period of time, and wherein the non-temporal locality implies that when the data is accessed from the address in memory,that the same data is accessed again within the time period. [13] The method (1200) of claim 12, wherein causing the processor to override (1240) the non-locality hints of the one or more memory access instructions during the second time period comprises sending a signal to an output logic of the processor to cause the output logic to translate one or more memory access instructions having the non-locality hint into one or more memory access instructions without the non-locality hint during the second time period. [14] The method (1200) of claim 12, further comprising, in response to the one or more performance monitoring values ​​exceeding the corresponding threshold, enabling non-locality memory access command handling for the second time period. [15] The method (1200) of claim 14, wherein the non-locality memory access instruction handling for a first memory access instruction having the non-locality hint comprises one or more of the following: a return of data to the core that bypasses a processor cache; and a return of a data section below a cache line to the core. [16] The method (1200) of claim 12, further comprising enabling nonlocality memory access instruction handling for a next time period after a plurality of time periods in which the nonlocality hints are disabled. [17] The method (1200) of claim 16, further comprising: Obtaining at least one performance metric for a rare access buffer for the next time period; and Enabling nonlocality memory access instruction handling for a time period following the next time period based at least in part on the at least one performance metric for the infrequent access buffer. [18] A computer-readable storage medium comprising computer-readable instructions which, when executed, serve to implement a method according to any one of claims 12 to 17. [19] Device comprising means for carrying out a method according to any one of claims 12 to 17. [20] System (100) comprising: a processor (202) having a core (104) for processing memory access instructions (214) that include non-locality hints to indicate non-locality of data associated with the memory access instructions, the core (104) having one or more performance monitors, the non-locality hints indicating that the data has at least one of non-spatial locality and non-temporal locality, the non-spatial locality implying that when the data is accessed from an address in a system memory (118), adjacent data is not likely to be accessed within a time period, and the non-temporal locality implying that when the data is accessed from the address in the system memory (118), the same data is not likely to be accessed again within the time period; a memory control unit (246) for issuing requests to a system memory (118), the memory control unit (246) including a locality control unit (347) for receiving the memory access commands (214) having the non-locality indications and for overriding the non-locality indications of the memory access commands (214) having the non-locality indications for a second period of time based at least in part on at least one performance metric for a first period of time obtained from the one or more performance monitoring units; and the system memory (118) coupled to the memory control unit (246). [21] The system (100) of claim 20, wherein in the second time period and in response to the non-locality hints of the memory access instructions (214) being overridden, the memory controller (246) is to return a plurality of full-width cache lines to the core (104) via a cache hierarchy, and in a third time period in which non-locality hints are not to be overridden, the memory controller (246) is to bypass caching and return data less than a cache line wide to the core (104) for one or more memory access instructions (214) that have non-locality hints. [22] The system (100) of claim 20, wherein, in response to a determination to override the non-locality hints of the memory access instructions (214) having the non-locality hints for the second period of time, the memory controller (246) is to issue an override signal to the core to cause the core (104) to translate the memory access instructions (214) having the non-locality hints into memory access instructions without non-locality hints. [23] Establishment comprising: Means for receiving performance monitoring information of a processor (202) for a first period of time; means for comparing one or more performance monitoring values ​​of the performance monitoring information with a corresponding threshold value; and Means for causing the processor (202) to override non-locality indications of one or more memory access instructions (214) during a second period of time based at least in part on a first result of comparing the one or more performance monitoring values ​​of the performance monitoring information with the corresponding threshold, wherein the non-locality indications indicate that data associated with the one or more memory access instructions (214) has at least one of non-spatial locality and non-temporal locality, wherein the non-spatial locality implies that when the data is accessed from an address in a memory (118), adjacent data is not likely to be accessed within a period of time, and wherein the non-temporal locality implies that when the data is accessed from the address in the memory (118),it is not likely that the same data will be accessed again within the time period. [24] The apparatus of claim 23, further comprising means for sending a signal to an output means for causing the output means to convert one or more memory access instructions (214) having the non-locality indication into one or more memory access instructions without the non-locality indication during the second time period. [25] The apparatus of claim 23, further comprising means for enabling non-locality memory access command servicing for the second time period based at least in part on a second result of comparing the one or more performance monitoring values ​​of the performance monitoring information to the corresponding threshold.

Images