Step-based prefetcher circuit for prefetching next step into cache memory based on identified cache access step pattern, and related processor-based systems and methods

Abstract

A stride-based prefetcher circuit for prefetching data of a next stride of a cache read request to a cache memory based on a stride pattern identified in the cache read request, and related processor-based systems and methods are disclosed. A stride-based prefetcher circuit ("prefetcher circuit") observes a cache read request to a cache memory at runtime to determine whether there is a stride mode. In response to detecting the stride mode, the prefetcher circuit prefetches data from the higher level memory from the next storage location in the detected stride and loads the prefetched data into the cache memory. This is more likely that a subsequent cache read request to the cache memory will be more likely to continue with the same stride when it is detected that there is a stride in the cache read request to the cache memory. As a result, the cache hit rate of the cache memory may increase.

Description

Priority application

[0001] This application claims priority to U.S. patent application serial number 17 / 821,016, filed on August 19, 2022, and entitled “STRIDE-BASED PREFETCHERCIRCUITS FOR PREFETCHING NEXT STRIDE(S)INTO CACHE MEMORY BASED ON IDENTIFIED CACHE ACCESS STRIDE PATTERNS, AND RELATED PROCESSOR-BASED SYSTEMS AND METHODS,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] The technical field of the present disclosure relates to cache memory in processor-based systems (e.g., graphics processing unit (GPU)-based systems, central processing unit (CPU)-based systems), and specifically, the field of the present disclosure relates to methods of improving cache hit rates in cache memory. Background Art

[0003] A microprocessor, also known as a processing unit (PU), performs computing tasks in various applications. One type of traditional microprocessor or PU is a central processing unit (CPU). Another type of microprocessor or PU is a dedicated processing unit called a graphics processing unit (GPU). The GPU is designed with specialized hardware to accelerate the rendering of graphics and video data for display. The GPU can be implemented as an integrated component of a general-purpose CPU, or as a discrete hardware component separated from the CPU. The PU executes software instructions that instruct the processor to obtain data from a location in the memory and perform one or more processor operations using the obtained data. The result can then be stored in the memory. For example, the memory can be a cache memory local to the PU, a shared local cache between PUs in a PU block, a shared cache between multiple PU blocks, and / or a system memory in a processor-based system. Cache memory, which can also be referred to as a "cache," is a smaller and faster memory that stores copies of data stored at frequently accessed memory addresses in the main memory or higher-level cache memory to reduce memory access latency. Therefore, the PU can use cache memory to reduce memory access time.

[0004] When the data requested by a memory read request is present in cache memory (i.e., a cache "hit"), system performance can be improved by retrieving the data from the cache rather than accessing the system memory more slowly. Conversely, if the requested data is not found in the cache (resulting in a cache "miss"), the requested data must be read from a higher-level cache or system memory. Frequent occurrences of cache misses can result in reduced system performance, which may negate the advantage of using a cache in the first place. The cache hit rate of a cache memory can often be improved by increasing the size of the cache memory, because the increased size of the cache memory can store more cache lines, thereby increasing the likelihood of a cache hit. However, increasing the size of the cache memory increases the cost in terms of area and power consumption. Summary of the invention

[0005] Aspects disclosed herein include a stride-based prefetcher circuit for prefetching the next stride of a cache read request into a cache memory based on a stride pattern identified in the cache read request. Related processor-based systems and methods are also disclosed. In exemplary aspects, a stride-based prefetcher circuit ("prefetcher circuit") is provided as part of a processor-based system. The processor-based system may include a central processing unit (CPU) and / or a dedicated processing unit, such as a graphics processing unit (GPU). The prefetcher circuit is coupled to the cache memory so that the prefetcher circuit can observe cache read requests to the cache memory. The prefetcher circuit observes a series of cache read requests at runtime to determine whether the addresses of such cache read requests present a stride pattern with previously observed cache read requests. It is easier to more accurately detect the stride pattern in the cache read request at runtime than based on program instructions at compile time. If the prefetcher circuit observes the stride pattern in a specified number of cache read requests, the prefetcher circuit considers that there is a stride pattern in such cache read requests. In response to detecting a stride pattern in a cache read request, a prefetcher circuit prefetches data from a higher level memory (e.g., a higher level cache memory or a system memory) from one or more next storage locations in the detected stride. The prefetcher circuit then causes the prefetched data from the one or more next storage locations in the detected stride to be written to a cache entry in the cache memory. It is believed that there is a higher probability that when a stride is present in a cache read request, subsequent cache read requests to the cache memory are more likely to continue with the same stride. Therefore, the prefetched data written into the cache entry in the cache memory may be more useful than data evicted from an overwritten cache entry of the cache memory. Therefore, the cache hit rate of the cache memory can be increased, thereby forming a higher performance cache memory.

[0006] In other exemplary aspects, if the stride pattern continues to be observed in subsequent cache read requests to the cache memory, the prefetcher circuit continues to prefetch data from one or more next storage locations in the detected stride and writes it to the cache memory. If the next cache read request to the cache memory is observed to not follow the currently detected stride, the prefetcher circuit can ignore the currently detected stride as the detected stride and interrupt prefetching until a new stride in a subsequent cache read request is detected. If the prefetcher circuit detects a new stride in such a subsequent cache read request, the prefetcher circuit will again prefetch data from one or more next storage locations of the new stride in a higher level memory and as a write request to the cache memory, cause the prefetched data to be written to the cache memory. In exemplary aspects, the process will continue in terms of prefetching data for the detected stride, then not prefetching data when the detected stride is no longer observed, and returning to prefetching a new detected stride, and so on.

[0007] In other exemplary aspects, once the prefetcher circuit observes that the total number of consecutive cache read requests to the cache memory that follow the currently detected stride exceeds a propagation count threshold, the prefetcher circuit can designate the currently detected stride as having a higher confidence. In response, the prefetcher circuit can be configured to propagate the prefetch request to multiple next storage locations of the detection stride in each prefetch request (e.g., four (4) next storage locations in the detection stride) in response to the next cache read request that matches the currently detected stride. This is because there is an even higher level of confidence that the prefetched data for the current stride is useful in the cache memory for subsequent cache read requests before being evicted. Again, in one example, if the next cache read request to the cache memory is observed not to follow the currently detected stride, the prefetcher circuit will ignore the currently detected stride as a detected stride and not prefetch data. If the prefetcher circuit detects a new stride in such a subsequent cache read request, the prefetcher circuit will again prefetch data for the new stride in the newly detected stride pattern and cause the prefetched data at the next storage location of the stride to be written to the cache memory. Prefetching for the newly detected stride may be reset with a lower confidence so that multiple prefetch requests are not issued until a total number of consecutive cache read requests to the cache memory following the currently detected stride exceeds a propagation count threshold.

[0008] In another example, the prefetcher circuit can also be configured to adaptively turn off and reopen prefetching. The prefetcher circuit can be configured to adaptively turn off and reopen prefetching based on tracking the total number of prefetch propagations that occur on a given interval (observation interval) of observed read requests to the cache memory. Prefetch propagation refers to the prefetching performed by the prefetcher circuit on the observed stride pattern. If the total number of prefetch propagations is less than a specified propagation threshold within a given observation interval of observed cache read requests, this means that prefetching may not be so useful because fewer strides are observed to be prefetched. In this case, the prefetcher circuit can be configured to turn off or interrupt data prefetching while still continuing to detect strides in subsequent cache read requests and tracking prefetch propagation that would occur if prefetching was turned on or activated. The prefetcher circuit can be configured to: if the total number of prefetch propagations is greater than a specified propagation threshold on a given observation interval of observed cache read requests, then reopen or continue data prefetching. Adaptive prefetching can reduce or avoid jitter of cache memory.

[0009] The prefetcher circuitry may also be configured to selectively observe and track strides on a per-thread basis. In this aspect, the prefetcher circuitry may be configured to determine, on a per-thread basis, whether strides exist on cache read requests for a given thread that issued the cache read request. This example is in contrast to only observing and detecting strides in cache read requests that are not thread filtered. Cache read requests that are not tracked on a per-thread basis may appear not to follow strides because cache read requests for different threads may be interleaved and appear non-contiguous for a given thread, and also result in increased propagation confidence, as described above.

[0010] In this regard, in an exemplary aspect, a prefetcher circuit is disclosed. The prefetcher circuit includes an access bus coupled to a cache access bus, the cache access bus coupled to a cache memory in a memory system. The prefetcher circuit is configured to observe a plurality of first cache read requests including a plurality of first memory addresses on the access bus. The prefetcher circuit is also configured to detect a first stride in the plurality of first cache read requests. The prefetcher circuit is also configured to, in response to detecting the first stride in the plurality of first cache read requests, issue one or more prefetch requests to prefetch one or more first data to be written to the cache memory from a second memory of the memory system at one or more next memory addresses after a last memory address in the plurality of first memory addresses with the detected first stride.

[0011] In another exemplary aspect, a method for prefetching data into a cache memory based on a stride pattern detected in a cache read request to the cache memory is disclosed. The method includes observing a plurality of first cache read requests including a plurality of memory addresses on an access bus coupled to a cache access bus, the cache access bus coupled to a cache memory in a memory system. The method also includes detecting a first stride in the plurality of first cache read requests. The method also includes, in response to detecting the first stride in the plurality of first cache read requests, issuing one or more prefetch requests to prefetch one or more first data to be written to the cache memory from a second memory of the memory system at one or more next memory addresses after a last memory address in the plurality of memory addresses with the detected first stride.

[0012] In another exemplary aspect, a processor-based system is disclosed. The processor-based system includes a memory system. The memory system includes a first cache memory. The memory system also includes a second memory, the second memory being a higher level memory than the first cache memory. The processor-based system also includes a cache access bus coupled to the cache memory. The processor-based system also includes a processor configured to issue a plurality of first cache read requests, the plurality of first cache read requests including a plurality of first memory addresses to be asserted on the cache access bus. The processor-based system also includes a prefetcher circuit. The prefetcher circuit is configured to observe the plurality of first cache read requests on the cache access bus. The prefetcher circuit is further configured to detect a first stride in the plurality of first cache read requests. The prefetcher circuit is further configured to, in response to detecting the first stride in the plurality of first cache read requests, issue one or more prefetch requests to prefetch one or more first data to be written to the cache memory from a second memory of the memory system at one or more next memory addresses after a last memory address in a plurality of memory addresses with the detected first stride. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram of an exemplary graphics processing unit (GPU) system including an exemplary cache memory system including a cache memory and an exemplary stride-based prefetcher circuit, wherein the stride-based prefetcher circuit is configured to detect a stride pattern in observed cache read requests and prefetch data for a next stride of the detected strided cache read requests from a higher level memory into the cache memory;

[0014] Figure 2 yes Figure 1 a more detailed block diagram of a cache memory system in which the cache memory and the stride-based prefetcher circuit are included;

[0015] Figure 3 It is a depiction Figure 1 and Figure 2 a flowchart of an exemplary process for a stride-based prefetcher circuit in which a cache read request to a cache memory is observed to detect a stride and data at a next storage location of the detected stride is prefetched from a higher level memory into the cache memory;

[0016] Figure 4 is a block diagram of an exemplary pre-fetch buffer circuit including a plurality of pre-fetch buffer entries for storing information to identify Figure 1 and Figure 2 cache read requests observed by a stride-based prefetcher circuit in the embodiment of the present invention, and tracking a detected stride of the observed cache read requests;

[0017] Figure 5 It shows that it can be Figure 2 The prefetcher circuit in Figure 4 A flowchart of an exemplary prefetch process performed by a prefetch buffer circuit in a cache memory to detect a stride in a cache read request and prefetch data of a next storage location of the detected stride into a cache (e.g., a cache memory);

[0018] Figure 6 yes Figure 1 and Figure 2 Examples of cache read requests observed by the stride-based prefetcher circuit in Figure 4 The corresponding information corresponding to the observed cache read request is stored in the prefetch buffer entry in the prefetch buffer circuit in the prefetch buffer circuit for performing Figure 5 Prefetch processing in

[0019] Figure 7 It shows that it can be Figure 2 a flow chart of an exemplary adaptive prefetch process performed by a stride-based prefetcher circuit in order to adaptively prefetch data into a cache memory based on a detected stride in a cache read request and based on whether a number of prefetch propagations exceeds a propagation threshold;

[0020] Figure 8 yes Figure 1 and Figure 2Examples of cache read requests observed by the stride-based prefetcher circuit in and Figure 4 The corresponding information corresponding to the observed cache read request stored in the prefetch buffer entry in the prefetch buffer circuit in the for performing Figure 7 Adaptive prefetching process in

[0021] Fig. 9 is a block diagram of an exemplary processor-based system including a plurality of central processing units (CPUs) and a memory system including a cache memory system and a system memory, the cache memory system including a hierarchy of local and shared cache memories, and wherein a stride-based prefetcher circuit may also be included to detect a stride in an observed cache read request and prefetch data at a next memory location of the detected stride from a higher level memory into the cache memory, the stride-based prefetcher circuit including but not limited to Figure 1 and Figure 2 The stride-based prefetcher circuit in Figure 3 and Figure 5-8 any of the exemplary processes in ; and

[0022] Fig.10 is a block diagram of another exemplary processor-based system that may include a cache memory system that includes a plurality of cache memories and wherein a stride-based prefetcher circuit may also be included to detect a stride pattern in observed cache read requests and prefetch data at a next memory location of the detected stride from a higher level memory into the cache memory, the stride-based prefetcher circuit including but not limited to Figure 1 and Figure 2 The stride-based prefetcher circuit in Figure 3 and Figure 5-8 Any of the exemplary procedures described herein may be performed. DETAILED DESCRIPTION

[0023] Referring now to the accompanying drawings, several exemplary aspects of the present disclosure are described. As used herein, the term "exemplary" means "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" should not be construed as being preferred or advantageous over other aspects.

[0024] Aspects disclosed herein include a stride-based prefetcher circuit for prefetching the next stride of a cache read request into a cache memory based on a stride pattern identified in a cache read request. Related processor-based systems and methods are also disclosed. In an exemplary aspect, a stride-based prefetcher circuit ("prefetcher circuit") is provided as part of a processor-based system. The processor-based system may include a central processing unit (CPU) and / or a dedicated processing unit, such as a graphics processing unit (GPU). The prefetcher circuit is coupled to the cache memory so that the prefetcher circuit can observe cache read requests to the cache memory. The prefetcher circuit observes a series of cache read requests at runtime to determine whether the address of such a cache read request presents a stride pattern with a previously observed cache read request. This can make it easier to more accurately detect the stride pattern in the cache read request at runtime, rather than based on program instructions at compile time. If the prefetcher circuit observes the stride pattern in a specified number of cache read requests, the prefetcher circuit considers that there is a stride pattern in such a cache read request. In response to detecting a stride pattern in a cache read request, a prefetcher circuit prefetches data from a higher level memory (e.g., a higher level cache memory or a system memory) from one or more next storage locations in the detected stride. The prefetcher circuit then causes the prefetched data from the one or more next storage locations in the detected stride to be written to a cache entry in the cache memory. This is because it is believed to be more likely that when a stride is present in a cache read request, subsequent cache read requests to the cache memory may be more likely to continue with the same stride. Therefore, the prefetched data written into the cache entry in the cache memory may be more useful than data evicted from an overwrite cache entry of the cache memory. Therefore, the cache hit rate of the cache memory may be increased, thereby forming a higher performance cache memory.

[0025] In other exemplary aspects, if the stride pattern continues to be observed in subsequent cache read requests to the cache memory, the prefetcher circuit continues to prefetch data from one or more next storage locations in the detected stride and writes it to the cache memory. If the next cache read request to the cache memory is observed to not follow the currently detected stride, the prefetcher circuit can ignore the currently detected stride as the detected stride and interrupt prefetching until a new stride in a subsequent cache read request is detected. If the prefetcher circuit detects a new stride in such a subsequent cache read request, the prefetcher circuit will again prefetch data from one or more next storage locations of the new stride in the higher level memory and as a write request to the cache memory, cause the prefetched data to be written to the cache memory. In exemplary aspects, the process will continue in terms of prefetching the data of the detected stride, and then not prefetching data when the detected stride is no longer observed, and returning to prefetching the new detected stride, and so on.

[0026] In this regard, Figure 1 1 is a block diagram of an exemplary graphics processing unit (GPU) based system 100 that includes a GPU 102 that may be provided in an integrated circuit (IC) chip 104. The GPU 102 includes a portion of a memory system 106 in the GPU based system 100. The memory system 106 includes a cache memory system 108 that includes both a shared high level cache memory 110 and a shared low level cache memory 112 that are integrated into the GPU 102. The GPU based system 100 includes a plurality of processors 114(1)-114(N) (e.g., shader processors (SPs)) that are configured to execute programmable software to perform graphics related tasks (e.g., vertex processing, pixel processing). The GPU 102 in this example also includes rendering support circuits 116(1)-116(R) that may be called upon to perform specialized tasks (including frame merging and blending) for the processors 114(1)-114(N). The GPU 102 also includes dedicated processing circuitry that is part of the GPU pipeline to perform dedicated, non-programmable tasks, including a triangle setup engine (TSE) 118(1), 118(2), a rasterizer (RAS) 120(1), 120(2), and a low-resolution Z culler (LRZ) 122(1), 122(2). The RAS 120(1), 120(2) can take three (3) vertices of a triangle and generate coordinates of points within the triangle by interpolation. The LRZ 122(1), 122(2) can perform culling to reduce the rendering work of the GPU 102.

[0027] Continue to refer Figure 1 , the processors 114(1)-114(N) in the GPU 102 are respectively configured to issue memory access requests to the cache memory system 108 to improve memory access performance. The processors 114(1)-114(N) can issue memory read requests to read data stored in the memory system 106. The processors 114(1)-114(N) can also issue memory write requests to write data to the memory system 106. In this example, the memory requests from the processors 114(1)-114(N) are first provided to the low level cache memory 112 in the GPU 102. The memory read and write requests are first provided to the low level cache memory 112 in the memory system 106. The memory write request writes data to the low level cache memory 112, wherein the cache entry in the low level cache memory 112 is evicted to the high level cache memory 110. If the memory read request to the lower level cache memory 112 results in a cache miss, the memory read request is passed to the higher level cache memory 110. If the memory read request through the higher level cache memory 110 also results in a cache miss, the memory read request is passed to the next higher level cache memory or system memory (which has storage locations for the entire physical address space). The read data retrieved as a result of the cache miss to the lower level cache memory 112 is inserted into the lower level cache memory 112 so that the read data is available if another memory request is for the same memory address of the memory read request.

[0028] Cache misses in the cache memory system 108 in the GPU 102 result in reduced system performance. The cache hit rate of the cache memory system 108 can generally be improved by increasing the size of the cache memories (e.g., the lower level cache memories 112 and the higher level cache memories 110) because the increased size of the cache memories can store more cache lines, thereby increasing the likelihood that the cache memories contain data at the address of the memory read request. However, increasing the size of the cache memories in the cache memory system 108 of the GPU 102 comes at an increased cost in terms of increased area and power consumption in the GPU 102.

[0029] Therefore, in order to improve the cache memory (e.g. Figure 1To improve the cache hit rate of the low-level cache memory 112 in the GPU 102 in the CPU 102, a stride-based prefetcher circuit can be used. A stride-based prefetcher circuit (an example of which is discussed herein) can be used to prefetch the next stride into the cache memory based on the identified cache access stride pattern. This is done by Figure 2 As shown in the example. Figure 2 It shows that it can be Figure 2 The memory system 200 is provided in the cache memory system 108 in the GPU-based system 100. Figure 2 The memory system 200 provides Figure 1 The memory system 200 includes a low level cache memory 112 and a high level cache memory 110 of the cache memory system 108. In this example, the memory system 200 includes a stride-based prefetcher circuit 202 ("prefetcher circuit 202"), which includes a prefetcher circuit 202 such as Figure 1 In a processor-based system of the GPU-based system 100 in FIG. 1 . In one example, the prefetcher circuit 202 is a circuit implemented only in a discrete hardware circuit that does not include a processor or other controller other than executing software, so that the prefetcher circuit 202 can operate more efficiently. The prefetcher circuit 202 is coupled to an access bus 204, and the access bus 204 is coupled to a cache access bus 206. The cache access bus 206 is configured to receive memory access requests (i.e., cache read requests 208R and cache write requests 208W) for memory access to the lower level cache memory 112. In this example, there are twelve (12) access ports 210 (1)-210 (12) to the lower level cache memory 112, so that twelve (12) different process threads 212 can access the lower level cache memory 112 simultaneously. The lower level cache memory 112 includes an arbiter circuit 214 that arbitrates cache access requests between different process threads 212 and memory banks 216 in the lower level cache memory 112. Cache misses in response to cache read requests 208R and cache evictions in response to cache write requests 208W to the lower level cache memory 112 are communicated to the higher level cache memory 110.

[0030] Continue to refer Figure 2In this example, the prefetcher circuit 202 is configured to observe the cache read request 208R on its access bus 204 by coupling its access bus 204 to the cache access bus 206. The prefetcher circuit 202 observes the memory address of the cache read request 208R on the access bus 204. The prefetcher circuit 202 observes a series of cache read requests 208R at runtime to determine whether the memory addresses of such cache read requests 208R exhibit a striding pattern. The striding pattern in the read request refers to the same distance between the memory locations in the cache read request, that is, the memory address in the read request is incremented sequentially by the same distance of bytes or memory location units. For example, if three (3) consecutive cache read requests 208R would have memory addresses that increment by consecutive distances of four (4) bytes or units of storage locations (e.g., memory addresses 0x10000, 0x10004, 0x10008), then such three (3) consecutive cache read requests 208R would have a stride pattern with a stride of four (4). If the prefetcher circuit 202 observes the stride pattern in a specified number of cache read requests 208R, then the prefetcher circuit 202 deems the stride pattern to be present in such cache read requests 208R. In response to detecting the stride pattern in the cache read requests 208R to the lower level cache memory 112, the prefetcher circuit 202 issues a prefetch request 220 to prefetch data 218 of one or more next storage locations of the detected stride from a higher level memory (e.g., the higher level cache memory 110) in the memory system 200. For example, if three (3) consecutive cache read requests 208R have memory addresses 0x10000, 0x10004, 0x10008 (thus, observed to have a stride of four (4)), then in this example, the prefetcher circuit 202 may be configured to prefetch data 218 at memory address 0x10012. This is because the subsequent cache read request 208R asserted on the access bus 204 and provided to the lower level cache memory 112 will most likely be the next memory address of 0x10012.

[0031] Continue to refer Figure 2, as a cache write request 208W to the lower level cache memory 112, the prefetcher circuit 202 loads the prefetch data 218 into the lower level cache memory 112. In this way, the data for the next memory address in the detected stride is loaded into the lower level cache memory 112 and is available in the lower level cache memory 112. This can reduce the chance of a cache miss if the next(s) cache read request 208R has a next memory address that is part of the detected stride of the previous cache read request 208R. Even if the lower level cache memory 112 is full, so that cache entries in the lower level cache memory 112 must be evicted to make room for the prefetch data 218, the prefetch data 218 is more useful than the evicted data. Therefore, the efficiency can be increased. Figure 2 The cache hit rate of the low-level cache memory 112 in the memory 112 is increased to form a higher performance cache memory. In one example, if the prefetched data 218 written to the low-level cache memory 112 already exists in the low-level cache memory 112, the low-level cache memory 112 can be configured to detect this situation and overwrite the existing data or ignore the write request for the prefetched data 218. In addition, the prefetcher circuit 202 may be more likely to detect the stride pattern more accurately at runtime rather than in instructions at compile time, for example. Therefore, the prefetcher circuit 202 is suitable for detecting the stride pattern at runtime to issue data prefetch requests, which may result in an increase in the cache hit rate.

[0032] Figure 3 It is shown Figure 2 Flowchart of an exemplary process 300 of the prefetcher circuit 200 in FIG. 1 that observes a cache read request 208R to a cache memory to detect a stride and prefetches data at the next storage location of the detected stride from a higher level memory into the cache memory. Figure 2 The example memory system 200 in FIG. Figure 3 In this aspect, the process 300 involves: the prefetcher circuit 200 observes a plurality of first cache read requests 208R including a plurality of memory addresses on an access bus 204 coupled to a cache access bus 206, wherein the cache access bus 206 is coupled to a lower level cache 112 ( Figure 3 302 in the process 300). The process 300 also involves: detecting a first stride ( Figure 3 Then, in response to detecting a first stride ( Figure 3The process 300 involves the prefetcher circuit 202 issuing one or more prefetch requests 220 to prefetch one or more first prefetch data 218 (i) to be written to the cache memory 112 from the second high level cache memory 110 in the memory system 200 at one or more next memory addresses after the last memory address in the plurality of memory addresses with the detected first stride. Figure 3 Then, the prefetcher circuit 202 or other circuit may write the one or more first prefetch data 218 prefetched in the memory system 200 into the cache memory 112 ( Figure 3 310 in FIG.

[0033] Figure 2 The prefetcher circuit 202 in the memory system 200 may be configured to continue to issue prefetch requests 220 in response to an observed cache read request 208R, wherein the cache read request 208R exhibits the same detected stride as a previous cache read request 208R. In this case, the prefetcher circuit 202 issues another prefetch request 220 to prefetch the second data (set) to be written to the cache memory 112 at the second next memory address after the second memory address of the observed cache read request 208R with the detected first stride. After having detected the stride pattern in the previous cache read request 208R, if it is determined that the other cache read request 208R does not have the stride detected in the previous cache read request 208R, the prefetcher circuit 202 may be configured to not issue a prefetch request 220 for the other cache read request 208R. The prefetcher circuit 202 may be configured not to prefetch data until a new stride pattern is detected in a further cache read request 208R. If the prefetcher circuit 202 detects a new stride again in a new cache read request 208R, the prefetcher circuit 202 will again issue a prefetch request 220 to prefetch data from one or more next storage locations of the newly detected stride, and load the prefetched data into the cache memory 112 as a write request to the cache memory 112. In an exemplary aspect, this process will continue in the following aspects: the prefetcher circuit 202 prefetches data for the detected stride in the observed cache read request 208R, and then aborts the prefetch when the detected stride is no longer observed, and returns to prefetching the newly detected stride, and so on.

[0034] As described above, the prefetcher circuit 202 is configured to observe cache read requests to the cache memory and determine whether the memory address of the cache read request exhibits a stride pattern. Therefore, the prefetcher circuit 202 can be configured to track incoming cache read requests (including their memory addresses) so that the memory address of the next incoming cache read request can be compared to determine whether a stride pattern exists. In response, the prefetcher circuit 202 can prefetch data at the memory location of the next stride based on the stride detected in the cache read request.

[0035] In this regard, Figure 4 is a block diagram of an exemplary prefetch buffer circuit 400 that may be provided in the prefetcher circuit 202 (or otherwise accessible by the prefetcher circuit) to track observed cache read requests, e.g. Figure 2 208R in the cache read request 208R. The prefetch buffer circuit 400 has a plurality of prefetch buffer entries 402(0)-402(X), and the prefetcher circuit 202 can access each of the plurality of prefetch buffer entries 402(0)-402(X) to store and access certain information about the observed cache read request. The prefetcher circuit 202 can use the prefetch buffer circuit 400 to detect a stride pattern in the observed cache read request. For example, a stride threshold of at least three (3) cache read requests may be required so that each memory address is a distance apart from each other at the same stride, thereby declaring that a stride pattern exists in such cache read requests. Therefore, as discussed in more detail below, in this example, the prefetch buffer circuit 400 can be used to store information about these three (3) observed cache read requests 208R so that their memory addresses can be compared with each other to determine whether a stride pattern exists between these cache read requests.

[0036] In this regard, if Figure 4As shown in , each prefetch buffer entry 402(1)-402(X) in the prefetch buffer circuit 400 includes a current address field 404, which is configured to store the current memory address of the observed cache read request. As discussed in more detail below, when the current cache read request observed by the prefetcher circuit 202 does not exhibit a striding pattern with the previous cache read request stored in the prefetch buffer entry 402(1)-402(X), the prefetcher circuit 202 can allocate a new, available prefetch buffer entry 402(1)-402(X) for the current cache read request 208R in the prefetch buffer circuit 400. The prefetcher circuit 202 stores the memory address of the current cache read request in the current address field 404 of the allocated prefetch buffer entry 402(1)-402(X). Also in this example, each prefetch buffer entry 402(1)-402(X) in the prefetch buffer circuit 400 also includes a valid field 406. The valid field 406 is configured to store a valid indicator (e.g., a '1' bit is valid, a '0' bit is invalid) to indicate whether the corresponding prefetch buffer entry 402(1)-402(X) is valid. When the prefetcher circuit 202 is reset, the valid field 406 of all prefetch buffer entries 402(1)-402(X) may be initially set to an invalid state (e.g., a '0' bit) to indicate invalidity, and thus the prefetcher circuit 202 will not consider the information in such invalid prefetch buffer entries 402(1)-402(X) to be available for searching the prefetch buffer circuit 400. If the prefetch buffer entry 402(1)-402(X) is allocated for the current cache read request, the valid field 406 of the allocated prefetch buffer entry 402(1)-402(X) may be set to a valid state (e.g., a '1' bit) to indicate a valid state, which means that the prefetch buffer entry 402(1)-402(X) contains information about the observed cache read request. The valid fields 406 of the prefetch buffer entries 402(1)-402(X) may be set to an invalid state as a way to deallocate or erase the prefetch buffer entries 402(1)-402(X) and make them available for storing information about future observed cache read requests. For example, the prefetcher circuit 202 may be configured to set all valid fields 406 of the prefetch buffer entries 402(1)-402(X) to an invalid state to clear the prefetch buffer circuit 400 after a certain time or a certain number of observed cache read requests 208R exceed a reset threshold. This may eliminate any skew that may have built up over time in the information stored in the prefetch buffer circuit 400.

[0037] Continue to refer Figure 4In this example, each prefetch buffer entry 402(1)-402(X) in the prefetch buffer circuit 400 further includes a stride field 408. The stride field 408 is configured to indicate a detected stride of a cache read request observed by the prefetch circuit 202 and stored in the prefetch buffer entry 402(1)-402(X). For example, as discussed below, when the current cache read request observed by the prefetch circuit 202 and stored in the newly allocated prefetch buffer entry 402(1)-402(X) does not exhibit a stride pattern, the stride field 408 for the allocated prefetch buffer entry 402(1)-402(X) may be set to zero (0). This means that the corresponding cache read request has not been detected as part of the cache read stride pattern. When the prefetcher circuit 202 observes the memory address of the next second cache read request, the prefetcher circuit 202 may search the prefetch buffer entries 402(1)-402(X) in the prefetch buffer circuit 400 to determine whether a masked version of the memory address of the second cache read request (e.g., with the least significant bits (LSBs) masked) corresponds to the current address field 404 stored in the prefetch buffer entry 402(1)-402(X). For example, if the second cache read request 208R is part of a series of cache read requests that exhibit a strided pattern, the masked memory address of the second cache read request 208R will match the current address stored in the current address field 404 of the prefetch buffer entry 402(1)-402(X). Therefore, in this case, rather than assigning a separate prefetch buffer entry 402(1)-402(X) to the second cache read request, the matching prefetch buffer entry 402(1)-402(X) is updated. The stride field 408 of the matching prefetch buffer entries 402(1)-402(X) is set to the memory address of the second cache read request minus the memory address previously stored in the current address field 404 as the stride value of the second cache read request. The current address field 404 of the matching prefetch buffer entries 402(1)-402(X) is updated to the memory address of the second cache read request. The stride valid field 410 in the matching prefetch buffer entries 402(1)-402(X) remains in an invalid state (e.g., a '0' bit) because, in this example, three (3) consecutive cache read requests with the same stride need to be observed before declaring the presence of a stride pattern.

[0038] If the memory address of the next third cache read request is observed to match the memory address stored in the current address field 404 of the matching prefetch buffer entry 402(1)-402(X) described in the previous paragraph, the prefetcher circuit 202 determines whether the memory address of the third cache read request minus the memory address stored in the matching prefetch buffer entry 402(1)-402(X) also matches the stride value stored in the stride field 408. If so, this means that the third cache read request exhibits a stride pattern with the first two cache read requests. In this case, the stride valid field 410 in the matching prefetch buffer entry 402(1)-402(X) is set to a valid state (e.g., a '1' bit), which means that the stride of the matching prefetch buffer entry 402(1)-402(X) has been detected. The current address field 404 of the matching prefetch buffer entry 402(1)-402(X) is updated to the memory address of the third cache read request 208R. The stride fields 408 of the matching prefetch buffer entries 402(1)-402(X) remain with the same stride value. Also in this case, the prefetcher circuit 202 is configured to prefetch data from at least one next storage location, which is the memory address of the third cache request updated in the current address field 404 of the matching prefetch buffer entry 402(1)-402(X), plus the stride value stored in the stride field 408 of the matching prefetch buffer entry 402(1)-402(X). The prefetched data may be Figure 2 The pre-fetched data 218 in is then written to the lower level cache memory 112 to be available in the lower level cache memory.

[0039] In this manner, the prefetcher circuit 202 allocates and searches the prefetch buffer entries 402(1)-402(X) in the prefetch buffer circuit 400 to track observed cache read requests. The prefetcher circuit 202 searches the prefetch buffer entries 402(1)-402(X) in the prefetch buffer circuit 400 to determine whether a subsequently observed cache read request 208R exhibits a stride pattern with a previously observed cache read request 208R stored in the prefetch buffer circuit 400.

[0040] In this example, Figure 4The prefetch buffer circuit 400 in the example also has other optional entries in the prefetch buffer entries 402(1)-402(X). For example, each prefetch buffer entry 402(1)-402(X) in the present example also includes a least recently used (LRU) information field 412. The LRU information field 412 can be used to store information about the most recent use of the corresponding prefetch buffer entry 402(1)-402(X). This allows the prefetcher circuit 202 to search the prefetch buffer entry 402(1)-402(X) in the LRU information field 412 to determine the least recently used prefetch buffer entry 402(1)-402(X) when allocating a new prefetch buffer entry for a newly observed cache read request 208R. When allocating a new prefetch buffer entry 402(1)-402(X) for a newly observed cache read request, the prefetcher circuit 202 may evict the least recently used prefetch buffer entry 402(2)-402 (as indicated by the LRU information stored in its LRU information field 412). As discussed above, a new prefetch buffer entry 402(1)-402(X) is allocated for a cache read request for which a matching memory address is not yet stored in the current address field 404 of a valid prefetch buffer entry 402(1)-402(X) (i.e., its valid field 406 is set to a valid state). The LRU information field 412 of each of the prefetch buffer entries 402(1)-402(X) may act as an LRU stack such that the LRU information field 410 of the most recently used prefetch buffer entry is updated each time it matches a newly observed cache read request. Therefore, the least recently used prefetch buffer entry 402 ( 1 )- 402 (X) will have LRU information stored in its LRU information field 412 that has not been updated for the longest time compared to other valid prefetch buffer entries 402 ( 1 )- 402 (X).

[0041] In addition, if Figure 4 As shown, in this example, each pre-fetch buffer entry 402(1)-402(X) in the pre-fetch buffer circuit 400 also includes an optional thread identification (ID) field 414. As previously discussed, processor-based systems (e.g., Figure 1Different threads in the GPU system 100 in the prefetch buffer circuit 400 may issue cache read requests 208R to the cache memory. When creating or updating a prefetch buffer entry 402(1)-402(X) in the prefetch buffer circuit 400, a thread ID field 414 may be used to store the thread ID of the initiator of the cache read request observed by the prefetcher circuit 202. Thus, when the prefetcher circuit 202 searches the prefetch buffer entries 402(1)-402(X) in the prefetch buffer circuit 400 to determine whether the currently observed cache read request 208R matches an existing prefetch buffer entry 402(1)-402(X), the thread ID of the currently observed cache read request may also be compared with the thread ID stored in the thread ID field 414 of the matching prefetch buffer entry 402(1)-402(X). In this example, it is detected whether the currently observed cache read request has a stride as part of the stride pattern only when the thread IDs of the newly observed cache read request 208R and the previous cache read request 208R stored in the matching prefetch buffer entries 402(1)-402(X) are considered to be prefetch buffer entries 402(1)-402(X) that match the currently observed cache read request.

[0042] In this example, if Figure 4 As shown in FIG. 4 , the prefetch buffer circuit 400 further has a plurality of global entries 416, which are used to store information for the prefetch circuit 202 to perform adaptive prefetching. Adaptive prefetching is that the prefetch circuit 202 adaptively enables and disables prefetching based on the effectiveness of stride detection observed at runtime. Figure 7 and Figure 8, describing adaptive prefetching in more detail. The global entry 416 includes an observation interval field 418, which is configured to store an observation interval of multiple cache read requests so that information on the observation interval can be analyzed as part of an adaptive prefetch operation. The observation interval field 418 can be reset when the number of observed cache read requests 208R is equal to the observation interval. The observation interval stored in the observation interval field 418 can be a programmed value. The global entry 416 also includes a propagation threshold field 420 configured to store a propagation threshold. The propagation threshold stored in the propagation threshold field 420 can be a programmed value. The global entry 416 also includes a propagation total number field 422, which is configured to store the total number of observed cache read requests 208R that caused the prefetcher circuit 202 to propagate (i.e., issue) a data prefetch request at the next storage location of the detected stride. The total number of propagations stored by the prefetcher circuit 202 in the propagation total number field 422 may be compared to the propagation threshold stored in the propagation threshold field 420 to determine the consistency (or inconsistency) of the stride pattern in the observed cache read request. If the total number of prefetch propagations stored by the prefetcher circuit 202 in the propagation total number field 422 does not exceed the propagation threshold stored in the propagation threshold field 420, the prefetcher circuit 202 may adaptively disable prefetching even if the observed cache read request 208R is detected as part of the stride pattern. If the total number of prefetch propagations stored by the prefetcher circuit 202 in the propagation total number field 422 thereafter exceeds the propagation threshold stored in the propagation threshold field 420, the prefetcher circuit 202 may adaptively enable prefetching again for the observed cache read request 208R detected as part of the stride pattern.

[0043] Figure 5 is a flowchart illustrating an exemplary prefetch process 500, Figure 2 The prefetcher circuit 202 may use Figure 4 The process 500 is performed by the pre-fetch buffer circuit 400 in the cache memory to detect a stride in a cache read request and pre-fetch the data of the next storage location of the detected stride into the cache memory (e.g., Figure 2 In this aspect, as Figure 5 As shown in FIG. 2 , in response to the prefetcher circuit 202 observing the cache read request 208R, the prefetcher circuit 202 Figure 4 The pre-fetch buffer circuit 400 in the pre-fetch buffer circuit 400 searches for pre-fetch buffer entries 402(1)-402(X)( Figure 5The prefetcher circuit 202 determines whether the memory address of the cache read request 208R matches the memory address stored in the current address field 404 of the prefetch buffer entry 402(1)-402(X) ( Figure 5 If there is no match, the prefetcher circuit 202 allocates a new, available prefetch buffer entry 402(1)-402(X) in the prefetch buffer circuit 400 for the current cache read request 208R, as previously discussed above ( Figure 5 402(X). The prefetcher circuit 202 stores the memory address of the current cache read request 208R in the current address field 404 of the allocated prefetch buffer entry 402(1)-402(X). The prefetcher circuit 202 stores the valid indicator in the valid field 406 of the allocated prefetch buffer entry 402(1)-402(X). The prefetcher circuit 202 sets the stride value in the stride field 408 to zero (0). The prefetcher circuit 202 updates the LRU information in the LRU information field 412 of the allocated prefetch buffer entry 402(1)-402(X) to indicate that the allocated prefetch buffer entry 402-402(X) is the most recently used prefetch buffer entry. Since the prefetch buffer circuit 400 has been previously initialized, the stride valid field 410 has already stored the stride invalid state in the allocated prefetch buffer entry 402(1)-402(X). The thread ID of the initiator of the current cache read request 208R is stored in the thread ID field 414 of the allocated prefetch buffer entry 402(1)-402(X) if that feature exists.

[0044] Continue to refer Figure 5 , if the memory address of the next second current cache request 208R observed by the prefetch circuit 202 matches the memory address stored in the current address field 404 of the prefetch buffer entry 402(1)-402(X) ( Figure 5 ), which means that the next current cache request 208 may be part of a stride pattern in a cache read request. However, this cannot be determined at this time. In this regard, the prefetcher circuit 202 checks whether the stride value in the stride field 408 in the matching prefetch buffer entry 402(1)-402(X) is zero (0) ( Figure 5If so, this means that a stride involving the second current cache request 208R has not been detected. In this case, the prefetcher circuit 202 sets the stride value in the stride field 408 of the matching prefetch buffer entry 402(1)-402(X) to be the memory address of the second current cache request 208R minus the memory address stored in the current address field 404 of the matching prefetch buffer entry 402(1)-402(X) ( Figure 5 Thus, if the next third current cache request 208R observed by the prefetch circuit 202 is found to match a prefetch buffer entry 402(1)-402(X) ( Figure 5 ), and the stride value in the stride field 408 of the matching prefetch buffer entry 402(1)-402(X) is not equal to zero (0) ( Figure 5 402(X), which means that the third cache read request 208R may be part of a series of cache read requests with a detected stride. In this case, the prefetcher circuit 202 determines whether the memory address of the third cache read request 208R minus the memory address in the current address field 404 of the matching prefetch buffer entry 402(1)-402(X) is equal to the stride value ( Figure 5 402(1)-402(X)). If so, this means that the third cache read request 208R is part of a series of cache read requests having a detected stride. Therefore, in this example, the stride threshold for detecting a stride pattern is three (3), which means that three (3) cache read requests 208R exhibit a stride pattern. In this case, the prefetcher circuit 202 adds the memory address of the third cache read request 208R to the stride value in the stride field 408 in the current address field 404 of the matching prefetch buffer entry 402(1)-402(X). The prefetcher circuit 202 sets the stride valid field 410 to valid ( Figure 5 The prefetcher circuit 202 also issues a prefetch request 220 to prefetch data at the next storage location plus the stride value ( ) stored in the stride field 408 in the matching prefetch buffer entry 402 ( 1 )-402 (X). Figure 5 514 in FIG.

[0045] Note that in another example, if the total number of prefetch requests that are continuously propagated (issued) by the prefetcher circuit 202 exceeds the propagation count threshold, the prefetcher circuit 202 may be configured to fetch more data further ahead in response to the detected stride in the observed cache read request 208R. For example, if the propagation count threshold is equal to eight (8), and the prefetcher circuit 202 observes eight (8) consecutive cache read requests 208R (which are detected as having strides to prefetch data), the prefetcher circuit 202 may be configured to prefetch multiple storage locations (e.g., storage location + stride, storage location + 2x stride) from the stride to be written to the cache memory. This is because if the number of prefetch requests that are continuously propagated (issued) by the prefetcher circuit 202 exceeds the number of prefetch requests that are continuously propagated (issued) by the prefetcher circuit 202, this means that there is a higher level of confidence that the stride pattern will continue to exist for subsequent cache read requests 208R. Thus, prefetch efficiency may be achieved by propagating multiple prefetch requests at the current memory address of the current cache read request 208R plus its stride instead of just a single prefetch request.

[0046] Figure 6 is an example of a cache read request 600 observed by the prefetcher circuit 202, and Figure 4 The corresponding information corresponding to the observed cache read request 208R stored in the pre-fetch buffer circuit 400 in FIG. 4 is shown to show a further example. Figure 6 As shown in FIG. 6 , in an initial state 602, the prefetch buffer entries 402(1)-402(X) are initialized such that their current address fields 404 are zero (0), the valid field 406 is set to invalid (zero (0)), the stride field 408 is set to (0), and the stride valid field 410 is set to invalid (zero (0)). Then, in response to an observed first cache read request 208R(1) of memory address 'A' that does not match the prefetch buffer entry 402(1)-402(X), the prefetcher circuit 202 stores, for the allocated prefetch buffer entry 402 of the first cache read request 208R(1), a valid indicator (1) in the valid field 406, and the memory address of the first cache read request 208R(1) in the current address field 404. Because the stride value in the stride field 408 is zero (0), no data prefetch is issued.

[0047] Then, in response to observing that the second cache read request 208R(2) with a memory address of 'A+B' matches the prefetch buffer entry 402 allocated for the first cache read request 208R(1), the prefetcher circuit 202 updates the allocated prefetch buffer entry 402: the stride value 'B' in the stride field 408, and the memory address of the second cache read request 208R(2) in the current address field 404 is 'A+B'. Because the stride value in the stride field 408 is still zero (0), no data prefetch is issued. Then, in response to observing that the third cache read request 208R(3) with a memory address of 'A+2B' matches the prefetch buffer entry 402 allocated for the first cache read request 208R(1), the prefetcher circuit 202 updates the allocated prefetch buffer entry 402: the stride valid indicator ('1') in the stride valid field 410, and the memory address in the current address field 404 is 'A+4B'. The reason why the memory address in the current address field 404 is not stored as the memory address of the third cache read request 208R(3) which is A+2B is because, in this example, the prefetcher circuit 202 issued two (2) prefetch requests for storage locations A+2B and A+4B. The prefetcher circuit 202 then increments the total number of propagations issued.

[0048] Then, in response to observing that the fourth cache read request 208R(4) with a memory address of 'A+4B' matches the prefetch buffer entry 402 allocated for the first cache read request 208R(1), the prefetcher circuit 202 updates the allocated prefetch buffer entry 402 to the memory address in the current address field 404 such as 'A+6B'. The reason why the memory address in the current address field 404 is not stored as the fourth cache read request 208R(4) such as A+4B is because, in this example, the prefetcher circuit 202 issues two (2) prefetch requests for storage locations A+5B and A+6B in response to the stride mode of 'B'. The prefetcher circuit 202 then increments the total number of propagations re-issued.

[0049] Figure 7 is a flow chart depicting an exemplary adaptive prefetching process 700, which may be performed by Figure 2 The prefetcher circuit 202 in the embodiment of the present invention performs an adaptive prefetch process 700 to adaptively prefetch data into the cache memory based on the detected stride in the cache read request. As discussed above and discussed in more detail below, the prefetcher circuit 202 can adaptively enable and disable prefetching based on whether the total number of prefetch propagations in a given observation interval exceeds a propagation threshold. As discussed above, Figure 4The prefetch buffer circuit 400 in has a propagation total number field 422 that is configured to store the total number of observed cache read requests 208R that caused the prefetch circuit 202 to propagate (i.e., issue) a data prefetch request at the next storage location of the detected stride. The propagation total number stored by the prefetch circuit 202 in the propagation total number field 422 can be compared to the propagation threshold stored in the propagation threshold field 420 to determine the consistency (or inconsistency) of the stride pattern in the observed cache read requests. If the total number of prefetch propagations stored by the prefetch circuit 202 in the propagation total number field 422 does not exceed the propagation threshold stored in the propagation threshold field 420 during the observation interval, the prefetch circuit 202 can adaptively disable prefetching even if the observed cache read request 208R is detected as part of the stride pattern. If the total number of prefetch propagations stored by the prefetcher circuit 202 in the propagation total field 422 thereafter exceeds the propagation threshold stored in the propagation threshold field 420, the prefetcher circuit 202 may adaptively enable prefetching again for observed cache read requests 208R detected as part of a stride pattern.

[0050] In this regard, Figure 7 As shown in , if the number of observed cache read requests 208R is less than (i.e., within) the Figure 4 The current observation interval ( Figure 7 ), the prefetch mode 706 is not changed. If the prefetch mode 706 is set to allow prefetching, the prefetch mode 704 is set to the prefetch enabled state. In this example, if the prefetch mode 706 is set to the prefetch enabled state, the prefetcher circuit 202 is configured to only prefetch data (e.g., as described above). For example, the prefetch mode 706 may be a memory circuit in the prefetch buffer circuit 400. Once the number of observed cache read requests 208R is not less than (i.e., is outside of) the current observation interval ( Figure 7 ), the prefetcher circuit 202 is configured to determine whether the prefetch mode 706 should be adaptively changed from its current setting. In this case, the prefetcher circuit 202 determines that the observed propagation total of the cache read request 208R in the propagation total field 422 is not less than (i.e., is outside) the current observation interval ( ) stored in the observation interval field 418 in the prefetch buffer circuit 400. Figure 5 Then, the prefetcher circuit 202 determines whether the total number of propagations is less than Figure 4 The propagation threshold value stored in the propagation threshold field 420 in Figure 7If so, the prefetcher circuit 202 sets the prefetch mode 706 to the prefetch enabled state again ( Figure 7 The prefetcher circuit 202 sets the propagation total in the propagation total field 422 to zero (0) and begins counting propagations again ( Figure 7 710 in FIG.

[0051] If the prefetch circuit 202 determines whether the total number of propagations is not less than Figure 4 The propagation threshold value stored in the propagation threshold field 420 in Figure 7 ), the prefetcher circuit 202 sets the prefetch mode 706 to the prefetch disabled state ( Figure 7 The prefetcher circuit 202 sets the propagation total in the propagation total field 422 to zero (0) and begins counting propagations again ( Figure 7 In the event that the prefetch mode 706 is in the prefetch disabled state, the prefetcher circuit 202 will not issue the prefetch request 220, but will continue to track the observed cache read request 208R in the prefetch buffer circuit 400.

[0052] Figure 8 is an example of a cache read request 800 observed by the prefetcher circuit 202, and Figure 4 The pre-fetch buffer circuit 400 stores the Figure 6 , corresponding to the same observed cache read requests 208R(1)-208R(4) shown in FIG. Figure 8 In the example of , the prefetch mode is set to the prefetch disabled state. Therefore, although the prefetcher circuit 202 will continue to search and update the prefetch buffer entries 402 allocated for the cache read requests 208R(1)-208R(4) in the prefetch buffer circuit 400, as previously described in Figure 6 As described in , the prefetcher circuit 200 will not prefetch data at the next storage location in response to the third cache read request 208R(3) and the fourth cache read request 208R(4), such as Figure 6 The previous discussion of the observed cache read request 600 and the above discussion of Figure 6 The details described also apply to Figure 8 , instead of setting the prefetch mode to the prefetch disabled state when a stride pattern is detected in the cache read requests 208R(1)-208R(4).

[0053] A stride-based prefetcher circuit for prefetching data for a next stride of a cache read request into a cache memory based on a stride pattern identified in the cache read request, including but not limited to Figure 2and Figure 3 The stride-based prefetcher circuit 202 in FIG. 1 and may be configured according to, but not limited to, Figure 3 and Figure 5-8 The stride-based prefetcher circuit may be provided in or integrated into any processor-based device according to any aspect disclosed herein. Examples include, but are not limited to, a set-top box, an entertainment unit, a navigation device, a communication device, a fixed location data unit, a mobile location data unit, a global positioning system (GPS) device, a mobile phone, a cellular phone, a smart phone, a session initiation protocol (SIP) phone, a tablet, a tablet phone, a server, a computer, a portable computer, a mobile computing device, a laptop computer, a wearable computing device (e.g., a smart watch, a health or fitness tracker, glasses, etc.), a desktop computer, a personal digital assistant (PDA), a monitor, a computer monitor, a television, a tuner, a radio unit, a satellite radio unit, a music player, a digital music player, a portable music player, a digital video player, a video player, a digital video disc (DVD) player, a portable digital video player, an automobile, a vehicle component, an avionics system, a drone, and a multirotor aircraft.

[0054] A stride-based prefetcher circuit may also be provided in other types of processor-based systems for prefetching data for the next stride of a cache read request into a cache memory based on a stride pattern identified in the cache read request. Fig. 9is a block diagram of an exemplary processor-based system 900 that includes a plurality of central processing units (CPUs) (multi-CPU) processors 902 that include a plurality of CPUs 904(0)-904(N) and a hierarchical memory system. As part of the hierarchical memory system, for example, CPU 904(0) includes a dedicated local cache memory 906, which may be a level 2 (L2) cache memory. CPUs 904(1), 904(2) and CPUs 904(N-1), 904(N) are configured to interface with respective local shared cache memories 906S(0)-906S(X), which may also be, for example, L2 cache memories. If a data read request requested by a CPU 904(0)-904(N) results in a cache miss to the respective cache memory 906, 906S(0)-906S(X), the read request may be transferred to the next level cache memory, which in this example is the shared cache memory 908. As an example, the shared cache memory 908 may be a level three (L3) cache memory. The cache memories 906, the local shared cache memories 906S(0)-906S(X), and the shared cache memory 908 are part of a cache memory system 910. An internal interconnect bus 912 (which may be a coherent bus) is provided to allow each of the CPUs 904(0)-904(N) to access the shared cache memories 906S(0)-906S(X) (if shared to the CPUs 904(0)-904(N)), the shared cache memories 908, and other shared resources coupled to the interconnect bus 912. The processor-based system 900 may include a prefetcher circuit 913 configured to observe cache read requests in the cache memory system 910 and detect a stride pattern in the cache read request. In response to a stride detected in the cache read request, the prefetcher circuit 913 may prefetch data of a next memory address of a next stride of the detected stride into the cache memory system 910. The prefetcher circuit 913 may be Figure 2 and Figure 3 The prefetcher circuit 202 in the embodiment of the present invention may be configured according to but not limited to Figure 3 and Figure 5-8 Any of the exemplary processes 300, 500, 600, 700, 800 in and operating according to any aspect disclosed herein.

[0055] Continue to refer Fig. 9, the processor-based system 900 in this example also includes a sniff controller 914 that is also coupled to the interconnect bus 912. The sniff controller 914 is a circuit that monitors or sniffs cache bus transactions on the interconnect bus 912 to maintain cache coherence between cache memories 906, 906S(0)-906S(X), 908 in the cache memory system 910. Other shared resources that the CPUs 904(0)-904(N) can access via the interconnect bus 912 may include input / output (I / O) devices 916 and system memory 918 (e.g., dynamic random access memory (DRAM)). If a read request issued by a CPU 904(0)-904(N) results in a cache miss in each level of cache memory 906, 906S(0)-906S(X), 908 accessible to CPU 904(0)-904(N), the read request is serviced by the system memory 918 and the data associated with the read request is installed in the cache memory 906, 906S(0)-906S(X), 908 associated with the requesting CPU 904(0)-904(N).

[0056] Fig.10 Another example of a processor-based system 1000 is illustrated, where the system 1000 may include a prefetcher circuit 1002 configured to observe cache read requests of a cache memory 1004 and detect a stride pattern in the cache read request. And in response to the stride detected in the cache read request, the prefetcher circuit 1002 may prefetch data for a next memory address of a next stride of the cache request that is to be written to the cache memory 1004. In this example, the processor-based system 1000 may be formed as an IC 1006 and a system on chip (SoC) 1008. The processor-based system 1000 includes a central processing unit (CPU) 1010, which includes one or more processors 1012, which may also be referred to as CPU cores or processor cores. The CPU 1010 may have a cache memory 1004 coupled to the CPU 1010 for fast access to temporarily stored data. The CPU 1010 is coupled to the system bus 1014 and can interconnect the master devices and slave devices included in the processor-based system 1000. As is well known, the CPU 1010 communicates with these other devices by exchanging address, control, and data information on the system bus 1014. For example, the CPU 1010 can transmit a bus transaction request to the memory controller 1016 as an example of a slave device. Although Fig.10 Although not shown, multiple system buses 1014 may be provided, each of which constitutes a different structure.

[0057] Other master devices and slave devices may be connected to the system bus 1014. Fig.10 As shown in , for example, these devices may include a memory system 1020 (which includes a memory controller 1016 and a memory array 1018), one or more input devices 1022, one or more output devices 1024, one or more network interface devices 1026, and one or more display controllers 1028. The input device 1022 may include any type of input device, including but not limited to input keys, switches, voice processors, etc. The output device 1024 may include any type of output device, including but not limited to audio, video, other visual indicators, etc. The network interface device 1026 can be any device configured to allow data to be exchanged with the network 1030. The network 1030 can be any type of network, including but not limited to a wired or wireless network, a private or public network, a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), Bluetooth TM Networks and the Internet. The network interface device 1026 may be configured to support any type of communication protocol desired.

[0058] The CPU 1010 may also be configured to access a display controller 1028 through the system bus 1014 to control information sent to one or more displays 1032. The display controller 1028 sends information to be displayed to the display 1032 via one or more video processors 1034, which processes the information to be displayed into a format suitable for the display 1032. The display 1032 may include any type of display, including but not limited to a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, a light emitting diode (LED) display, and the like.

[0059] It will also be appreciated by those skilled in the art that the various exemplary logic blocks, modules, circuits, and algorithms described in conjunction with the disclosed aspects herein can all be implemented as electronic hardware, instructions stored in a memory or another computer-readable medium, and executed by a processor or other processing device or a combination of the two. The memory disclosed herein can be a memory of any type and size, and can be configured to store any type of information desired. In order to clearly depict this interchangeability, the above various exemplary components, blocks, modules, circuits, and steps are generally described around their functions. As for how this function is implemented, it depends on the specific application, design selection, and / or the design constraints imposed on the entire system. A skilled person can implement the described functions in various ways for each specific application, but these implementation decisions should not be interpreted as departing from the scope of the present disclosure.

[0060] A processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components or any combination thereof designed to perform the functions described herein may be used to implement or execute the various illustrative logic blocks, modules and circuits described in conjunction with the aspects disclosed herein. The processor may be a microprocessor, or the processor may be any conventional processor, controller, microcontroller or state machine. The processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, a combination of one or more microprocessors and a DSP core, or any other such structure).

[0061] The aspects disclosed herein may be embodied as hardware, instructions stored in hardware, and the aspects disclosed herein may be located in, for example, random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of computer-readable media known in the art. An exemplary storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium may also be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The ASIC may be located in a remote station. Alternatively, the processor and the storage medium may also be present in a remote station, a base station, or a server as discrete components.

[0062] It should also be noted that the operational steps described in any exemplary aspect herein are intended to provide examples and discussions. The described operations can be performed in many different sequences different from the sequence shown. In addition, the operations described in a single operational step can actually be performed with multiple different steps. In addition, one or more operational steps discussed in the exemplary aspects can be combined. It should be understood that the operational steps shown in the flow chart can be modified in many different ways, which is obvious to those of ordinary skill in the art. It should also be understood by those of ordinary skill in the art that information and signals can be represented using any one of a variety of different techniques and skills. For example, the data, instructions, commands, information, signals, bits, symbols, and chips mentioned throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or light particles, or any combination thereof.

[0063] In order to enable any person of ordinary skill in the art to implement or use the present disclosure, the above description is made around the present disclosure. It is obvious to those of ordinary skill in the art that various modifications to the disclosure are made, and the general principles defined herein may be applied to other variations. Therefore, the present disclosure is not intended to be limited to the examples and designs described herein, but is consistent with the widest range of principles and novel features disclosed herein.

[0064] Examples of implementation are described in the following numbered clauses: 1. A prefetcher circuit, comprising: an access bus coupled to a cache access bus coupled to a cache memory in a memory system; The prefetcher circuit is configured to: observing a plurality of first cache read requests including a plurality of first memory addresses on the access bus; detecting a first stride in the plurality of first cache read requests; and In response to detecting the first stride in the plurality of first cache read requests: One or more prefetch requests are issued to prefetch one or more first data to be written to the cache memory from a second memory of the memory system at one or more next memory addresses after a last memory address of the plurality of first memory addresses with the detected first stride. 2. The prefetcher circuit of clause 1, further configured to: observing a second cache read request on the access bus including a second memory address; determining whether the second cache read request has the first stride; and In response to determining that the second cache read request has the first stride: A second prefetch request is issued to prefetch one or more second data to be written to the cache memory from the second memory in the memory system at one or more second next memory addresses after the second memory address with the detected first stride. 3. The prefetcher circuit of clause 1, further configured to: observing a second cache read request on the access bus including a second memory address; determining whether the second cache read request has the first stride; and In response to determining that the second cache read request does not have the first stride, no prefetch request is issued to prefetch data based on the second cache read request. 4. The prefetcher circuit of clause 3, further configured to: observing a plurality of third cache read requests including a plurality of third memory addresses on the access bus subsequent to the second cache read request; determining, based on the second cache read request, whether the plurality of third cache read requests have a second stride; and In response to determining that the plurality of third cache read requests have the second stride: A second prefetch request is issued to prefetch one or more second data to be written to the cache memory from the second memory in the memory system at one or more next second memory addresses after the last memory address in the plurality of third memory addresses with the detected second stride. 5. The prefetcher circuit of any of clauses 1-4, configured to detect the first stride in the plurality of first cache read requests by being configured to: determining that a number of the plurality of first cache read requests having the first stride exceeds a stride threshold; and In response to determining that the number of the plurality of first cache read requests having the first stride exceeds the stride threshold, the first stride in the plurality of first cache read requests is detected. 6. The prefetcher circuit of clause 3, further comprising a prefetch buffer circuit, the prefetch buffer circuit comprising a plurality of prefetch buffer entries; In response to determining that the second cache read request does not have the first stride, the prefetcher circuit is further configured to: assign a first prefetch buffer entry of the plurality of prefetch buffer entries in the prefetch buffer circuit to the second cache read request. 7. The prefetcher circuit of clause 6, wherein, in response to determining that the second cache read request does not have the first stride, the prefetcher circuit is further configured to: determining whether to allocate each of the plurality of prefetch buffer entries in the prefetch buffer circuit; and in response to determining that each of the plurality of prefetch buffer entries in the prefetch buffer circuit is allocated: evicting a least recently used prefetch buffer entry from the plurality of prefetch buffer entries; and The evicted prefetch buffer entry is assigned to the second cache read request. 8. A prefetcher circuit as described in clause 6 or 7, wherein, in response to determining that the second cache read request does not have the first stride, the prefetcher circuit is further configured to: associate a thread identification associated with the second cache request to the first prefetch buffer entry. 9. A prefetcher circuit according to any of clauses 6-8, wherein the prefetcher circuit is further configured to: observing, subsequent to the second cache read request, a third cache read request on the access bus including a third memory address; determining whether the prefetch buffer circuit contains the first prefetch buffer entry of the plurality of prefetch buffer entries corresponding to the third cache read request; and In response to determining that the prefetch buffer circuit contains the first prefetch buffer entry corresponding to the third cache read request: determining that the first prefetch buffer entry has an associated stride; and In response to the first prefetch buffer entry not having an associated stride, a second stride is associated with a second prefetch buffer entry that is the second memory address subtracted from the third memory address. 10. The prefetcher circuit of clause 9, wherein the prefetcher circuit is further configured to: In response to determining that the second cache read request does not have the first stride, the prefetcher circuit is further configured to associate a second thread identification associated with the second cache request to the first prefetch buffer entry; and The prefetch buffer circuit includes the first prefetch buffer entry corresponding to the third cache read request among the plurality of prefetch buffer entries by further being configured to determine that a third thread identifier associated with the third cache read request matches a second thread identifier associated with the first prefetch buffer entry. 11. A prefetcher circuit according to clause 9 or 10, wherein the prefetcher circuit is further configured to: observing, subsequent to the second cache read request, a fourth cache read request on the access bus including a fourth memory address; determining that the prefetch buffer circuit contains the first prefetch buffer entry corresponding to the fourth cache read request among the plurality of prefetch buffer entries; and In response to determining that the prefetch buffer circuit contains the first prefetch buffer entry corresponding to the fourth cache read request: determining that the first prefetch buffer entry has an associated stride; and In response to the first prefetch buffer entry having the associated stride: A second prefetch request is issued to prefetch one or more second data to be written to the cache memory at one or more second next memory addresses starting from an address of the fourth memory address added to the second stride. 12. The prefetcher circuit of clause 11, wherein the prefetcher circuit is further configured to: In response to determining that the second cache read request does not have the first stride, the prefetcher circuit is further configured to associate a second thread identification associated with the second cache request to the first prefetch buffer entry; and The prefetch buffer circuit is further configured to determine whether a fourth thread identifier associated with the fourth cache read request matches the second thread identifier associated with the first prefetch buffer entry, thereby determining that the prefetch buffer circuit contains the first prefetch buffer entry corresponding to the fourth cache read request among the plurality of prefetch buffer entries. 13. The prefetcher circuit according to clause 11 or 12, further configured to: tracking a number of cache read requests observed on the access bus; and In response to the number of observed cache read requests being greater than a reset threshold, each prefetch buffer entry of the plurality of prefetch buffer entries in the prefetch buffer circuit is invalidated. 14. A prefetcher circuit according to any one of clauses 1 to 13, configured to: issue multiple prefetch requests to prefetch multiple first data to be written to the cache memory from a second memory of the memory system at multiple next memory addresses after the last memory address in the multiple first memory addresses with the detected first stride. 15. The prefetcher circuit according to any of clauses 1 to 14, further configured to: tracking a total number of one or more prefetch requests issued successively for the plurality of first cache read requests; determining whether the total number of one or more prefetch requests issued consecutively exceeds a propagation count threshold; observing a second cache read request on the access bus including a second memory address; detecting the first stride in the second cache read request; and In response to detecting that the second cache read request has the first stride and determining that the total number of one or more prefetch requests issued consecutively exceeds the propagation count threshold: A second prefetch request is issued to prefetch second data to be written to the cache memory from the second memory of the memory system at a plurality of second next memory addresses after the second memory address. 16. The prefetcher circuit of any of clauses 1-15, further configured to set the prefetch mode to a prefetch enabled state; and The prefetcher circuit is configured to: in response to detecting the first stride in the plurality of first cache read requests and the prefetch mode is the prefetch enabled state: The one or more prefetch requests are issued to prefetch the one or more first data to be written to the cache memory from a second memory in the memory system at the one or more next memory addresses after the last memory address in the plurality of first memory addresses with the detected first stride. 17. The prefetcher circuit of clause 16, further configured to: tracking a total number of the one or more prefetch requests over an observation interval; determining whether a total number of the one or more prefetch requests observed in the observation interval does not exceed a propagation threshold; and In response to a total of the one or more prefetch requests observed in the observation interval not exceeding the propagation threshold, setting the prefetch mode to a prefetch disabled state; and The prefetcher circuit is configured to: in response to detecting the first stride in the plurality of first cache read requests and the prefetch mode is a prefetch disabled state: The one or more prefetch requests are not issued so as not to prefetch the one or more first data to be written to the cache memory from a second memory in the memory system at the one or more next memory addresses after the last memory address in the plurality of first memory addresses with the detected first stride. 18. The prefetcher circuit of clause 17, further configured to: tracking a second total number of the one or more prefetch requests observed on the access bus during the observation interval; determining whether the second total number of the one or more prefetch requests in the observation interval exceeds the propagation threshold; and In response to the second total number of the one or more prefetch requests in the observation interval exceeding the propagation threshold, the prefetch mode is set to a prefetch enabled state. 19. The prefetcher circuit of any of clauses 1-18, wherein the prefetcher circuit is further configured to write one or more data to the cache memory. 20. A prefetcher circuit as claimed in any of clauses 1 to 19, wherein the second memory is a higher level memory than the cache memory. 21. A prefetcher circuit as described in any of clauses 1-20, integrated into a device selected from the group consisting of: a set-top box; an entertainment unit; a navigation device; a communication device; a fixed location data unit; a mobile location data unit; a global positioning system (GPS) device; a mobile phone; a cellular phone; a smartphone; a Session Initiation Protocol (SIP) phone; a tablet computer; a phablet; a server; a computer; a portable computer; a mobile computing device; a wearable computing device; a desktop computer; a personal digital assistant (PDA); a monitor; a computer monitor; a television set; a tuner; a radio unit; a satellite radio unit; a music player; a digital music player; a portable music player; a digital video player; a video player; a digital video disc (DVD) player; a portable digital video player; an automobile; a vehicle component; an avionics system; an unmanned aerial vehicle; and a multirotor aircraft. 22. A method for prefetching data into a cache memory based on a stride pattern detected in a cache read request to the cache memory, comprising: observing a plurality of first cache read requests including a plurality of memory addresses on an access bus coupled to a cache access bus coupled to a cache memory in a memory system; detecting a first stride in the plurality of first cache read requests; and In response to detecting the first stride among the plurality of first cache read requests: One or more prefetch requests are issued to prefetch one or more first data to be written to the cache memory from a second memory of the memory system at one or more next memory addresses after a last memory address of the plurality of memory addresses with the detected first stride. 23. The method according to clause 22, further comprising: observing a second cache read request on the access bus including a second memory address; determining whether the second cache read request has the first stride; and In response to determining that the second cache read request has the first stride: A second prefetch request is issued to prefetch one or more second data to be written to the cache memory from the second memory in the memory system at one or more second next memory addresses after the second memory address with the detected first stride. 24. The method according to clause 22, further comprising: observing a second cache read request on the access bus including a second memory address; determining whether the second cache read request has the first stride; and In response to determining that the second cache read request does not have the first stride, no prefetch request is issued to prefetch data based on the second cache read request. 25. The method according to clause 24, further comprising: observing a plurality of third cache read requests including a plurality of third memory addresses on the access bus subsequent to the second cache read request; determining, based on the second cache read request, whether the plurality of third cache read requests have a second stride; and In response to determining that the plurality of third cache read requests have the second stride: A second prefetch request is issued to prefetch one or more second data to be written to the cache memory from the second memory in the memory system at one or more next second memory addresses after the last memory address in the plurality of third memory addresses with the detected second stride. 26. The method of any of clauses 22-25, wherein detecting the first stride in the plurality of first cache read requests comprises: determining that a number of the plurality of first cache read requests having the first stride exceeds a stride threshold; and In response to determining that the number of the plurality of first cache read requests having the first stride exceeds the stride threshold, the first stride in the plurality of first cache read requests is detected. 27. A method according to any one of clauses 22 to 26, comprising: issuing a plurality of prefetch requests to prefetch a plurality of first data to be written to the cache memory from a second memory of the memory system at a plurality of next memory addresses after the last memory address of the plurality of first memory addresses with a detected first stride. 28. A method according to any one of clauses 22 to 27, further comprising: tracking a total number of one or more prefetch requests issued successively for the plurality of first cache read requests; determining whether the total number of one or more prefetch requests issued consecutively exceeds a propagation count threshold; observing a second cache read request on the access bus including a second memory address; detecting the first stride in the second cache read request; and In response to detecting that the second cache read request has the first stride and determining that the total number of one or more prefetch requests issued consecutively exceeds the propagation count threshold: A second prefetch request is issued to prefetch second data to be written to the cache memory from the second memory of the memory system at a plurality of second next memory addresses after the second memory address. 29. A method according to any of clauses 22-28, further comprising setting the prefetch mode to a prefetch enabled state; and In response to detecting the first stride in the plurality of first cache read requests, and the prefetch mode is the prefetch enabled state: The one or more prefetch requests are issued to prefetch the one or more first data to be written to the cache memory from a second memory in the memory system at the one or more next memory addresses after the last memory address in the plurality of first memory addresses with the detected first stride. 30. The method according to clause 29, further comprising: tracking a total number of the one or more prefetch requests over an observation interval; determining whether a total number of the one or more prefetch requests observed in the observation interval does not exceed a propagation threshold; and In response to a total of the one or more prefetch requests observed in the observation interval not exceeding the propagation threshold, setting the prefetch mode to a prefetch disabled state; and In response to detecting the first stride in the plurality of first cache read requests, and the prefetch mode is a prefetch disabled state: The one or more prefetch requests are not issued so as not to prefetch the one or more first data to be written to the cache memory from a second memory in the memory system at the one or more next memory addresses after the last memory address in the plurality of first memory addresses with the detected first stride. 31. The method according to clause 30, further comprising: tracking a second total number of the one or more prefetch requests observed on the access bus during the observation interval; determining whether the second total number of the one or more prefetch requests in the observation interval exceeds the propagation threshold; and In response to the second total number of the one or more prefetch requests in the observation interval exceeding the propagation threshold, the prefetch mode is set to a prefetch enabled state. 32. A processor-based system comprising: A memory system comprising: a first cache memory; and a second memory, the second memory being a higher level memory than the first cache memory; a cache access bus coupled to the cache memory; a processor configured to issue a plurality of first cache read requests comprising a plurality of first memory addresses to be asserted on the cache access bus; and A prefetcher circuit, the prefetcher circuit being configured to: observing the plurality of first cache read requests on the cache access bus; detecting a first stride in the plurality of first cache read requests; and In response to detecting the first stride in the plurality of first cache read requests: One or more prefetch requests are issued to prefetch one or more first data to be written to the cache memory from a second memory of the memory system at one or more next memory addresses after a last memory address of a plurality of memory addresses with the detected first stride. 33. The processor-based system of clause 32, wherein the second memory comprises a second cache memory. 34. The processor-based system of clause 32, wherein the second memory comprises a second cache memory, the second cache memory being a last level cache memory. 35. The processor-based system of clause 32, wherein the second memory comprises system memory containing the entire physical memory address space for the processor.

Claims

1. A prefetcher circuit, comprising: an access bus coupled to a cache access bus coupled to a cache memory in a memory system; The prefetcher circuit is configured to: observing a plurality of first cache read requests including a plurality of first memory addresses on the access bus; detecting a first stride in the plurality of first cache read requests; as well as In response to detecting the first stride in the plurality of first cache read requests: One or more prefetch requests are issued to prefetch one or more first data to be written to the cache memory from a second memory in the memory system at one or more next memory addresses after a last memory address in the plurality of first memory addresses with the detected first stride.

2. The prefetcher circuit according to claim 1, further configured to: observing a second cache read request on the access bus including a second memory address; determining whether the second cache read request has the first stride; and In response to determining that the second cache read request has the first stride: A second prefetch request is issued to prefetch one or more second data to be written to the cache memory from the second memory in the memory system at one or more second next memory addresses after the second memory address with the detected first stride.

3. The prefetcher circuit according to claim 1 , further configured to: observing a second cache read request on the access bus including a second memory address; determining whether the second cache read request has the first stride; and In response to determining that the second cache read request does not have the first stride, a prefetch request to prefetch data based on the second cache read request is not issued.

4. The prefetcher circuit according to claim 3, further configured to: observing a plurality of third cache read requests including a plurality of third memory addresses on the access bus subsequent to the second cache read request; determining whether the plurality of third cache read requests have a second stride based on the second cache read request; and In response to determining that the plurality of third cache read requests have the second stride: A second prefetch request is issued to prefetch one or more second data to be written to the cache memory from the second memory in the memory system at one or more next second memory addresses after the last memory address in the plurality of third memory addresses with the detected second stride.

5. The prefetcher circuit of claim 1 , configured to detect the first stride in the plurality of first cache read requests by being configured to: determining that a number of the plurality of first cache read requests having the first stride exceeds a stride threshold; and In response to determining that the number of the plurality of first cache read requests having the first stride exceeds the stride threshold, the first stride in the plurality of first cache read requests is detected.

6. The prefetcher circuit of claim 3, further comprising a prefetch buffer circuit, the prefetch buffer circuit comprising a plurality of prefetch buffer entries; in, In response to determining that the second cache read request does not have the first stride, the prefetcher circuit is further configured to assign a first prefetch buffer entry of the plurality of prefetch buffer entries in the prefetch buffer circuit to the second cache read request.

7. The prefetcher circuit according to claim 6, wherein: In response to determining that the second cache read request does not have the first stride, the prefetcher circuit is further configured to: determining whether each of the prefetch buffer entries in the plurality of prefetch buffer entries in the prefetch buffer circuit is allocated; and In response to determining that each of the prefetch buffer entries in the plurality of prefetch buffer entries in the prefetch buffer circuit is allocated: evicting a least recently used prefetch buffer entry from the plurality of prefetch buffer entries; and The evicted prefetch buffer entry is assigned to the second cache read request.

8. The prefetcher circuit according to claim 6, wherein: In response to determining that the second cache read request does not have the first stride, the prefetcher circuit is further configured to associate a thread identification associated with the second cache request with the first prefetch buffer entry.

9. The prefetcher circuit according to claim 6, wherein: The prefetcher circuit is further configured to: observing, subsequent to the second cache read request, a third cache read request on the access bus including a third memory address; determining whether the prefetch buffer circuit includes the first prefetch buffer entry corresponding to the third cache read request among the plurality of prefetch buffer entries; and In response to determining that the prefetch buffer circuit contains the first prefetch buffer entry corresponding to the third cache read request: determining that the first prefetch buffer entry has an associated stride; and In response to the first prefetch buffer entry not having an associated stride, a second stride is associated with a second prefetch buffer entry that is the second memory address subtracted from the third memory address.

10. The prefetcher circuit of claim 9, wherein: The prefetcher circuit is further configured to: In response to determining that the second cache read request does not have the first stride, the prefetcher circuit is further configured to associate a second thread identification associated with the second cache request to the first prefetch buffer entry; and The prefetch buffer circuitry is further configured to determine that a third thread identification associated with the third cache read request matches a second thread identification associated with the first prefetch buffer entry, thereby determining that the prefetch buffer circuitry contains the first prefetch buffer entry of the plurality of prefetch buffer entries corresponding to the third cache read request.

11. The prefetcher circuit of claim 9, wherein: The prefetcher circuit is further configured to: observing a fourth cache read request including a fourth memory address on the access bus after the second cache read request; determining that the prefetch buffer circuit contains the first prefetch buffer entry corresponding to the fourth cache read request among the plurality of prefetch buffer entries; and In response to determining that the prefetch buffer circuit contains the first prefetch buffer entry corresponding to the fourth cache read request: determining that the first prefetch buffer entry has an associated stride; and In response to the first prefetch buffer entry having the associated stride: A second prefetch request is issued to prefetch one or more second data to be written to the cache memory from one or more second next memory addresses starting from an address of the fourth memory address added to the second stride.

12. The prefetcher circuit of claim 11, wherein: The prefetcher circuit is further configured to: In response to determining that the second cache read request does not have the first stride, the prefetcher circuit is further configured to associate a second thread identification associated with the second cache request to the first prefetch buffer entry; and The prefetch buffer circuit is further configured to determine whether a fourth thread identifier associated with the fourth cache read request matches the second thread identifier associated with the first prefetch buffer entry, thereby determining that the prefetch buffer circuit contains the first prefetch buffer entry corresponding to the fourth cache read request among the plurality of prefetch buffer entries.

13. The prefetcher circuit of claim 11 , further configured to: tracking a number of cache read requests observed on the access bus; and In response to the number of observed cache read requests being greater than a reset threshold, each of the plurality of prefetch buffer entries in the prefetch buffer circuit is invalidated.

14. The prefetcher circuit of claim 1 , configured to issue a plurality of prefetch requests to prefetch a plurality of first data to be written to the cache memory from a second memory in the memory system at a plurality of next memory addresses after the last memory address among the plurality of first memory addresses with the detected first stride.

15. The prefetcher circuit of claim 1 , further configured to: tracking a total number of one or more prefetch requests issued successively for the plurality of first cache read requests; determining whether the total number of one or more prefetch requests issued consecutively exceeds a propagation count threshold; observing a second cache read request on the access bus including a second memory address; detecting the first stride in the second cache read request; as well as In response to detecting that the second cache read request has the first stride and determining that the total number of one or more prefetch requests issued consecutively exceeds the propagation count threshold: A second prefetch request is issued to prefetch second data to be written to the cache memory from the second memory in the memory system at a plurality of second next memory addresses after the second memory address.

16. The prefetcher circuit of claim 1 , further configured to set a prefetch mode to a prefetch enabled state; and The prefetcher circuit is configured to, in response to detecting the first stride in the plurality of first cache read requests and the prefetch mode is the prefetch enabled state: The one or more prefetch requests are issued to prefetch the one or more first data to be written to the cache memory from a second memory in the memory system at the one or more next memory addresses after the last memory address in the plurality of first memory addresses with the detected first stride.

17. The prefetcher circuit of claim 16, further configured to: tracking a total number of the one or more prefetch requests during an observation interval; determining whether a total number of the one or more prefetch requests observed in the observation interval does not exceed a propagation threshold; as well as responsive to a total of the one or more prefetch requests observed in the observation interval not exceeding the propagation threshold, setting the prefetch mode to a prefetch disabled state; as well as The prefetcher circuit is configured to, in response to detecting the first stride in the plurality of first cache read requests and the prefetch mode is a prefetch disabled state: The one or more prefetch requests are not issued to prefetch the one or more first data to be written to the cache memory from a second memory in the memory system at the one or more next memory addresses after the last memory address in the plurality of first memory addresses with the detected first stride.

18. The prefetcher circuit of claim 17, further configured to: tracking a second total number of the one or more prefetch requests observed on the access bus during the observation interval; determining whether the second total number of the one or more prefetch requests in the observation interval exceeds the propagation threshold; and In response to the second total number of the one or more prefetch requests in the observation interval exceeding the propagation threshold, the prefetch mode is set to a prefetch enabled state.

19. The prefetcher circuit of claim 1, wherein: The prefetcher circuit is further configured to write one or more data to the cache memory.

20. The prefetcher circuit of claim 1, wherein: The second memory is a higher level memory than the cache memory.

21. The prefetcher circuit of claim 1 , integrated into a device selected from the group consisting of: a set-top box; an entertainment unit; a navigation device; a communication device; a fixed location data unit; a mobile location data unit; a global positioning system (GPS) device; a mobile phone; a cellular phone; a smartphone; a Session Initiation Protocol (SIP) phone; a tablet computer; a phablet; a server; a computer; a portable computer; a mobile computing device; a wearable computing device; a desktop computer; a personal digital assistant (PDA); a monitor; a computer monitor; a television set; a tuner; a radio unit; a satellite radio unit; a music player; a digital music player; a portable music player; a digital video player; a video player; a digital video disc (DVD) player; a portable digital video player; an automobile; a vehicle component; an avionics system; an unmanned aerial vehicle; and a multirotor aircraft.

22. A method of prefetching data into a cache memory based on a stride pattern detected in a cache read request to the cache memory, comprising: observing a plurality of first cache read requests including a plurality of memory addresses on an access bus coupled to a cache access bus coupled to a cache memory in a memory system; detecting a first stride in the plurality of first cache read requests; as well as In response to detecting the first stride in the plurality of first cache read requests: One or more prefetch requests are issued to prefetch one or more first data to be written to the cache memory from a second memory in the memory system at one or more next memory addresses after a last memory address in the plurality of memory addresses with the detected first stride.

23. The method according to claim 22, further comprising: observing a second cache read request on the access bus including a second memory address; determining whether the second cache read request has the first stride; as well as In response to determining that the second cache read request has the first stride: A second prefetch request is issued to prefetch one or more second data to be written to the cache memory from the second memory in the memory system at one or more second next memory addresses after the second memory address with the detected first stride.

24. The method of claim 22, further comprising: observing a second cache read request on the access bus including a second memory address; determining whether the second cache read request has the first stride; as well as In response to determining that the second cache read request does not have the first stride, a prefetch request to prefetch data based on the second cache read request is not issued.

25. The method according to claim 24, further comprising: observing a plurality of third cache read requests including a plurality of third memory addresses on the access bus subsequent to the second cache read request; determining, based on the second cache read request, whether the plurality of third cache read requests have a second stride; as well as In response to determining that the plurality of third cache read requests have the second stride: A second prefetch request is issued to prefetch one or more second data to be written to the cache memory from the second memory in the memory system at one or more next second memory addresses after the last memory address in the plurality of third memory addresses with the detected second stride.

26. The method of claim 22, wherein: Detecting the first stride in the plurality of first cache read requests comprises: determining that a number of the plurality of first cache read requests having the first stride exceeds a stride threshold; and In response to determining that the number of the plurality of first cache read requests having the first stride exceeds the stride threshold, the first stride in the plurality of first cache read requests is detected.

27. The method of claim 22, comprising: A plurality of prefetch requests are issued to prefetch a plurality of first data to be written to the cache memory from a second memory in the memory system at a plurality of next memory addresses after the last memory address in the plurality of first memory addresses with the detected first stride.

28. The method of claim 22, further comprising: tracking a total number of one or more prefetch requests issued successively for the plurality of first cache read requests; determining whether the total number of one or more prefetch requests issued consecutively exceeds a propagation count threshold; observing a second cache read request on the access bus including a second memory address; detecting the first stride in the second cache read request; as well as In response to detecting that the second cache read request has the first stride and determining that the total number of one or more prefetch requests issued consecutively exceeds the propagation count threshold: A second prefetch request is issued to prefetch second data to be written to the cache memory from the second memory in the memory system at a plurality of second next memory addresses after the second memory address.

29. The method of claim 22, further comprising setting a prefetch mode to a prefetch enabled state; and In response to detecting the first stride in the plurality of first cache read requests, and the prefetch mode is the prefetch enabled state: The one or more prefetch requests are issued to prefetch the one or more first data to be written to the cache memory from a second memory in the memory system at the one or more next memory addresses after the last memory address in the plurality of first memory addresses with the detected first stride.

30. The method of claim 29, further comprising: tracking a total number of the one or more prefetch requests during an observation interval; determining whether a total number of the one or more prefetch requests observed in the observation interval does not exceed a propagation threshold; as well as responsive to a total of the one or more prefetch requests observed in the observation interval not exceeding the propagation threshold, setting the prefetch mode to a prefetch disabled state; as well as In response to detecting the first stride in the plurality of first cache read requests, and the prefetch mode is a prefetch disabled state: The one or more prefetch requests are not issued to prefetch the one or more first data to be written to the cache memory from a second memory in the memory system at the one or more next memory addresses after the last memory address in the plurality of first memory addresses with the detected first stride.

31. The method of claim 30, further comprising: tracking a second total number of the one or more prefetch requests observed on the access bus during the observation interval; determining whether the second total number of the one or more prefetch requests in the observation interval exceeds the propagation threshold; as well as In response to the second total number of the one or more prefetch requests in the observation interval exceeding the propagation threshold, the prefetch mode is set to a prefetch enabled state.

32. A processor-based system comprising: A memory system comprising: a first cache memory; and a second memory, the second memory being a higher level memory than the first cache memory; a cache access bus coupled to the cache memory; a processor configured to issue a plurality of first cache read requests comprising a plurality of first memory addresses to be asserted on the cache access bus; and A prefetcher circuit, the prefetcher circuit being configured to: observing the plurality of first cache read requests on the cache access bus; detecting a first stride in the plurality of first cache read requests; and In response to detecting the first stride among the plurality of first cache read requests: One or more prefetch requests are issued to prefetch one or more first data to be written to the cache memory from a second memory in the memory system at one or more next memory addresses after a last memory address in a plurality of memory addresses with the detected first stride.

33. The processor-based system of claim 32, wherein: The second memory includes a second cache memory.

34. The processor-based system of claim 32, wherein: The second memory includes a second cache memory, and the second cache memory is a last level cache memory.

35. The processor-based system of claim 32, wherein: The second memory comprises a system memory containing the entire physical memory address space for the processor.

Images