Cache management based on reuse distance

Abstract

A cache [110] of a processor [102] includes a cache controller [112] for implementing a cache management policy for insertion and replacement of cache lines of the cache. The cache management policy assigns a replacement priority level to each cache line in at least a subset of cache lines in a region of the cache based on a comparison of a number of accesses to a cache set [116] having a way storing the cache line since the cache line was last accessed and a reuse distance determined for the region of the cache, wherein the reuse distance represents an average number of accesses to the cache set between accesses to any given cache line of a given cache set of the region.

Description

Background Art

[0001] Processing systems use caches to temporarily buffer data from memory or mass storage devices for rapid access. Because caches have limited storage capacity, cache management policies are often employed to guide the selection of cache lines for replacement when the corresponding cache area is full. However, some conventional cache management policies, such as those based on the least recently used (LRU) or heavily referenced interval prediction (RRIP) principles, are inefficient when handling non-routine accesses to cache lines or require relatively complex circuit implementations, which can limit their applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] By referencing the accompanying drawings, the present disclosure will be better understood, and its numerous features and advantages will become apparent to those skilled in the art.The use of the same reference numerals in different drawings indicates similar or identical items.

[0003] Figure 1 is a block diagram of a processing system with a cache that implements a reuse distance-based cache management policy, according to some embodiments.

[0004] Figure 2 According to some embodiments Figure 1 Block diagram of the cache reuse distance calculation component.

[0005] Figure 3 is shown according to some embodiments for calculating Figure 1 and Figure 2 A flowchart of a method for determining the current reuse distance of a corresponding area of a cache.

[0006] Figure 4 is a block diagram illustrating an alternative implementation of a reused distance calculation component, according to some embodiments.

[0007] Figure 5 is a flow chart illustrating a method of maintaining a row access counter for a corresponding cache line, according to some embodiments.

[0008] Figure 6 is a flow chart illustrating a method for assigning replacement priority levels to cache lines based on current reuse distances and corresponding line access count values, according to some embodiments.

[0009] Figure 7 is a flow chart illustrating a method for selecting a cache line for replacement using replacement priority based on reuse distance, according to some embodiments. DETAILED DESCRIPTION

[0010] Figure 1-Figure 7Systems and techniques are described for implementing a cache management policy for a processing system's cache that is based on a predicted "reuse distance" for corresponding regions of the cache. As described herein, the reuse distance for a corresponding region of a cache is a representation of the historical average number of accesses to a given cache line of a given cache set of that cache region between accesses to that cache set. This reuse distance is then used as a predictor or other indicator of how many times a given cache line in a given cache set is likely to be accessed again, and this information, along with information about the cache line's recent access history, is used to assign a replacement priority to that cache line. The replacement priority assigned to the cache line in this manner is then used to select a cache line for replacement according to the cache management policy when the corresponding cache set is fully occupied. This cache management approach based on reuse distance can provide accurate and efficient prioritization for replacing cache lines that may be used in a cyclic or streaming access pattern, and in a manner that is relatively simple to implement in hardware.

[0011] Figure 1A processing system 100 employing reuse distance-based cache management according to some embodiments is shown. Processing system 100 includes a processor 102 coupled to a memory subsystem 104, wherein memory subsystem 104 includes one or more system memories, scratch pad memories, disk drives, or other mass storage devices. Processor 102 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a digital signal processor (DSP), or a combination thereof. Processor 102 includes one or more execution pipelines 106 (e.g., CPU cores) and a cache hierarchy of one or more caches, including cache 108. Cache 108 includes a cache array 110 and a cache controller 112. Cache array 110 includes a plurality of entries 114 to store cache lines (i.e., temporarily buffered blocks of data) for access by one or more execution pipelines 106. In at least one embodiment, cache 108 is a set associative cache such that cache line entries 114 are arranged in a plurality of cache sets 116, each cache set 116 having a plurality of ways, each way being operable for a cache line entry 114 to store a corresponding cache line, and such that any cache line associated with a memory address mapped to a cache set 116 can be stored in any way of the cache set 116. In the illustrated example, cache 108 implements four ways, Way 0 through Way 3, but more or fewer ways may be implemented. Each cache set 116 includes additional fields, such as a tag field 118 for each way of the set, wherein the tag field 118 stores a portion of the memory address associated with the valid cache line stored in the corresponding way (if any), status bits, control bits, etc.

[0012] The cache controller 112 operates to maintain various fields of the cache array 110 based on the activity of one or more execution pipelines 106, including receiving and storing data blocks as cache lines, accessing cache lines for use or modification by one or more execution pipelines 106, accessing cache lines for eviction or flushing to the memory subsystem 104, and the like. As part of this process, the cache controller 112 implements a cache management policy 120 that controls the prioritization of cache lines for replacement or other eviction and controls the selection of candidate cache lines for replacement or other eviction based on such prioritization. In at least one embodiment, the cache management policy 120 uses a "reuse distance" determined for a corresponding region of the cache 108 (which may be a portion (e.g., a quarter) of the cache 108 or the entire cache 108) and a recent access history of cache lines in the corresponding region to determine the replacement priority of the cache line. Although the reuse distance is determined for a portion of the cache in this embodiment, the cache management policy 120 can also be applied to the entire cache 108. As described above, at a high level, the reuse distance represents the average number of accesses to a cache set 116 between accesses to a particular cache line within the cache set 116 in the corresponding cache region. That is, the reuse distance represents a prediction of the average number of cache accesses that are likely to occur for a given cache set before a given cache line of the corresponding region of cache 108 is accessed again. To this end, the cache management policy 120 implements three phases, each of which operates independently and concurrently with the others: a reuse distance determination phase 122, a replacement priority assignment phase 124, and a cache line replacement phase 126. The reuse distance determination phase 122 provides a calculation of the current reuse distance for each applicable region of cache 108, and is referred to below in its entirety. Figure 2-Figure 4 The replacement priority assignment stage 124 provides assignment of replacement priorities to cache lines based on the current reuse distances calculated in the most recent calculation cycle of stage 122 and is described below with reference to Figure 5 and Figure 6 The cache line replacement stage 126 provides selection of a cache line for replacement based on an assigned replacement priority, which is determined from the most recent prioritization cycle of stage 124 and will be referred to below. Figure 7 Describe in more detail.

[0013] In at least one embodiment, cache 108 employs a set of counters 128 for calculating the current reuse distance at each iteration of stage 122 and for determining the replacement priority of a cache line at each cycle of stage 124. The set of counters 128 includes a set access counter 130 and a row access counter 132 for each way of cache array 110 (i.e., each cache line), or, alternatively, for each way of a subset of cache sets designated as representative cache sets of cache 108 for sampling purposes (e.g., every Xth cache set of the region, where X is an integer greater than 1). The set access counter 130 stores a set access count value that indicates the number of times the cache set 116 associated with the corresponding cache line has been accessed since the corresponding cache line was inserted or last accessed. The row access counter 132 stores a row access count value that indicates the number of times the corresponding cache line has been accessed since it was inserted into cache 108 or since it was reset in response to the start of the next computation cycle. In some embodiments, the counter set 128 also includes an Nth access counter 134 that counts the number of accesses to the corresponding cache set 116 before being reset (and triggering further operations, as described below) after the Nth counted access, where N represents a programmable or otherwise specified integer greater than one (N>1).

[0014] Figure 2 An implementation of a reuse distance calculation component 200 implemented by the cache controller 112 and accessing the counter set 128 is shown in accordance with some embodiments. The reuse distance calculation component 200 includes a set accounting component 202 for each cache set 116 for calculating the reuse distance of the corresponding region of the cache 108. This can include each cache set 116 in the cache region, or a representative subset of the cache sets 116 of the cache region. In the depicted example, X cache sets 116 are represented (X>=1), with set accounting components 202-0, 202-1, and 202-3 shown for cache set 0, cache set 1, and cache set X-1 of the represented cache sets 116. The reuse distance calculation component 200 also includes an accumulator 204 and an averaging / scaling component 206. In one embodiment, the reuse distance calculation component 200 further includes a hit counter 207 for counting the number of cache hits for the corresponding region of the cache 108 in the current calculation cycle, and triggering the reuse distance calculation when the number of cache hits reaches a programmable or otherwise specified value K. For illustration, the hit counter 207 can be implemented as a countdown counter that is reset to K for each calculation cycle, decrements for each cache hit of the representative cache set, and triggers the reuse distance calculation when it reaches 0.

[0015] The configuration of the set accounting component 202-3 for set X-1 is illustrated and represents the configuration of each set accounting component 202 with respect to its corresponding cache set 116. As shown, the set accounting component 202-3 includes a set of comparators 208, one comparator for each way in the corresponding cache set 116, and selection logic 214 (depicted as a multiplexer for ease of illustration). Thus, for the depicted example of a cache 108 having a set with four ways, the set accounting component 202-3 includes four comparators 208. Each comparator 208 includes an input coupled to receive an address value from the tag field of the corresponding way of the set (e.g., one of the tag fields 118-0 through 118-3 for ways 0 through 4, respectively) and an input coupled to receive an address value from a tag field 210 of a cache probe 212 submitted to the cache 108 by the execution pipeline 106. Each comparator 208 also has an output that asserts when the address value from the tag field 118 of the corresponding way matches the address value of the tag field 210 of the cache probe 212; that is, the comparator 208 associated with the way of the cache set 116 that is the target of the cache probe 212 (that is, provides a cache "hit" for the cache probe 212) has its output asserted, while the other comparators 208 of the set accounting component 202-3 remain deasserted. In this way, the output of the comparator 208 identifies the way containing the cache line with an address that matches the address represented in the cache probe 212.

[0016] As described above, the counter set 128 includes a set access counter 130 for each way of each representative cache set 116 for use in reuse distance calculations. Thus, for set X-1 associated with the illustrated set accounting component 202-3, the counter set 128 includes four set access counters 130-0, 130-1, 130-2, and 130-3 for ways 0, 1, 2, and 3, respectively. Each of the set access counters 130-0 through 130-3 stores a set access count value representing the number of accesses to set X-1 since a cache line in the corresponding way was inserted or last accessed, as described in more detail below.

[0017] The selection logic 214 includes a plurality of select inputs, each of which is coupled to receive a currently selected access count value for a corresponding one of the set access counters 130 of the cache set 116. Thus, in a four-way cache set 116, the selection logic 214 has four select inputs, one for receiving a count value from the set access counter 130-0, one for receiving a count value from the set access counter 130-1, one for receiving a count value from the set access counter 130-2, and one for receiving a count value from the set access counter 130-3. The selection logic 214 also includes a select control input coupled to the output of the comparator 208 and an output coupled to the accumulator 204. Thus, the selection logic 214 operates to select one of the access count values from the set access counters 130-0 through 130-3 for output to the accumulator 204 based on which comparator 208 (if any) has an asserted output. That is, the way of the cache set 116 having a tag address that matches the tag address of the cache probe 212 triggers the selection logic 214 to output the counter value of the set access counter 130 associated with the way to the accumulator 204 .

[0018] The accumulator 204 operates to accumulate the set access counter values received from each set accounting component 202 and provides the resulting updated accumulated value to the averaging / scaling component 206. In response to a triggering event (e.g., every Kth access to the representative cache set of the region), the averaging / scaling component 206 operates to average the most recently updated accumulated values for the number of accesses to the representative cache set of the region since the last calculation cycle to generate an average set access count value from the accumulated values. The average set access count value can be obtained via, for example, a series of shift operations, and in some embodiments, the averaging / scaling component 206 scales the resulting average set access count value using a specified scaling factor. The resulting averaged / scaled set access count value is then used as the current reuse distance 216 for the corresponding region of the cache 108.

[0019] Figure 3 A more detailed depiction of the cache controller 112 is shown in accordance with some embodiments. Figure 23. The method 300 of the reuse distance calculation process employed by the reuse distance calculation component 200 of the embodiment of the present invention is shown. At block 302, the cache controller 112 monitors the operation of the cache 108 to determine whether a cache line has been inserted into one of the representative cache sets 116. At block 304, the cache controller 112 monitors the operation of the cache 108 to determine whether a cache line in one of the representative cache sets 116 has been accessed. If the cache line has been accessed, then at block 306, the set access counter 130 value associated with the way of the cache set 116 storing the accessed cache line is sent to the accumulator 204. When the cache line has been inserted or accessed, at block 308, the cache controller 112 resets the set access counter 130 associated with the way of the cache set 116 storing the inserted or accessed cache line to a predetermined value (e.g., 0), and at block 310, the cache controller 112 increments the set access counter 130 associated with every other way of the cache set 116. That is, inserting a cache line into a way of the representative cache set 116 or accessing a cache line resets the set access counter 130 for that way of the cache set 116, while also causing the set access counter 130 for every other way of the cache set 116 to be incremented. Alternatively, the cache controller 112 increments the set access counter 130 associated with each way of the cache set 116 and then resets the set access counter 130 associated with the way of the cache set 116 storing the inserted or accessed cache line to a predetermined value. Thus, in this manner, the cache controller 112 monitors, for each cache line in the representative cache set 116, the number of accesses to the representative cache set 116 since the cache line was inserted or last accessed.

[0020] Additionally, at block 312, the cache controller 112 monitors cache hits (which may include accesses referenced at block 302) to ways of the representative cache set 116. In at least one embodiment, cache hits to ways of the representative cache set 116 are signaled via the comparator 208 of the set accounting component 202 associated with the representative cache set 116 because when a cache probe (e.g., cache probe 212, Figure 2 ) matches the address in the tag field 118 of the corresponding way, the output of the corresponding comparator 208 asserts, thereby signaling that the cache probe has hit the way of the representative cache set 116. In response to such a cache hit, at block 314, the selection logic 214 of the set accounting component 202 associated with the hit cache set 116 outputs the value of the set access counter 130 associated with the hit way of the hit cache set 116 to the accumulator 204, so that the accumulator 204 adds the input value to the previously accumulated value in the current calculation cycle.

[0021] For each cache hit to the representative cache set 116, the processes of blocks 312 and 314 are repeated until a Kth cache hit to the corresponding region of cache 108 is detected in the current computation iteration (via, for example, hit counter 207), where K is a programmable or otherwise specified integer value greater than 1 (K>1). For example, K can be set to 64, so that the set count accumulation process continues until a 64th cache hit to the corresponding region of cache 108 occurs. In response to determining at block 316 that a Kth cache hit to the region of cache 108 has occurred for the current computation cycle, then at block 318, an averaging / scaling component averages the current accumulated value from accumulator 204 across the K cache hits for the current computation cycle and, in some embodiments, scales the average by multiplying the average by a factor. In implementations where K and any scaling factors are powers of 2, averaging / scaling component 206 can be implemented as right-shift and left-shift logic. For illustration, when K is set to 64 (2^6) according to the previous example, averaging of the accumulated value can be performed by shifting the accumulated value right by 6 bits and then left by one bit. More generally, when K is equal to 2^M (M is a positive integer), the averaging / scaling component 206 can shift the currently updated accumulated value left by M bits and then right by one bit to obtain the average set access count in the K cache accesses to the representative cache set. Similarly, scaling the average value by 2 can be achieved by shifting the obtained average value left by 1 bit. The obtained average (and scaled) value is then set as the current reuse distance 216 of the corresponding area of the cache 108. The scaling of the average value takes into account the fact that there may be differences between the set access count values of the various ways in the set, resulting in a reuse distance lower than the set access count values of some rows in the set. As explained in more detail below, once the set access count value of a given row exceeds the reuse distance of the cache, the row is more likely to be evicted. Scaling the average value can be used to prevent certain rows in the set from being inadvertently prioritized for replacement. As the current computation cycle ends, at block 320 , the cache controller 112 resets the various components used in the ended computation cycle, such as the hit counter 207 , the set access counter 130 , the row access counter 132 , and the accumulator 204 , and the process then returns to block 312 for the next computation cycle.

[0022] Figure 4 1 shows a set accounting component 202 implemented by the cache controller 112 for each representative cache set 116 according to at least one embodiment. Figure 2 ) is an alternative implementation of Figure 2 The implementation method, Figure 4An alternative implementation of the set accounting component 202 includes a comparator 208 operable to assert their respective outputs in response to an address in the tag field 118 of the corresponding way matching an address in the tag field 210 of a received cache probe 212, and also includes selection logic 414 (depicted as a multiplexer for ease of illustration) that utilizes the output of the comparator 208 as its selection control input. However, rather than using a counter large enough to account for every access to the corresponding cache set during a calculation cycle for the set access counter 130, Figure 4 Instead, the illustrated implementation increments the set access counter 130 every Nth access to the cache set 116, thereby allowing a smaller counter to be used for the set access counter 130. To facilitate counting accesses by N, the illustrated implementation of the set accounting component 202 also includes an Nth access counter 404 that is incremented (or reset to N and decremented for each access) each time the cache set 116 is accessed during the current calculation cycle. Furthermore, to compensate for the fact that only every Nth access to a cache set 116 triggers an increment of the set access counter 130 for that cache set, the illustrated implementation employs a shift register 406 and an adder 408 between the output of each set access counter 130 and the corresponding input to the selection logic 414 to adjust for this sampled cache access method. Thus, assuming N is a power of 2 (2^j), the set access count value output from the set access counter 130 is left shifted by j bits and added to the current hit count represented in the Nth access counter 404, and the resulting value is fed to a corresponding input of the selection logic 414. The selection logic 414 is then operable to select one of the input values for output to the accumulator 204 based on the state of the output of the comparator 208, which in turn indicates which way (if any) of the representative cache set 116 is the target of the Nth cache probe 212.

[0023] Figure 5 and Figure 6 Together, the following are shown for use in the replacement priority assignment phase 124 ( Figure 1 ) during which the cache controller 112 implements the operation of the cache management policy 120. Figure 5A method 500 for maintaining row access counters 132 for a representative cache set 116 during a prioritization cycle is shown, in accordance with some embodiments. As described above, each row access counter 132 represents the number of times a cache line stored in a way associated with the row access counter 132 has been accessed since the start of the current prioritization cycle. Thus, as the prioritization cycle begins, at block 502, the cache controller 112 monitors for insertions of cache lines into the ways of the representative cache set 116. In response to such insertions, at block 504, the cache controller 112 resets the row access counter 132 associated with the target way of the representative cache set 116. Thereafter, at block 506, the cache controller 112 monitors for accesses to the cache line. In response to detecting an access, at block 508, the cache controller 112 increments the row access counter 132 for the accessed cache line. In some embodiments, row access counter 132 is implemented as a one-bit or two-bit saturating counter to reduce the hardware requirements for row access counter 132 and thus count at most one access (for a one-bit counter implementation) or at most three accesses (for a two-bit counter implementation). In other embodiments, more than two bits are used for row access counter 132 to facilitate counting a larger number of accesses to any given cache line.

[0024] Go to Figure 6 , a method 600 is shown for assigning a replacement priority to a cache line based on a current row access count of the cache line in the corresponding row access counter 132 and based on a current reuse distance, according to some embodiments. The method 600 is initiated by an event (block 602), such as a determination that a cache line is to be replaced, which serves as a trigger to determine the replacement priority of newly inserted cache lines during the current prioritization cycle and to re-determine the replacement priority of previously existing cache lines. In response to the trigger, the prioritization process is initiated by selecting cache lines of a region of the cache 108 according to a selection sequence (e.g., direct sequential selection, pseudo-random selection, etc.) at block 604.

[0025] At block 606, the cache controller 112 accesses the set access counter 130 of the way storing the selected cache line and compares the count contained therein with the current reuse distance 216. If the set access counter 130 is not greater than the current reuse distance 216, this indicates that the cache line has not yet reached the reuse distance and, therefore, may be reused in the future. Therefore, if the set access counter 130 is less than the current reuse distance, then at block 608, the cache controller 112 accesses the row access counter 132 associated with the way storing the selected cache line and determines whether the value stored therein is greater than zero (i.e., determines whether the cache line has been reused after insertion). If so, given the tendency of some data to be accessed repeatedly, the cache line is likely to be reused again. Therefore, if it is determined at block 608 that the row access counter 132 is greater than zero, then at block 610, the cache line is assigned a replacement priority level of 3 (for the following, it is assumed that the lower the replacement priority level value, the less suitable the corresponding cache line is as a candidate for replacement, and therefore the greater the likelihood that the corresponding cache line will be selected for replacement). Otherwise, if the row access counter 132 is equal to zero, this indicates that the cache line has not been reused; however, since the reuse distance of the cache line has not yet been reached, there is still some possibility that the cache line will be reused in the future. In this case, at block 612, the cache line is assigned a replacement priority level of 1 (a priority level of 1 indicates a greater likelihood of replacement selection than a replacement priority level of 3).

[0026] Returning to block 606, if it is determined that the access count represented by the set access counter 130 is greater than the current reuse distance, this means that the cache line has reached the reuse distance but has not been reused since then. Therefore, at block 614, the cache controller 112 determines whether the cache line has been accessed by accessing the row access counter 132 of the way used to store the cache line. If the row access counter 132 is greater than zero, this means that the cache line has been reused at least once since being inserted into the cache 108 and is therefore likely to be reused again. However, since it has exceeded the reuse distance, its reuse probability is reduced. Therefore, if the row access counter 132 is greater than zero, then at block 616, the selected cache line is assigned a replacement priority level of 2 (a replacement priority level of 2 indicates a greater likelihood of replacement selection than a replacement priority level of 3 and a lower likelihood of selection than a replacement priority level of 1). Otherwise, if the count is zero, this means that the cache line has not been reused since being inserted and has exceeded the reuse distance, and is therefore less likely to be reused in the future. In this case, at block 618, the cache line is assigned a replacement priority level of 0 (a replacement priority level of 0 represents the greatest likelihood of replacement selection in this example).

[0027] Figure 71 shows a schematic diagram representing a cache line replacement phase 126 ( FIG. Figure 1 ) is a method 700 of operation of a cache controller 112 of the embodiment of the present invention. For the following, recall that, for purposes of illustration, an increase in the numerical value of the replacement priority level corresponds to an increase in the priority of retaining the corresponding cache line, and conversely, a decrease in the numerical value of the replacement priority level corresponds to an increase in the priority or likelihood of evicting the corresponding cache line. At block 702, a load operation or a store operation is performed and results in the generation of a data block to be stored as a cache line in the cache 108. Therefore, further at block 702, the cache controller 112 determines a cache set 116 that can be used to store the cache line based on an address associated with the cache line. At block 704, the cache controller 112 determines whether there is an available way in the identified cache set 116 (i.e., whether there is a way that does not currently store a valid cache line). If so, then at block 706, the cache controller 112 inserts the cache line into an available way of the cache set 116. If the cache set 116 is a representative cache set, then the insertion of the cache line into the cache set 116 triggers certain counting operations, as described above with reference to Figure 3 Block 302 and Figure 5 As described in block 502.

[0028] Otherwise, if there are no available ways in the identified cache set 116, the cache controller 112 determines whether to evict the current cache line in the cache set 116 or bypass caching of the cache line based on the replacement priority level assigned during the replacement priority assignment phase 124 of the cache management policy 120, as described above. Therefore, at block 708, the cache controller 112 determines whether the cache 108 supports cache bypass (also known as "selective caching" or "cache exclusion"). If cache bypass is supported, at block 710, the cache controller 112 determines whether the cache set 116 contains any cache lines that are assigned a replacement priority level of 0. If not, at block 712, the cache controller 112 may selectively bypass caching of the cache line in the cache 108 (e.g., by preventing any caching of the cache line, or by providing the cache line to a lower level cache for storage). In another embodiment, if it is determined that the cache line is part of a stream (eg, the current reuse distance is small (0 or 1)), then cache bypass may be selected if there is also no cache line with a replacement priority level of 1.

[0029] If cache bypass is not supported, or there are no cache lines with a sufficiently low priority ranking to justify cache bypass, then at block 714, the cache controller 112 selects the cache line with the lowest replacement priority in the cache set 116 as a replacement candidate. In the event that there are two or more cache lines with the same lowest replacement priority, the cache controller 112 may select one of the cache lines pseudo-randomly, based on a specified selection order, or select the way with the largest set access count. At block 716, the cache controller 112 replaces or otherwise evicts the selected candidate cache line with a new cache line in the corresponding way of the cache set 116. This action is a cache line insertion and therefore triggers the cache line insertion described above with reference to FIG. Figure 3 Block 302 and Figure 5 Block 502 describes certain counting operations.

[0030] In some embodiments, certain aspects of the technology described above are implemented by one or more processors of a processing system that executes software. The software includes one or more executable instruction sets stored or otherwise tangibly embodied on a non-transitory computer-readable storage medium. The software includes instructions and certain data that manipulate one or more processors to execute one or more aspects described above when executed by one or more processors. Non-transitory computer-readable storage media include, for example, magnetic or optical disk storage devices, solid-state storage devices (such as flash memory, cache, random access memory (RAM) or one or more other non-volatile memory devices), etc. The executable instructions stored on the non-transitory computer-readable storage medium may present source code, assembly language code, object code, or other instruction formats that are interpreted or otherwise executed by one or more processors.

[0031] According to one aspect, a method for managing a cache of a processing system includes determining, by a cache controller of the cache, a reuse distance for a region of the cache, the reuse distance representing an average number of accesses to the cache set between accesses to a given cache line of a given cache set of the cache. The method also includes assigning, by the cache controller, a replacement priority level to each cache line in at least a subset of the cache lines of the region of the cache based on the reuse distance and a count of cache hits for the cache line.

[0032] According to another aspect, a method for managing a cache of a processing system includes implementing, by a cache controller of the cache, a cache management policy for insertion and replacement of cache lines of the cache, the cache management policy providing: assigning a replacement priority level to each cache line in at least a subset of cache lines in a region of the cache based on a comparison of a number of accesses to a cache set having ways storing the cache line since the cache line was last accessed and a reuse distance determined for the region of the cache, the reuse distance representing an average number of accesses to the cache set between accesses to any given cache line of a given cache set of the region.

[0033] According to yet another aspect, a processor includes a cache memory comprising a plurality of cache sets, each cache set having a plurality of ways configured to store corresponding cache lines. The processor also includes a cache controller configured to implement a cache management policy for insertion and replacement of cache lines of the cache, the cache management policy providing for assigning a replacement priority level to each cache line in at least a subset of cache lines in a region of the cache based on a comparison of a number of accesses to the cache set having the ways storing the cache line since the cache line was last accessed and a reuse distance determined for the region of the cache, the reuse distance representing an average number of accesses to the cache set between accesses to any given cache line of a given cache set of the region.

[0034] Computer-readable storage media include any non-transitory storage media or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tapes, or magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS)-based storage media. Computer-readable storage media may be embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard drive), removably attached to a computing system (e.g., an optical disc or flash memory based on a universal serial bus (USB)), or coupled to a computer system via a wired or wireless network (e.g., a network accessible storage device (NAS)).

[0035] It should be noted that not all activities or elements described above in the general description are required, a portion of a particular activity or device may not be required, and one or more other activities may be performed, or elements other than those described may be included. In addition, the order in which the activities are listed is not necessarily the order in which the activities are performed. In addition, the corresponding concepts have been described with reference to specific embodiments. However, it will be understood by those skilled in the art that various modifications and changes may be made without departing from the scope of the present disclosure as set forth in the appended claims. Therefore, this specification and drawings are to be regarded as illustrative rather than restrictive, and all such modifications are intended to be included within the scope of the present disclosure.

[0036] The benefits, other advantages, and solutions to problems have been described above with reference to specific embodiments. However, none of the benefits, advantages, solutions to problems, and any features that may make any benefit, advantage, or solution to the problem appear or become more prominent, should be construed as being key, required, or essential features of any or all of the claims. Furthermore, the specific embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners that will be apparent to those skilled in the art having the benefit of the teachings herein. No limitation to the details of construction or design shown herein is intended, except as described in the appended claims. It is therefore apparent that the specific embodiments disclosed above may be altered or modified, and all such variations are considered to be within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the appended claims.

Claims

1. A method for managing a cache of a processing system, comprising: determining, by a cache controller of the cache, a reuse distance for a region of the cache, the reuse distance representing an average of at least a first number of accesses to a given cache set of the cache each time a given cache line of the given cache set is accessed and a second number of accesses to the given cache set each time the given cache line is accessed, wherein the first number is different from the second number; and The cache controller assigns a replacement priority level to each cache line in at least a subset of the cache lines of the region of the cache based on the reuse distance and a count of cache hits for the cache line.

2. The method of claim 1, further comprising: A cache line is selected for replacement in the region of the cache based on the replacement priority level assigned to the cache.

3. The method of claim 1 or claim 2, wherein determining the reuse distance comprises: In response to each cache hit to a cache set in at least a subset of the cache sets of the region of the cache: adding a set access count value associated with a way of the cache set containing the cache line that is the target of the cache hit to an accumulated value to generate an updated accumulated value; resetting the set access count value associated with the way of the cache set containing the cache line that is the target of the cache hit; as well as incrementing set access counts associated with other ways of the cache set; as well as In response to detecting a specified number of cache hits to the at least one subset of the cache set: averaging the updated accumulated value according to the specified number of cache hits to generate an average set access count value; as well as A reuse distance is determined based on the average set visit count value.

4. The method of claim 3, further comprising: In response to detecting the specified number of cache hits to the at least one subset of the cache set: scaling the average set access count value by a specified factor to generate a scaled average set access count value; and Wherein determining the reuse distance comprises determining the reuse distance based on the scaled average set visit count value.

5. The method of claim 3 , wherein assigning a replacement priority level to each cache line in at least a subset of the cache lines of the region of the cache comprises: assigning a first replacement priority level to the cache line in response to determining that the set access count value associated with the cache line is not greater than the reuse distance and in response to determining that a number of accesses to the cache line since insertion in the way is greater than zero; In response to determining that the set access count value associated with the cache line is not greater than the reuse distance and in response to determining that the number of accesses to the cache line since insertion in the way is equal to zero, assigning a second replacement priority level to the cache line; and The second replacement priority level indicates a greater possibility of replacement selection than the first replacement priority level.

6. The method of claim 5 , wherein assigning a replacement priority level to each cache line in at least a subset of the cache lines of the region of the cache further comprises: assigning a third replacement priority level to the cache line in response to determining that the set access count value associated with the cache line is greater than the reuse distance and in response to determining that a number of accesses to the cache line since insertion in the way is greater than zero; assigning a fourth replacement priority level to the cache line in response to determining that the set access count value associated with the cache line is greater than the reuse distance and in response to determining that the number of accesses to the cache line since insertion in the way is equal to zero; wherein the third replacement priority level represents a greater likelihood of replacement selection than the first replacement priority level; and The fourth replacement priority level indicates a greater possibility of replacement selection than the second replacement priority level.

7. The method of claim 6, further comprising: A cache line is selected for replacement in the region of the cache based on the replacement priority level assigned to the cache.

8. The method of claim 1, wherein the region of the cache comprises the entirety of the cache.

9. A processor comprising: a cache comprising a plurality of cache sets, each cache set having a plurality of ways configured to store corresponding cache lines; and a cache controller configured to implement a cache management policy for insertion and replacement of cache lines of the cache, the cache management policy providing for assigning a replacement priority level to each cache line in at least a subset of cache lines in the region of the cache based on a comparison of a number of accesses to a cache set having a way storing the cache line since the cache line was last accessed and a reuse distance determined for the region of the cache, the reuse distance representing an average of at least a first number of accesses to the region of the given cache set per access to the given cache line of the given cache set and a second number of accesses to the given cache set per access to the given cache line, wherein the first number is different from the second number.

10. The processor of claim 9, wherein the cache comprises: a plurality of set access counters, each set access counter being associated with a corresponding way of a cache set in at least a subset of cache sets of the region of the cache and configured to store a set access count value representing a number of accesses to the cache set since the cache line stored in the corresponding way was inserted or last accessed; an accumulator configured to accumulate the set access count value for each way from a cache set in the at least one subset of cache sets that is a target of a cache hit in an accumulated value; and An averaging / scaling component is configured to, in response to detecting a specified number of cache hits to the at least one subset of the cache set, determine the reuse distance based on averaging the accumulated value over the specified number of cache hits.

11. The processor of claim 10, wherein the cache further comprises: A plurality of row access counters, each row access counter being associated with a corresponding way of a cache set in the at least one subset of cache sets and configured to store a row access count value representing a number of accesses to the cache line stored in the corresponding way.

12. The processor of claim 11 , wherein the cache controller is configured to implement the cache management policy by: Assigning a replacement priority level to each cache line in at least a subset of cache lines of the region of the cache based on a comparison of the set access count value associated with the cache line and the reuse distance, and based on a determination whether the line access count value of the way storing the cache line is greater than or equal to zero.

13. The processor of claim 12 , wherein the cache controller is configured to assign replacement priority levels to cache lines of a cache set by: assigning a first replacement priority level to the cache line in response to determining that the set access count value associated with the cache line is not greater than the reuse distance and in response to determining that a line access count associated with the cache line is greater than zero; assigning a second replacement priority level to the cache line in response to determining that the set access count value associated with the cache line is not greater than the reuse distance and in response to determining that a line access count associated with the cache line is equal to zero; assigning a third replacement priority level to the cache line in response to determining that the set access count value associated with the cache line is greater than the reuse distance and in response to determining that the line access count associated with the cache line is greater than zero; assigning a fourth replacement priority level to the cache line in response to determining that the set access count value associated with the cache line is greater than the reuse distance and in response to determining that the line access count is equal to zero; wherein the second replacement priority level represents a greater likelihood of replacement selection than the first replacement priority level and the third replacement priority level; wherein the third replacement priority level represents a greater likelihood of replacement selection than the first replacement priority level; and The fourth replacement priority level indicates a greater possibility of replacement selection than the second replacement priority level.

14. The processor of claim 13 , wherein the cache controller is configured to implement the cache management policy by: A cache line of the cache set is selected for replacement based on a comparison of the replacement priority levels assigned to each cache line in the cache set.

15. The processor of claim 10, wherein: The specified number of cache hits is equal to 2^M, where M is an integer greater than 1; and The averaging / scaling component is configured to determine the reuse distance by right shifting the accumulated value by M bits.

Images