Cash management based on reuse period

Abstract

The cache [110] of the processor [102] includes a cache controller [112] for implementing a cache management policy for inserting and replacing cache lines in the cache. The cache management policy assigns a replacement priority level to each cache line of at least a subset of cache lines in a region of the cache based on a comparison of the number of accesses to a cache set [116] having a way storing the cache line since the cache line was last accessed and a reuse period determined for the region of the cache, the reuse period representing the average number of accesses to a given cache set of the region between accesses to a given cache line of the given cache set.

Description

Background Art

[0001] A processing system uses a cache to temporarily buffer data from a memory or a mass storage device for fast access. Since the cache has a limited storage capacity, generally, a cache management policy is used to indicate the selection of a cache line for replacement when the corresponding area of the cache becomes full. However, some conventional cache management policies, such as those based on the principle of least recently used (LRU) or predicted recurrence interval (RRIP), are not efficient in dealing with irregular accesses to cache lines, or may require relatively complex circuit implementations that limit their application.

[0002] By referring to the accompanying drawings, the present disclosure can be better understood, and many of its features and advantages will become apparent to those skilled in the art. When the same reference numerals are used in different drawings, similar or identical items are indicated.

Brief Description of the Drawings

[0003]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

[0004] FIGS. 1 - 7 illustrate a system and technique for implementing a cache management policy for a cache of a processing system based on a predicted "reuse period" for a corresponding region of the cache. As described herein, the reuse period of a corresponding region of the cache is a representation of the history of the average number of accesses to that corresponding region of the cache to a given cache line of a cache set. Next, this reuse period is used as a predictor or other indicator of the likely number of accesses to occur to a given cache set before that given cache line of that cache set is accessed again, and this information is used together with information regarding the most recent access history of the cache line to assign an exchange priority to the cache line. Next, according to the cache management policy, the exchange priority of the cache line thus assigned is used to select a cache line for replacement when the corresponding cache set is fully occupied. This reuse period - based approach to cache management can provide an accurate and effective prioritization for replacing cache lines that are likely to be used in a periodic pattern or streaming access pattern and in a relatively simple manner for implementation in hardware.

[0005] FIG. 1 is a diagram showing a processing system 100 that uses cache management based on a reuse period according to some embodiments. The processing system 100 includes a processor 102 coupled to a memory subsystem 104, and the memory subsystem 104 includes one or more system memories, scratchpad memories, disk drives, or other mass storage devices. The processor 102 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a digital signal processor (DSP), or a combination thereof. The processor 102 includes one or more execution pipelines 106 (e.g., CPU cores) and a cache hierarchy of one or more caches including a cache 108. The cache 108 includes a cache array 110 and a cache controller 112. The cache array 110 includes a plurality of entries 114 that store cache lines (i.e., blocks of temporary buffer data) for access by one or more execution pipelines 106. In at least one embodiment, the cache 108 is a set-associative cache, whereby cache line entries 114 are arranged in a plurality of cache sets 116, each cache set 116 having a plurality of ways, each way being a cache line entry 114 that operably stores a corresponding cache line, and any cache line related to a memory address mapped to the cache set 116 can be stored in any way of the cache set 116. In the illustrated example, the cache 108 implements four ways (way 0 to way 3), but more or fewer ways can be implemented. Each of the cache sets 116 includes additional fields (such as a tag field 118) for each way of the set, and the tag field 118 stores a part of a memory address, status bits, control bits, etc. related to a valid cache line stored in the corresponding way (if present).

[0006] The cache controller 112 operates to maintain various fields of the cache array 110 based on the activities of one or more execution pipelines 106, and its operations include receiving and storing data blocks as cache lines, accessing cache lines for use or modification by one or more execution pipelines 106, accessing cache lines for eviction or flushing to the memory subsystem 104, and the like. As part of this process, the cache controller 112 implements a cache management policy 120, and the cache management policy 120 controls the prioritization of cache lines for replacement or other eviction, and based on such prioritization, controls the selection of candidate cache lines for replacement or other eviction. In at least one embodiment, the cache management policy 120 utilizes a determined "reuse period" for a corresponding region of the cache 108 (which region can be a part (e.g., 1 / 4) or all of the cache 108) and the most recent history of accesses to cache lines within that corresponding region to determine the replacement priority of that cache line. Although the reuse period is determined for a part of the cache in this embodiment, the cache management policy 120 can be applied to the entire cache 108. As described above, at a high level, this reuse period represents the average number of accesses to the cache set 116 of the corresponding cache region between accesses to a particular cache line within the cache set 116. That is, the reuse period represents an estimate of the average number of cache accesses that are likely to occur to any given cache set of the corresponding cache region of the cache 108 before accessing the same cache line of the cache set again. For this purpose, the cache management policy 120 implements three stages. Each of the three stages (reuse period determination stage 122, replacement priority assignment stage 124, and cache line replacement stage 126) operates independently and simultaneously of the other phases.The reuse period determination stage 122 provides the calculation of the current (latest) reuse period for each applicable area of the cache 108, and will be described in detail below with reference to FIGS. 2 to 4. The replacement priority assignment stage 124 provides the assignment of replacement priorities to cache lines based on the current (latest) reuse period calculated in the current calculation cycle of stage 122, and will be described in further detail below with reference to FIGS. 5 and 6. The cache line replacement stage 126 provides the selection of cache lines for replacement based on the assignment of replacement priorities determined from the current prioritization cycle of stage 124, and will be described in further detail below with reference to FIG. 7.

[0007] In at least one embodiment, cache 108 uses counter set 128 to calculate the current reuse period for each iteration of stage 122 and to determine the replacement priority of cache lines in each cycle of stage 124. This counter set 128 includes a set access counter 130 and a line access counter 132, which are included for each way of cache array 110 (i.e., for each cache line), or alternatively, for each way of a subset of cache sets designated as representative cache sets of cache 108 for sampling purposes (e.g., for each Xth cache set of a region, where X is an integer greater than 1). The set access counter 130 stores a set access count value representing the number of accesses that have occurred to cache set 116 associated with the corresponding cache line since the cache line was inserted or last accessed. The line access counter 132 stores a line access count value representing the number of times the corresponding cache line has been accessed since it was inserted into cache 108 or since it was reset in response to the start of the next calculation cycle. In some embodiments, counter set 128 further includes an Nth access counter 134 that counts the number of accesses to the corresponding cache set 116 before being reset after the Nth counted access (and before triggering further operations as described below), where N represents a programmable integer or a specific integer greater than 1 (N>1).

[0008] FIG. 2 is a diagram showing an embodiment of a reuse period calculation component 200 implemented by a cache controller 112 and accessing a counter set 128 according to some embodiments. The reuse period calculation component 200 includes a set account component 202 for each cache set 116 that is used to calculate the reuse period of a corresponding area of the cache 108. This may include each of the cache sets 116 within the cache area, or a representative subset of the cache sets 116 of the cache area. In the illustrated example, X (X≧1) cache sets 116 are represented, and the set account components 202-0, 202-1, 202-3 are shown corresponding to cache set 0, cache set 1, and cache set X-1 of the represented cache sets 116. The reuse period calculation component 200 further includes an accumulator 204 and an averaging / scaling component 206. In one embodiment, the reuse period calculation component 200 includes a hit counter 207 that counts the number of cache hits to the corresponding area of the cache 108 for the current calculation cycle, and triggers the reuse period calculation when the number of cache hits reaches a programmably or otherwise specified value K. By way of example, the hit counter 207 can be implemented as a countdown counter that resets to K for each calculation cycle, decrements for each cache hit to a representative cache set, and triggers the reuse period calculation when the count value reaches 0.

[0009] The configuration of the set account component 202-3 of set X-1 is shown, representing the configuration of each of the set account components 202 with respect to the corresponding cache set 116. As shown, the set account component 202-3 includes a set of corresponding comparators 208, one for each way of the corresponding cache set 116, and a selection logic 214 (shown as a multiplexer for ease of explanation). Thus, for the illustrated example of cache 108 having a set with four ways, the set account component 202-3 includes four comparators 208. Each comparator 208 includes an input coupled to receive an address value from the tag field of the corresponding way of the set (e.g., one of the tag fields 118-0 to 118-3 for ways 0 to 4, respectively), and an input coupled to receive an address value from the tag field 210 of the cache probe 212 sent from the execution pipeline 106 to the cache 108. Further, each comparator 208 has an output that is asserted when the address value from the tag field 118 of the corresponding way matches (equals) the address value of the tag field 210 of the cache probe 212. That is, the comparator 208 associated with the way of the cache set 116 that is the target of the cache probe 212 (i.e., provides a cache "hit" to the cache probe 212) asserts its output, while the other comparators 208 of the set account component 202-3 remain unasserted. In this way, the output of the comparator 208 identifies the way that includes the cache line having an address that matches the address represented in the cache probe 212.

[0010] As described above, the counter set 128 includes a set access counter 130 for each way of the representative cache set 116 used for reuse period calculation. Therefore, for set X-1 related to the illustrated set account component 202-3, the counter set 128 includes four set access counters 130-0, 130-1, 130-2, and 130-3 for ways 0, 1, 2, and 3 respectively. As will be described in more detail below, each of the set access counters 130-0 to 130-3 stores a set access count value representing the number of accesses to set X-1 since the cache line of the corresponding way was inserted or last accessed.

[0011] The selection logic 214 includes a plurality of selection inputs, and each of the selection inputs is coupled to receive a current selected access count value of a corresponding one of the set access counters 130 of the cache set 116. Therefore, in the four-way cache set 116, the selection logic 214 has four selection inputs (i.e., a selection input for receiving a count value from the set access counter 130-0, a selection input for receiving a count value from the set access counter 130-1, a selection input for receiving a count value from the set access counter 130-2, and a selection input for receiving a count value from the set access counter 130-3). Further, the selection logic 214 includes a selection control input coupled to the output of the comparator 208 and an output coupled to the accumulator 204. Therefore, the selection logic 214 operates to select one of the input selected access count values from the set access counters 130-0 to 130-3 for output to the accumulator 204, and based on that operation, the comparator 208 has an asserted output if present. That is, the way of the cache set 116 having a tag address that matches the tag address of the cache probe 212 triggers the selection logic 214 to output the counter value of the set access counter 130 related to that way to the accumulator 204.

[0012] The accumulator 204 operates to accumulate the set access counter values received from various set account components 202 and provide the resulting updated cumulative value to the averaging / scaling component 206. In response to a trigger event (e.g., an event that occurs every Kth access to the representative cache set of a region), the averaging / scaling component 206 operates to average the current updated cumulative value for the number of accesses to the representative cache set of the region since the last calculation cycle to generate an average set access counter value from the cumulative value. The average set access counter value can be obtained, for example, by a series of shift operations, and in some embodiments, the averaging / scaling component 206 scales the resulting average set access counter value using a specified magnification factor. Next, the resulting averaged / scaled set access counter value functions as the current reuse period 216 for the corresponding region of the cache 108.

[0013] Figure 3 is a diagram that more particularly illustrates a method 300 showing a reuse period calculation process used by the reuse period calculation component 200 of FIG. 2 of the cache controller 112, according to some embodiments. In block 302, the cache controller 112 monitors the operation of the cache 108 to determine whether a cache line is inserted into one of the representative cache sets 116. In block 304, the cache controller 112 monitors the operation of the cache 108 to determine whether a cache line in one of the representative cache sets 116 is being accessed. If a cache line is being accessed, in block 306, the value of the set access counter 130 is associated with the way of the cache set 116 that stores the accessed cache line, which is sent to the accumulator 204. When a cache line is inserted or accessed, in block 308, the cache controller 112 resets the set access counter 130 associated with the way of the cache set 116 that stores the inserted or accessed cache line to a predetermined value (e.g., 0), and in block 310, the cache controller 112 increments the set access counters 130 associated with all other ways of the cache set 116. That is, the insertion of a cache line or the access to a cache line in a way of the representative cache set 116 resets the set access counter 130 of that way of the cache set 116 and at the same time increments the set access counters 130 of all other ways of the cache set 116. Alternatively, the cache controller 112 increments the set access counter 130 associated with each way of the cache set 116 and then resets the set access counter 130 associated with the way of the cache set 116 that stores the inserted or accessed cache line to a predetermined value. Thus, in this way, the cache controller 112 monitors the number of accesses made to the representative cache set 116 for each cache line of the representative cache set 116 since the cache line was inserted or last accessed.

[0014] Further, in block 312, cache controller 112 monitors for cache hits to the ways of representative cache set 116 (which may include the accesses referenced in block 302). In at least one embodiment, a cache hit to the ways of representative cache set 116 is signaled via comparator 208 of set account component 202 associated with representative cache set 116, whereupon, when the address represented in the cache probe (e.g., cache probe 212 of FIG. 2) matches the address of the tag field 118 of the corresponding way, the output of the corresponding comparator 208 is asserted, and in turn, signals that the cache probe has hit in that way of representative cache set 116. In response to such a cache hit, in block 314, selection logic 214 of set account component 202 associated with hit cache set 116 outputs the value of set access counter 130 associated with the hit way of hit cache set 116 to accumulator 204, and immediately thereafter, accumulator 204 adds the input value to the previous accumulated value for the current calculation cycle.

[0015] The processes of blocks 312, 314 are repeated for each cache hit to the representative cache set 116 in the current computational iteration until the Kth cache hit to the corresponding region of cache 108 is detected (e.g., by hit counter 207), where K is an integer value greater than 1 that is programmable or otherwise specified (K>1). For example, K can be set to 64, such that the set count accumulation process continues until the 64th cache hit to the corresponding region of cache 108 occurs. In block 316, in response to determining that the Kth cache hit to the region of cache 108 has occurred for the current computational cycle, in block 318, the averaging / scaling component averages the current cumulative value (accumulated value) from accumulator 204 for the K cache hits utilized for the current computational cycle, and in some embodiments, scales the average value by multiplying the average value by a coefficient. In embodiments where K and any multiplier are powers of 2, the averaging / scaling component 206 can be implemented as left and right shift logic. By way of illustration, if K is set to 64 (2 6 ) as in the previous example, the averaging of the cumulative value can be performed by shifting the cumulative value 6 bits to the right and then 1 bit to the left. More generally, if K is 2 MWhen equal to a positive integer M, the averaging / scaling component 206 can obtain the average set access count for K cache accesses to the representative cache set by shifting the current update accumulation value left by M bit positions and then shifting it right by one bit position. Similarly, scaling the average value by 2 can be achieved by shifting the resulting average value left by one bit. Next, the resulting average value (and scale value) is set as the current (latest) reuse period 216 for the corresponding region of the cache 108. In the scaling of the average value, there are differences in the set access count values for the individual ways of the set, and as a result, there may be a reuse period lower than the set access count value for a part of the lines of the set. As will be explained in more detail below, when the set access count value of a given line exceeds the reuse period of the cache, that line is likely to be evicted. Using the scaling of the average value can prevent some lines of the set from being preferentially replaced incorrectly. When the current calculation cycle ends, in block 320, the cache controller 112 resets various components (hit counter 207, set access counter 130, line access counter 132, accumulator 204, etc.) used in the ended calculation cycle, and then the process returns to block 312 for the next calculation cycle.

[0016] FIG. 4 is a diagram showing an alternative embodiment of the set account component 202 (FIG. 2) implemented by the cache controller 112 for each representative cache set 116, according to at least one embodiment. Similar to the embodiment of FIG. 2, the alternative embodiment of the set account component 202 in FIG. 4 includes a comparator 208 operable to assert its respective output in response to the address of the tag field 118 of the corresponding way matching the address of the tag field 210 of the received cache probe 212, and further includes selection logic 414 (shown as a multiplexer for ease of explanation) that utilizes the output of the comparator 208 as its selection control input. However, instead of using a counter for the set access counter 130 that is large enough to account for all accesses to the corresponding cache set during a computational cycle, in the embodiment of FIG. 4, a smaller counter is used for the set access counter 130 by incrementing the set access counter 130 for every Nth access to the cache set 116. To facilitate counting accesses by N, the illustrated embodiment of the set account component 202 further includes an Nth access count 404, which is incremented (or reset to N and decremented for each access) for each access to the cache set 116 during the current computational cycle. Further, to compensate for the fact that the increment of the set access counter 130 for a cache set is triggered only for every Nth access to the cache set 116, in the illustrated embodiment, a shift register 406 and an adder 408 are used between each output of the set access counter 130 and the corresponding input to the selection logic 414 to adjust this sampled cache access approach. Thus, when N is a power of two (2 j) Assuming that it is so, the set access count value output from the set access counter 130 is shifted left by j bits, and then added to the current hit count represented by the Nth access count 404, and the resulting value is fed to the corresponding input of the selection logic 414. Next, the selection logic 414 is operable to select one of the input values for output to the accumulator 204 based on the output state of the comparator 208, and further represents whether any way of the representative cache set 116 is the target of the Nth cache probe 212 if it exists.

[0017] Figures 5 and 6 together show the operation of cache controller 112 for implementing cache management policy 120 during exchange priority assignment stage 124 (FIG. 1) according to some embodiments. FIG. 5 shows a method 500 for maintaining line access counters 132 of representative cache set 116 during a prioritization cycle according to some embodiments. As described above, each of the line access counters 132 represents the number of times the cache line stored in the way associated with the line access counter 132 has been accessed since the current prioritization cycle started. Thus, at the start of the prioritization cycle, in block 502, cache controller 112 monitors the insertion of cache lines into the ways of representative cache set 116. In response to such an insertion, in block 504, cache controller 112 resets the line access counter 132 associated with the target way of representative cache set 116. Thereafter, in block 506, cache controller 112 monitors accesses to the cache lines. In response to detection of an access, in block 508, cache controller 112 increments the line access counter 132 of the accessed cache line. In some embodiments, the line access counter 132 is implemented as a 1-bit saturation counter or a 2-bit saturation counter to reduce the hardware requirements of the line access counter 132, and thus count either up to 1 access (in the case of the 1-bit counter embodiment) or up to 3 accesses (in the case of the 2-bit counter embodiment). In other embodiments, more than 3 bits are utilized for the line access counter 132 to facilitate counting a large number of accesses to any given cache line.

[0018] Referring to FIG. 6, a method 600 for assigning replacement priorities to cache lines, according to some embodiments, is shown based on the current line access count of the corresponding line access counter 132 and based on the current reuse period. The method 600 begins (block 602) upon an event such as a determination that a cache line is to be replaced, determines the replacement priority of a cache line newly inserted during the current prioritization cycle, which serves as a trigger to re-determine the replacement priority of a previously existing cache line. In response to the trigger, the prioritization process begins, in block 604, by selecting cache lines in the region of the cache 108 according to a selection sequence (e.g., straight order selection, pseudo-random selection, etc.).

[0019] In block 606, cache controller 112 accesses set access counter 130 that stores the selected cache line and compares the count contained therein with the current reuse period 216. If set access counter 130 is not greater than the current reuse period 216, this indicates that the cache line has not yet reached the reuse period and can thus be reused in the future. Therefore, if set access counter 130 is less than the current reuse period, in block 608, cache controller 112 accesses line access counter 132 associated with the way that stores the selected cache line to determine whether the value stored therein is greater than zero (i.e., to determine whether the cache line has been reused since it was inserted). In such a case, considering that some data tends to be repeatedly accessed, there is a possibility that this cache line will be reused again. Therefore, in block 608, if line access counter 132 is determined to be greater than zero, in block 610, replacement priority level 3 is assigned to the cache line (in the following description, it is assumed that as the replacement priority level value decreases, the corresponding cache line becomes less suitable as a replacement candidate, and thus the replacement selection possibility of the corresponding cache line increases). Otherwise, if line access counter 132 is equal to zero, this indicates that the cache line has not been reused. However, since the cache line has not yet reached the reuse period, there is still a possibility that the cache line will be reused in the future. In such a case, in block 612, replacement priority level 1 is assigned to the cache line (priority level 1 indicates a greater replacement selection possibility than replacement priority level 3).

[0020] Returning to block 606, if it is determined that the access count represented by the set access counter 130 is greater than the current reuse period, this means that the cache line has already reached the reuse period but has not been reused thereafter. Thus, in block 614, the cache controller 112 determines whether the cache line has been accessed by accessing the line access counter 132 of the way that stores the cache line. If the line access counter 132 is greater than zero, this means that the cache line has been reused at least once since it was inserted into the cache 108 and, thus, may be reused again. However, the likelihood of reuse is suppressed by the fact that it has already passed the reuse period. Thus, if the line access counter 132 is greater than zero, in block 616, the selected cache line is assigned an exchange priority level 2 (the exchange priority level 2 indicates that there is a greater likelihood of being selected for replacement than the exchange priority level 3 and a lower likelihood than the exchange priority level 1). Otherwise, if the count is equal to zero, this means that the cache line has not been reused since it was inserted, has already passed the reuse period, and, thus, is unlikely to be reused in the future. In such a case, in block 618, the cache line is assigned an exchange priority level 0 (in this example, the exchange priority level 0 represents the highest likelihood of being selected for replacement).

[0021] FIG. 7 is a diagram showing a method 700 of operation of a cache controller 112 for a cache line replacement stage 126 (FIG. 1) of a cache management policy 120 according to some embodiments. In the following description, for the sake of explanation, it is to be recalled that an increase in the numerical value of the replacement priority level corresponds to an increase in the priority of holding the corresponding cache line, and conversely, a decrease in the numerical value of the replacement priority level corresponds to an increase in the priority or likelihood of evicting the corresponding cache line. In block 702, a load operation or a store operation is performed, resulting in the generation of a data block to be stored as a cache line of cache 108. Thus, further, in block 702, cache controller 112 determines a cache set 116 available for storing the cache line based on the address associated with the cache line. In block 704, cache controller 112 determines whether there is an available way in the identified cache set 116 (i.e., determines whether there is a way that does not currently store a valid cache line). In such a case, in block 706, cache controller 112 inserts the cache line into the available way of cache set 116. As described above with reference to block 302 of FIG. 3 and block 502 of FIG. 5, if cache set 116 is a representative cache set, the insertion of this cache line into cache set 116 triggers a specific counting operation.

[0022] Instead, if the ways of the identified cache set 116 are not available, the cache controller 112 determines whether to evict the current cache line of the cache set 116 or bypass the caching of the cache line based on the replacement priority level assigned during the replacement priority assignment stage 124 of the cache management policy 120 as described above. Thus, at block 708, the cache controller 112 determines whether cache bypass (also known as "selective caching" or "cache deletion") is supported by the cache 108. If cache bypass is supported, at block 710, the cache controller 112 determines whether the cache set 116 contains a cache line assigned to replacement priority level 0. If not supported, at block 712, the cache controller 112 can select to bypass the caching of the cache lines of the cache 108 (e.g., by preventing the caching of the cache lines or by providing the cache lines to a lower-level cache for storage). In another embodiment, if it is determined that the cache line is part of a streaming process (e.g., if the current reuse period is small (0 or 1)), similarly, if there is no cache line at replacement priority level 1, cache bypass can be selected.

[0023] If cache bypass is not supported, or if there are no cache lines at a priority level low enough to justify cache bypass, in block 714, cache controller 112 selects a cache line of a cache set 116 having the lowest replacement priority level as a replacement candidate. If there are two or more cache lines at the same lowest replacement priority level, cache controller 112 can randomly select one of the cache lines pseudo-randomly based on a specified selection order, or can select the way having the largest set access count. In block 716, cache controller 112 replaces or evicts the selected candidate cache line with a new cache line of the corresponding way of cache set 116. This action is a cache line insertion and triggers a specific counting operation as described above with reference to block 302 of FIG. 3 and block 502 of FIG. 5.

[0024] In some embodiments, some aspects of the above techniques may be implemented by one or more processors of a processing system that executes software. The software may be stored in a non-transitory computer-readable storage medium or may include one or more sets of executable instructions tangibly embodied on a non-transitory computer-readable storage medium. The software, when executed by one or more processors, can include instructions and specific data for operating one or more processors to perform one or more aspects of the above techniques. The non-transitory computer-readable storage medium can include, for example, magnetic or optical disk storage devices, such as solid state storage devices like flash memory, cache, random access memory (RAM), or one or more other non-volatile memory devices. The executable instructions stored on the non-transitory computer-readable storage medium can be in source code, assembly language code, object code, or other instruction formats interpretable or executable by one or more processors.

[0025] According to one aspect, a method for managing a cache of a processing system includes determining, by a cache controller of the cache, a reuse period of a region of the cache, the reuse period representing an average number of accesses to the cache set of the cache between accesses to a predetermined cache line of a predetermined cache set. The method further includes assigning, by the cache controller, an exchange priority level to each cache line of at least a subset of the cache lines of the region of the cache based on the reuse period and a count of cache hits to the cache lines.

[0026] According to another aspect, a method for managing a cache of a processing system includes implementing, by a cache controller of the cache, a cache management policy for inserting and replacing cache lines of the cache, the cache management policy providing an assignment of an exchange priority level to each cache line of at least a subset of the cache lines within the region of the cache based on a comparison of the number of accesses to a cache set having a way of storing a cache line since the cache line was last accessed and a determined reuse period of the region of the cache, the reuse period representing an average number of accesses to a predetermined cache set of the region between accesses to any predetermined cache line of the cache set.

[0027] According to yet another aspect, the processor includes a cache including a plurality of cache sets, each cache set having a plurality of ways configured to store corresponding cache lines. The processor further includes a cache controller configured to implement a cache management policy for inserting and replacing cache lines of the cache, the cache management policy providing an assignment of replacement priority levels to each cache line of at least a subset of the cache lines within the cache region based on a comparison of the number of accesses to a cache set having a way storing the cache line since the cache line was last accessed and a reuse period determined for the region of the cache, the reuse period representing an average number of accesses to a given cache set of the region between accesses to any given cache line of the cache set.

[0028] A computer-readable storage medium includes any non-transitory storage medium or combination of non-transitory storage media that is accessible by a computer system during use to provide instructions and / or data to the computer system. Such storage media include, but are not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-ray (registered trademark) disc), magnetic media (e.g., floppy (registered trademark) disc, magnetic tape, magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS)-based storage media. A computer-readable storage medium (e.g., system RAM or ROM) may be built into a computing system, a computer-readable storage medium (e.g., magnetic hard drive) may be fixedly attached to a computing system, a computer-readable storage medium (e.g., optical disc or universal serial bus (USB)-based flash memory) may be removably attached to a computing system, or a computer-readable storage medium (e.g., network-accessible storage (NAS)) may be coupled to a computer system via a wired or wireless network.

[0029] In addition to what has been described above, not all of the activities or elements described in the general description are required, and in some cases, some activities or parts of a device may not be required, and one or more additional activities may be performed, and one or more additional elements may be included. It should be noted that the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will understand that various changes and modifications can be made without departing from the scope of the invention as set forth in the claims. Therefore, the specification and drawings should be considered in an illustrative rather than a limiting sense, and all such modifications are intended to be included within the scope of the invention.

[0030] Advantages, other benefits, and solutions to problems have been described above with respect to specific embodiments. However, advantages, benefits, solutions to problems, and any features that may give rise to or manifest any advantage, benefit, or solution are not to be construed as important, essential, or indispensable features of any or all of the claims. Further, since the disclosed invention can be modified and practiced in a manner that will be apparent to those skilled in the art having the benefit of the teachings herein, but in a different yet similar manner, the specific embodiments described above are merely illustrative. There is no limitation as to the details of the construction or design shown herein other than as set forth in the appended claims. Therefore, it is clear that the specific embodiments described above may be changed or modified, and all such modifications are considered to be within the scope of the disclosed invention. Accordingly, the protection sought here is set forth in the appended claims.

Claims

1. A method for managing a cache of a processing system, wherein a cache controller of the cache determines a reuse period of a region of the cache, the reuse period representing an average number of accesses to the cache to a predetermined cache set of the cache between accesses to a predetermined cache line of the predetermined cache set, and determining the reuse period includes adding a set access count value associated with a way of the cache set including the cache line that is a target of the cache hit to a cumulative value in response to each cache hit in at least a subset of cache sets of the region of the cache to generate an updated cumulative value, and the cache controller assigns an exchange priority level to each cache line of at least a subset of the cache lines of the region of the cache based on the reuse period and a count of cache hits to the cache line. A method.

2. further comprising selecting a cache line for exchange in the region of the cache based on the exchange priority level assigned to the cache. The method of Claim 1.

3. Determining the reuse period includes in response to each cache hit in at least a subset of cache sets of the region of the cache, resetting the set access count value associated with the way of the cache set including the cache line that is a target of the cache hit, incrementing the set access count value associated with another way of the cache set, and in response to detecting a predetermined number of cache hits to at least a subset of the cache sets, Averaging the update cumulative value by the predetermined number of cache hits to generate an average set access count value; determining a reuse period based on the average set access count value, comprising: The method according to claim 1 or 2.

4. in response to detecting a predetermined number of cache hits to at least a subset of the cache set; further comprising scaling the average set access count value by a predetermined factor to generate a scaled average set access count value; wherein determining the reuse period includes determining the reuse period based on the scaled average set access count value. The method according to claim 3.

5. Assigning an exchange priority level to each cache line of at least a subset of the cache lines in the region of the cache includes: assigning a first exchange priority level to the cache line in response to determining that the set access count value associated with the cache line is not greater than the reuse period and in response to determining that the number of accesses to the cache line since being inserted into the way is greater than zero; assigning a second exchange priority level to the cache line in response to determining that the set access count value associated with the cache line is not greater than the reuse period and in response to determining that the number of accesses to the cache line since being inserted into the way is equal to zero, including: wherein the second exchange priority level indicates a greater likelihood of being selected for replacement than the first exchange priority level. The method according to claim 3.

6. Assigning an exchange priority level to each cache line of at least a subset of the cache lines in the region of the cache includes: In response to determining that the set access count value associated with the cache line is greater than the reuse period, and in response to determining that the number of accesses to the cache line since being inserted into the way is greater than zero, assigning a third replacement priority level to the cache line; further comprising, in response to determining that the set access count value associated with the cache line is greater than the reuse period, and in response to determining that the number of accesses to the cache line since being inserted into the way is equal to zero, assigning a fourth replacement priority level to the cache line; The third replacement priority level indicates that it is more likely to be selected for replacement than the first replacement priority level; The fourth replacement priority level indicates that it is more likely to be selected for replacement than the second replacement priority level. The method of claim 5.

7. further comprising selecting a cache line for replacement in the region of the cache based on the replacement priority level assigned to the cache; The method of claim 6.

8. The region of the cache includes the entire cache; The method of claim 1.

9. A cache including a plurality of cache sets, each cache set having a plurality of ways configured to store corresponding cache lines; a cache controller configured to implement a cache management policy for inserting and replacing cache lines of the cache. The cache management policy provides an assignment of replacement priority levels to each cache line of at least a subset of the cache lines within the cache region based on a comparison between the number of accesses to a cache set having a way of storing the cache line since the cache line was last accessed, and the reuse period determined for the cache region. The reuse period represents the average number of accesses to the predetermined cache set between accesses to a predetermined cache line of the predetermined cache set of the region. The cache comprises a plurality of set access counters, each set access counter being associated with a corresponding way of a cache set of at least a subset of the cache sets of the cache region, and configured to store a set access count value representing the number of accesses to the cache set since the cache line stored in the corresponding way was inserted or last accessed. A processor.

10. The cache is an accumulator configured to accumulate the set access count values from each way of the cache sets in at least a subset of the cache sets that are targets of cache hits into a cumulative value, and an averaging / scaling component configured to determine the reuse period based on an average of the cumulative value by the predetermined number of cache hits, in response to detecting the predetermined number of cache hits to at least a subset of the cache sets. The processor of claim 9.

11. The cache is Comprising a plurality of line access counters, each line access counter being associated with a corresponding way of a cache set of at least a subset of the cache sets, and configured to store a line access count value representing the number of accesses to the cache line stored in the corresponding way. The processor of claim 10.

12. The cache controller Based on a comparison of the set access count value associated with the cache line and the reuse period, and based on a determination of whether the line access count value of the way storing the cache line is zero or more, by assigning an exchange priority level to each cache line of at least a subset of the cache lines in the region of the cache, it is configured to implement the cache management policy. The processor of claim 11.

13. The cache controller In response to determining that the set access count value associated with the cache line is not greater than the reuse period, and in response to determining that the line access count associated with the cache line is greater than zero, assigning a first exchange priority level to the cache line; In response to determining that the set access count value associated with the cache line is not greater than the reuse period, and in response to determining that the line access count associated with the cache line is equal to zero, assigning a second exchange priority level to the cache line; In response to determining that the set access count value associated with the cache line is greater than the reuse period, and in response to determining that the line access count associated with the cache line is greater than zero, assigning a third exchange priority level to the cache line; In response to determining that the set access count value associated with the cache line is greater than the reuse period, and in response to determining that the line access count is equal to zero, assign a fourth replacement priority level to the cache line. is configured to assign a replacement priority level to a cache line of a cache set. The second replacement priority level indicates that it is more likely to be selected for replacement than the first replacement priority level and the third replacement priority level. The third replacement priority level indicates that it is more likely to be selected for replacement than the first replacement priority level. The fourth replacement priority level indicates that it is more likely to be selected for replacement than the second replacement priority level. The processor of claim 12.

14. The cache controller is configured to implement the cache management policy by selecting a cache line of a cache set for replacement based on a comparison of the replacement priority levels assigned to each cache line of the cache set. The processor of claim 13.

15. The predetermined number of the cache hits is 2 Ｍ is equal to, M is an integer greater than 1, The averaging / scaling component is configured to determine the reuse period by shifting the cumulative value to the right by M bit positions. The processor of claim 10.

Images