Cache management based on reuse distance

Abstract

The cache (110) of the processor (102) includes a cache controller (112) that implements a cache management policy for inserting and replacing cache lines in the cache. The cache management policy assigns replacement priority levels to each cache line of at least a subset of cache lines within the cache area based on a comparison of the number of accesses to a cache set (116) having a way to store the cache line since the cache line was last accessed and a reuse distance determined for the area of ​​the cache, wherein the reuse distance represents the average number of accesses to a predetermined cache set of the area between any predetermined cache line of the cache set.

Description

Background Technology

[0001] Processing systems employ caches to temporarily buffer data from memory or mass storage devices for fast access. Since caches have limited storage capacity, cache management policies are typically employed to direct the selection of cache lines for replacement when a corresponding area of ​​the cache becomes full. However, some conventional cache management policies, such as those based on least recently used (LRU) or re-reference interval prediction (RRIP) principles, require relatively complex circuit implementations that are less efficient when handling irregular accesses to cache lines or may limit their applicability. Brief explanation of the drawing

[0002] The present disclosure may be better understood with reference to the accompanying drawings, and many of its features and advantages will become apparent to those skilled in the art. The use of the same reference numerals in different drawings indicates similar or identical items. FIG. 1 is a block diagram of a processing system having a cache that implements a reuse distance-based cache management policy according to some embodiments. FIG. 2 is a block diagram showing a reuse distance calculation component of FIG. 1 according to some embodiments. FIG. 3 is a flowchart illustrating a method for calculating the current reuse distance of corresponding regions of the caches of FIG. 1 and FIG. 2 according to some embodiments. FIG. 4 is a block diagram showing an alternative implementation of a reuse distance calculation component according to some embodiments. FIG. 5 is a flowchart illustrating a method for maintaining a line access counter for a corresponding cache line according to some embodiments. FIG. 6 is a flowchart illustrating a method for assigning an alternative priority level to a cache line based on the current reuse distance and the corresponding line access count value according to some embodiments. FIG. 7 is a flowchart illustrating a method for selecting a cache line for replacement using reuse distance-based replacement priorities according to some embodiments. Specific details for implementing the invention

[0003] FIGS. 1 through 7 illustrate systems and techniques for implementing a cache management policy for a cache of a processing system based on a predicted "reuse distance" for a corresponding region of the cache. As described herein, the reuse distance for a corresponding region of the cache is an expression of the historical average number of accesses to a specific cache set of a region of the cache between accesses to a specific cache line of that cache set. This reuse distance is then used as a predictor or other indicator of how many accesses are likely to occur to a specific cache set before a specific cache line within that cache set is accessed again, and this information, along with information regarding the recent access history of the cache line, is used to assign replacement priorities to the cache line. Subsequently, the replacement priorities of the cache lines assigned in this manner are used according to the cache management policy to select cache lines for replacement when the corresponding cache sets are fully occupied. This reuse distance-based approach to cache management can provide accurate and effective prioritization in a relatively simple manner for the replacement of cache lines likely to be used in cycle or streaming access patterns, and for implementation in hardware.

[0004] FIG. 1 illustrates a processing system (100) employing reuse distance-based cache management according to some embodiments. The processing system (100) includes a processor (102) coupled to a memory subsystem (104), the memory subsystem (104) including one or more system memory, scratchpad memory, disk drive, or other mass storage device. The processor (102) is, for example, a central processing unit (CPU), a graphics processing unit (GPU), an acceleration processing unit (APU), a digital signal processor (DSP), or a combination thereof. The processor (102) includes one or more execution pipelines (106) (e.g., CPU cores) and a cache hierarchy of one or more caches including a cache (108). The cache (108) includes a cache array (110) and a cache controller (112). The cache array (110) includes a plurality of entries (114) for storing cache lines (i.e., blocks of temporarily buffered data) for access by one or more execution pipelines (106). In at least one embodiment, the cache (108) is a set associative cache such that cache line entries (114) are arranged into a plurality of cache sets (116)—each cache set (116) has a plurality of ways, and each way is a cache line entry (114) operable to store a corresponding cache line—and any cache line associated with a memory address mapped to the cache set (116) can be stored in any way of the cache set (116). In the illustrated example, the cache (108) implements four ways (ways 0 through 3), but more or fewer ways may be implemented. Each cache set (116) includes additional fields such as a tag field (118) for each way of the set, wherein the tag field (118) stores a portion of the memory address associated with an effective cache line stored in the corresponding way (if any), status bits, control bits, etc.

[0005] The cache controller (112) operates to maintain various fields of the cache array (110) based on activities of one or more execution pipelines (106), including receiving and storing blocks of data as cache lines, accessing cache lines for use or modification by one or more execution pipelines (106), and accessing cache lines for evicting or flushing of the memory subsystem (104). As part of this process, the cache controller (112) implements a cache management policy (120) that controls the prioritization of cache lines for replacement or other evicting and controls the selection of candidate cache lines for replacement or other evicting based on such prioritization. In at least one embodiment, the cache management policy (120) uses the recent history of accesses to a cache line to determine a "reuse distance" determined for a corresponding area of ​​the cache (108) (this area may be a part of the cache (108) (e.g., 1 / 4) or the entire cache (108)) and a replacement priority for a cache line within that corresponding area. In this embodiment, the reuse distance is determined for a part of the cache, but the cache management policy (120) may also be applied to the entire cache (108). As described above, at a higher level, this reuse distance represents the average number of accesses to a cache set (116) within a corresponding cache area between accesses to a specific cache line within a cache set (116). That is, the reuse distance represents a prediction of the average number of cache accesses likely to occur in a cache set before a specific cache line of any given cache set in the corresponding area of ​​the cache (108) is accessed again. To this end, the cache management policy (120) implements three stages, each of which operates independently and simultaneously with the other stages: the reuse distance determination stage (122), the replacement priority assignment stage (124), and the cache line replacement stage (126).The reuse distance determination stage (122) provides for the calculation of the current reuse distance for each applicable area of ​​the cache (108) and is described in more detail below with reference to FIGS. 2 through 4. The replacement priority assignment stage (124) provides for the assignment of replacement priorities to cache lines based on the current reuse distance calculated in the most recent calculation cycle of stage (122) and is described in more detail below with reference to FIGS. 5 and 6. The cache line replacement stage (126) provides for the selection of cache lines for replacement based on the assigned replacement priorities determined from the most recent priority cycle of stage (124) and is described in more detail below with reference to FIG. 7.

[0006] In at least one embodiment, the cache (108) employs a counter set (128) to be used to calculate the current reuse distance in each iteration of stage (122) and to determine alternative priorities for cache lines in each cycle of stage (124). This counter set (128) includes a set access counter (130) and a line access count (132) for each way of the cache array (110) (i.e., for each cache line), or alternatively, for each way of a subset of cache sets designated as representative cache sets of the cache (108) for sampling purposes (e.g., for every Xth cache set of the area, where X is an integer greater than 1). The set access counter (130) stores a set access count value indicating the number of times access to the cache set (116) associated with the corresponding cache line has occurred since the cache line was inserted or last accessed. A line access counter (132) stores a line access count value indicating the number of times a corresponding cache line has been accessed since it was inserted into the cache (108) or since it was reset in response to the start of the next computation cycle. In some embodiments, the counter set (128) further includes an Nth access counter (134) that counts the number of accesses to a corresponding cache set (116) before being reset after the Nth counted access (and before triggering additional operations as described below), where N represents a programmable or otherwise specified integer greater than 1 (N > 1).

[0007] FIG. 2 illustrates an implementation of a reuse distance calculation component (200) that is implemented by a cache controller (112) according to some embodiments and accesses a counter set (128). The reuse distance calculation component (200) includes a set accounting component (202) for each cache set (116) used to calculate the reuse distance for a corresponding area of ​​the cache (108). This may include each cache set (116) within the cache area, or a representative subset of the cache sets (116) in the cache area. In the illustrated example, X cache sets (116) are shown (X >= 1), and set accounting components (202-0, 202-1, and 202-3) are illustrated for cache set 0, cache set 1, and cache set X-1 among the shown cache sets (116). The reuse distance calculation component (200) further includes an accumulator (204) and an averaging / scaling component (206). In one embodiment, the reuse distance calculation component (200) also includes a hit counter (207) which counts the number of cache hits for a corresponding area of ​​the cache (108) during the current calculation cycle and triggers the reuse distance calculation when the number of cache hits reaches a programmable or otherwise specified value K. For example, the hit counter (207) may be implemented as a countdown counter that is reset to K for every calculation cycle, decreases for each cache hit for a representative cache set, and triggers the reuse distance calculation when it reaches 0.

[0008] The configuration of the set accounting component (202-3) for set X-1 is illustrated, which represents the configuration of each set accounting component (202) for the corresponding cache set (116). As illustrated, the set accounting component (202-3) includes a set of comparators (208)—each for each way of the corresponding cache set (116)—and selection logic (214) (illustrated as a multiplexer for ease of illustration). Accordingly, for the illustrated example of a cache (108) having sets having four ways, the set accounting component (202-3) includes four comparators (208). Each comparator (208) includes a combined input for receiving an address value from a tag field of a corresponding way of a set (e.g., tag fields (118-0 to 118-3) are each for ways 0 to 4), and an input for receiving an address value from a tag field (210) of a cache probe (212) submitted to the cache (108) from the execution pipeline (106). Each comparator (208) further has an output that is asserted when an address value from a tag field (118) of a corresponding way matches an address value from a tag field (210) of a cache probe (212); That is, the comparator (208) associated with the way of the cache set (116) that is the target of the cache probe (212) (i.e., providing a cache "hit" for the cache probe (212)) asserts its output, while other comparators (208) of the set accounting component (202-3) remain unasserted. In this way, the outputs of the comparators (208) identify a way containing a cache line having an address that matches the address appearing in the cache probe (212).

[0009] As described above, the counter set (128) includes a set access counter (130) for each way of each representative cache set (116) used for calculating the reuse distance. Accordingly, for a set X-1 associated with the illustrated set accounting component (202-3), the counter set (128) includes four set access counters (130-0, 130-1, 130-2, and 130-3) for ways 0, 1, 2, and 3, respectively. Each of the set access counters (130-0 to 130-3) stores a set access count value indicating the number of accesses to set X-1 since the cache line of the corresponding way was inserted or last accessed, as described in more detail below.

[0010] The selection logic (214) includes a plurality of selection inputs, each coupled to receive the current selection access count value of a corresponding set access counter (130) for the cache set (116). Accordingly, in a 4-way cache set (116), the selection logic (214) has four selection inputs, one for receiving a count value from the set access counter (130-0), one for receiving a count value from the set access counter (130-1), one for receiving a count value from the set access counter (130-2), and one for receiving a count value from the set access counter (130-3). The selection logic (214) further includes a selection control input coupled to the outputs of the comparators (208), and an output coupled to the accumulator (204). Accordingly, the selection logic (214) operates to select one of the input select access count values ​​from the set access counters (130-0 to 130-3) for output to the accumulator (204) based on which comparator (208) (if any) has an asserted output. That is, the way of the cache set (116) having a tag address that matches the tag address of the cache probe (212) triggers the selection logic (214) to output the counter value of the set access counter (130) associated with that way to the accumulator (204).

[0011] The accumulator (204) operates to accumulate set access counter values ​​received from various set accounting components (202) and to provide the resulting updated accumulated value to the averaging / scaling component (206). In response to a trigger event (e.g., every Kth access to the representative cache sets of a region), the averaging / scaling component (206) operates to average the most recently updated accumulated value over the number of accesses to the representative cache sets of the region since the last calculation cycle to generate an average set access count value from the accumulated value. The average set access count value may be obtained, for example, through a sequence of shift operations, and in some embodiments, the averaging / scaling component (206) scales the resulting average set access count value using a specified scaling factor. The resulting averaged / scaled set access count value then functions as the current reuse distance (216) for the corresponding region of the cache (108).

[0012] FIG. 3 illustrates a method (300) that further illustrates a reuse distance calculation process employed by the reuse distance calculation component (200) of FIG. 2 of a cache controller (112) according to some embodiments. In block (302), the cache controller (112) monitors the operations of the cache (108) to determine whether a cache line has been inserted into one of the representative cache sets (116). In block (304), the cache controller (112) monitors the operations of the cache (108) to determine whether a cache line within one of the representative cache sets (116) has been accessed. If the cache line has been accessed, in block (306), the value of the set access counter (130) associated with the way of the cache set (116) storing the accessed cache line is sent to the accumulator (204). When a cache line is inserted or accessed, in block (308), the cache controller (112) resets the set access counter (130) associated with the way of the cache set (116) storing the inserted or accessed cache line to a predetermined value (e.g., 0), and in block (310), the cache controller (112) increments the set access counter (130) associated with all other ways of the cache set (116). That is, the insertion of a cache line into a way of the representative cache set (116) or the access to a cache line resets the set access counter (130) for that way of the cache set (116), while also incrementing the set access counter (130) for all other ways of the cache set (116). Alternatively, the cache controller (112) increments the set access counter (130) associated with each way of the cache set (116) and then resets the set access counter (130) associated with the way of the cache set (116) that stores the inserted or accessed cache line to a predetermined value.Accordingly, in this manner, the cache controller (112) monitors, for each cache line in the representative cache set (116), the number of accesses performed on the representative cache set (116) since the cache line was inserted or last accessed.

[0013] Furthermore, in block (312), the cache controller (112) monitors a cache hit for a way of a representative cache set (116) (which may include access referenced in block (302)). In at least one embodiment, a cache hit for a way of a representative cache set (116) is signaled through the comparators (208) of the set accounting component (202) associated with the representative cache set (116) in such a way that when an address appearing on a cache probe (e.g., cache probe (212) in FIG. 2) matches with an address in the tag field (118) of the corresponding way, the output of the corresponding comparator (208) is asserted, thereby signaling that the cache probe has hit that way of the representative cache set (116). In response to such a cache hit, in block (314), the selection logic (214) of the set accounting component (202) associated with the hit cache set (116) outputs the value of the set access counter (130) associated with the hit method of the hit cache set (116) to the accumulator (204), and the accumulator (204) adds the input value to the previous accumulator value during the current calculation cycle.

[0014] The process of blocks (312 and 314) is repeated for each cache hit for the representative cache set (116) until the Kth cache hit for the corresponding area of ​​the cache (108)—where K is a programmable or otherwise specified integer value greater than 1 (K > 1)—is detected in the current calculation iteration (e.g., via the hit counter (207)). For example, K can be set to 64, and accordingly, the count accumulation process continues until the 64th cache hit for the corresponding area of ​​the cache (108) occurs. In block (316), in response to determining that the Kth cache hit for the area of ​​the cache (108) has occurred during the current calculation cycle, in block (318), the averaging / scaling component averages the current accumulated value from the accumulator (204) over the K cache hits used for the current calculation cycle, and in some embodiments, scales this averaged value by multiplying this averaged value by a factor. In embodiments where K and any scaling factor are powers of 2, the averaging / scaling component (206) can be implemented as right and left shift logic. For example, when K is set to 64 (2^6) in each previous example, averaging of the accumulated value can be performed by right-shifting the accumulated value by 6 bits and then left-shifting it by 1 bit. More generally, when K is equal to 2^M (M is a positive integer), the averaging / scaling component (206) can left-shift the currently updated accumulated value by M bit positions and then right-shift it by 1 bit position to obtain the average set access count for K cache accesses for representative cache sets. Similarly, scaling of the average value by 2 can be achieved by left-shifting the resulting average value by 1 bit. Then, the resulting average (and scaled) value is set as the current reuse distance (216) for the corresponding area of ​​the cache (108).Scaling the average takes into account situations where there may be discrepancies between the set access count values ​​for individual ways within a set, so that the reuse distance becomes less than the set access count value for some of the lines within the set. As described in more detail below, if the set access count value for a given line exceeds the cache's reuse distance, the line is more likely to be evicted. Scaling the average can be used to prevent some lines within a set from being unintentionally prioritized for replacement. When the current computation cycle ends, in block (320), the cache controller (112) resets the various components used in the ended computation cycle, such as the hit counter (207), set access counters (130), line access counters (132), and accumulator (204), and then the process returns to block (312) for the next computation cycle.

[0015] FIG. 4 illustrates an alternative implementation of a set accounting component (202) (Fig. 2) implemented by a cache controller (112) for each representative cache set (116) according to at least one embodiment. The alternative implementation of the set accounting component (202) of FIG. 4 includes comparators (208) operable to assert their respective outputs in response to an address in the tag field (118) of a corresponding way matching an address in the tag field (210) of a received cache probe (212), as in the implementation of FIG. 2, and further includes selection logic (414) (illustrated as a multiplexer for ease of illustration) that uses the outputs of the comparators (208) as selection control inputs. However, the implementation of FIG. 4 does not use counters for set access counters (130) that are large enough to account for all accesses to the corresponding cache set during the computation cycle, but instead allows smaller counters to be used for set access counts (130) by incrementing the set access counters (130) for every Nth access to the cache set (116). To enable counting of accesses by N, the illustrated implementation of the set accounting component (202) further includes an Nth access counter (404) that is incremented for each access to the cache set (116) during the current computation cycle (or reset to N and decremented for each access). Furthermore, to compensate for the fact that only every Nth access to the cache set (116) triggers an increment of the set access counters (130) of the cache set, the illustrated embodiment employs a shift register (406) and an adder (408) between the output of each set access count (130) and the corresponding input to the select logic (414) to adjust this sampled cache access approach.Accordingly, assuming N is a power of 2 (2^j), the set access count value output from the set access counter (130) is shifted by j bits and then added to the current hit count appearing in the Nth access count (404), and this resulting value is supplied to the corresponding input of the selection logic (414). Then, the selection logic (414) is operable to select one of the input values ​​for the output to the accumulator (204) based on the states of the outputs of the comparators (208), which in turn indicate which way of the representative cache set (116) (if any) is the target of the Nth cache probe (212).

[0016] FIGS. 5 and FIGS. 6 together illustrate the operation of a cache controller (112) for implementing a cache management policy (120) during an alternative priority assignment stage (124) ( FIG. 1) according to some embodiments. FIG. 5 illustrates a method (500) for maintaining line access counters (132) for representative cache sets (116) during a priority cycle according to some embodiments. As described above, each line access counter (132) indicates the number of times a cache line stored in the way associated with the line access counter (132) has been accessed since the start of the current priority cycle. Accordingly, at the start of the priority cycle, in block (502), the cache controller (112) monitors the insertion of a cache line into the way of the representative cache set (116). In response to this insertion, in block (504), the cache controller (112) resets the line access counter (132) associated with the target way of the representative cache set (116). Then, in block (506), the cache controller (112) monitors access to the cache line. In response to detecting access, in block (508), the cache controller (112) increments the line access counter (132) for the accessed cache line. In some embodiments, the line access counter (132) is implemented as a 1-bit or 2-bit saturation counter to reduce hardware requirements for the line access counters (132), and accordingly counts at most one access (for a 1-bit counter implementation) or at most three accesses (for a 2-bit counter implementation). In other embodiments, more than 2 bits are used for the line access counters (132) to enable counting of a greater number of accesses for any given cache line.

[0017] Referring to FIG. 6, a method (600) for assigning replacement priorities to a cache line based on the current line access count in the corresponding line access counter (132) and based on the current reuse distance is shown according to some embodiments. The method (600) is initiated by an event such as a decision that a cache line will be replaced (block (602)), which functions as a trigger to determine replacement priorities for newly inserted cache lines during the current prioritization cycle and to re-determine replacement priorities for previously existing cache lines. In response to the trigger, the prioritization process is initiated by the selection of a cache line in an area of ​​the cache (108) according to a selection sequence in block (604), e.g., linear order selection, pseudo-random selection, etc.

[0018] In block (606), the cache controller (112) accesses the set access counter (130) of the way storing the selected cache line and compares the count contained therein with the current reuse distance (216). If the set access counter (130) is not greater than the current reuse distance (216), it indicates that the cache line has not yet reached the reuse distance and may be reused later. Accordingly, if the set access counter (130) is less than the current reuse distance, in block (608), the cache controller (112) accesses the line access count (132) associated with the way storing the selected cache line and determines whether the value stored therein is greater than 0 (i.e., determines whether the cache line has been reused since insertion). If so, there is a possibility that this cache line will be reused again in terms of the tendency of some data to be accessed repeatedly. Accordingly, if the line access counter (132) is determined to be greater than 0 in block (608), the cache line is assigned replacement priority level 3 in block (610) (for the following, it is assumed that the lower the replacement priority level value, the less suitable the corresponding cache line is as a candidate for replacement, and thus the greater the likelihood of the corresponding cache line being selected for replacement). Otherwise, if the line access counter (132) is equal to 0, this indicates that the cache line has not been reused, but since the reuse distance has not yet been reached for the cache line, the possibility of the cache line being reused in the future still exists. In this case, the cache line is assigned replacement priority level 1 in block (612) (priority level 1 indicates a greater likelihood of selection for replacement than replacement priority level 3).

[0019] Returning to block (606), if the access count indicated by the set access counter (130) is determined to be greater than the current reuse distance, this means that the cache line has already reached the reuse distance but has not been reused since. Accordingly, in block (614), the cache controller (112) determines whether the cache line has been accessed by accessing the line access counter (132) for the way storing the cache line. If the line access counter (132) is greater than 0, this means that the cache line has been reused at least once since insertion into the cache (108) and thus has the potential to be reused again. However, the potential for reuse is reinforced by the fact that the reuse distance has already been passed. Accordingly, if the line access counter (132) is greater than 0, the selected cache line is assigned replacement priority level 2 in block (616) (replacement priority level 2 indicates a greater likelihood of selection for replacement than replacement priority level 3, and a lower likelihood of selection for replacement than replacement priority level 1). Otherwise, if the count is equal to 0, this means that the cache line has not been reused since insertion and is unlikely to be reused in the future as it has already passed the reuse distance. In this case, the cache line is assigned replacement priority level 0 in block (618) (in this example, replacement priority level 0 indicates the highest likelihood of selection for replacement).

[0020] FIG. 7 illustrates a method (700) for the operation of a cache controller (112) for a cache line replacement stage (126) (Fig. 1) of a cache management policy (120) according to some embodiments. In the following, for example, it is recalled that a numerical increase in the replacement priority level corresponds to an increasing priority for maintaining the corresponding cache line, and conversely, a numerical decrease in the replacement priority level corresponds to an increasing priority or possibility for evicting the corresponding cache line. In block (702), a load operation or a save operation is performed, and as a result, a data block to be stored as a cache line in the cache (108) is created. Accordingly, also in block (702), the cache controller (112) determines a cache set (116) available for storing the cache line based on the address associated with the cache line. In block (704), the cache controller (112) determines whether there is a way available in the identified cache set (116) (i.e., whether there is a way that is not currently storing an effective cache line). If so, in block (706), the cache controller (112) inserts a cache line into an available way of the cache set (116). If the cache set (116) is a representative cache set, inserting this cache line into the cache set (116) triggers specific counting operations as described above with reference to block (302) of FIG. 3 and block (502) of FIG. 5.

[0021] Otherwise, if no way is available in the identified cache set (116), the cache controller (112) determines whether to evict the current cache line in the cache set (116) or bypass the caching of the cache line based on the alternate priority levels assigned during the alternate priority assignment stage (124) of the cache management policy (120) as described above. Accordingly, in block (708), the cache controller (112) determines whether cache bypass (also known as “optional caching” or “cache exclusion”) is supported by the cache (108). If cache bypass is supported, in block (710), the cache controller (112) determines whether the cache set (116) contains any cache lines assigned alternate priority level 0. Otherwise, in block (712), the cache controller (112) may choose to bypass caching of a cache line in the cache (108) (e.g., by preventing any caching of the cache line, or by providing the cache line to a lower-level cache for storage). In another embodiment, if it is determined that the cache line is part of a streaming process (e.g., if the current reuse distance is small (0 or 1)), cache bypass may be chosen if there are no cache lines with alternate priority level 1.

[0022] If cache bypass is not supported, or if there are no arbitrary cache lines of a priority level low enough to justify cache bypass, in block (714), the cache controller (112) selects a cache line within the cache set (116) having the lowest replacement priority level as a replacement candidate. If two or more cache lines have the same lowest replacement priority level, the cache controller (112) may select one of the cache lines pseudo-randomly based on a specified selection order, or select a way having the maximum set access count. In block (716), the cache controller (112) replaces the selected candidate cache line with a new cache line in the corresponding way of the cache set (116) or otherwise evicts it. This action is cache line insertion, which accordingly triggers specific counting operations as described above with reference to block (302) of FIG. 3 and block (502) of FIG. 5.

[0023] In some embodiments, specific aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software comprises one or more sets of executable instructions stored on a non-transient computer-readable storage medium or otherwise tangibly implemented. When executed by one or more processors, the software comprises instructions and specific data that manipulate one or more processors to perform one or more aspects of the techniques described above. The non-transient computer-readable storage medium includes, for example, magnetic or optical disk storage devices, solid-state storage devices such as flash memory, caches, random access memory (RAM), or other non-volatile memory devices or devices. The executable instructions stored on the non-transient computer-readable storage medium may be in source code, assembly language code, object code, or other instruction formats that are interpreted by one or more processors or otherwise executable.

[0024] According to one embodiment, a method for managing a cache of a processing system comprises the step of determining a reuse distance for a region of a cache by a cache controller of a cache—the reuse distance represents the average number of accesses to a set of caches of a cache between accesses to a set of caches of a predetermined cache line. The method further comprises the step of assigning an alternate priority level to each cache line of at least a subset of cache lines of a region of a cache by the cache controller based on the reuse distance and a count of cache hits for a cache line.

[0025] According to another aspect, a method for managing a cache of a processing system comprises the step of implementing a cache management policy for inserting and replacing cache lines of a cache by a cache controller of the cache, wherein the cache management policy provides for assigning replacement priority levels to each cache line of at least a subset of cache lines within a cache region based on a comparison of the number of accesses to a cache set having a way to store a cache line since the cache line was last accessed and a reuse distance determined for a region of the cache, wherein the reuse distance represents the average number of accesses to a predetermined cache set of a region between any predetermined cache line of the cache set.

[0026] According to another aspect, the process comprises a cache comprising a plurality of cache sets, each cache set having a plurality of ways configured to store corresponding cache lines. The processor further comprises a cache controller configured to implement a cache management policy for inserting and replacing cache lines of the cache, wherein the cache management policy provides for assigning replacement priority levels to each cache line of at least a subset of cache lines within a region of the cache based on a comparison of the number of accesses to a cache set having a way storing a cache line since the cache line was last accessed and a reuse distance determined for a region of the cache, and the reuse distance represents the average number of accesses to a certain cache set of a region between any certain cache line of the cache set.

[0027] Computer-readable storage media include any non-transient storage media, or a combination of non-transient storage media, that are accessible by a computer system in use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., CD (compact disc), DVD (digital versatile disc), Blu-ray disc), magnetic media (e.g., floppy disk, magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS)-based storage media. Computer-readable storage media may be embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., magnetic hard drive), detachably attached to a computing system (e.g., optical disc or USB (Universal Serial Bus)-based flash memory), or coupled to a computer system via a wired or wireless network (e.g., Network Accessible Storage (NAS)).

[0028] It should be noted that not all of the operations or elements described above may be required in the general description, that specific operations or parts of a device may not be required, and that in addition to the described operations or elements, one or more additional operations may be performed or elements may be included. Furthermore, the order in which the operations are listed is not necessarily the order in which they are performed. Also, concepts have been described with reference to specific embodiments. However, those skilled in the art understand that various modifications and changes may be made without departing from the scope of the disclosure as set forth in the claims below. Accordingly, the specification and drawings are to be understood as illustrative rather than restrictive, and all such modifications are intended to be included within the scope of the disclosure.

[0029] Benefits, other advantages, and solutions to problems have been described in detail with respect to specific embodiments. However, any feature(s) that give rise to or more clearly reveal any benefits, advantages, solutions to problems, and any benefit, advantage, or solution should not be construed as critical, essential, or essential features of any or all claims. Furthermore, the specific embodiments disclosed above are merely illustrative, as the subject matter disclosed may be modified and practiced in different but equivalent ways that are apparent to those skilled in the art who benefit from the teachings of this specification. It is not intended to be limited to the details of configuration or design presented in this specification other than those described in the claims below. Accordingly, it is evident that the specific embodiments disclosed above may be substituted or modified, and that all such variations are considered to be within the scope of the subject matter disclosed. Accordingly, the protection sought in this specification is as set forth in the claims below.

Claims

Claim 1 A method for managing a cache of a processing system, comprising: determining a reuse distance for a region of the cache by a cache controller of the cache, wherein the reuse distance represents the average number of accesses to the cache set of the cache between accesses to a predetermined cache line of a predetermined cache set; and assigning an alternative priority level to each cache line of at least a subset of the cache lines of the region of the cache by the cache controller based on the reuse distance and a count of cache hits for the cache line. Claim 2 A method according to claim 1, further comprising the step of selecting a cache line for replacement in an area of ​​the cache based on the replacement priority level assigned to the cache. Claim 3 The method of claim 1, wherein the step of determining the reuse distance comprises: in response to each cache hit in a cache set among at least a subset of cache sets of the region of the cache: adding a set access count value associated with a way of the cache set including the cache line that is the target of the cache hit to the accumulated value to generate an updated accumulated value; resetting the set access count value associated with the way of the cache set including the cache line that is the target of the cache hit; and incrementing the set access count values ​​associated with other ways of the cache set; and in response to detecting a specific number of cache hits for at least a subset of the cache sets: averaging the updated accumulated value by the specific number of cache hits to generate an average set access count value; and determining the reuse distance based on the average set access count value. Claim 4 In paragraph 3, in response to detecting the number of specific cache hits for at least a subset of the cache sets: the method further comprises the step of generating a scaled average set access by scaling the average set access count value by a specific factor; and the step of determining the reuse distance comprises the step of determining the reuse distance based on the scaled average set access count value. Claim 5 In claim 3, the step of assigning a replacement priority level to each cache line of at least a subset of the cache lines of the region of the cache comprises: assigning a first replacement priority level to the cache line in response to a determination that the set access count value associated with the cache line is not greater than the reuse distance, and in response to a determination that the number of accesses to the cache line stored in the way is greater than 0; and assigning a second replacement priority level to the cache line in response to a determination that the set access count value associated with the cache line is not greater than the reuse distance, and in response to a determination that the number of accesses to the cache line stored in the way is equal to 0; wherein the second replacement priority level indicates a greater possibility of selection for replacement than the first replacement priority level. Claim 6 In claim 5, the step of assigning a replacement priority level to each cache line of at least a subset of the cache lines of the region of the cache comprises: assigning a third replacement priority level to the cache line in response to a determination that the set access count value associated with the cache line is greater than the reuse distance and in response to a determination that the number of accesses to the cache line stored in the way is greater than 0; and assigning a fourth replacement priority level to the cache line in response to a determination that the set access count value associated with the cache line is greater than the reuse distance and in response to a determination that the number of accesses to the cache line stored in the way is equal to 0; wherein the third replacement priority level indicates a greater likelihood of selection for replacement than the first replacement priority level; and the fourth replacement priority level indicates a greater likelihood of selection for replacement than the second replacement priority level. Claim 7 A method according to claim 6, further comprising the step of selecting a cache line for replacement in an area of ​​the cache based on the replacement priority level assigned to the cache. Claim 8 A method according to claim 1, wherein the area of ​​the cache includes the entire cache. Claim 9 A processor comprising: a cache including a plurality of cache sets, wherein each cache set has a plurality of ways configured to store corresponding cache lines; and a cache controller configured to implement a cache management policy for inserting and replacing cache lines of said cache, wherein the cache management policy provides for assigning replacement priority levels to each cache line of at least a subset of cache lines within said cache region based on a comparison of the number of accesses to a cache set having a way storing said cache line since said cache line was last accessed and a reuse distance determined for said cache region, said reuse distance represents the average number of accesses to said cache set of said region between any given cache line of said cache set. Claim 10 In claim 9, the cache comprises: a plurality of set access counters, each set access counter being associated with a corresponding way of each cache set of at least a subset of cache sets of the region of the cache and configured to store a set access count value indicating the number of accesses to the cache set since the cache line stored in the corresponding way was inserted or last accessed; an accumulator configured to accumulate the set access count values ​​from each way of the cache set of at least a subset of the cache sets that is the target of the cache hit as an accumulator value; and an averaging / scaling component configured to determine the reuse distance based on the averaging of the accumulator value by the specific cache hit number in response to detecting a specific cache hit number for at least a subset of the cache sets. Claim 11 A processor according to claim 10, wherein the cache further comprises: a plurality of line access counters—each line access counter being associated with a corresponding way of each cache set of at least a subset of cache sets and configured to store a line access counter value indicating the number of accesses to said cache line stored in said corresponding way. Claim 12 A processor configured to implement the cache management policy in paragraph 11, wherein the cache controller: assigns an alternative priority level to each cache line of at least a subset of cache lines in the region of the cache, based on a comparison of the set access count value associated with the cache line and the reuse distance, and based on a determination of whether the line access count value for the way storing the cache line is 0 or greater. Claim 13 In paragraph 12, the cache controller is configured to assign a first replacement priority level to the cache line in response to a determination that the set access count value associated with the cache line is not greater than the reuse distance, and in response to a determination that the line access count associated with the cache line is greater than 0; to assign a second replacement priority level to the cache line in response to a determination that the set access count value associated with the cache line is not greater than the reuse distance, and in response to a determination that the line access count associated with the cache line is equal to 0; to assign a third replacement priority level to the cache line in response to a determination that the set access count value associated with the cache line is greater than the reuse distance, and in response to a determination that the line access count associated with the cache line is greater than 0; and to assign a fourth replacement priority level to the cache line in response to a determination that the set access count value associated with the cache line is greater than the reuse distance, and in response to a determination that the line access count is equal to 0; and the second replacement priority level is A processor that indicates that the possibility of selection for replacement is greater than that of the first replacement priority level and the third replacement priority level; that the third replacement priority level indicates that the possibility of selection for replacement is greater than that of the first replacement priority level; and that the fourth replacement priority level indicates that the possibility of selection for replacement is greater than that of the second replacement priority level. Claim 14 In paragraph 13, the cache controller is configured to implement the cache management policy by selecting a cache line of the cache set for replacement based on a comparison of replacement priority levels assigned to each cache line in the cache set. Claim 15 A processor according to claim 10, wherein the specific cache hit number is equal to 2^M, where M is an integer greater than 1; and wherein the averaging / scaling component is configured to determine the reuse distance by right-shifting the accumulated value by M bit positions. Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete Claim 21 delete

Images