Cache management based on access type priority

Abstract

Systems, apparatuses, and methods for cache management based on access type priority are disclosed. A system includes at least a processor and a cache. During program execution phase, certain access types are more likely to result in demand hits in the cache than other access types. A demand hit is a load hit and a store hit to the cache. A runtime analysis mechanism is used to find which access types are more likely to cause demand hits. Based on the analysis result, cache lines that are likely to be accessed in the future are retained based on their recent access types. The goal is to increase demand hits, thus improving system performance. An effective cache replacement policy can potentially reduce redundant data movement, thus improving system performance and reducing energy consumption.

Description

Background Technology

[0001] This invention was carried out with government support under the Lawrence Livermore National Security PathForward project (Master Contract No. DE-AC52-07NA27344 and Subcontract No. B620717 awarded by the U.S. Department of Energy). The U.S. government holds certain rights to this invention.

[0002] Description of related technologies

[0003] Computer systems use main memory typically formed from inexpensive and high-density Dynamic Random Access Memory (DRAM) chips. However, DRAM chips suffer from relatively long access times. To improve performance, data processors typically include at least one local high-speed memory called a cache. The cache stores blocks of data that the processor frequently accesses. As used herein, a “block” is a set of bytes stored in a contiguous memory location that is treated as a unit for consistency purposes. As used herein, the terms “cache block,” “block,” “cache line,” and “line” are interchangeable. In some implementations, a block can also be a unit of allocation and deallocation within the cache. The number of bytes in a block varies depending on design choices and can be of any size. Additionally, the terms “cache tag,” “cache line tag,” and “cache block tag” are interchangeable.

[0004] Because caches have limited storage capacity, cache management strategies determine which cache lines to replace when a corresponding cache region is full. Effective cache management strategies are crucial for improving application performance. However, some conventional cache management strategies, such as those based on the Least Recently Used (LRU) principle, are inefficient at handling irregular accesses to cache lines or require relatively complex circuit implementations that may limit their applicability. Attached Figure Description

[0005] The advantages of the methods and mechanisms described herein can be better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0006] Figure 1 It is a block diagram of one implementation method of a computing system.

[0007] Figure 2 It is a block diagram of one implementation method of a computing system.

[0008] Figure 3 It is a block diagram of an implementation of a cache that tracks the last access type of each cache line.

[0009] Figure 4 This is a block diagram of an implementation of a cache that maintains a counter to track how many demand hits each access type causes.

[0010] Figure 5 This is a generalized flowchart illustrating an implementation of a method for prioritizing cache lines based on the last access type.

[0011] Figure 6 This is a generalized flowchart illustrating one implementation of a method for selecting cache lines for eviction. Detailed Implementation

[0012] Numerous specific details are set forth in the following description to provide a thorough understanding of the methods and mechanisms presented herein. However, those skilled in the art will recognize that various implementations can be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques have not been shown in detail to avoid obscuring the methods described herein. It should be understood that, for the sake of brevity and clarity, the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some elements may be enlarged relative to other elements.

[0013] This paper discloses various systems, devices, and methods for cache management based on access type priority. In one implementation, a system includes at least a processor and a cache. During program execution, certain access types are more likely to produce a cache demand hit than others. A demand hit is a cache load hit or store hit. Runtime analysis mechanisms are used to determine which access types are more likely to cause a demand hit. Based on the analysis results, cache lines that are likely to be accessed in the future are retained based on their most recent access type. The goal is to increase demand hits, thereby improving system performance. Effective cache replacement strategies can potentially reduce redundant data movement, thereby improving system performance and reducing energy consumption.

[0014] Now for reference Figure 1A block diagram of one implementation of a computing system 100 is shown. In one implementation, the computing system 100 includes at least a processor 110, a configuration 120, an input / output (I / O) interface 125, a memory interface 130, peripheral devices 135, and a memory subsystem 140. In other implementations, the computing system 100 may include other components, the components shown may be omitted, and / or the computing system 100 may be arranged differently. In one implementation, each processor 110 includes a cache subsystem 115. The cache subsystem 115 has any number of cache levels with any of a variety of types of caches that may vary depending on the implementation. In some cases, one or more caches in the cache hierarchy of the cache subsystem 115 may be located in other locations outside the processor 110.

[0015] In one implementation, one or more caches of the cache subsystem 115 employ a cache management scheme based on access type priority. For example, in one implementation, the cache controller determines which access types are more likely to cause a demand hit. A demand hit is a cache hit caused by a load or store operation. During the analysis phase, the cache controller determines from multiple access types which access type causes the fewest demand hits. Then, during replacement, the cache controller attempts to evict cache lines from multiple access types that have a recorded last access type that matches the access type with the fewest demand hits. In other words, the cache controller dynamically determines the access type most likely to have occurred before a load hit or store hit to the cache line. The cache controller then protects cache lines most recently accessed by the access type most likely to have occurred before a load hit or store hit. Further details regarding the techniques used for managing cache replacement strategies based on access type priority will be provided throughout the remainder of this disclosure.

[0016] Processor 110 represents any number and type of processing units (e.g., Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processor (DSP), Field Programmable Gate Array (FPGA), Application-Specific Integrated Circuit (ASIC)). Memory subsystem 140 includes any number and type of memory devices. For example, memory types in memory subsystem 140 may include High Bandwidth Memory (HBM), Non-Volatile Memory (NVM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), NAND Flash Memory, NOR Flash Memory, Ferroelectric Random Access Memory (FeRAM), etc. I / O interface 125 represents any number and type of I / O interface (e.g., Peripheral Component Interconnect (PCI) bus, PCI Expansion (PCI-X), PCIe (PCI High Speed) bus, Gigabit Ethernet (GBE) bus, Universal Serial Bus (USB)). Various types of peripheral devices 135 can be coupled to I / O interface 125. Such peripheral devices 135 include (but are not limited to) monitors, keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network interface cards, and so on.

[0017] In various implementations, computing system 100 is any of a computer, laptop computer, mobile device, game console, server, streaming device, wearable device, or various other types of computing system or device. It should be noted that the number of components in computing system 100 varies depending on the implementation. For example, in other implementations, there are components related to… Figure 1 The number shown is greater than that of a given component. It should also be noted that in other implementations, the computing system 100 includes... Figure 1 Other components not shown. Additionally, in other implementations, the computing system 100 is coupled with... Figure 1 The different ways of structuring are shown.

[0018] Now go to Figure 2 A block diagram of one implementation of a computing system 200 is shown. As shown, system 200 represents chips, circuits, components, etc., of a desktop computer 210, a laptop computer 220, a server 230, a mobile device 240, etc. Other components are possible and envisioned. In the illustrated implementation, system 200 includes a cache subsystem 115 (… Figure 1 At least one instance of ). Although Figure 2Not shown, but system 200 may also include any number and type of other components, such as one or more processors, one or more memory devices, one or more peripheral devices, etc. Cache subsystem 115 includes any number of cache levels employing a cache management scheme based on access type priority.

[0019] Now for reference Figure 3 The diagram illustrates a block diagram of one implementation of a cache 300 that tracks the last access type for each cache line. In various implementations, cache 300 is a low-latency, high-bandwidth memory separate from system memory. In some implementations, cache 300 is used as a cache memory subsystem (e.g., Figure 1 The cache 300 is the last level of the cache subsystem (115). In other implementations, cache 300 is another level within the cache subsystem.

[0020] When cache 300 receives a read or write request, it performs a lookup of tag memory 302 using the tag of the address targeted by the request. If the lookup results in a hit and an access is performed to a cache line in data storage 304, the access type is stored in the last access type field 308 of the corresponding entry 306 in tag memory 302. In one embodiment, tag memory 302 and data storage 304 are organized as an array, but other organization is possible and envisioned. In some embodiments, each of tag memory 302 and data storage 304 is a distinct entity, while in other embodiments, they are combined in a single storage entity (device). In either case, tag memory 302 and data storage 304 may simply be referred to as data storage or data storage device. In one implementation, different possible access types include load, store, prefetch, and write-back. As used herein, a “load operation” or “load access” is defined as an operation that specifies the transfer of data from a memory location to a processor or execution unit. A “store operation” or “store access” is defined as an operation that specifies the transfer of data from a processor or execution unit to a memory location. A "prefetch operation" or "prefetch access" is defined as an operation that specifies the transfer of data from a memory location to a cache before data is requested via a demand operation. If based on incorrect predictions, the data targeted by a prefetch access may ultimately not be used by the execution unit. A "write-back operation" or "write-back access" is defined as transferring a dirty cache line to a lower-level cache or memory location. A "dirty cache line" is defined as a cache line that has been modified but has not yet been written to a lower-level cache or main memory.

[0021] If a lookup for the second 302 fails and a cache line will be allocated for the request, the cache controller 320 determines which cache line to evict so that a new cache line can be stored. In one implementation, the cache controller 320 uses the last access type field 308 in each entry in the second 302 of the corresponding group to determine which cache line to evict from the second 304. For example, in one implementation, the cache controller 320 retrieves a sorted priority 350 that lists the priority associated with each access type. Once the cache controller 320 has retrieved the sorted priority 350, it searches for a cache line with the lowest last access type priority based on the sorted priority 350. Note that the cache controller 320 may also be referred to as a control unit or control logic.

[0022] In one implementation, cache 300 includes counter 340 to track the number of demand hits for each access type. A demand hit refers to a load or store operation that hits an entry in second 308. In one implementation, counter 340 includes counters for each different access type. For example, when a demand access hits a cache line whose last access type field 308 is encoded using prefetch encoding, the prefetch access type counter 340 increments. In one implementation, the value of counter 340 is used to determine the priority of each access type. For example, the larger the counter value, the higher the priority of the corresponding access type. In one implementation, cache controller 320 periodically retrieves the values ​​of counter 340 and sorts these values ​​in descending order. Then, cache controller 320 generates a sorted priority 350 based on the sorted values ​​of counter 340, where the highest priority is assigned to the access type whose counter has the highest value. Other access types are assigned priorities in descending order based on the values ​​of their counters.

[0023] An example of the last access type encoding used in one implementation is shown in the last access type encoding table 330. For example, bit "00" indicates that the last access type is load. For bit "01", this indicates that the last access type is store. For bit "10", this indicates that the last access type is prefetch. For bit "11", this indicates that the last access type is write-back. In other implementations, the last access type field 308 of the entry in the second 302 may have a number of bits other than two. Furthermore, in other implementations, encodings other than those shown in the last access type encoding table 330 may be used.

[0024] Now go to Figure 4This diagram illustrates one implementation of a cache 400 that maintains a counter to track how many demand hits are caused by each access type. When a lookup is performed on cache 400 for a given address, various structures of cache 400 are accessed using a portion of the address's tag 410, set 415, and offset 420. The set portion 415 of the address is used to select a given set of cache 400, and then comparator 425 compares the tag portion 410 of the address with tags stored in the road 405 of the given set. In the example shown, the given set of cache 400 includes four roads 405 (roads 0 through 3), but more or fewer roads may be implemented in other caches.

[0025] If a match is found in one of the paths 405 of the label portion 410 of the address, the last access type is retrieved from the corresponding last access type field 430. If the access is a demand hit (i.e., a load hit or a storage hit), the last access type is provided to the comparison logic 440. A counter is incremented according to the last access type. For example, if the last access type is a load, the load counter 445 is incremented. If the last access type is a storage hit, the storage counter 450 is incremented. If the last access type is a prefetch, the prefetch counter 455 is incremented. Otherwise, if the last access type is a write-back, the write-back counter 460 is incremented.

[0026] When the number of accesses reaches a programmable threshold, the cache controller executes a sorting phase 465 to sort counters 445, 450, 455, and 460 in descending order of their counts. Then, the cache controller executes a priority allocation phase 470 to assign priorities to the sorted counters 445, 450, 455, and 460. The counter with the highest count receives the highest priority, the counter with the second highest count receives the second highest priority, and so on. Then, when the cache controller is searching for cache lines to be evicted, the priorities are used to determine the replacement strategy. In other words, when the cache controller needs to evict a cache line, it evicts the cache line with the lowest priority.

[0027] Now for reference Figure 5 This illustrates one implementation of a method 500 for prioritizing cache lines based on their last access type. For illustrative purposes, the steps in this implementation are shown in sequence. Figure 6 Those steps. However, it should be noted that in various implementations of the described method, one or more of the described elements may be performed simultaneously, in a different order than shown, or completely omitted. Other additional elements may also be performed as needed. Any of the various systems or devices described herein is configured to implement method 500.

[0028] Counters are maintained by the cache controller for each different access type (Box 505). In one implementation, there are four counters for four separate access types: load, store, prefetch, and write-back. In other implementations, a different number of counters may track a different number of different access types. For example, in another implementation, multiple counters may track multiple different types of loads, where a first counter tracks loads from a first application, a second counter tracks loads from a second application, and so on. Other counters may track store loads from each individual application, prefetch loads from each individual application, and write-back loads from each application. In other implementations, the cache controller may track other types of access types. In one implementation, all groups in the cache share the same group's access type counters. In another implementation, there are multiple groups of counters, where the counter for each group is shared by a portion of the cache group. In yet another implementation, a single group's counter tracks several representative groups instead of the entire cache.

[0029] When a cached request results in a demand hit (e.g., load hit, store hit), the last access type field corresponding to the hit cache line is retrieved (box 510). Next, a counter corresponding to the access type specified by the retrieved last access type field is incremented (box 515). If fewer than a threshold number of cache accesses were performed (condition box 520, "No" branch), method 500 returns to box 510. If the number of cache accesses has reached the threshold number (condition box 520, "Yes" branch), the access type counters are sorted in descending order, and priority is assigned to access types based on the sorting order (box 525). The priority is then used to determine the replacement strategy for selecting cache lines to be evicted (box 530). For example, if the prefetch access type counter has the lowest value among all counters, the cache controller will prioritize evicting cache lines last accessed by a prefetch. In another example, if the load access type counter has the highest value among all counters, the cache controller will attempt to retain those cache lines last accessed by a load. Method 500 concludes after box 530. It should be noted that method 500 can be executed periodically or in response to the detection of an event (e.g., the start of a new phase of the application) to update the priorities of different access types.

[0030] Now go to Figure 6This illustrates one implementation of a method 600 for selecting cache lines for eviction. The cache controller detects a cache miss for a given request (box 605). In response to the detected miss, the cache controller searches for cache lines to be evicted to make room for the data targeted by the given request (box 610). Next, the cache controller determines which access type has the lowest priority based on the most recent tracking interval (box 615). In various implementations, the interval is a given time period, a given number of clock cycles, a given number of transactions or accesses, etc. Figure 5 The discussion associated with method 500 describes an example of determining which access type has the lowest priority based on the most recent tracking interval. The cache controller then searches for cache lines with a last access type field that matches the lowest priority access type (box 620).

[0031] If a cache line with a Last Access Type field set as the lowest priority access type is found (condition box 625, "Yes" branch), the cache controller evicts this cache line (box 630). Otherwise, if no cache line with a Last Access Type field set as the lowest priority access type is found (condition box 625, "No" branch), the cache controller determines which of the remaining access types has the lowest priority (box 640). If a cache line with a Last Access Type field set as the current low priority access type is found (condition box 645, "Yes" branch), the cache controller evicts this cache line (box 650). Otherwise, if no cache line with a Last Access Type field set as the current low priority access type is found (condition box 645, "No" branch), method 600 returns to box 640, where the cache controller determines which of the remaining access types has the lowest priority. After boxes 630 and 650, the cache controller stores the new cache line in the path of the evicted cache line (box 635). Method 600 ends after box 650.

[0032] In various implementations, the methods and / or mechanisms described herein are implemented using program instructions from a software application. For example, program instructions executable by a general-purpose processor or a special-purpose processor are envisioned. In various implementations, such program instructions are represented by a high-level programming language. In other implementations, the program instructions are compiled from a high-level programming language into binary, intermediate, or other forms. Alternatively, program instructions describing the behavior or design of the hardware are written. Such program instructions are represented by a high-level programming language such as C. Alternatively, a hardware design language (HDL) such as Verilog is used. In various implementations, the program instructions are stored on any of a variety of non-transitory computer-readable storage media. During use, the computing system can access the storage media to provide the program instructions to the computing system for program execution. Generally, such a computing system includes at least one or more memories and one or more processors configured to execute the program instructions.

[0033] It should be emphasized that the above implementations are merely non-limiting examples. Many variations and modifications will be apparent to those skilled in the art once the above disclosure is fully understood. The appended claims are intended to cover all such variations and modifications.

Claims

1. A system comprising: a processor; and a cache configured to: maintain, for each of a plurality of cache lines, a last access type indication; maintain a plurality of counters, wherein each of the plurality of counters tracks a number of each of a plurality of different access types; assign a lowest priority to a given access type corresponding to a counter having a lowest number of demand hits, wherein a demand hit is a hit on the cache for a load operation or a store operation; and in response to detecting a cache miss, evict a cache line having a last access type field set to the given access type assigned the lowest priority in preference.

2. The system of claim 1, wherein in response to detecting a demand hit on a first cache line, the cache is further configured to: increment a first counter in response to detecting that the first cache line has a last access type field set to a prefetch access type; and increment a second counter in response to detecting that the first cache line has a last access type field set to a writeback access type.

3. The system of claim 1, wherein the cache is configured to: assign a highest priority to a first access type having a counter with a highest number of demand hits; and preferentially retain a cache line having a last access type field set to the first access type.

4. The system of claim 1, wherein the cache is further configured to order the plurality of counters according to a number of demand hits per counter within a recent time interval.

5. The system of claim 1, wherein the cache is further configured to store the last access type indication in a last access type field of a first tag entry of a corresponding cache line.

6. The system of claim 1, wherein the given access type is a prefetch access type.

7. The system of claim 1, wherein the given access type is a writeback access type.

8. A method comprising: maintaining, by a cache, for each of a plurality of cache lines, a last access type indication; maintaining a plurality of counters, wherein each of the plurality of counters tracks a number of each of a plurality of different access types; assigning a lowest priority to a given access type corresponding to a counter having a lowest number of demand hits, wherein a demand hit is a hit on the cache for a load operation or a store operation; and in response to detecting a cache miss, evict a cache line having a last access type field set to the given access type assigned the lowest priority in preference.

9. The method of claim 8, wherein in response to detecting a demand hit on a first cache line, the method further comprises: incrementing a first counter in response to detecting that the first cache line has a last access type field set to a prefetch access type; and incrementing a second counter in response to detecting that the first cache line has a last access type field set to a writeback access type. ​ ​ 10. The method of claim 8, further comprising: assigning a highest priority to a first access type having a counter of the plurality of counters having a highest number of demand hits; and preferentially retaining a cache line having a last access type field set to the first access type.

11. The method of claim 8, further comprising ordering the plurality of counters according to a number of demand hits per counter in a recent time interval.

12. The method of claim 8, further comprising storing the last access type indication in a last access type field of a first tag entry of a corresponding cache line.

13. The method of claim 8, wherein the given access type is a prefetch access type.

14. The method of claim 8, wherein the given access type is a writeback access type.

15. A cache comprising: a data storage area; and a cache controller configured to: maintain, in the data storage area, a last access type indication for each of a plurality of cache lines in the data storage area; maintain a plurality of counters, wherein each of the plurality of counters tracks a number of each of a plurality of different access types; assign a lowest priority to a given access type corresponding to a counter having a lowest number of demand hits, wherein a demand hit is a hit to the cache for a load operation or a store operation; and in response to detecting a cache miss, preferentially evict a cache line having a last access type field set to the given access type assigned the lowest priority.

16. The cache of claim 15, wherein in response to detecting a demand hit to a first cache line, the cache controller is further configured to: increment a first counter in response to detecting that the first cache line has a last access type field set to a prefetch access type; and increment a second counter in response to detecting that the first cache line has a last access type field set to a writeback access type.

17. The cache of claim 15, wherein the cache controller is further configured to: assign a highest priority to a first access type having a counter of the plurality of counters having a highest number of demand hits; and preferentially retain a cache line having a last access type field set to the first access type.

18. The cache of claim 15, wherein the cache controller is further configured to order the plurality of counters according to a number of demand hits per counter in a recent time interval.

19. The cache of claim 15, wherein the given access type is a prefetch access type.

20. The cache of claim 15, wherein the given access type is a writeback access type.

Images