System and method for efficient prediction-based cache line processing

Abstract

The present disclosure provides a data management method for a processor, the processor being assigned a first cache, a second cache, and a behavior history table. The method comprises: tracking reuse information stored in at least one of the first cache and the second cache to learn cache lines; recording the reuse information in the behavior history table; and determining a placement strategy for future operations to be performed on a plurality of cache lines stored in the first cache and the second cache based on the reuse information in the behavior history table. The present disclosure also provides a multiprocessor system and a computer-readable storage medium.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] U.S. Provisional Application No. 62 / 588,715, filed on November 20, 2017, in the U.S. Patent and Trademark Office, is incorporated herein by reference in its entirety. Technical Field

[0003] At least some example embodiments of the present inventive concepts relate generally to methods and systems for accessing data in computer memory devices and, more particularly, to mechanisms and techniques for predicting data unit behavior and applying modifications or, alternatively, optimizations based on those predictions. Background Art

[0004] Today's processors are often equipped with caches that can store copies of data and instructions stored in some high-capacity memory. A popular example of such high-capacity memory today is dynamic random access memory (DRAM). In this article, the term "memory" will be used to collectively refer to all existing and future memory implementations. Cache memory, or simply "cache," is typically built from a much smaller and faster memory than other memory implementations, and can then only keep a copy of a small portion of the data stored in main memory or secondary storage at any given time. Today, caches are often implemented using SRAM, and large caches can be implemented using DRAM. The caches described herein can be implemented using any existing and future memory technology.

[0005] Typically, a computer system's memory system includes a series of caches, with larger, slower caches (referred to herein as higher-level caches) located closer to the main memory and smaller, faster caches (referred to herein as lower-level caches) located closer to the processor. This configuration is often referred to as a cache hierarchy, memory hierarchy, or memory system. Each level in the cache hierarchy is called a cache level. Figure 1 is a block diagram showing a portion of a computer system. Figure 1 In the example shown, the computer system is a multi-processor system in which each CPU 101 is assigned its own first level private cache 102 (L1 cache). Figure 2A more detailed example of a CPU 101 and L1 cache 102 is described. A private cache is a cache in which the space for a data element can only respond to the activity of the CPU local to the cache (or a CPU located in the same node as the cache, where the cache is private to that node). In this example, the second level cache 103 is shared by all CPUs 101 and can contain data and instructions accessed by all CPUs 101. Because the space for a data element can be made responsive to the activity of any CPU 101, cache 103 is a global cache. Furthermore, cache 103 is a shared cache in that each data element can only exist in one shared copy (whereas each private cache can have its own replica). Memory 105 also stores instructions and data that are accessed by all CPUs 101. Physical addresses (or PA for short) are typically used to access (main) memory 105, while addresses generated by CPU 101 are typically virtual addresses. For example Figure 1 The illustrated system, a computer system having multiple CPUs and multiple dedicated caches, requires an efficient mechanism for finding a requested data unit in one of the caches and for keeping multiple copies of the data stored in different nodes consistent.

[0006] In addition to the main memory and cache memory, such a computer system typically includes one or more auxiliary storage devices. These auxiliary storage devices may include one or more hard disk drives, optical drives, flash drives, etc., collectively represented here by disk 104. Disk or auxiliary storage 104 can store several orders of magnitude more data than memory 105, but cannot be directly accessed using physical addresses. If CPU 101 wants to access data stored on disk 104, the virtual memory system ( Figure 1 ) moves large blocks of data (often called pages) from disk 104 to memory 105 and creates translations from the virtual addresses corresponding to the pages to physical addresses. Special types of translation caches ( Figure 1 A memory 105 (not shown) (often called a "translation look-aside buffer" or TLB) caches translation mappings from virtual pages to physical pages. The virtual memory system can be viewed as a cache system that stores a portion of the data located on disk 104 in memory 105.

[0007] Figure 2 is a block diagram used to explain an example cache hierarchy for a computer system. Figure 2 Shown from Figure 1201 and data TLB (DTLDB, 205), and CPU core 202. Level 1 cache 102 is shown as including instruction cache 200 and data cache 206 interposed between CPU 204 and level 2 (and potentially higher level) cache 207. Figure 2 Also shown is a high capacity memory 208 connected to the cache hierarchy. Figure 2 The computer system in is shown as a uniprocessor (ie, single processor) system. Figure 2 The computer system in may be part of a multi-processor system.

[0008] When CPU 101 requests data that is present in the cache (called a cache hit), the request can be serviced much faster than when accessing data that is not present in the cache (called a cache miss). Generally, an application running on CPU 101 that experiences fewer cache misses will execute faster and consume less energy than the same application that experiences more cache misses (assuming the two versions of the application have similar properties). Consequently, considerable effort has been invested in techniques for avoiding cache misses, and implementations have been designed that can make cache hits and cache misses efficient in terms of access latency and the energy consumed by these memory transactions.

[0009] In order to understand how to develop solutions to this cache hit / miss problem, some understanding of cache organization and terminology will be useful. Typically, data is installed into the cache in fixed blocks that are larger than the processor's word size, called cache lines. Common cache line sizes today are, for example, 32, 64, and 128 bytes, but as will be appreciated by those skilled in the art, larger and smaller cache line sizes exist for various cache implementations. For some cache implementations, the cache line size can also be variable.

[0010] A common way to organize data placement in memory 208 is to statically map each data word to reside in a specific cache line. Each cache typically has an index that identifies the portion of the cache where each cache line can reside, called a set. A set can contain space for holding one or more cache lines simultaneously. The number of cache lines a set can hold is called its associativity. Typically, the associativity of all sets in a cache is the same. Such caches are often called set-associative caches. Associativity can also vary between sets. Each space within a set that can hold a cache line is called a way.

[0011] In order to determine the identity of each cache line stored in each group, the cache lines in the cache each have some identifier associated with them. A common example of such an identifier is an address tag. When looking up a particular cache line in a cache, its address can be used to determine the group of cache in which it may reside. The address tag of the cache line of the corresponding group is compared with the tag portion of the address used to identify the cache line (see, for example, Figure 3 The ADDR address tag is a memory address that is used to determine whether the desired cache line resides in the cache and, if so, in which way it resides, that is, in which space in the set that can hold the cache line. Typically, the size of this ADDR address tag is quite large and can be in the range of 30-40 bits, which is 6-10% of the typical cache line size.

[0012] Typically, each cache has a built-in strategy for determining which cache lines to keep in a set and which cache lines to evict (also called replacements) to make room for new cache lines to be introduced into the set. This is called a replacement policy. The cache line that is replaced is called a victim cache line or a replaced cache line. Popular replacement policies used in conjunction with cache memories include (but are not limited to) least recently used (LRU), pseudo-LRU, and random replacement policies.

[0013] An inclusive cache hierarchy requires that a copy of a data block (e.g., a cache line) present in one cache level, such as the L1 cache, also be present in higher cache levels (where higher refers to cache levels with a number higher than 1), such as the L2 and L3 caches. An exclusive cache hierarchy only has one copy of a data block (e.g., a cache line) present in the entire cache hierarchy, while a non-inclusive hierarchy can have a mix of the two strategies. In both exclusive and non-inclusive cache hierarchies, a cache line is typically installed in the next higher cache level upon eviction from a given cache level.

[0014] With this background of caches and cache lines in mind, the discussion now continues by explaining the methods used to identify caches such as Figure 1 and Figure 2 A conventional technique for specifying a cache location within a cache hierarchy as shown. Figure 3 is a block diagram for explaining a conventional implementation of a two-level cache hierarchy including a set-associative cache and a set-associative data translation lookaside buffer (TLB). Figure 3An example of the organization of a computer system node is shown, including a CPU 301, a DTLB 303, an L1 cache 307, and an L2 cache 311. CPU 301 generates a memory request containing a virtual address 302 (VADDR), which is used to perform an associative lookup in DTLB 303. In this example, virtual address 302 is divided into three parts: a P_OFFSET (page offset, e.g., consisting of low-order bits), a tag, and an index. The index portion of virtual address 302 is used to identify a bank within DTLB 303 where the address translation can be stored.

[0015] The DTLB 303 used in this example is shown with a bidirectional associative organization, showing two entries for the identified group. Each entry consists of an address tag (ATAG) 304 and a physical page frame (PPF) 305. The tag portion of the virtual address 302 is compared with the ATAG 304 of each entry of the identified group. Logic 318 determines whether there is a match for any entry and, if so, controls MUX 306 to select the corresponding PPF 305. Physical address PADDR 316 is formed by concatenating the PPF 305 selected by MUX 306 with the P_OFFSET portion of the virtual address 302. Alternatively, if none of the entries of the identified group match the tag portion of the virtual address 302, a TLB fill operation is performed and the required translation entry is introduced into DTLB 303.

[0016] PADDR 316 is used to perform a lookup in L1 cache 307. More specifically, the index portion of PADDR 316 is used to identify the set that can store the cache line containing the requested data. (It should be noted that this L1 index may contain different bits than the DTLB index.) The depicted L1 cache 307 has a bidirectional associative organization, with two entries in the identified set. Each entry consists of an address tag (ATAG) 308 and the data 309 for the corresponding cache line. The tag portion of PADDR 316 is compared with the ATAG of each entry in the identified set. Logic 319 determines whether there is a match for any entry and controls MUX 310 to select the corresponding data. If none of the entries match (indicated by an L1 miss 317), a lookup is required in L2 cache 311. While the division of PADDR 316 into index and tag portions for an L2 cache lookup may differ from the division performed for an L1 cache lookup, the remaining steps for an L2 cache lookup are generally similar to those performed for an L1 cache lookup. If a miss in the L2 cache is determined, a new lookup in a higher level cache or an access to memory may be required. It may be noted that if the index portion of PADDR 316 used by the L1 cache consists entirely of P_OFFSET bits, then accesses to the L1 cache 307 may begin before the PPF information from the MUX 306 is available. This is often referred to as a virtually indexed physically tagged (VIPT) cache.

[0017] Those skilled in the art will understand that each has Figure 3 Multiple "nodes" of a memory hierarchy similar to the memory hierarchy shown in FIG can be connected together to form a coherent multiprocessor system. Figure 1 Another example of a multiprocessor is shown in , where each CPU has its own dedicated L1 cache. This further complicates the task of locating where the requested data resides, as it is no longer sufficient to always search for the data in the next higher cache level. A mechanism is needed for searching the caches of other nodes and for keeping multiple copies of the datum stored in different nodes consistent.

[0018] As further background information, Figure 4is a block diagram illustrating an example implementation of a tag-less cache. Such a cache relies on location information (LI) (sometimes referred to as a cache line pointer CP) corresponding to a requested cache line of a matching cache table entry (CTE) (sometimes referred to as a region location) to indicate in which cache the requested cache line resides and at which location within that cache. This causes the computing system to perform a direct cache lookup in the appropriate cache. Figure 4 The example implementation of the sparsely tagged cache shown in FIG includes a two-level cache hierarchy and a two-level cache location buffer (CLB) hierarchy, wherein the first level is accessed virtually and the second level is accessed physically.

[0019] For example, in Figure 4 In the system shown (which may be a node in a multiprocessor system), location information may distinguish between the L1 cache and the L2 cache. It may also be possible to distinguish between other caches ( Figure 4 (not shown) to distinguish between. Figure 4 An sparsely tagged node is shown that can be connected to other nodes in a multiprocessor system having a two-level cache hierarchy of set associative caches and a CPU (which can contain one or more traditional caches, described here as L0), where the CPU 401 generates an address (ADDR) 402. Figure 4 Also shown is a set-associative CLB in a two-level CLB hierarchy, where a first-level CLB CLB1 410 and a second-level CLB CLB2 420 are indexed by ADDR. As used in this disclosure, the term "CLB1" may refer to a first-level CLB, and the term "CLB2" may refer to a second-level CLB. Each entry in CLB1 410 includes at least an address tag (AT) 411 and a cache location table (CLT) 412. This may store additional information, such as region information (RI) 413. CLT 412 stores cache line location information (LI) for the corresponding region (also called a micropage). Figure 4Four LIs per region are shown as an example. Empirical studies by the inventors have shown that 8 to 16 LIs per region (i.e., 8 to 16 data units (cache lines) per region) are cost-effective and efficient. The location information encodes the location of the corresponding cache line that can be found in a multiprocessor system. For example, the location information may include an identifier of the cache in which the data unit can be found. The location information may also include the location where the data unit can be found within the cache. Alternatively, the location information may include an identifier of the node in which the data unit can be found, or include some symbols that provide information about its location, such as the symbol "MEM" indicating that the data unit can be found in memory. In at least some cases, the symbol "unknown" may indicate that the location of the corresponding data unit is not recorded in the CLT corresponding to the region. MUX 416 selects the CLT with AT 411 that matches the tag portion of ADDR 402, and MUX 415 selects the LI 414 corresponding to the requested cache line based on the L-OFFSET portion of ADDR 402. If the cache identifier stored in the selected L1 corresponds to L2 cache 440, an address 417 for accessing the requested data unit in L2 cache 440 is formed using the index portion of address 402 and a portion of location information 414. In the example of a set-associative L2 cache 440, this portion contains information corresponding to the associated way in which the requested data unit is stored. If the cache identifier stored in the selected L1 corresponds to L1 cache 430, an address 418 for accessing the requested data unit in L1 cache 430 is formed in a similar manner.

[0020] For a cache hierarchy composed of set associative caches with the same associativity, the number of bits in each LI depends on the number of ways and the number of caches of the cache hierarchy covered by the CLB. One LI value may be reserved for invalid mode ("MEM"). In this embodiment, the number of LI bits is equal to the base 2 logarithm of the number of ways plus the base 2 logarithm of the number of cache levels covered by the CLB plus 1 (i.e., log2(ways) + log2(levels) + 1). If the number of ways varies for different cache levels, or if caches other than set associative caches are used, those skilled in the art will understand that other LI representations are possible. Typically, the size of the LI is much smaller than the size of a typical address tag. A 2-level cache hierarchy with 16-way associative caches at each level can be encoded using 6 LI bits.

[0021] exist Figure 4In the example shown in FIG, each entry in L1 cache 430 and L2 cache 440 has a back pointer (BP) (432 and 442, respectively) associated with its cache line (DATA) (431 and 441, respectively). In this example, the BP pointers point to their respective associated CTEs in CLB2 420. Each CTE in CLB2 420 contains an address tag (AT) 421, a CLT 422, and a CLB1 pointer (CIP) 423. If a CLB2 entry has a corresponding CLT residing on CLB1 410, its CIP will point to that entry. In this case, CLB1 410 will contain the most up-to-date information for that region. In this case, the CLE in CLB1 is referred to as the active CLB entry, while the CLB entry for the region in CLB2 420 contains a passive entry. If CLB1 410 contains an active entry for a region, CLB2 420 will be required to contain a passive entry for that region.

[0022] According to at least one example embodiment of the inventive concepts, when no matching CTE is found in CLB1 410, a corresponding CTE is looked up in CLB2 420 and copied to CLB1, while the old entry in CLB1 is copied to CLB2 420, which is referred to as overflow and fill or simply overflow / fill ( Figure 4 Overflow and fill in a traditional inclusive cache). For example, overflow is performed when a data unit is evicted to the next level to make room for a new data unit, which can be brought in (filled) from the next cache level. Similar overflow / fill activities are performed between different CLB levels, where CLB entries are moved between CLB levels because the CLB levels form an inclusive hierarchy. The directory DIR can be located at the root of the CLB hierarchy and use the highest-level CLB for overflow / fill.

[0023] Access to the requested data unit may be initiated in parallel with the overflow / fill activity.

[0024] The contents of the CLBs of a multi-level CLB implementation can be kept consistent with the locations of the data units of the multi-level cache system they manage. Data units moved within the cache hierarchy (including but not limited to movements caused by requests for data units from the CPU, evictions of data units, prefetching activities, and coherency activities) result in updates to their corresponding location information in one or more CLBs. For example, in Figure 4In the example, evicting a data unit 441 from cache 440 may include following its associated BP 442 pointer to locate its corresponding CTE (which, in this example, is in CLB2 420) and updating its corresponding location information to point to the new location of the data unit (e.g., storing the MEM symbol). Furthermore, if the CIP pointer of the CLE corresponding to CLB2 is valid, following the CIP to locate its corresponding CTE in CLB1 410 and updating its location information associated with the cache line. According to at least one example embodiment of the present inventive concepts, if there is a valid CIP pointer, the entry in CLB2 is not updated. For example, the CLT may be updated so that only the active CLT is updated.

[0025] As will be appreciated from the foregoing description, use of the above-mentioned sparsely tagged cache system (i.e., a cache system in which data units stored in the cache do not have address tags associated therewith) will result in many different types of transactions occurring when the CPU 401 requests various cache lines located in different storage devices. Figure 4 The node depicted in FIG can be a subsystem (or node) of a larger multiprocessor system composed of multiple such cache nodes. Keeping the cached data in multiple nodes consistent requires a cache coherence protocol, such as implementing snooping or directory-based coherence. The coherence protocol sends a coherence request to the cache coherence protocol. Figure 4 Node depicted (external request). These requests typically go first to CLB2 420, which may determine that the cache system does not have the requested cache line (CLB2 miss or CLB2 hit an entry with an inactive CIP and the location information of the requested cache line holds a numeric memory pattern). If so, no further action is required with respect to the data caches L1 and L2 (called coherence filtering), but the corresponding LI in the active CLT still needs to be updated to track the new location associated with the data unit. Otherwise, the CLB2 lookup may provide the requested location information (CLB2 hit and the location information of the requested cache line holds the location where the cache line is stored), or it may determine that CLB1 410 stores the associated location information (CLB2 hit and valid CIP). In the latter case, a lookup is required in CLB1 410 to determine the location of the requested cache line or the cache line does not reside in the cache subsystem. (The corresponding location information in CLB1 contains the memory pattern). Depending on the nature of the coherence request, such as an invalidate request, a read request, or an exclusive read request, the coherence protocol will perform some operations on the requested data unit and may change its state. It may also be necessary to update the corresponding LI in the active CLT to track the new location associated with the data unit.

[0026] Although Figure 4 The node shown in FIG is composed of two cache levels, but those skilled in the art will appreciate that other numbers of levels can be used to assemble a node, and that a node can contain more than one CPU. Furthermore, the number of cache levels and the number of CLB levels in a node do not need to be the same.

[0027] Figure 5 is a block diagram illustrating a portion of a computer system including two CPUs connected to a two-level cache hierarchy and a two-level cache location buffer (CLB) hierarchy. For example, Figure 5 The computer system in FIG. 5 includes two CPU nodes (580 and 590), each having a CLB (CLB1A 503 and CLB1B 513) and private L1 caches (504 and 514) in their own L1 cache hierarchy. The system also includes a globally shared L2 cache 523 (sometimes called a last-level cache, LLC) that is shared between the two CPUs 501 and 511.

[0028] A lookup in CLB1 (513 or 503) selects a single L1 based on the address generated by their respective CPUs (501, 511), for example using techniques similar to CLB1 410. As indicated by the three arrows starting from the two CLT entries in CLB1A 503 and CLB1B 513 shown, the selected L1 can identify that the location is in the corresponding L1 cache (504 and 514, respectively), in the L2 cache 523, or in another node (represented by 570 and 571, respectively). In this example, CLB1A 503 identifies that its access data is 505 in the shared L2 cache 523, while CLB1A 503 identifies that its access data is 505 in its L1 514 (shown by the solid arrows).

[0029] Figure 6 is a block diagram illustrating a portion of a computer system including a tag-less cache hierarchy having a single monolithic last level cache. Figure 6 , Figure 6A general description of a tag-less multiprocessor memory system is shown. This example includes N nodes, ranging from node 1 601 to node N 699. Each node has X levels of CLBs and Y levels of dedicated caches (i.e., caches dedicated to that node). The nodes are connected to each other via a network-on-chip (NoC) 650 circuit. The NoC 650 also connects the nodes to a directory (DIR) 660, a global last-level cache (LLC) 670, and memory 680. The DIR 660 is organized similarly to the CLBs and has entries consisting of at least an address tag (AT) 661 and a cache location table (CLT) 662. Entries may also contain additional information, such as region information (RI) 663, which will be described in more detail below. The highest-level CLBs in a node (CLB-X 630, CLB-X 693) overflow into / fill from the DIR. The DIR also plays a central role in the coherence protocol that keeps the contents of the caches and CLBs consistent and coherent.

[0030] The CLB and DIR can be thought of as forming an inclusive "cache hierarchy" that caches metadata containing information about the contents of the data cache hierarchy. The data hierarchy, including L-1 through LY and LLC, can be thought of as a separate cache hierarchy, without the inclusion attribute applied to it. For example, it can be non-inclusive, inclusive, or exclusive.

[0031] exist Figure 6 In the example shown, each node has one CPU (CPU-1 600, CPU-1 696), which may contain zero, one, or more conventional tag-based caches, depicted as LO-D and LO-1. A memory request that cannot be satisfied by any conventional LO cache will generate a lookup in the node's CLB-1 (e.g., 610) to search for the corresponding CLB entry CLT, which may be performed using a method similar to Figure 4640), LLC (670), memory 680, or any of the other nodes (node ​​2 690 to node N 699) as the location of the data unit it is tracking. The LI may also identify the location of the data unit in the identified cache. The LI may also indicate whether the data unit resides in any of the conventional caches L0-D or L0-I of the CPU 600 connected to the node.

[0032] If a cache location (or memory) is identified as where the data can be found, the request can be satisfied by reading the cache directly without consulting the directory (DIR) 660. If a node is identified as where the requested data can be found, the request is sent to the CLB-X of that node and the request is fulfilled with the help of the CLB-X. Figure 4 The external requests discussed are similarly satisfied. However, there is still no need to consult the directory to find the data. This is different from most traditional directory-based consistency schemes, in which the directory is consulted and updated for accesses outside the node.

[0033] despite this, Figure 6 While one CPU is shown in each node, those skilled in the art will recognize that a node may contain any number (including zero) of CPUs, GPUs, accelerators, or other devices that can access memory, such as I / O devices. Furthermore, the configuration and size of CLBs and caches may vary significantly between nodes.

[0034] According to at least one example embodiment of the present invention, there may be a strict hierarchical search level by level in an inclusive CLB / DIR hierarchy. If the location information is not found at level CLB-i, a search is performed at the next level CLB-(i+1). If the location information is not found in the highest level (CLB-X) of the node, a search is performed in the DIR. If the location information is found in the CLB level or DIR, a read request may be sent to the data location identified by the corresponding location information of that level, and a new CLE entry corresponding to the requested area is created in CLB-1. The CLB / DIR hierarchy is inclusive. This means that if there is an available CLB entry at level CLB-L, an entry must be installed at level CLB-(L+1). The CLB entry closest to the CPU is in the active state. The CLB entry may be found in cache level K (and lower caches, i.e., Figure 6higher caches in ) and a certain CLB level C (and all higher CLB levels, i.e. Figure 6 There is forced inclusion between caches (lower locations in the L-1 620 and LY 640). For example, in order for data to be available in any cache L-1 620 through LY 640, there must be a corresponding entry in CLB-X 630. If this entry in CLB-X 630 is evicted, all data in the corresponding region must be evicted from caches L-1 620 through LY 640. This is called forced eviction. In addition, all CLB entries from the same node down to CLB-1 610 must also be evicted.

[0035] It should be noted that there is no containment requirement between cache levels within a node (e.g., cache L-1 620 to LY 640) or between LLC 670 and caches in a node (e.g., cache L-1 620 to LY 640). For example, valid data corresponding to a particular address may be present in L1 cache 620, but not in cache LY 640 or LLC cache 670. This enables cache bypass optimizations, where, for example, streaming data only needs to be installed in L1 cache 620 (actually, only in L1, if it exists), and not in other levels. However, the corresponding CLB entries (typically 50 times smaller than the corresponding data area) need to be installed in all levels.

[0036] Directory (DIR) 660 contains CLB-like information: address tags (AT) 661, cache location table (CLT) 662, and region information (RI) 663. While these fields have the same names as CLB fields (e.g., CLB-X 630), their contents may differ from the corresponding CLB fields. RI field 663 of DIR 660 may contain N so-called presence bits (PBs) that indicate which nodes are tracking the region, where N corresponds to the number of nodes in the system. If bit K in the PB is set, it means that a corresponding entry exists in CLB-X for node K. This further means that the cached data may exist in any of cache levels L-1 through LY for node K, and that a corresponding entry may also exist in CLB-1 for node K, enabling the CPU in node K to access any cache line of the corresponding region. While the description relates to an N-bit implementation of the PB information, those skilled in the art will recognize that many scalable techniques for directory implementations (including, but not limited to, coarse-grained, bounded pointers, and linked lists) can be used to achieve similar functionality.

[0037] The PB bit can be used to classify regions. If one of the PB bits of the corresponding entry in directory 660 is set, the region is classified as a private region (PR). Empirical studies have shown that, on average, approximately 80% of all accessed regions are PR regions, a widely selected research benchmark. Classifying a region as PR means that the data in that region can only exist in one node whose corresponding PB bit is set, and it is also guaranteed that no other node can access the data from that region at that point in time. Node 1 601, whose PB bit is the only set bit in the DIR, can be notified that the corresponding region is a private region (PR), and the region classification can be recorded in its corresponding region information (RI) in its CLB (e.g., in CLB-1 610 to CLB-X 630). PR regions can be accessed efficiently in many ways. For example, global consistency is not required to add write permissions to a node for a data unit that exists in the node, for which the node currently only has read permissions (to upgrade the data unit from state S to state E), because only one node may have a cached copy of the data for the data unit, and no other node can access any data unit in the region. Furthermore, the movement of data elements of a PR region can be achieved efficiently because data can be moved up and down the entire cache hierarchy (e.g., performing an eviction from LX to LLC or moving data from LLC to L-1) without notifying any structures outside the node. This movement also needs to be recorded in the node's local CLB (610 to 630). This means that the CLT information of the PR region stored in the node's CLB (610 to 630) can be different (e.g., more recent) than the CLT information of the region stored in the DIR 660.

[0038] When the second node (e.g., 690) accesses the PR region (empirical studies have shown that this is a relatively rare event), it will miss in all of its CLBs 691 to 693 and will send a request to DIR 660 to obtain the appropriate location information L1 required to access the data and allow its own CLB entries to be created in CLBs 691 to 693. Since this request is for the PR region, DIR 660 may not have the latest information and will need to obtain the latest location information from Node 1 601, ensure that Node 1 601's CLB no longer marks the region as PR, update its own CLT information for the region, set the PB bit corresponding to Node 2 690, and send information about the appropriate CLB for the region to Node 2 690. Node 2 690 can now create its own local CLB entry for the region. Before sending the latest location information to DIR 660, Node 1 601 may have to complete all of its ongoing direct data memory requests for the region.

[0039] If another node (e.g., node N 699) accesses the region, it will also miss in all of its CLBs and will also send a request to DIR 660, which now has the latest information about the region and can reply with the appropriate information to node N 699, and will also set the PB bit for the region for the requesting node (i.e., node N 699).

[0040] A region with more than one PB bit set is classified as a shared region (SR). This means that multiple nodes have CLBs that are tracking the location information of the region. It also means that the nodes with the corresponding PB bits set can store data units of the region in any of their caches (e.g., 620 to 640). If the region is classified as SR, a request from the CPU (e.g., 600) to access data units that cannot be satisfied locally within the node (e.g., by caches L-1 620 to LY 640) may need to initiate a global coherence transaction.

[0041] Globally coherent read transactions to SR or PR regions (reads that cannot be satisfied by local caches (e.g., caches L-1 620 to LY 640)) can utilize location information retrieved from its local CLB (e.g., CLB-1 610 to CLB-X 630) to directly access the data without involving the DIR 660 or communicating with any node outside of that node that can be identified by the location information. This is referred to herein as a direct-to-master access. Empirical studies have shown that 75% of all global coherent transactions to SR regions are direct-to-master read transaction accesses. Therefore, if 20% of CPU-initiated global transactions are for SR pages and 75% do not require access to the DIR, then when using at least some of the cache structures and techniques mentioned above, 95% of global coherent transactions do not require access to the DIR, whereas for traditional directory-based protocols, all global coherent transactions require access to the DIR.

[0042] Because an SR region has several nodes that track the locations of its data units, the movement of their data units needs to be reflected in all nodes tracking the region (i.e., those nodes with active CLB entries for the region) and the location information of the moved data units updated in these nodes. The location information of the CLBs of different nodes tracking data units should be updated in a consistent manner so that the CLBs agree on the location of the data (called its primary location). It is also possible or optional to guarantee that the data unit location accessed based on the location information retrieved from the CLB contains the correct data (called determinism).

[0043] A common reason for this type of data unit movement is a globally consistent write, where a requesting node that does not currently have write permission for a data unit requests write permission through a globally consistent request (e.g., an invalidate request or an exclusive read request). After the request is completed, the data unit will reside only in the requesting node.

[0044] A global consistency write can be sent by a requesting node (e.g., 601) to a directory 660, which locates the corresponding directory entry to retrieve the corresponding PB bit for the region and forward the request to all nodes (except the requesting node) with the corresponding PB bit set, which are called slave nodes (e.g., nodes 690 and 699). When a slave node ensures that any future requests for the data unit originating from the slave node will be directed to the requesting node, the slave node sends an ACK message. This can be achieved, for example, by changing the location information of the data unit in their CLB to point to the requesting node (601), which will become the new master node. The ACK message can be sent to the DIR 660 or directly to the requesting node 601. In the case where the ACK message is sent to the DIR 660, the DIR 660 aggregates the ACK message and sends an ACK to the requesting node once all ACKs are received. Once the requesting node knows that all ACKs have been received, it has obtained exclusive write permission for the cache line and completes the request. After completing the request, a completion message can be sent to the DIR 660.

[0045] Another common reason for data movement is eviction (also known as replacement). Node-local evictions (e.g., eviction from L-1 620 to LY 640) are handled locally, tracked by their local CLB entries, and are not visible outside the node. However, the global eviction of a master data unit (e.g., from location 641 in LY 640 to a new location 671 in LLC 670) must be reflected by all nodes tracking that region (i.e., all nodes with CLB entries for that region). Global eviction can be achieved by first copying the value of the data unit stored in its old location (641) to the new location (671), while keeping the data stored in the old location (641) valid. A global consistency eviction request with information about the new location (671) is then sent from the requesting node (e.g., 601) to the DIR (660), which forwards the request to the slave nodes (e.g., 690 and 699). Once the slave nodes update their corresponding location information to the new location (671), they send an ACK message. Once the requesting node knows that all ACKs have been sent, it obtains permission to evict the data unit and can reclaim the old position and complete the request.Once the request is completed, a completion message can be sent to the DIR 660.

[0046] The slave node may not send an ACK message until some outstanding requests for the requested cache line (eg, all direct requests to the master) have completed.

[0047] The directory 660 can implement a blocking mechanism that ensures that there is at most one outstanding global request of a particular type per data unit. Examples of such request types may be global coherence write and global coherence eviction requests. The block on the cache line ends after the request completes, or as otherwise specified by the coherence protocol. The blocking mechanism can be precise, such as implemented with one lock per data unit in the directory, or it can be highly approximate, where addresses hash to a finite pool of locks, and blocking a data unit blocks access to all other data units that hash to the same lock.

[0048] Blocking can be done at a region granularity rather than a data unit granularity, and some CLB / directory overflow / fill requests will also need to block until their updates have completed. This ensures that globally consistent write requests are not processed while nodes and directories are overflowing / filling CLB information.

[0049] Another useful region classification is a region where no PB bit associated with the region is set in the DIR 660. This means that the region data units will not exist in any of the node's caches L-1 to LY, and no node can currently access its data units. This is called a non-tracking region (UR). The data in the UR region can still be cached in the LLC. The data units in the UR region can be evicted from the LLC 670 without any global consistency request. Regions that do not have corresponding entries in the DIR 660 are classified as memory regions (MR). The data units in the MR region can neither be cached in the node nor in the LLC. Replacing the CLT entry from the DIR 660 means that all data units in the region should be forcefully evicted from the LLC.

[0050] Figure 6 The cache system may be a tag-less cache system, i.e., a cache system in which data units stored in the caches (e.g., caches L-1 through LY and LLC of a node) do not have address tags associated with them. Instead, location information with address tags associated with them identifies the location where the data unit is stored. Figure 6 The traditional caches in (LO-1 and LO-D) can still have address tags associated with their cache data cells.

[0051] Figure 7 is a block diagram illustrating a generalized sparsely tagged cache hierarchy having many slices of last level cache. Figure 7 and Figure 6 The difference is that the LLC has been cut into LLC slices (LLC-1 771, LLC-2 772 and LLC-N 779) and placed on the side of the network on chip (NoC) 750 close to the node. Each slice is configured with a replacement counter (RC) 775, 776 and 777 respectively. These counters count the number of cache replacements in each LLC slice. It should be noted that Figure 7 The example shown in places LLC slices outside of each node (indicated by the dashed boxes). Even though each LLC slice (e.g., 771) is adjacent to each node and can be accessed from the node's CPU (700) with very low latency, they can still be considered part of a large global LLC. Evicting a region entry from CLB-X (e.g., 730) does not necessarily require that the corresponding region data residing in an LLC slice (e.g., 771) close to the node be forcibly evicted. In addition, a node (e.g., 701) can store location information in its CLB (e.g., 710) that directly points to a location in a remote LLC slice adjacent to the remote node (e.g., the location in 772) and access that data without indirections through DIR 760 or any CLB of that node (790).

[0052] The existing LLC implementation method is reminiscent of Figure 7 (Only insofar as LLC is physically divided into banks that are physically placed close to the nodes, Figure 7 ion of the LLC memory banks). However, current solutions (e.g., server chips from Intel) place data cells in LLC banks based on their addresses, which results in 25% of accesses that hit in an LLC bank hitting in a bank close to a node, assuming four nodes. The cache access techniques described according to at least some of the examples discussed above (e.g., based on location information pointers) allow for sufficient flexibility in placing data cells and are independent of the addresses of the data cells, which may increase the local portion of LLC hits. One option is to replicate all shared data in all LLC slices where the shared data is accessed. However, replicating all shared data in all LLC slices will consume LLC capacity. As mentioned earlier, 20% of the area is typically shared. Assuming that all shared areas need to be replicated in all nodes, a four-node system will occupy 80% of the LLC capacity for storing replicated LLC data. This may be suboptimal, prompting the need for smarter LLC allocation / replication decisions.

[0053] When a data unit is first allocated to an LLC slice, it would appear beneficial to always place it in the LLC slice adjacent to the CPU accessing it. However, because threads running on different CPUs may have different LLC capacity requirements, it may sometimes be beneficial to place the data unit in a remote LLC slice, thereby "stealing" some of its local LLC capacity. In at least one example, allocation decisions are made based on LLC slice cache pressure. Each LLC slice has replacement counters (RC 775, 776, 777) that record the number of replacements that have occurred in that slice. In at least one example, these counters are periodically monitored to determine the cache pressure of each LLC slice, with higher counts indicating higher pressure. When allocating LLC space for a data unit, if its cache pressure is lowest, space is always allocated in the local LLC slice. If not, a configurable portion of the allocation is made in the remote LLC slice currently having the lowest pressure, with the remainder allocated in the local LLC slice. In at least one example, allocation of space in a remote LLC slice occurs only under certain circumstances, such as when the cache pressure of the local LLC slice is above a certain threshold.

[0054] In traditional cache topologies, a single last-level cache is shared by multiple nodes (e.g. Figure 1 L2 103 or Figure 6 LLC 670), there is typically at most one copy of each data unit residing in the LLC (called a shared cache), while caches dedicated to a node (also called private caches) (e.g. Figure 1 The L1 caches 102) can each store duplicate copies of the same data unit. Figure 7 An LLC slice may choose to store a single copy of a data unit in one of the LLC slices and have all nodes access it using the same location information, or may choose to store multiple so-called replicated copies of the data unit in multiple LLC slices.

[0055] For example, data unit 774 may be the only LLC copy of a data unit, and both CLT 712 in CLB-1 710 and the CLT in CLB-1 791 have associated location information pointing to location 774. This is referred to herein as shared LLC data. After deciding to replicate the data, the copy of the data may be placed in location 773 and the associated location information in CLT 712 may be changed to point to location 773. The data unit stored in 773 is referred to as a local LLC data unit. Local data stored in 773 can only be accessed indirectly through a lookup in the CLB local to Node 1 701, while global data stored in location 774 can still be accessed directly, for example, by using the location information pointing to location 774, via Node N 799.

[0056] In at least one example, Figure 7 The cache system of is a tag-less cache system, i.e., a cache system in which data units stored in the caches (e.g., caches L-1 to LY and LLC of a node) do not have address tags associated with them. Instead, location information with address tags associated with them identifies the location where the data unit is stored. Figure 7 The traditional caches in (LO-1 and LO-D) can still have address tags associated with their cache data cells.

[0057] Summary and reference Figures 3 to 7 As will be appreciated by those skilled in the art from the discussion of cache hierarchies today, current implementations of cache hierarchies can require many costly operations and moves on data cells. The cost of some of these operations depends on which operations have previously been applied to the data cell. For example, if a data cell enters the L1 cache in a writable state, a read request for the data cell will reduce the cost of a subsequent write operation to the data cell. Furthermore, the cost of data movement depends on the placement decisions made by previous operations.

[0058] Therefore, it is desirable to provide systems and methods that predict future operations and enable more optimal choices to be made for current operations. Summary of the Invention

[0059] According to at least one example embodiment of the present inventive concept, data units are classified as belonging to a data set and are assumed to have similar behavior to other data units of the same data set. The current behavior of the data set is recorded by counting the occurrence of certain events for the data units belonging to the data set. The data set is predicted to have similar behavior in the future, which prompts the application of certain modifications or (alternatively) optimizations for future operations on the data set. One such modification or (alternatively) optimization includes future per-data set placement decisions for a two-dimensional non-uniform cache architecture (NUCA) array. At least some example embodiments of the present inventive concept also include a general method for predicting various behaviors of each data set, for which known modifications or (alternatively) optimizations exist and can be applied.

[0060] According to at least some example embodiments of the present inventive concepts, a data management method for a processor, the processor being assigned a first cache, a second cache, and a behavior history table, the method comprising: tracking reuse information learning cache lines stored in at least one of the first cache and the second cache; recording the reuse information in the behavior history table; and determining a placement strategy regarding future operations to be performed on a plurality of cache lines stored in the first cache and the second cache based on the reuse information in the behavior history table.

[0061] According to at least some example embodiments of the present inventive concepts, a multiprocessor system includes: a first cache and a second cache having different cache levels from each other; and at least one processor core configured to determine a placement strategy regarding future operations to be performed on a plurality of cache lines stored in the first cache and the second cache based on reuse information recorded in a behavior history table, wherein the reuse information is information regarding reuse of learning cache lines stored in at least one of the first cache and the second cache.

[0062] According to at least some example embodiments of the present inventive concepts, a non-transitory computer-readable storage medium includes instructions that, when executed by a processor assigned with a first cache, a second cache, and a behavior history table, cause the processor to perform operations including the following steps: tracking reuse information learning cache lines stored in at least one of the first cache and the second cache; recording the reuse information in the behavior history table; and determining a placement strategy regarding future operations to be performed on a plurality of cache lines stored in the first cache and the second cache based on the reuse information in the behavior history table. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The above and other features and advantages of the exemplary embodiments of the present invention will become more apparent by describing in detail exemplary embodiments of the present invention with reference to the accompanying drawings. The accompanying drawings are intended to illustrate exemplary embodiments of the present invention and should not be interpreted as limiting the intended scope of the claims. Unless explicitly stated, the accompanying drawings should not be considered to be drawn to scale.

[0064] Figure 1 is a block diagram illustrating a portion of a computer system;

[0065] Figure 2 is a block diagram used to explain an example cache hierarchy for a computer system;

[0066] Figure 3is a block diagram for explaining a conventional implementation of a two-level cache hierarchy including a set-associative cache and a set-associative data translation lookaside buffer (TLB);

[0067] Figure 4 is a block diagram illustrating an example implementation of a less-tagged cache;

[0068] Figure 5 is a block diagram illustrating a portion of a computer system including two CPUs connected to a two-level cache hierarchy and a two-level cache location buffer (CLB) hierarchy;

[0069] Figure 6 is a block diagram illustrating a portion of a computer system including a tag-less cache hierarchy including a single monolithic last level cache;

[0070] Figure 7 is a block diagram illustrating a generalized sparsely tagged cache hierarchy having many slices of last level cache;

[0071] Figure 8 is a block diagram illustrating a portion of a computer system including a tag cache hierarchy extended to support future behavior prediction (FBP) according to at least some example embodiments of the present inventive concepts;

[0072] Figure 9 is a block diagram illustrating a portion of a computer system including a tag-less cache hierarchy extended to support future behavior prediction (FBP) according to at least some example embodiments of the inventive concepts;

[0073] Figure 10 is a block diagram illustrating three alternative ways of implementing a behavior history table (BHT) according to at least some example embodiments of the inventive concepts;

[0074] Figure 11 is a block diagram illustrating a portion of a computer system including a non-uniform cache architecture (NUCA) cache system according to at least some example embodiments of the inventive concepts, wherein both L2 and L3 are non-uniform caches;

[0075] Figure 12 is a block diagram illustrating a behavior history table (BHT) for NUCA placement according to at least some example embodiments of the inventive concepts; and

[0076] Figure 13 is a behavior history table (BHT) illustrating general prediction and modification or (alternatively) optimization in a computer system according to at least some example embodiments of the present inventive concepts.

[0077] Block diagram of . DETAILED DESCRIPTION

[0078] As is conventional in the field of the present invention, embodiments are described in terms of functional blocks, units and / or modules and illustrated in the accompanying drawings. It will be understood by those skilled in the art that these blocks, units and / or modules are physically implemented by electronic (or optical) circuits, such as logic circuits, discrete components, microprocessors, hard-wired circuits, memory elements, wiring connections, etc. that can be formed using semiconductor-based manufacturing technology or other manufacturing technologies. In the case of blocks, units and / or modules implemented by microprocessors or the like, software (e.g., microcode) can be used to program them to perform the various functions discussed herein, and can optionally be driven by firmware and / or software. Alternatively, each block, unit and / or module can be implemented by dedicated hardware, or can be implemented as a combination of dedicated hardware that performs certain functions and a processor (e.g., one or more programmed microprocessors and related circuits) that performs other functions. Moreover, without departing from the scope of the present invention, each block, unit and / or module of the embodiment can be physically divided into two or more interactive and discrete blocks, units and / or modules. Furthermore, without departing from the scope of the inventive concept, the blocks, units and / or modules of the embodiments may be physically combined into more complex blocks, units and / or modules.

[0079] During the execution of an application, many costly operations and movements are performed on data units. The cost of some of these operations depends on which operations have previously been applied to the data unit, for example, if a data unit enters the L1 cache in a writable state, then a read request for the data unit will reduce the cost of subsequent write operations to the data unit. In addition, if it is known that the area is likely to remain dedicated in the future, operations applied to the dedicated area can be handled more ideally or (alternatively) optimally. In addition, the cost of data movement depends on the placement decisions made by previous operations. For example, if data that is likely to be reused is placed in a faster cache than data that is less likely to be reused, then future operations are more likely to find the requested data in the fast cache. Therefore, it is desirable to provide such a system and method that predicts future operations and is able to make a more desirable choice or (alternatively) the best choice for the current operation.

[0080] According to at least some example embodiments of the present invention, a future behavior prediction (FBP) mechanism may be used to predict such future operations. According to at least one example embodiment of the present invention, the FBP is constructed by a combination of some or all of the following components:

[0081] 1. Identifying Data Sets: The behavior of each individual cache line can be tracked. Another alternative is to track the behavior of groups of cache lines (referred to herein as data sets) that are believed to have similar behavior. According to at least one example embodiment of the present inventive concepts, data units located near each other in the address space are determined to belong to the same data set. For example, according to at least some example embodiments, the address space can be divided into N different groups of consecutive addresses. Furthermore, the N groups of addresses can correspond to N data sets, such that data units having addresses included in a particular group of the N address groups are considered to belong to the data set corresponding to that particular group of the N data sets. According to at least one example embodiment of the present inventive concepts, each data set can be identified with the assistance of a programmer, a compiler, and / or a runtime system. According to at least one example embodiment of the present inventive concepts, the program counter (PC) value (i.e., the value stored in the PC or the instruction address) of the instruction that brought the cache line into the cache hierarchy from memory or from a cache level above a certain FBP level threshold is used to identify the data set to which the cache line belongs. According to at least another example embodiment of the present inventive concepts, the PC value that caused the TLB fault of the page where the data resides is used to identify the data set of that page. According to at least another example embodiment of the present invention, for a region where data resides, the PC value that caused a CLB miss at a CLB level is used to identify a data set for that region. According to at least another example embodiment of the present invention, the PC value of the instruction that generated at least one of the following "cache line requests" that initiated the start of a hardware prefetch flow is used to identify the data set. According to at least another example embodiment of the present invention, call stack information (e.g., identification of the PC value of the last function call) is used to identify the data set. According to at least one example embodiment of the present invention, two or more of the above-described schemes are combined to identify the data set. Those skilled in the art will recognize that, to save storage space, both the call stack and the PC value can be represented by a subset of their address bits or by some other transformation function that uses their address bits as input. According to at least one example embodiment of the present invention, the data set is identified by a data set identifier (DID). According to at least one example embodiment of the present invention, the DID is composed at least in part of some bits from the CP, some call stack information, and / or some address bits of the address range.

[0082] 2. Detecting special usage: One or more types of special usage of cache lines can be detected and recorded. For example, according to at least one example embodiment of the present invention, the number of special usages of a cache line or data set (e.g., read accesses) of a certain type is tracked and recorded by a counter that counts the number of occurrences of special usage of the cache line or data set. Each type of cache line usage may be recorded as a special usage. The types of such special usage to be tracked and recorded include, but are not limited to, read accesses, write accesses, cache allocations, cache evictions, cache evictions of cache lines that have never been reused, region transitions from dedicated to shared regions, transitions of cache lines from read-only to write-only, the number of cache lines currently resident in the cache hierarchy, or the number of regions or pages currently resident in the cache hierarchy. According to at least one example embodiment of the present invention, the reuse information consists of a single reuse bit that records whether a cache line, region, or page has been accessed since it was initially installed (or stored) at a particular level. According to at least one example embodiment of the present invention, the reuse of cache lines at a particular cache level is determined by looking at the reuse information of the cache line when the cache line is replaced. Those skilled in the art will appreciate that more specific usage types can be tracked and recorded, and that a complete list need not be enumerated. According to at least one example embodiment of the present invention, each core records some specific uses of cache lines, while other specific uses are recorded for the entire system. Modern computers are typically equipped with a large number of event counters capable of counting a large number of different hardware events. All of these events can also be recorded using the described mechanism.

[0083] 3. Selective Learning: Sometimes, recording every unique usage for all cache lines may be too costly. According to at least one example embodiment of the present invention, so-called learning cache lines are selected, and unique usages are collected only for these cache lines. According to at least one example embodiment of the present invention, the learning cache lines are randomly selected. According to at least one example embodiment of the present invention, only cache lines belonging to certain pages, regions, or other types of address ranges (which may be referred to as learning pages, learning regions, or learning address ranges) are learning cache lines. According to at least one example embodiment of the present invention, each such learning page, region, or address range is randomly selected. According to at least one example embodiment of the present invention, each such page, region, or address range is labeled as a learning address range or learning cache line. Learning cache lines can also be selected based on which data set (DID) they belong to. According to at least one example embodiment of the present invention, all cache lines are learning cache lines. Several of the above selection methods can also be combined.

[0084] According to at least one example embodiment of the present inventive concept, a learning cache line is operated in a special manner. For example, a learning cache line may be installed in all cache levels, while the remaining cache lines are installed only in levels identified by a specific placement strategy (e.g., a placement strategy associated with its DID). According to at least one example embodiment of the present inventive concept, special use is detected only for the learning cache line, as described above in "2. Detecting Special Use."

[0085] 4. Record special reuse: When a special use of a learning cache line is detected, the detection is recorded in a behavior history table (BHT). According to at least one example embodiment of the present inventive concept, a behavior history table (BHT) is used to record data reuse. The BHT collects reuse information from learning cache lines of different cache levels. In one embodiment, each entry in the BHT is associated with a BHT identifier (BHTI), which is at least partially a dataset identifier (DID). Each BHT has some behavior counters (BC) that are updated each time a corresponding special use is recorded for a dataset associated with a BHT entry. The BHT can be organized as an associated storage indexed by some of the BHTI bits and marked by some of the BHTI bits. The BHT can also be organized as a table indexed by some of the BHTI bits, but without markings.

[0086] When a special usage of a learning cache line is detected, an associated BHT entry is selected at least in part by using the DID associated with the cache line. A behavior counter (BC) of the selected BHT entry corresponding to the detected special usage is incremented or decremented.

[0087] 5. History-Based Policy: Based on the reuse information collected in the BHT, a policy can be determined for future operations on certain cache lines, regions, pages, or other address ranges. For example, the policy can be based on the assumption that the counter values ​​collected for a dataset will be representative of the dataset's future behavior. For example, the counters of one or more BHT entries can be periodically checked, and a policy can be determined for future accesses to the dataset corresponding to the BHT entry or entries. For example, for dataset Z, identified by a DID, which has shown good reuse (e.g., reuse equal to or above a threshold that can be set based on empirical analysis) at cache level X but not at cache level Y, the corresponding future policy is to install dataset Z in cache level X instead of cache level Y. In another example, for dataset A, identified by a DID, which shows more frequent reuse than dataset B, identified by a different DID, when accessing a cache with variable response times (e.g., a non-uniform cache architecture (NUCA)), the future policy is to install dataset A in the faster portion of the cache and dataset B in the slower portion of the cache. In yet another example, for a dataset C identified by a DID that has shown better reuse than a dataset D identified by a different DID, and where it has been identified that dataset C is primarily accessed by CPU P, a future strategy is to install dataset C in a cache or a portion of a cache that has shorter access times with respect to CPU P, after which appropriate placement of dataset D is determined.

[0088] The latest policy decision for each BHT entry can be stored with the BHT entry. For example, before making an installation decision for a cache line of dataset A identified as a particular DID, that DID can be used to look up the corresponding entry in the BHT and its latest policy for guiding cache line installation.

[0089] Figure 8 is a block diagram illustrating a portion of a computer system including a tag cache hierarchy extended to support future behavior prediction (FBP) according to at least some example embodiments of the inventive concepts. Figure 8 An example of an implementation of Future Behavior Prediction (FBP) in a conventional cache hierarchy (i.e., a tagged cache, not a tag-less cache) according to at least one example embodiment of the inventive concept is shown, which can be viewed as Figure 3 Although Figure 8 Only two cache levels are shown, but the functionality can be generalized to more than two cache levels. Figure 8The two-level cache hierarchy shown in FIG can also constitute a node in a multiprocessor system constructed from many such nodes. Each cache line in the L1 cache 810 and the L2 cache 820 has been extended with the following fields: a reuse information field R (813 and 823) for recording the reuse behavior of the cache line while the cache line is resident in the cache; a learning bit L (811, 821) for indicating whether the corresponding cache line is a learning cache line; and a data set DID field (812, 822) for identifying the data set having the corresponding cache line. Figure 8 The remaining unnumbered elements of the cache hierarchy are separated by Figure 3 The corresponding elements shown in FIG. 1 function in the same manner and reference is made to the preceding description of those elements.

[0090] A behavior history table (BHT) 870 is also added. Each entry in table 870 contains a data set identifier DID 871 for identifying the data set associated with each entry and used, for example, as an address tag to allow associative lookups in the BHT 870 structure, as well as some reuse information collected for the data set. In this embodiment example, counters (873, 875) are shown that count the number of learning cache lines with reuse at each level (1R, 2R, 3R...). Counters (874, 876) are also shown that count the number of unused learning cache lines (1U, 2U...). Based on the counter value, a placement strategy for the data set is selected. The current placement strategy is stored in the strategy field POL 872. According to at least one example embodiment of the present invention, the strategy is represented by a bit at each level in the cache hierarchy to indicate whether the data set identified by DID871 should be installed in that cache level. Those skilled in the art will appreciate that a variety of different implementation options (including Figure 10 Some embodiments in ) implement similar functionality, which will be discussed in more detail below.

[0091] Figure 10 is a block diagram illustrating three alternative ways of implementing a behavior history table (BHT) according to at least some example embodiments of the present inventive concepts. Figure 10Example (A) shows a group-associative BHT 1000, where a set (shown as the top-level set in this example) is selected based in part on some indexing function of the DID, and all address tags of that set are compared to some lookup key to determine a hit. Parts of the DID are used as address tags 1001 and 1004 to identify group-associative entries in BHT 1000. In this example, assume hit 1001. As described above, its BHT entry contains a set of counters and some policy POL 1002 determined by previous counter values. In example (B), some portion of the DID is used to select an index for accessing the BHT. The single entry selected will represent that DID and all other DIDs with the same indexing function without performing any comparison (in this example, the top-level entry is indexed using the DID and used, for example, using POL 1012, to determine the policy). Example (C) is a system with one BHT 1020 and multiple policy tables (PT 1022 and PT 1023), potentially distributed near each CPU core.

[0092] According to at least one example embodiment of the present inventive concepts, FBP is used to make placement decisions for a cache hierarchy with four cache levels (4kB, 32kB, 256kB, and 8MB, respectively). Each cache entry is extended to store a learning bit (L), one or more reuse bits, and a data set identifier consisting of the 12 least significant bits of the PC value, which identifies the cache line being entered from memory into the cache hierarchy. The BHT is organized as a 256-set associative cache with 4 ways each. A BHT entry contains a 6-bit DID tag, a 4-bit policy field (one for each of the four cache levels), and two 6-bit counters, U and R, for each cache level. When either counter reaches its maximum value or (alternatively) a threshold, if the corresponding R counter value is above a threshold (e.g., 48), a decision is made to install the data in the corresponding cache level. In some applications, FBP according to these embodiments has been shown to significantly reduce installs per cache level. On average, FBP performs 50% fewer installs compared to a standard cache hierarchy without a placement policy.

[0093] Figure 9 is a block diagram illustrating a portion of a computer system including a tag-less cache hierarchy extended to support future behavior prediction (FBP) according to at least some example embodiments of the inventive concepts. Figure 9 An example of an implementation of FBP in a less-tagged cache hierarchy according to at least another example embodiment of the inventive concept is shown, which can be viewed as Figure 4 Although Figure 9Only two cache levels are shown, but the functionality can be extended to more than two cache levels. Each cache line in the L1 cache 930 and L2 cache 940 has been extended with a reuse information field R 934 and R 944, respectively, which records the reuse behavior of the cache line while it resides in each cache level, and a data set DID field that identifies the data set of the cache line. In this example, a data set identifier (DID) has not yet been added to each cache line in the L1 cache and L2 cache. Instead, each entry in CLB1 910 and CLB2 920 has been extended with DID information 914 and DID information 915 associated with each entry. The C2P pointer 932, 942 of each cache line in the L1 cache 930 and L2 cache 940 points to an entry in CLB2 920 (which in turn can point to CLB1 910). The DID 915 of the associated CLB 2 entry determines the data set ID for each cache line in the L1 cache 930 and the L2 cache 940 . Figure 9 The remaining elements in this section are the same as those described above. Figure 8 Those described function in the same manner, to which description reference is made for the sake of brevity.

[0094] and Figure 8 A similar behavior history table (BHT) 970 is also added to Figure 9 Each entry in the table contains a data set DID 971 for identifying the data set associated with each table entry, as well as some reuse information collected for the data set. In this example, counters that count the number of reuses at each level 1R, 2R, 3R... are shown and referenced as 973, 975, 977, respectively. Counters that count the number of unused cache lines 1U, 2U... are also shown and referenced as 974 and 976. Based on the counter values, a placement strategy 972 for the data set is shown. Those skilled in the art will understand that similar functionality can be achieved using a variety of different implementation options.

[0095] Figure 8 and Figure 9 The BHT shown in FIG. 1 can be part of a multiprocessor configuration, where the CPU and cache shown in the figure constitute a node that is part of a multi-core configuration built from two or more such nodes. In such a configuration, the BHT can be local to a node and collect information about the specific usage of that node, or global to a multiprocessor and collect information about the specific usage of all nodes in the multiprocessor. According to at least one example embodiment of the present inventive concepts, a multiprocessor can have both a BHT that is local to its nodes and a global BHT.

[0096] As previously described, a data set can be identified, at least in part, by the PC value of an instruction that generates at least one of the cache line requests that causes the hardware prefetch flow to begin. Similar to any other data set described in accordance with one or more exemplary embodiments of the present inventive concepts, this data set will be selected for learning accesses and the optimal placement strategy across cache levels will be learned for the prefetched data set, just like any other data set.

[0097] So-called non-uniform cache architectures (NUCA) are becoming increasingly common. NUCA refers to multiprocessor systems in which one or more cache levels, physically distributed across the cores, are logically shared among the cores. In a NUCA system, a core's access time to "its slice" of the shared NUCA cache is shorter than access time to other slices of the shared NUCA cache. Figure 11 is a block diagram illustrating a portion of a computer system including a non-uniform cache architecture (NUCA) cache system according to at least some example embodiments of the inventive concepts, wherein both L2 and L3 are non-uniform caches. Figure 11 A NUCA multi-core system is shown, in which CPUs 1101 and 1120 have dedicated L1 caches 1102 and 1122, respectively. The L2 cache is a logically shared NUCA cache implemented by separate L2 slices (e.g., 1103 and 1123), which are connected to the CPUs via a switch 1140. Each CPU (e.g., 1101 and 1120) can access all L2 slices, but the access time to their L2 slices (1103 and 1123, respectively) is shorter. Each L2 slice also has an adjacent L3 slice. The L3 slices form a logically shared NUCA cache in a similar manner, that is, the access time of CPU 1101 to its slice's L3 1104 is shorter than the access time to any other L3 slice. However, the access time to the L3 slice is substantially longer than the access time to the adjacent L2 slice. This L2 / L3 NUCA structure with two levels of NUCA cache is called a two-dimensional NUCA array.

[0098] It is beneficial if cache lines can be placed close to the cores that access them. It is also beneficial to place the most frequently reused cache lines in the L2 cache rather than the L3 cache. Figure 11 In the illustrated NUCA, the access cost to each L2 slice and L3 slice may be determined based on the latency cost, communication cost, and energy cost for accesses starting from each CPU and accessing each L2 slice and L3 slice.

[0099] The NUCA-aware placement (NAP) algorithm is a specialized implementation of FBP for desired or (alternatively) optimal cache line placement in NUCA systems. The initial NAP description targets tag-less NUCA systems, e.g. Figure 6 、 Figure 7 and Figure 9 The system depicted in FIG is modified to have an L2 NUCA cache and an L3 NUCA cache and its BHT table 970 is replaced by a NUCA history table (NHT) 1210, which will be referred to below. Figure 12 Similar NAP algorithms can also be applied to marker-based NUCA systems, e.g. Figure 8 The NUCA cache is modified to have an L2 NUCA cache and an L3 NUCA cache and its BHT table 870 is replaced by a NUCA history table (NHT) 1210, which will be referred to below. Figure 12 More detailed explanation.

[0100] NAP uses DID to identify the dataset of each region, such as Figure 9 As shown (e.g., 914 and 915), and may have a specifically allocated learning region, which is marked by a dedicated L bit (not explicitly shown) in the region information RI field (913).

[0101] Figure 12 is a block diagram illustrating a behavior history table (BHT) for NUCA placement according to at least some example embodiments of the present inventive concepts. Figure 12 , Figure 12 The NUCA History Table (NHT) 1210 used by the NAP placement mechanism is shown. Similar to the FBP, this table can be represented in a variety of ways. Figure 12 The representation shown in is similar to Figure 10 The FBP representation (B) of (1010). The goal of the NAP is to determine the policy for indexing the relevant DIDs for NHT entries (POL 1211). To help determine the policy, each NHT entry stores a number of counters that are updated by the particular use of the cache line associated with the entry.

[0102] According to at least one example embodiment of the present inventive concept, assuming that there are four CPUs (cores) in the system, each core has a reuse counter, such as Figure 121212, 1213, 1214, 1215, respectively. According to at least one example embodiment of the present inventive concept, the size counter S (1217) is used to estimate the size of the data structure associated with the NHT entry. According to at least one example embodiment of the present inventive concept, the "unused counter" counts the number of cache lines that were replaced before a single reuse of the cache line occurred.

[0103] Each time the learning cache line in L2 or L3 associated with the entry is accessed by the corresponding core, the per-core reuse counter of the NAP entry is incremented. According to at least one example embodiment of the present inventive concept, each counter is incremented only for a specific type of access (e.g., only for read accesses). According to at least one example embodiment of the present inventive concept, each counter is incremented for all accesses, not just for learning cache lines.

[0104] Each time a data unit associated with the entry is brought into the cache system, the size counter of the NAP entry is incremented, and each time a data unit (e.g., a data unit associated with the entry) is evicted from the cache system, the size counter of the NAP entry is decremented. According to at least one example embodiment of the present inventive concepts, each time a CLB region associated with the entry is allocated or evicted at a level of the CLB hierarchy, the size counter of the NAP entry is incremented or decremented. According to at least one example embodiment of the present inventive concepts, each time a page associated with the entry is allocated or evicted at a level of the TLB hierarchy, the size counter of the NAP entry is incremented or decremented. According to at least one example embodiment of the present inventive concepts, the allocation and eviction of some other data entity associated with the entry will increment and decrement the size counter.

[0105] According to at least one example embodiment of the present inventive concepts, an NHT entry includes an "unused" counter U 1216. Each time a data unit that has never been reused at a particular cache level is evicted from that cache level, the "unused" counter 1216 is incremented. According to at least one example embodiment of the present inventive concepts, each time a data unit that has never been reused at that cache level is evicted from a cache level, for example, when a data unit has never been reused at the L2 or L3 level and is evicted to a cache level higher than the L3 or to memory, the unused counter is incremented. The unused counter 1216 can be used to determine whether a data set should bypass the L2 / L3 cache and be installed only in the L1 cache.

[0106] Periodically, the placement strategy in the NUCA hierarchy is re-evaluated based on data collected in the NHT 1210. This could be, for example, after a certain number of instructions have been executed, after a number of memory accesses have been performed, after a certain number of execution cycles, or when some counter reaches a threshold or (alternatively) a predetermined value. Those skilled in the art will appreciate that many other forms of determining the next placement re-evaluation may be used.

[0107] During placement re-evaluation, NHT entries are sorted according to a priority. According to at least one example embodiment of the present invention, NHT entries are sorted by their total reuse count in relationship to their size, for example, by dividing their total reuse count by their size count or estimating the relationship in some other manner. According to at least one example embodiment of the present invention, the total reuse count can be calculated by adding the individual per-core reuse counters 1211, 1212, 1213, 1214. According to at least one example embodiment of the present invention, the total reuse count is recorded by a separate counter in each NHT entry.

[0108] During placement reevaluation, a placement strategy is determined for each data set in the NHT in a priority order, where each data set corresponds to an NHT entry. The data set with the highest priority is placed in the cache with the lowest cost function relative to the core or cores accessing the data set. According to at least one example embodiment of the present invention, the cost function takes into account the latency and / or communication cost from the core to the cache. According to at least one example embodiment of the present invention, a power estimate for access from the core to the cache is taken into account. According to at least one example embodiment of the present invention, an estimated size of the data set is taken into account. If the data set size is deemed suitable to fit into the selected cache, a portion of the cache proportional to the size of the data set is marked as in use. If the data set size is deemed too large to fit into the selected cache, the entire cache is marked as used, and the remainder of the data set is fit into the cache using a second low-cost function, and so on, until the entire data set has been fit. According to at least one example embodiment of the present inventive concept, the fraction of data sets that fit into each cache is recorded as a data set placement policy, for example, 25% of the data sets are placed in the L2 cache slice of CPU 1, 25% of the data sets are placed in the L2 cache slice of CPU 2, and 50% of the data sets are placed in the L3 cache slice of CPU 1. When the highest priority data set has been placed, the second highest priority data set is placed in a cache that has not been marked as used, and so on, until all data sets that are not considered to bypass L2 / L3 are placed.

[0109] According to at least one example embodiment of the present inventive concepts, some data sets are determined to bypass the L2 / L3 NUCA cache and not be placed in any of its caches. According to at least one example embodiment of the present inventive concepts, the remaining data sets are placed based on a distribution of data set sizes across the caches. According to at least one example embodiment of the present inventive concepts, the placement strives to achieve the same ratio between the size of the data set placed in each cache and the actual size of the data set. According to at least one example embodiment of the present inventive concepts, the placement strives to achieve the same cache pressure across the cache slices, where cache pressure can be measured, for example, as the number of evictions from a cache per use relative to its size. According to at least one example embodiment of the present inventive concepts, the placement strives to achieve a desired or (alternatively) predetermined relationship between the cache pressure of one level of cache (e.g., L2) and some other level of cache (e.g., L3). According to at least one example embodiment of the present inventive concepts, the placement strives to achieve the same replacement age among cache lines replaced from all caches, which is defined as how long a cache line remains unused in a cache before being replaced. According to at least one example embodiment of the present inventive concepts, placement will strive to achieve a desired or (alternatively) predetermined relationship between the replacement ages of one level of cache (e.g., L2) and some other level of cache (e.g., L3).

[0110] The newly determined placement is recorded as a new placement policy and is recorded as a policy associated with each data set in, for example, the policy field 1211 of the NHT entry for the corresponding data set and / or in a separate policy table similar to 1022 or 1023 or having some other representation. Future installations of data into the NUCA hierarchy will follow the placement policy, for example, 25% of the data set is installed in the slice of CPU1 in the L2 cache, 25% of the data set is installed in the slice of CPU2 in the L2 cache, and 50% of the data set is installed in the slice of CPU1 in the L3 cache.

[0111] According to at least one exemplary embodiment of the present inventive concept, the size and frequency of reuse of each dataset are estimated. New global placement decisions are periodically made. First, the dataset with the highest reuse rate per unit size is placed in its most favorable position. Then the dataset with the second highest frequency / size is placed, and so on, until all known datasets are placed using a simple eager packing algorithm. The goal of placement is to place the dataset with the highest probability of reuse close to the core that will use it.

[0112] While this discussion focuses on predicting future access patterns for a dataset and leveraging that prediction to implement Figure 11However, those skilled in the art will recognize that the described approach can be generalized to predict many other types of future behavior and apply modifications or (alternatively) optimization strategies for future operations.

[0113] Figure 13 is a block diagram illustrating a behavior history table (BHT) for general prediction and modification or (alternatively) optimization in a computer system according to at least some example embodiments of the inventive concepts. Figure 13 A Generic History Table (GHT) is depicted, wherein a DID is used to associate a data set identified by the DID with a GHT entry. A GHT entry contains counters that count, for example, any of the so-called hardware counters present in most modern computer systems, where a hardware counter is configured to count one of a number of different events. Any existing or future counter that counts events can be used as such a counter. Figure 13 As shown, the counting of these events is organized as counting events associated with the dataset DID in which the event occurred. Although two counters are shown: CTR1 and CTR2 (1312, 1313), the number of counters per GHT entry is not limited to two. The counters can be used to determine a certain policy for, for example, a dataset stored in the GHT entry itself (as shown in 1311). However, there are many other ways that each dataset counter can be used, and their policies can be organized in many other ways, including but not limited to Figure 10 Those skilled in the art will understand how to use this method to detect a variety of future behaviors, including but not limited to: mostly read cache lines, mostly dedicated areas, mostly written cache lines, mostly migrated shared cache lines, mostly producer-consumer cache lines, mostly write-once cache lines, mostly read-once cache lines, mostly regions or pages of cache line accesses, mostly sparse cache lines, mostly compressible cache lines, etc., which can be predicted using this method. Most of these future behaviors have known modifications or (optionally) optimizations that can be applied to future uses of the corresponding data set.

[0114] For clarity, most of the description herein generally describes techniques for locating a cache line and returning it to the requesting CPU. This description does not detail the various ways in which a requested word contained within a cache line is selected and returned to the CPU. However, various methods for selecting a cache line contained within a cache line and returning the requested cache line to the CPU are known to those skilled in the art.

[0115] For clarity, most of the descriptions herein describing the processing of data in a cache hierarchy describe an exclusive cache hierarchy. Those skilled in the art will appreciate that one or more exemplary embodiments of the present inventive concept can be expanded to also include inclusive and non-exclusive cache hierarchies.

[0116] Although one or more example embodiments of the above-described inventive concepts are useful in connection with both single-processor systems and multi-processor systems, such as those described above with respect to Figure 1 and Figure 2 The ones shown and described, but mainly in association with a single processor system illustrate one or more example embodiments of the inventive concept. However, those skilled in the art will understand that one or more example embodiments of the inventive concept illustrated in conjunction with a single processor system are not limited to such implementation.

[0117] Although described above in the context of certain example computer architectures, caches exist in Figures 8 to 13 The present invention may be used in many other settings both inside and outside the example computer system shown in , and it will be understood by those skilled in the art that at least some of the example embodiments of the invention conceived above in the context of a computer system may also be applied to such other contexts. An example of such use is a virtual memory system that caches data from a slow, high-capacity memory (such as a disk or FLASH memory) into a faster and smaller high-capacity memory that can be implemented using dynamic RAM. Other examples of caching in a computer system include (but are not limited to) disk caches, web caches, and name caches. The organization and caching mechanisms of such caches may differ from the caches discussed above, for example, with variations in the size of the groups, the implementation of the groups, and the associativity. Regardless of the implementation of the caching mechanism itself, at least some of the example embodiments of the invention conceived are equally applicable to implementing various caching schemes.

[0118] The methods or flow charts provided in this application may be implemented in a computer program, software, or firmware clearly embodied in a computer-readable storage medium for execution by a general-purpose computer or processor.

[0119] Having thus described example embodiments of the present invention, it will be apparent that the example embodiments of the present invention may be modified in a variety of ways. Such variations are not to be regarded as a departure from the intended spirit and scope of the example embodiments of the present invention, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the appended claims.

Claims

1. A data management method for a multi-core processor system, the multi-core processor system comprising a plurality of processor cores, a plurality of caches, and a behavior history table, the plurality of caches comprising a plurality of first caches in a first cache level and a plurality of second caches in a second cache level, the method comprising: tracking reuse information of a learning cache line stored in at least one cache of the plurality of first caches or the plurality of second caches; Recording the reuse information in the behavior history table; determining a placement strategy regarding future operations to be performed on a plurality of cache lines stored in the plurality of first caches or the plurality of second caches based on the reuse information in the behavior history table, wherein the second cache level is at a higher level than the first cache level, Each of the plurality of first caches is dedicated to a corresponding processor core among the plurality of processor cores, The plurality of second caches are shared by the plurality of processor cores, The reuse information includes a plurality of reuse counters corresponding to the plurality of processor cores respectively, and Each of the plurality of reuse counters corresponds to a different one of the plurality of processor cores; Each time the corresponding processor core accesses the learning cache line stored in the plurality of second caches, the reuse counter is incremented; Calculating a total reuse count by accumulating the reuse counter; The total reuse count is recorded by a separate counter in the behavior history table; and During placement re-evaluation, entries of the behavior history table are sorted, wherein the entries of the behavior history table are sorted by the total reuse count in relation to their size.

2. The data management method for a multi-core processor system according to claim 1, further comprising: The multi-core processor system updates at least one behavior counter among a plurality of behavior counters included in the reuse information each time a usage type corresponding to the at least one behavior counter occurs with respect to at least one of the learning cache lines.

3. The data management method for a multi-core processor system according to claim 2, further comprising: The multi-core processor system updates the at least one behavior counter each time at least one of the learning cache lines is accessed by a read request.

4. The data management method for a multi-core processor system according to claim 1 , further comprising: The determined placement strategy is stored in the behavior history table.

5. The data management method for a multi-core processor system according to claim 1 , further comprising: At least some cache lines are randomly selected from among the plurality of cache lines stored in at least one cache among the plurality of first caches or the plurality of second caches as the learning cache lines.

6. The data management method for a multi-core processor system according to claim 1, in, The plurality of processor cores include a first core and a second core, and Among them, the access time of the first core to at least one cache among the multiple first caches is shorter than the access time of the second core to at least one cache among the multiple first caches, and the access time of the second core to at least one cache among the multiple second caches is shorter than the access time of the first core to at least one cache among the multiple second caches.

7. A multi-core processor system comprising: Multiple processor cores; Multiple caches; as well as Behavior history table, The plurality of caches include a plurality of first caches of a first cache level and a plurality of second caches of a second cache level higher than the first cache level, the first cache level and the second cache level being different cache levels from each other, wherein at least one processor core among the plurality of processor cores is configured to determine a placement strategy regarding future operations to be performed on a plurality of cache lines stored in at least one cache among the plurality of first caches or the plurality of second caches based on the reuse information recorded in the behavior history table; wherein the reuse information is information on reuse of a learning cache line stored in at least one of the plurality of first caches or the plurality of second caches, The reuse information includes a plurality of reuse counters corresponding to the plurality of processor cores respectively. wherein each of the plurality of reuse counters corresponds to a different processor core among the plurality of processor cores; Each of the plurality of first caches is dedicated to a corresponding processor core among the plurality of processor cores, Each time the corresponding processor core accesses the learning cache line stored in the plurality of second caches, the reuse counter is incremented. wherein a separate counter records a total reuse count calculated by accumulating the reuse counter in the behavior history table, and During placement re-evaluation, entries of the behavior history table are sorted, wherein the entries of the behavior history table are sorted by the total reuse count in relation to their size.

8. The multi-core processor system according to claim 7, wherein: The at least one processor core is configured to update at least one reuse counter of the plurality of reuse counters each time a usage type corresponding to the at least one reuse counter occurs with respect to at least one of the learning cache lines.

9. The multi-core processor system according to claim 7, wherein: The behavior history table includes at least one unused counter, and the at least one processor core is configured to update the at least one unused counter each time at least one of the learning cache lines is replaced before a single reuse of the learning cache line occurs.

10. The multi-core processor system according to claim 7, wherein: The behavior history table includes a policy field storing a policy determined by the at least one processor core according to the placement policy.

11. The multi-core processor system according to claim 7, wherein: The behavior history table includes a data set identifier indicating an address tag corresponding to the learning cache line.

12. The multi-core processor system according to claim 7, wherein: Each of the learning cache lines is extended with a reuse information field configured to store reuse information for the learning cache line.

13. The multi-core processor system according to claim 7, wherein: Each learning cache line is extended with a learning bit that indicates that the learning cache line is a learning cache line.

14. A non-transitory computer-readable storage medium comprising instructions, wherein a multi-core processor system comprises a plurality of processor cores, a plurality of caches, and a behavior history table, wherein the plurality of caches comprises a plurality of first caches of a first cache level and a plurality of second caches of a second cache level, and when the instructions are executed by at least one processor core of the multi-core processor system, the at least one processor core performs operations comprising the following steps: tracking reuse information of a learning cache line stored in at least one cache of the plurality of first caches or the plurality of second caches; Recording the reuse information in the behavior history table; as well as determining a placement strategy regarding future operations to be performed on a plurality of cache lines stored in the plurality of first caches or the plurality of second caches based on the reuse information in the behavior history table, wherein the second cache level is at a higher level than the first cache level, Each of the plurality of first caches is dedicated to a corresponding processor core among the plurality of processor cores, The plurality of second caches are shared by the plurality of processor cores, The reuse information includes a plurality of reuse counters corresponding to the plurality of processor cores respectively. Each of the plurality of reuse counters corresponds to a different one of the plurality of processor cores; Each time the corresponding processor core accesses the learning cache line stored in the plurality of second caches, the reuse counter is incremented; Calculating a total reuse count by accumulating the reuse counter; The total reuse count is recorded by a separate counter in the behavior history table; and During placement re-evaluation, entries of the behavior history table are sorted, wherein the entries of the behavior history table are sorted by the total reuse count in relation to their size.

Images