Lateral Persistence Directory State

Abstract

The computer-implemented method includes defining one or more processor units having a plurality of caches, each processor unit comprising a processor having at least one cache, each of the one or more processor units coupled together by an interconnect fabric; placing a plurality of cache lines into one or more match classes for each of the plurality of caches, each match class including a chronology vector; placing each cache line in the plurality of caches into a cluster of caches based on a plurality of scope domains; determining a first cache line to be evicted based on the chronology vector; and determining a target cache for installing the first cache line based on a scope of the first cache line and a saturation metric associated with the target cache, the scope of the first cache line being determined based on a lateral persistence tag bit.

Description

[Technical Field]

[0001] The present invention relates generally to data processing, and more particularly to lateral persistence directory state in symmetric multiprocessing computers. [Background technology]

[0002] Modern high-performance computer systems are typically implemented as multinode symmetric multiprocessing ("SMP") computers with many compute nodes. SMP is a multiprocessor computer hardware architecture in which two or more, typically many, identical processors are connected to a single shared main memory and controlled by a single operating system. Today, most multiprocessor systems use SMP architecture. In the case of multicore processors, the SMP architecture applies to the cores, treating them as separate processors. Processors may be interconnected using buses, crossbar switches, mesh networks, and the like. Each compute node typically contains several processors, each of which may have at least some local memory, at least some of which is accelerated by cache memory. Cache memory can be local to each processor, local to a compute node shared across two or more processors, or shared across the entire node. Summary of the Invention

[0003] In one aspect, a computer-implemented method includes: defining one or more processor units having a plurality of caches, each processor unit including a processor having at least one cache from the plurality of caches, each of the one or more processor units coupled together by an interconnect fabric; arranging, for each of the plurality of caches, a plurality of cache lines into one or more congruence classes, each congruence class in the one or more congruence classes including a chronology vector; arranging each cache line in the plurality of caches into a cluster of caches based on a plurality of scope domains; determining a first cache line to be evicted based on the chronology vector of the first cache line; and determining a target cache for installing the first cache line based on a scope of the first cache line and a saturation metric associated with the target cache, the scope of the first cache line being determined based on lateral persistence tag bits.

[0004] In another aspect, a computer-implemented method includes receiving a request to evict a first cache line from a first cache on a first microprocessor chip in a plurality of microprocessor chips in a processing drawer, the first cache line having a first set of lateral persistence bits that track a scope for the first cache line; determining a scope of the first cache line; identifying a target cache having a saturation metric, the target cache having a higher scope than the scope of the first cache line; and determining an action for the first cache line based on the saturation metric for the target cache and the scope of the first cache line.

[0005] Other aspects of the invention embody features of the above-described methods in computer systems and computer program products.

[0006] Additional technical features and advantages are realized through the techniques of the present invention. Embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed subject matter. For a better understanding, reference is made to the detailed description and drawings.

[0007] The particulars of the exclusive rights set forth herein are particularly pointed out and distinctly claimed in the claims at the conclusion of this specification. The foregoing and other features and advantages of embodiments of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram of a distributed symmetric multiprocessing (SMP) system that utilizes cache persistence in accordance with one or more embodiments of the present invention. [Figure 2]FIG. 2 is a block diagram of an exemplary target cache, in accordance with one or more embodiments of the present invention. [Figure 3] 1 is a flow diagram of a method for lateral cache persistence in accordance with one or more embodiments of the present invention. [Figure 4] 1 is a flow diagram of a method for lateral cache persistence in accordance with one or more embodiments of the present invention. [Figure 5] 1 is a flow diagram of a method for lateral cache persistence in accordance with one or more embodiments of the present invention. [Figure 6] FIG. 1 is a block diagram of a computer system in accordance with one or more embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0009] The diagrams depicted herein are exemplary. Many variations to the diagrams or operations described are possible without departing from the scope of the invention. For example, actions can be performed in a different order, or actions can be added, deleted, or modified. Also, the term "coupled" and variations thereof indicate that there is a communication path between two elements and do not imply a direct connection between elements without an intervening element / connection between them. All of these variations are considered part of this specification.

[0010] One or more embodiments of the present invention provide a system and method for reducing memory accesses in an SMP environment. Traditionally, reducing memory accesses is addressed by utilizing larger caches backing up smaller caches in an N-level vertical cache hierarchy. This has drawbacks because, at a particular cache level, not all caches are utilized at the same rate. Because not all caches are utilized at the same rate, this opens up the opportunity to utilize any underutilized cache space.

[0011] In one or more embodiments of the present invention, in an SMP environment, an aspect includes defining a lateral cache that can be used to survive cache evictions from peer lateral caches. These peer lateral caches can be divided into clusters of caches, with each cluster representing a persistence scope. An evicted cache line is first allowed to persist in the cluster of the peer cache at the next persistence scope. Then, as cache lines continue to be evicted from clusters, other clusters are available for storing the cache line until the evicted cache line reaches the last cluster and can be evicted to main memory. Lateral persistence tag bits are available to track the cache persistence scope for each cache line. In the case of a cache fetch miss, the lateral persistence directory tag is set to 0 when the cache line is first installed in the system cache. When a cache line is installed into a lateral cache within a cluster, or into another cluster of caches, the directory tag is incremented and set to the corresponding persistence scope to which the target cache belongs. A replacement algorithm / policy is implemented to determine at which level scope the cache line should be evicted. The target cache identifies within the scope level, and an adaptive LRU (least recently used) replacement policy then determines where to install the cache line in the matching class of the target cache.

[0012] FIG. 1 depicts a distributed symmetric multiprocessing (SMP) system 100 (hereinafter "system 100") according to one or more embodiments. System 100 may include four processing units, or "drawers." Each drawer 240-0, 240-1, 240-2, and 240-3 includes eight (8) microprocessor (CP) chips (202-0 through 202-7). Each CP chip may include eight (8) cores 204-0 through 204-7. Each core within a CP chip includes a private L1 cache 206-0 through 206-7 (including both an instruction cache and a data cache). These private L1 caches are backed by semi-private L2 caches 208-0 through 208-7. In one or more embodiments of the present invention, the semi-private L2 caches 208 may interact to provide an on-chip virtual L3 cache. Each processor drawer 240 contains up to eight CP chips 202 with a fully connected topology providing a virtual L4 cache. The virtual L3 and virtual L4 caches can be realized through a set of chip caching techniques that cluster independent physical L2 caches 208 within the chips 202 and drawers 240 to function as a unified shared victim cache.

[0013] In one or more embodiments of the present invention, the virtual L3 / L4 cache is implemented by defining a group / cluster of L2 caches within a CP chip, a group of CP chips, and / or a drawer for evicting cache lines from peer caches, i.e., cache lines are evicted from a first L2 to a peer L2 within the defined group / cluster of L2 caches according to a prescribed replacement policy described herein.

[0014] In one or more embodiments of the present invention, a peer's L2 cache (sometimes called a "lateral cache") can be divided into clusters 214 of caches called primary, secondary, and tertiary caches, which can be expanded to an infinite number of unique scopes. When a cache line is evicted from the L2 cache, the evicted cache line is allowed to persist within the clusters 214 of caches, passing through each cluster until it reaches the final cluster of caches. In one or more embodiments, lateral persistence (LP) tag bits are available to identify the scope of persistence. That is, the tag bits can indicate what the current persistence scope is and how many scopes the cache line can hop before being evicted to memory or referenced again by the processor cache. In the case of a fetch miss, the LP tag bit can be set to 0 when the cache line is first installed in the system cache. Also, when the same cache line is evicted from a cache, the cache line is persisted in any of the other caches in the next persistence scope (e.g., primary, secondary, or tertiary) in the lateral persistence tag bit set for each persistence scope.

[0015] In one or more embodiments of the present invention, each drawer 240 includes one or more cache clusters 214 that are utilized to persist cache lines when evicted from caches within the clusters 214. While the illustrative example shows one configuration of cache clusters 214, in one or more embodiments, the clusters 214 can include any number of L2 caches in any type of configuration, including across drawer L2 caches within a group / cluster. In one or more embodiments of the present invention, data in the cache is arranged into congruence classes that include multiple cache lines, the congruence classes include a chronology vector that is used to determine which entry should be evicted, and cache eviction is performed using an adaptive LRU replacement policy. If there is no empty partition to install on a local processor fetch miss, the replacement policy attempts to evict a cache line from the L2 cache associated with the processor that has been least recently used, based on the chronology vector. The target L2 cache can be identified by examining the utilization of the target L2 cache for the processing cores using the target L2 cache, as well as any other metrics for the target L2 cache. The target L2 cache may be selected from among the L2 caches in the cluster 214 based on which of the L2 caches in the cluster 214 has the lowest utilization. The cache utilization may be based on several factors, including, but not limited to, total cache accesses within a predefined period, frequency of cache evictions and / or writes, duration between cache accesses, evictions, and / or writes, number of lateral persistent cache lines installed in the cache, invalidations from local / remote cores, and the like (these factors may be referred to as saturation metrics).The method for selecting the least utilized cache can be implemented as a minimum number in a group of counters that track per-cache activity, or as an LRU policy for determining the cache that was most recently used within a time window. In one or more embodiments of the present invention, when a cache line is first evicted to a lateral cache in a cluster of caches, the LP bit can be set to 1, indicating that the cache line was evicted from the first cache to a target cache in the cluster of caches corresponding to the primary / first persistence scope. When the same cache line is evicted from the target cache to a new target cache in the next persistence scope (i.e., secondary castout (SCO)), the LP bit will be set to 2, and so on. When the same cache line is evicted from the target cache, a replacement policy can look to other clusters of caches to write the cache line to using the same cache utilization determination within the new cluster of caches. The new cache cluster may be in the same drawer 240-0 or in another drawer 240-1, 240-2, 240-3, depending on how the caches are virtually identified / defined as clusters / scopes. In one or more embodiments of the present invention, when a cache line is fetched by a processing core 204, the cache line may be written to the fetching core's L2 cache and the LP bit may be reset to 0. In one or more embodiments of the present invention, a lateral persistence and replacement policy may be implemented using a cache controller 212 to manage cache eviction between cache clusters 214 and to main memory 220. The cache controller 212 may be local within a drawer 240 or may be a distributed element within each cache cluster instance.

[0016] In one or more embodiments of the present invention, a replacement policy may be prioritized for the lateral cache 208 on the CP chip 202. The CP chip 202 may have one or more defined cache clusters 214, since there are eight on the CP chip. The replacement policy may first attempt to evict a cache line from the L2 cache 208 local to the CP chip 202 before searching other L2 caches, which may be on other CP chips 202. For example, consider three cache clusters 214, with a first cache cluster / primary scope and a second cache cluster / secondary scope residing on the first CP chip and a third cache cluster / tertiary scope residing on the second CP chip. The replacement policy may focus on the utilization of the L2 cache in the first cache cluster, which contains the cache from which the cache line is to be evicted. This may be defined as a first persistence scope. The first persistence scope focuses on the first cache cluster in which the cache to be evicted resides. The second persistence scope can be defined as any other group on the CP chip (e.g., the second cache cluster) that does not have a cache to evict, but may include the entire system. The third persistence scope can focus on groups on other CP chips in the drawer. The utilization of each L2 cache 208 in the cache cluster 214 can be analyzed to determine the target cache that will persist the cache eviction. When further evicting the same cache line from the target cache on the primary cluster, the replacement policy uses the same utilization analysis as before to select a cache from the secondary cluster so that the castout will be persisted. During subsequent castouts from the last scope, the line should be written to memory if modified; otherwise, it should simply be discarded.If there are no empty partitions in the target cache, the persistence installation may trigger a castout (cascading castout) in the target lateral cache, which is sent for persistence in subsequent scopes until an empty partition becomes available in the next scope or the castout chain reaches the final persistence scope. The processor can decide to divert all cascading castouts to memory under certain utilization thresholds or contention in the system. Several utilization thresholds can be used to determine whether to continue cache eviction within a particular level of scope. For example, if the cache utilization percentage in a first cache cluster is higher than a first threshold, the replacement policy will look to a second cache cluster on the same CP chip to evict the cache line. If the L2 cache utilization percentage in the second cache cluster is also higher than the first threshold, the replacement policy will look to a third cache cluster on a different CP chip, and so on.

[0017] In one or more embodiments of the present invention, the replacement policy executed by the cache controller 212 can determine a target cache for an evicted cache line by maintaining a counter (a “saturation counter”) for each cache in the system 200. The counter can track a saturation metric for each cache 208 in the system 200. A cache line can be persisted by first searching a target cache in the evicted cache line's home cache cluster 214. The counter for each cache 208 can be used to track the saturation metric for the cache. This saturation metric (i.e., utilization) can include the number of installs (fetch misses) in the cache from the core / cores attached to the cache, the number of installs of castouts from lateral caches (peer caches), etc., with the counter incrementing with each install event. This counter provides a metric for the combined activity of the cores attached to the cache and the castouts the cache has received from peer lateral caches. In the case of a cache eviction from a cache, the cache line is sent for persistence in the lateral cache with the lowest saturation counter value, or a lateral cache with a saturation counter value less than the first cache, which performs the eviction or broadcasts to a group of less active caches so that the less busy caches (at the time of the eviction) will accept the cache line.

[0018] In one or more embodiments of the present invention, a replacement policy determines a target cache for cache eviction based on the LP bit for the evicted cache line and the utilization rate of the lateral cache. Once the target cache is determined, the replacement policy is further utilized to determine where within the target cache the cache line should be placed. FIG. 2 depicts a block diagram of an exemplary target cache in accordance with one or more embodiments of the present invention. The exemplary target cache 200 is an 8-way cache capable of storing eight cache lines. The exemplary cache 200 utilizes an adaptive least recently used (LRU) algorithm to manage cache lines within the cache 200. LRU is a cache replacement algorithm that discards the least recently used cache line first whenever a write needs to be made to the cache 200. The LRU algorithm supports multiple installation locations, including MRU, Mid-LRU, Quarter-LRU, LRU, and any partial installation location in between. Additionally, the LP tag bit can be used to distinguish lines installed directly by the local processor from lateral castouts by peer caches from any scope. Scanning a given match class provides a measure of activity distribution between the core / cores attached to the cache 200 and castouts received from the lateral cache. Upon installation, all castouts persisted from the lateral cache initially get partial placement in the LRU tree. All lines installed by local processor fetches should get MRU placement. When the percentage of lines installed by lateral castouts exceeds a threshold of the total capacity of the match class, the newly persisted entries get larger partial / MRU placement. Larger partial and partial placement are configurable based on workload demands.This adaptive LRU placement policy dynamically reserves space in the cache to accommodate more local installations and locally rereferenced lines when the local processor is active, and more persisted entries when the local processor is idle. For example, if the minimum capacity threshold for persisted entries in the cache is set to 50%, persisted entries will get partial placement until the number of persisted entries in the match class (including the current installation 102) is less than 50% of the match class capacity. They will then get a larger partial / MRU placement. In the illustrated example, if the threshold is set to 75%, the current install cache line 102 is a persisted installation (LP greater than 0) and the total persisted installations are 4. Therefore, line 102 will still get partial MRU placement. Once the persisted installations reach 6, those that would become persisted installations will get MRU placement.

[0019] In one or more embodiments of the present invention, the determined utilization percentage of the target cache can affect how cache lines are installed and how many peer cache lines can be installed in the target cache. For example, if the utilization percentage of the target cache is very low (e.g., below a predefined threshold), the target cache can install most, but not all, lateral cache lines in the target cache. Multiple threshold utilization percentages can be defined that affect how many peer cache lines can be installed. The number of cache lines can be defined as a percentage of available memory (e.g., 50%, 75% of cache memory, etc.) or a number.

[0020] In one or more embodiments of the present invention, cache clusters 214 can be defined using various means, such as selecting caches within CP chip 202, selecting caches within drawer 240, or selecting caches across drawers 240, or a combination thereof. Defining cache clusters within a system can be based on the location of peer caches, such as implementing data sourcing latency, workload dispatch patterns, and / or OS / software directive hints. Scopes / cache clusters can change over time based on hypervisor hints and / or processor type attached to the cache and / or including the above directives. Additionally, not all clusters have the same number of L2 caches.

[0021] FIG. 3 depicts a flow diagram of a method 300 for lateral cache persistence according to one or more embodiments of the present invention. At least a portion of method 300 can be performed, for example, by processor 601 shown in FIG. 6. Method 300 includes, as shown in block 302, defining scopes and / or clusters for multiple caches in a symmetric multiprocessing (SMP) system. This includes having a data processing system including several processor units, each processor unit including a processor with associated high and low level caches (L2) coupled together by an interconnect fabric. Data in the cache is arranged into consistency classes that include several cache lines, and these consistency classes further include a chronology vector used to determine which entries should be evicted. The chronology vector (age bits) tracks the age of cache lines in the cache. Clusters of the cache are arranged into multiple scope domains, called primary, secondary, and tertiary, which are expandable to an infinite number of unique scopes. Cache eviction and, if a cache is determined to be evicted, the processing cache determines whether the entry should be laterally cast out (LCO) to a peer cache in the primary cast-out scope (PCO) or secondary cast-out scope (SCO), or written back to memory as a tertiary cast-out (TCO). At block 304, method 300 includes defining both a method and metrics for tracking cache activity within the system. Upon determining which cache line to evict, the processing cache determines whether the entry should be laterally cast out (LCO) to a peer cache in the primary cast-out scope (PCO) or secondary cast-out scope (SCO), or written back to memory as a tertiary cast-out (TCO). Within each of the target scopes, cache activity is tracked using a saturating counter that tracks the number of installs to each cache, as defined by processor misses.At block 306, the method 300 includes defining a threshold for the capacity of persisted entries within the match class. Cache lines installed for a given cache can be tracked by various requester types, such as a local processor fetch for a lateral castout from another cache. The install location of a cache line can be based on the percentage of lines installed by processor fetches for a lateral castout. When a cache line is installed in a cache on a lateral castout, the cache line is initially placed in a non-MRU (most recently used) location. If the percentage of cache lines installed by lateral castout exceeds a predefined threshold of the total capacity of the match class, the persisted cache entry gets a larger partial / MRU placement. At block 308, the method 300 includes receiving a request to evict a cache line (LCO) with a lateral persistence bit that tracks the scope of current persistence for the cache line. The current scope of a cache line is determined using the lateral persistence tag bit, which is set to zero when the line is installed or referenced again by the processor, and incremented each time the line is evicted from the current cache and persisted in one of the caches in the next scope.

[0022] In one or more embodiments of the present invention, method 300 includes, at block 310, determining a target cache for writing the LCO, where the target cache is in a higher persistence scope than the current scope. The current scope is determined based on cache activity using counters that track the number of installs to each cache, as defined by processor misses, and the least active cache is selected as the target for persisting the castout. The castout can be sent to a group of less active caches. The counters track the number of invalidations from local and / or remote cores (as defined by local evictions and / or invalidations from local / remote cores and / or peer cache evictions), the number of lateral castout installs, and the number of total castouts. The counters can be implemented, for example, as an LRU tree algorithm. In one or more embodiments of the present invention, method 300 includes decision block 312, which includes determining whether the target cache has no empty partitions for installation and then determining a cascading castout method to make room for the LCO installation. That is, sending an evicted entry for persistence may cause a castout in the target lateral cache if there are no empty partitions. In this case, cascading castouts are sent for persistence in subsequent groups until an empty partition is available in the next scope, or until the castout chain reaches the last persistence scope, or both. In this case, if the replacement algorithm determines that the system is in a contention / busy state, the cascading castout is bypassed to main memory. With the target cache determined with or without cascading and the cache line not written to memory, method 300 includes writing an LCO to the target cache and setting the LP tag bit in the scope of the target cache, as shown in block 314.Also, at block 316, the method 300 includes scanning the match class of the target cache for the number of persisted entries and placing the LCO in the target cache in a respective partial / larger partial MRU location based on a threshold. The chronology vector includes a cache replacement algorithm that supports multiple installation locations, including MRU, Mid-LRU, LRU, and any partial installation location in between. When the percentage of lines installed in the lateral castout exceeds the capacity threshold of the match class, the persisted entry gets a larger partial / MRU placement.

[0023] Additional processes may also be included. It should be understood that the processes depicted in Figure 3 represent examples, and that other processes may be added, or existing processes may be removed, modified, or rearranged, without departing from the scope of the present disclosure.

[0024] FIG. 4 depicts a flow diagram of a method 400 for lateral cache persistence in accordance with one or more embodiments of the present invention. At least a portion of the method 400 can be performed, for example, by the processor 601 shown in FIG. 6. The method 400 includes, as shown in block 402, defining one or more processor units having multiple caches, each processor unit including a processor having at least one cache from the multiple caches, and each of the one or more processor units coupled together by an interconnect fabric. At block 404, the method 400 includes, for each of the multiple caches, arranging multiple cache lines into one or more match classes, each match class in the one or more match classes including a chronology vector. The method 400 then includes arranging each cache line in the multiple caches into a cluster of caches based on the multiple scope domains, as shown in block 406. Furthermore, at block 408, the method 400 includes determining a first cache line to be evicted based on the chronology vector of the first cache line. Also, at block 410, the method 400 includes determining a target cache for installing the first cache line based on a scope of the first cache line and a saturation metric associated with the target cache, wherein the scope of the first cache line is determined based on the lateral persistence tag bit.

[0025] Additional processes may also be included. It should be understood that the processes depicted in Figure 4 represent examples, and that other processes may be added, or existing processes may be removed, modified, or rearranged, without departing from the scope of the present disclosure.

[0026] FIG. 5 depicts a flow diagram of a method 500 for lateral cache persistence in accordance with one or more embodiments of the present invention. At least a portion of method 500 can be executed, for example, by processor 601 shown in FIG. 6. As shown in block 502, method 500 includes receiving a request to evict a first cache line from a first cache on a first microprocessor chip in a processing drawer, the first cache line having a first set of lateral persistence bits tracking scope for the first cache line. At block 504, method 500 includes determining a scope of the first cache line. Furthermore, method 500 includes identifying a target cache having a saturation metric, the target cache having a higher scope than the scope of the first cache line, at block 506. Also, at block 508, the method 500 includes determining an action for the first cache line based on the saturation metric of the target cache and the scope of the first cache line.

[0027] Additional processes may also be included. It should be understood that the processes depicted in Figure 5 represent examples, and that other processes may be added, or existing processes may be removed, modified, or rearranged, without departing from the scope of the present disclosure.

[0028] Turning now to FIG. 6 , a computer system 600 according to an embodiment is generally illustrated. The computer system 600 can be an electronic computer framework comprising and / or employing any number of computing devices and networks, and combinations thereof, utilizing various communication technologies, as described herein. The computer system 600 can be easily scalable, extensible, and modular, allowing modifications to different services or reconfiguration of some features independently of others. The computer system 600 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, the computer system 600 can be a cloud computing node. The computer system 600 may be described in the general context of computer system-executable instructions, such as program modules, executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer system 600 may also be practiced in a distributed cloud computing environment where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

[0029] As shown in FIG. 6, computer system 600 includes one or more central processing units (CPUs) 601a, 601b, 601c, etc. (collectively or generally referred to as processors 601). Processor 601 can be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. Processor 601, also referred to as a processing circuit, is coupled to system memory 603 and various other components via system bus 602. System memory 603 can include read-only memory (ROM) 604 and random access memory (RAM) 605. ROM 604 may have subsequent versions, such as a basic input / output system (BIOS), coupled to system bus 602 for use by processor 601. RAM is read-write memory coupled to system bus 602 for use by processor 601. System memory 603 provides temporary memory space for the execution of instructions during operation. The system memory 603 may include random access memory (RAM), read-only memory, flash memory, or any other suitable memory system.

[0030] Computer system 600 includes an input / output (I / O) adapter 606 and a communications adapter 607 coupled to a system bus 602. I / O adapter 606 may be a small computer system interface (SCSI) adapter that communicates with a hard disk 608 and / or any other similar components. I / O adapter 606 and hard disk 608 are collectively referred to herein as mass storage 610.

[0031] Software 611 for execution by computer system 600 may be stored on mass storage 610. Mass storage 610 is an example of a tangible storage medium readable by processor 601, and software 611 is stored as instructions for execution by processor 601 to operate computer system 600, as described later herein with reference to various figures. Examples of computer program products and the execution of such instructions are discussed in more detail herein. Communications adapter 607 interconnects system bus 602 with network 612, which may be an external network, enabling computer system 600 to communicate with other such systems. In one embodiment, system memory 603 and a portion of mass storage 610 collectively store an operating system for coordinating the functions of the various components shown in FIG. 6, which may be any suitable operating system, such as IBM Corporation's z / OS or AIX operating systems. AIX and z / OS are trademarks of IBM Corporation.

[0032] Additional input / output devices are shown connected to system bus 602 via display adapter 615 and interface adapter 616. In one embodiment, adapters 606, 607, 615, and 616 may be connected to one or more I / O buses connected to system bus 602 through intermediate bus bridges (not shown). A display 619 (e.g., a screen or display monitor) is connected to system bus 602 by display adapter 615, which may include a graphics controller and a video controller to improve performance of graphics-intensive applications. A keyboard 621, mouse 622, speaker 623, etc., may be interconnected to system bus 602 via interface adapter 616, which may include, for example, a super I / O chip that combines multiple device adapters into a single integrated circuit. Suitable I / O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols such as Peripheral Component Interconnect (PCI). Thus, as configured in Figure 6, computer system 600 includes processing capabilities in the form of processor 601, storage capabilities including system memory 603 and mass storage 610, input means such as keyboard 621 and mouse 622, and output capabilities including speakers 623 and display 619.

[0033] In some embodiments, communications adapter 607 may transmit data using any suitable interface or protocol, such as an Internet Small Computer System Interface, among others. Network 612 may be a cellular network, a wireless network, a wide area network (WAN), a local area network (LAN), or the Internet, among others. External computing devices may connect to computer system 600 through network 612. In some examples, the external computing device may be an external web server or a cloud computing node.

[0034] It should be understood that the block diagram of Figure 6 is not intended to indicate that computer system 600 should include all of the components shown in Figure 6. Rather, computer system 600 may include any suitable fewer or additional components (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.) not illustrated in Figure 6. Furthermore, the embodiments described herein with respect to computer system 600 may be implemented with any suitable logic, and logic as referred to herein may include any suitable hardware (e.g., a processor, embedded controller, or application specific integrated circuit, among others), software (e.g., an application, among others), firmware, or any suitable combination of hardware, software, and firmware, in various embodiments.

[0035] Various embodiments of the present invention are described herein with reference to the associated drawings. Alternate embodiments of the present invention may be devised without departing from the scope of the present invention. In the following description and drawings, various connections and relationships (e.g., above, below, adjacent, etc.) between elements are described. These connections and / or relationships may be direct or indirect unless otherwise specified, and the present invention is not intended to be limited in this respect. Thus, connections of entities may refer to direct or indirect connections, and relationships between entities may be direct or indirect relationships. Moreover, the various tasks and process steps described herein may be combined into a more comprehensive procedure or process having additional steps or functions not specifically described herein.

[0036] One or more of the methods described herein may be performed using any technology or combination of technologies, such as discrete logic circuits having logic gates for performing logic functions based on data signals, application specific integrated circuits (ASICs) having appropriate combinatorial logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), or the like, each of which is well known in the art.

[0037] For the sake of brevity, conventional technology relevant to the implementation and use of aspects of the present invention may or may not be described in detail herein. In particular, various aspects of computing systems and specific computer programs for implementing various technical features described herein are well known. Thus, for the sake of brevity, many conventional implementation details are only briefly mentioned herein or are omitted entirely without providing details of well-known systems and / or processes.

[0038] In some embodiments, various functions or acts may be performed at a given location, or in conjunction with the operation of one or more devices or systems, or both. In some embodiments, some of a given function or act may be performed at a first device or location, and remaining functions or acts may be performed at one or more additional devices or locations.

[0039] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used herein, specify the presence of stated features, integers, steps, operations, elements, or components, or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof, or combinations thereof.

[0040] Corresponding structures, materials, acts, and equivalents of all means or step and functional elements in the following claims are intended to include any structure, material, or act for performing a function in combination with other claim elements as specifically claimed. This disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the precise form disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the disclosure. The embodiments were chosen and described in order to best explain the principles and practical application of the disclosure and to enable others skilled in the art to understand the disclosure of various embodiments with various modifications as suited to the particular uses envisioned.

[0041] The diagrams depicted herein are exemplary. There may be many variations to the diagrams or steps (or operations) described without departing from the scope of this disclosure. For example, actions may be performed in a different order, or actions may be added, deleted, or modified. Also, the term "coupled" refers to having a signal path between two elements and does not imply a direct connection between elements with no intervening elements / connections between them. All of these variations are considered to be part of this disclosure.

[0042] The following definitions and abbreviations will be used for interpreting the claims and this specification. As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having," "contains," or "containing," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a structure, mixture, process, method, article, or device that includes a list of elements is not necessarily limited to only those elements, but can include other elements not expressly listed or inherent in such structure, mixture, process, method, article, or device.

[0043] Additionally, the term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. The terms "at least one" and "one or more" are understood to include any integer greater than or equal to one, i.e., 1, 2, 3, 4, etc. The term "plurality" is understood to include any integer greater than or equal to two, i.e., 2, 3, 4, 5, etc. The term "connected" can include both an indirect "connected" and a direct "connected."

[0044] The terms "about," "substantially," "approximately," and variations thereof are intended to include the degree of error associated with measurement of a particular quantity based on equipment available at the time of filing this application. For example, "about" can include a range of ±8% or 5%, or 2% of a given value.

[0045] The present invention may be a system, method, or computer program product, or combination thereof, at any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions for causing a processor to carry out aspects of the present invention.

[0046] A computer-readable storage medium can be a tangible device capable of retaining and storing instructions for use by an instruction-execution device. A computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge-in-groove structures having instructions recorded thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein should not be construed as being signals that are transitory in nature, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted through wires.

[0047] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage on a computer-readable storage medium within the respective computing / processing device.

[0048] Computer-readable program instructions for carrying out the operations of the present invention may be source or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object code written in one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, C++, or the like, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry, including programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer readable program instructions by utilizing state information of the computer readable program instructions to individualize the electronic circuitry to implement aspects of the present invention.

[0049] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0050] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, the instructions of which execute on the processor of the computer or other programmable data processing apparatus to produce means for performing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium such that the computer-readable storage medium comprises an article of manufacture containing instructions for performing aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams, and can direct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner.

[0051] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to perform a series of operational steps on the computer, other programmable apparatus, or other device to produce a computer-executed process, the instructions executing on the computer, other programmable apparatus, or other device to perform the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0052] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing specified logical functions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by a dedicated hardware-based system that performs the specified functions or acts or executes a combination of dedicated hardware and computer instructions.

[0053] The description of various embodiments of the present invention has been presented for purposes of illustration and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical applications, or technical improvements over technology found in the marketplace, or to enable others skilled in the art to understand the embodiments described herein.

Claims

1. 1. A computer-implemented method comprising: defining one or more processor units having a plurality of caches, each processor unit comprising a processor having at least one cache from the plurality of caches, each of the one or more processor units being coupled together by an interconnect fabric; for each of the plurality of caches, arranging a plurality of cache lines into one or more match classes, each match class in the one or more match classes including a chronology vector; arranging each cache in the plurality of caches into a cluster of caches based on a plurality of scope domains; determining a first cache line to be evicted based on the chronology vector of the first cache line; determining a target cache for installing the first cache line based on a scope of the first cache line and a saturation metric associated with the target cache, wherein the scope of the first cache line is determined based on a lateral persistence tag bit; 20. A computer-implemented method comprising:

2. 2. The method of claim 1, further comprising: writing the first cache line to the target cache; and incrementing the lateral persistence tag bit for the first cache line.

3. writing the first cache line to the target cache; selecting a second cache line to be evicted from the target cache; determining a second scope for the second cache line; determining an eviction location for the second cache line based on the second scope, the eviction location including at least one of a lateral cache and a main memory; The method of claim 2 , comprising:

4. writing the first cache line to the target cache; determining a cache replacement policy including one or more cache install locations; determining a scope for the first cache line for the target cache, the scope determining an originator of a request to install the first cache line, the originator including at least one of a processor local to the target cache and a lateral cache; determining an install location for the first cache line based on a percentage of cache lines installed by the processor that are local to the target cache and a percentage of cache lines installed by a lateral cache; The method of claim 2 , comprising:

5. 5. The method of claim 4, wherein the one or more cache installation locations include a most recently used (MRU) location, a mid-least recently used (mid-LRU) location, and a least recently used (LRU) location.

6. determining the target cache for installing the first cache line; determining a target cluster of a cache based on the scope of the first cache line; selecting the target cache from the target cluster of caches; The method of claim 1 , comprising:

7. determining the target cluster of a cache based on the scope of the first cache line and a saturation metric associated with the target cache; determining a scope domain for each cluster of caches; selecting the target cluster of a cache based on a scope domain of the target cluster that is higher than the scope of the first cache line; The method of claim 6, comprising:

8. determining the target cluster of a cache based on the scope of the first cache line and a saturation metric associated with the target cache; analyzing a combined saturation metric for each cluster of caches, the combined saturation metric including the combined saturation metric for each cache within a cluster of caches; selecting a target cluster for the cache based on the combined saturation metric for the target cluster being the lowest combined saturation metric among each cluster of the cache; The method of claim 6, comprising:

9. selecting the target cache from the target cluster of caches; analyzing a saturation metric for each cache in said target cluster of caches; selecting the target cache based on the saturation metric for the target cache being the lowest saturation metric among each cache in the target cluster of caches; The method of claim 6, comprising:

10. The method of claim 1 , wherein the saturation metric comprises at least one of a number of installs defined by processor misses and a number of installs in cache from a lateral cache.

11. 1. A system comprising: one or more processor units having a plurality of caches, each processor unit comprising a processor having at least one cache from the plurality of caches, each of the one or more processor units coupled together by an interconnect fabric; for each of the plurality of caches, arranging a plurality of cache lines into one or more match classes, each match class in the one or more match classes including a chronology vector; arranging each cache in the plurality of caches into a cluster of caches based on a plurality of scope domains; determining a first cache line to be evicted based on the chronology vector of the cache line; determining a target cache for installing the first cache line based on a scope of the first cache line and a saturation metric associated with the target cache, wherein the scope of the first cache line is determined based on a lateral persistence tag bit; and a cache controller configured to implement the above.

12. 12. The system of claim 11, wherein the cache controller is further configured to write the first cache line to the target cache and increment the first lateral persistence bit tag for the first cache line.

13. writing the first cache line to the target cache; selecting a second cache line to be evicted from the target cache; determining a second scope for the second cache line; determining an eviction location for the second cache line based on the second scope, the eviction location including at least one of a lateral cache and a main memory; The system of claim 12 , comprising:

14. writing the first cache line to the target cache; determining a cache replacement policy including one or more cache install locations; determining an originator of a request to install the first cache line into the target cache, the originator including at least one of a processor local to the target cache and a lateral cache; determining an install location for the first cache line based on a percentage of cache lines installed by the processor that are local to the target cache and a percentage of cache lines installed by a lateral cache; The system of claim 12 , comprising:

15. determining the target cache for installing the first cache line; determining a target cluster of a cache based on the scope of the first cache line; selecting the target cache from the target cluster of caches; The system of claim 11 , comprising:

16. A computer program, the computer program causing one or more processors to perform a computer-implemented method according to any one of claims 1 to 10.

17. 1. A computer-implemented method comprising: receiving a request to evict a first cache line from a first cache on a first microprocessor chip in a plurality of microprocessor chips in a processing drawer, the first cache line having a first set of lateral persistence bits that tracks a scope for the first cache line; determining the scope of the first cache line; identifying a target cache having a saturation metric, the target cache having a higher scope than the scope of the first cache line; determining an action for the first cache line based on the saturation metric of the target cache and the scope of the first cache line; 20. A computer-implemented method comprising:

18. 18. The method of claim 17, wherein the action comprises writing the first cache line to the target cache and incrementing the first set of lateral persistence bits.

19. 18. The method of claim 17, wherein the action comprises rejecting the first cache line for the target cache.

20. A computer program that causes one or more processors to perform the computer-implemented method of any one of claims 17 to 19.

Images