A way-restriction bounded micro-tags collisions cache

Abstract

The presently disclosed subject matter discloses a cache component comprising a plurality of sets partitioned into a plurality of cells, each capable of storing a record composed of (i) a cacheline, and (ii) at least part of a cacheline address, wherein at least one set of said sets is associated with a vector of micro-tags, each of which is related to a respective record of said set, such that said vector is capable of including one or more duplicate micro-tags, wherein, for every duplicate micro-tag, the number of duplicate micro-tags is at least two but less than the number of cells of said at least one set.

Description

[0001] A WAY-RESTRICTION BOUNDED MICRO-TAGS COLLISIONS CACHE

[0002] TECHNICAL FIELD

[0003] The present invention relates to the field of data storage in general, and more particularly, to the field of cache components.

[0004] BACKGROUND

[0005] In computing, a cache is a hardware or software component that stores frequently or recently used data so that future requests for said data can be served faster. The data stored in a cache might be the result of an earlier computation or a copy of data stored elsewhere. By definition, a cache stores sets partitioned into cells, such that each cell is capable of storing a data record. The number of cells in a set determines the degree of associativity of the cache.

[0006] Though higher degrees of associativity are desirable since they reduce cache conflicts and increase hit rates, they incur high costs and high latency. Prior approaches attempted to overcome these drawbacks by performing prediction via micro-tags for records. When prediction fails, stored record tags may be compared either in parallel or serially. Such approaches were deemed insufficient for high degrees of associativity due to high circuit cost or variable latency.

[0007] Thus, there is a need in the art for a cache component capable of accommodating high degrees of associativity while reducing the number of reading and comparing events, as well as latency.

[0008] GENERAL DESCRIPTION

[0009] In accordance with a first aspect of the presently disclosed subject matter, there is provided a cache component comprising a plurality of sets partitioned into a plurality of cells, each capable of storing a record composed of (i) a cacheline, and (ii) at least part of a cacheline address, wherein at least one set of said sets is associated with a vector of micro-tags, each of which is related to a respective record of said set, such that said vector is capable of including one or more duplicate micro-tags, wherein, for every duplicate micro-tag, the number of duplicate micro-tags is at least two but less than the number of cells of said at least one set. In some cases, the micro-tags are each related to the at least part of a cacheline address of their respective record.

[0010] In some cases, the record further includes metadata attached to said cacheline.

[0011] In some cases, the at least part of a cacheline address contains a tag portion configured to represent the at least part of a cacheline address of said record.

[0012] In some cases, at least one micro-tag of said micro-tags is generated by executing a hash function on the tag portion of said at least one micro-tag's respective cacheline address.

[0013] In some cases, the cache component is a hardware component.

[0014] In some cases, the cache component is a cache in a cache hierarchy.

[0015] In some cases, the cache component is configured to perform at least one cache operation of: (i) a query operation for determining if a given record is included in said cache component, (ii) a delete operation for removing a given record from said cache component, (iii) an update operation for updating a given record of said cache component, or (iv) an insert operation for inserting a new record into said cache component.

[0016] In some cases, the cache component is further configured to perform at least one of: (v) an extract operation, combining said query operation and said delete operation, such that upon identifying that a given record is within said cache component, said given record is being deleted from said cache component, or (vi) an upsert operation, combining said update operation and said insert operation, such that upon said given record is within said cache component, said given record is being updated, otherwise it is being inserted into said cache component.

[0017] In some cases, the query operation includes: (a) obtaining a query record cacheline address; (b) generating a query record micro-tag from the tag portion of the obtained query record cacheline address; (c) comparing the generated query record micro-tag to the micro-tags of said vector of micro-tags associated with the set corresponding to the set-index of said query record cacheline address; (d) upon said comparison of step (c) yielding no match, providing an indication that said query record cacheline address is not within said cache component; (e) upon said comparison of step (c) yielding at least one match, comparing the tag of said query record cacheline address to the tags of the one or more records corresponding to the micro-tags of said vector found to match said query record micro-tag, and, (f) upon said comparison of step (e) yielding a match, providing an indication that said query record cacheline address is within said cache component, otherwise providing an indication that said query record cacheline address is not within said cache component.

[0018] In some cases, the comparison of step (e) is performed sequentially until a matching tag is found or until all records corresponding to the matched micro-tags of said vector have been compared.

[0019] In some cases, upon said comparison of step (e) yielding a match, in addition to said indication that said query record cacheline address is within said cache component, said query operation includes at least one of: (i) updating the metadata of the record matching said query record, (ii) returning the value of the record matching said query record, or (iii) returning at least part of the metadata of the record matching said query record.

[0020] In some cases, the delete operation includes: (a) obtaining a cacheline address of a record to be deleted; (b) generating a micro-tag from the tag portion of said cacheline address of the record to be deleted; (c) comparing said micro-tag to the micro-tags of the vector of micro-tags associated with the set corresponding to the set-index of said cacheline address of the record to be deleted; (d) upon said comparison of step (c) yielding at least one match, comparing the tag of said record to be deleted to the tags of the one or more records corresponding to the matched micro-tags of said vector, and, (e) upon said comparison of step (d) yielding a match, updating the metadata of the record of the matched tag of the at least one matched micro-tag to reflect its deletion from the set corresponding to the set-index of said cacheline address of the record to be deleted.

[0021] In some cases, the comparison of step (d) is performed sequentially until a matching tag is found or until all records corresponding to the matched micro-tags of said vector have been compared.

[0022] In some cases, prior to step (e), said delete operation includes outputting at least part of the record being deleted in said step (e).

[0023] In some cases, the insert operation includes: (a) obtaining a record to be inserted, including at least (i) a record to be inserted cacheline, and (ii) a record to be inserted cacheline address; (b) generating a micro-tag from the tag portion of said record to be inserted cacheline address; (c) comparing said micro-tag to the micro-tags of the vector of micro-tags associated with the set corresponding to the set-index of said record to be inserted cacheline address; (d) upon said comparison of step (c) yielding a number of matches equal to the number of identical micro-tags within said vector, evicting a record of said records associated with said identical micro-tags from said set, and inserting said record to be inserted in place of said evicted record; (e) upon said comparison of step (c) yielding a number of matches that is less than the number of identical micro-tags within said vector, determining whether the set corresponding to the set-index of said cacheline address of the record to be inserted is full; (f) upon said determination of step (e) being that said set is not full, inserting said record to be inserted to a vacant cell within said set; (g) upon said determination of step (e) being that said set is full, evicting a record of said records from said set, and inserting said record to be inserted in place of said evicted record.

[0024] In some cases, the update operation includes: (a) obtaining an update cacheline address and at least one of (i) at least part of an update cacheline, or (ii) at least part of update metadata; (b) generating an update micro-tag from the tag portion of said update cacheline address; (c) comparing said update micro-tag to the micro-tags of the vector of micro-tags associated with the set corresponding to the set-index of said update cacheline address; (d) upon said comparison of step (c) yielding at least one match, comparing the tag of said update cacheline address to the tags of the matched micro-tags of said vector, and, (e) upon said comparison of step (d) yielding a match, performing at least one of: (i) updating at least part of the metadata of the record associated with the matched micro-tag according to the at least part of update metadata, or (ii) replacing at least part of the cacheline of the record associated with the matched micro-tag with the at least part of the update cacheline.

[0025] In some cases, the cache component is a hardware component associated with (i) a metadata memory configured to store the metadata of the records stored within the cells of a given set, (ii) a K-memory configured to store the tags of the records stored within the cells of said given set, (iii) a probe box configured to perform step (c) of the delete operation, (iv) an EQtagbox configured to perform step (d) of the delete operation, and (v) an EDIT box configured to perform step (e) of the delete operation.

[0026] In some cases, the output of said probe box is in one-hot encoding and wherein said K-memory is configured to access the cells whose indices are in one-hot encoding, thus eliminating the need to decode said cell indices.

[0027] In some cases, the cache component is a hardware component associated with (i) a metadata memory configured to store the metadata of the records stored within the cells of a given set, (ii) a K-memory configured to store the tags of the records stored within the cells of said given set, (iii) a V-memory configured to store the cachelines of the records stored within the cells of said given set, (iv) a probe box configured to perform step (c) of the insert operation, (v) a victim^ box configured to determine the evicted record of step (d) of the insert operation, (vi) a victimset box configured to determine whether the set has a vacant cell, and if so, output the index of said vacant cell, and if set is full to determine the evicted record of step (g) of the insert operation, (vii) a multiplexer configured to output a cell index selected either from the output of the victimpbox or the output of victimset box, and (viii) an EDIT box configured to perform at least part of any of steps (d), (f), and (g) of the insert operation.

[0028] In some cases, the output of said multiplexer is in one-hot encoding and wherein said K-memory is configured to access the cells whose indices are in one-hot encoding, thus eliminating the need to decode said cell indices.

[0029] In some cases, the cache component is a hardware component associated with (i) a metadata memory configured to store the metadata of the records stored within the cells of a given set, (ii) a K-memory configured to store the tags of the records stored within the cells of said given set, (iii) a V-memory configured to store the cachelines of the records stored within the cells of said given set, (iv) a probe box configured to perform step (c) of the update operation, (vi) an EQtagbox configured to perform step (d) of the update operation, and (vii) an EDIT box configured to perform at least part of step (e) of the update operation.

[0030] In some cases, the output of said probe box is in one-hot encoding and wherein said K-memory is configured to access the cells whose indices are in one-hot encoding, thus eliminating the need to decode said cell indices.

[0031] In some cases, the vector is capable of including one or more pairs of identical micro-tags, and is incapable of including more than two identical micro-tags.

[0032] In some cases, the comparison of step (d) is performed sequentially until a matching tag is found or until all records corresponding to the matched micro-tags of said vector have been compared.

[0033] In some cases, the cache component is a hardware component associated with (i) a metadata memory configured to store the metadata of the records stored within the cells of a given set, (ii) a K-memory configured to store the tags of the records stored within the cells of said given set, (iii) a probe box configured to perform step (c) of the query operation, (iv) an EQtagbox configured to perform step (d) of the query operation, and (v) an EDIT box configured to perform step (e) of the query operation.

[0034] In some cases, the output of said probe box is in one-hot encoding and wherein said K-memory is configured to access the cells whose indices are in one-hot encoding, thus eliminating the need to decode said cell indices.

[0035] BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to understand the presently disclosed subject matter and to see how it may be carried out in practice, the subj ect matter will now be described, by way of non-limiting examples only, with reference to the accompanying drawings, in which:

[0037] Fig- 1 is an exemplary illustration of a partition of a cacheline address into a setindex and a tag (LSB on the right), in accordance with the presently disclosed subject matter;

[0038] Fig- 2 is an exemplary flowchart illustrating an example of a sequence of operations carried out by WRBC-cache of the presently disclosed subject matter to perform a query operation, in accordance with the presently disclosed subject matter;

[0039] Fig- 3 is an exemplary flowchart illustrating an example of a sequence of operations carried out by WRBC-cache of the presently disclosed subject matter to perform a delete operation, in accordance with the presently disclosed subject matter;

[0040] Fig. 4 is an exemplary flowchart illustrating an example of a sequence of operations carried out by WRBC-cache of the presently disclosed subject matter to perform an insert operation, in accordance with the presently disclosed subject matter;

[0041] Fig. 5 is an exemplary flowchart illustrating an example of a sequence of operations carried out by WRBC-cache of the presently disclosed subject matter to perform an update operation, in accordance with the presently disclosed subject matter;

[0042] Fig. 6 is an exemplary Julia pseudo-code for a WRBC-cache data- structure, in accordance with the presently disclosed subject matter;

[0043] Fig. 7 is an exemplary Julia pseudo-code for a WRBC-cache Map interface for the cache, in accordance with the presently disclosed subject matter;

[0044] Fig. 8 is an exemplary Julia pseudo-code for an insert operation, in accordance with the presently disclosed subject matter;

[0045] Fig. 9 is an exemplary Julia pseudo-code for finding a matching tag in a set, in accordance with the presently disclosed subject matter; Fig. 10 is an exemplary Julia pseudo-code for the basic cache operations (except insert), in accordance with the presently disclosed subject matter;

[0046] Fig. 11 is a block diagram schematically illustrating one example of a three-stage pipeline block diagram for processing at least one of the EXTRACT, DELETE, or QUERY operations, in accordance with the presently disclosed subject matter;

[0047] Fig. 12 is a block diagram schematically illustrating one example of a three-stage pipeline block diagram for processing an INSERT operation, in accordance with the presently disclosed subject matter; and,

[0048] Fig. 13 is a block diagram schematically illustrating one example of a three-stage pipeline block diagram for processing a VAL-UPDATE operation, in accordance with the presently disclosed subject matter.

[0049] DETAILED DESCRIPTION

[0050] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the presently disclosed subject matter. However, it will be understood by those skilled in the art that the presently disclosed subject matter may be practiced without these specific details. In other instances, well- known methods, procedures, and components have not been described in detail so as not to obscure the presently disclosed subject matter.

[0051] In the drawings and descriptions set forth, identical reference numerals indicate those components that are common to different embodiments or configurations.

[0052] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions utilizing terms such as “obtaining^ “generating”, “comparing^ “providing”, “updating”, “returning”, “performing”, “determining”, “inserting”, “deleting”, “replacing”, “outputting”, or the like, include action and / or processes of a computer that manipulate and / or transform data into other data, said data represented as physical quantities, e.g., such as electronic quantities, and / or said data representing the physical objects. The terms “computer”, “processor”, “processing resource”, “processing circuitry”, and “controller” should be expansively construed to cover any kind of electronic device with data processing capabilities, including, by way of non-limiting example, a personal desktop / laptop computer, a server, a computing system, a communication device, a smartphone, a tablet computer, a smart television, a processor (e.g. digital signal processor (DSP), a microcontroller, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), a group of multiple physical machines sharing performance of various tasks, virtual servers co-residing on a single physical machine, any other electronic computing device, and / or any combination thereof.

[0053] The operations in accordance with the teachings herein may be performed by a computer specially constructed for the desired purposes or by a general-purpose computer specially configured for the desired purpose by a computer program stored in a non- transitory computer readable storage medium. The term "non-transitory" is used herein to exclude transitory, propagating signals, but to otherwise include any volatile or nonvolatile computer memory technology suitable to the application.

[0054] As used herein, the phrase "for example," "such as", "for instance" and variants thereof describe non-limiting embodiments of the presently disclosed subject matter. Reference in the specification to "one case", "some cases", "other cases" or variants thereof means that a particular feature, structure or characteristic described in connection with the embodiment s) is included in at least one embodiment of the presently disclosed subject matter. Thus, the appearance of the phrase "one case", "some cases", "other cases" or variants thereof does not necessarily refer to the same embodiment s).

[0055] It is appreciated that, unless specifically stated otherwise, certain features of the presently disclosed subject matter, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the presently disclosed subject matter, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub-combination.

[0056] In embodiments of the presently disclosed subject matter, fewer, more and / or different stages than those shown in Figs. 2 to 5 may be executed. In embodiments of the presently disclosed subject matter one or more stages illustrated in Figs. 2 to 5 may be executed in a different order and / or one or more groups of stages may be executed simultaneously. Each module in Figs. 11 to 13 can be made up of any combination of software, hardware and / or firmware that performs the functions as defined and explained herein. The modules in Figs. 11 to 13 may be centralized in one location or dispersed over more than one location. In other embodiments of the presently disclosed subject matter, the system may comprise fewer, more, and / or different modules than those shown in Figs. 11 to 13. Any reference in the specification to a method should be applied mutatis mutandis to a system capable of executing the method and should be applied mutatis mutandis to a non-transitory computer readable medium that stores instructions that once executed by a computer result in the execution of the method.

[0057] Any reference in the specification to a system should be applied mutatis mutandis to a method that may be executed by the system and should be applied mutatis mutandis to a non-transitory computer readable medium that stores instructions that may be executed by the system.

[0058] Any reference in the specification to a non-transitory computer readable medium should be applied mutatis mutandis to a system capable of executing the instructions stored in the non-transitory computer readable medium and should be applied mutatis mutandis to method that may be executed by a computer that reads the instructions stored in the non-transitory computer readable medium.

[0059] By way of introduction, as a first point of emphasis, the presently disclosed subject matter presents a cache component capable of accommodating a version of way restriction intermediate between a version encompassing distinct micro-tags and a version encompassing unrestricted collision of micro-tags. The proposed cache, referred to hereinafter as WRBC-cache and described in detail below, bounds micro-tag collisions by a parameter / ?, i.e., at most p micro-tags may share the same value in a set (e.g., = 2).

[0060] A WRBC-cache may provide the following benefits:

[0061] (a) Low conflict rate. For appropriate values of associativity and micro-tag length, bounding micro-tag collisions by a small value > 1 achieves almost the same conflict rate as unbounded micro-tag collisions. Formally, for =2, the parameters k and £ should satisfy 2(> k2. For example, for k = 16 and f = 8, the hit rate with / ? = 2 is reduced by less than 0.1% compared to = k (see Table 2 below).

[0062] (b) Bounded latency and reduced cost. At most tags need to be read and compared.

[0063] (c) Obviating the need for a destination snoop filter in WRBC caches. Snoop filters are key-only filters put in front of the cache to reduce incoming / exiting coherency broadcasts. The comparison of the micro-tag [Aag(addr) with the micro-tags of the valid cachelines in a set can replace a destination snoop filter since easy-misses do not require extra work. The key advantage of such an approach is that the false-positive rate of such a solution does not degrade over time, in contrast to Counting Stream Registers (CSR), and there is no need to flush the cache. A multi-port WRBC cache (i.e., a WRBC-cache with many ports on the inputs) can issue many operations for only a few hits to be forwarded to the next pipeline stage.

[0064] It is to be of note that way restriction with distinct micro-tags may referred to as a special case of ? = 1 , while way restriction with unrestricted micro-tags may be referred to as a special case of p = k.

[0065] It is to be further of note that bounding micro-tag collisions may require the eviction of a colliding micro-tag to maintain the invariant of at most micro-tag collisions (such evictions are referred to hereinafter as p -evictions).

[0066] It is to be further of note that comparison of at most tags (whose micro-tags collide with a queried micro-tag, as explained in further detail below) can be done in parallel or sequentially.

[0067] As a second point of emphasis, the presently disclosed subject matter presents two circuit-level contributions that facilitate the implementation of WRBC-caches:

[0068] (a) Simple eviction policy for / ^-evictions. The presently disclosed subject matter proposes to manage a modulo counter per set that determines which of the colliding ways should be evicted in case of a / ?-eviction. This / ?-eviction policy can be viewed as an approximate LRU replacement policy.

[0069] (b) Avoid the back-to-back encoding of the way-index (resulting from micro-tag comparisons) and the decoding of the way-index in the tag memory. The presently disclosed subject matter proposes a hybrid representation of the address input to tag memory in which the set-index may be represented in binary representation and the wayindex may be represented in one-hot encoding. The advantage of this contribution is in reducing the delay of the path that ends in the tag memory.

[0070] As a third point of emphasis, the presently disclosed subject matter characterizes a condition that reduces VIPT-caches to PIPT-caches. This reduction eliminates the problem of false-negatives due to aliasing in VIPT caches. Particularly, the presently disclosed subject matter shows that if to#2(Page Size) then a VIPT-cache behaves as if it is physically indexed, and false negative cache misses due to aliasing do not occur. By way of a non-limiting example, presented merely for better understanding of the presently disclosed subject matter and not intended in any way to limit its scope, consider standard page size (i.e., 212bytes per page), and a setting in which s = 16, B = 16, and £ = 4. A WRBC-cache with ? = 2 requires 2(> k2, hence k = 4. Thus, the presently disclosed subject matter obtains a VIPT WRBC cache that stores k • s = 64 cachelines of B = 16 bytes (i.e., a one KiB cache) that is free of false-negatives.

[0071] As a fourth point of emphasis, the presently disclosed subject matter presents a “complete” basic set of cache operations sufficient to implement various cache functionalities including: inclusive / exclusive cache policies, write policies, cache coherency protocols, management of metadata, and the like. Consequently, the implementation of a cache hierarchy with cache coherency protocols is reduced to: (a) implementing a cache that supports the basic cache operations, and (b) implementing a cache controller that translates protocol directives to basic operations (e.g., microcoding).

[0072] Initially, attention is drawn to a brief overview of what a cache stores, what a cache hierarchy is, and how set-associative caches are organized.

[0073] WHAT DOES A CACHE STORE?

[0074] A cache stores a set of records that may optionally be, for example, 3 -tuples of the form (addr, val, mt), where addr is an address, val is a cacheline (optionally being B bytes long), and mt is metadata attached to a cacheline (e.g., dirty bit, valid bit, etc.).

[0075] symbol meaning typical value

[0076] B cacheline length in bytes 64

[0077] W main memory byte address in bits 48 addr cacheline address W - log2B bits val cacheline value B bv jt tes mt, cacheline metadata binary string k degree of associativity 4, 8, 16 s number of sets (row) 64, .. .. 1024

[0078] |f micro-tag length in bits 8

[0079] / .» a hash function for micro -tags -♦ {6, i)r bound on micro-tag collisions 1, 2, k

[0080] P page size in bytes 4096

[0081] ± null

[0082] Table 1: Notation

[0083] In some cases, it is assumed that the main memory may be byte-addressable and cachelines may be aligned to multiples of B, hence, a cacheline address may not include an offset that is logiS bits long.

[0084] By way of a non-limiting example, the main memory byte addresses may be I'V = 48bits = Sbytes long and cachelines may be B = 64 bytes. Since a cacheline is aligned to a multiple of B, the address of a cacheline is = 42 bits long. Reference to a byte within a cacheline is done by an offset in [0,1. ... . B 1} that is encoded bits.

[0085] Cache as a Data Structure

[0086] In its simplest form, a cache is a data- structure that maintains a set of key-value pairs subject to queries and insertions (for simplicity, the metadata is disregarded here).

[0087] A set C of key -value pairs may be, for example, a subset of ^x J7, where A={0,l }w' / og2Band V={0,l }8B. Query and insert operations may have the following semantics: •n7{E / ?F(a.da'r ): if then3exists a value val such that (addr, w.Z) t C, then return val ’ 1

[0088] ; otherwise, return (null value, as indicated in table 1 above).

[0089] • INSERTiaddr, val) : update the cache: where victim may be, for example, either 1 or some (addr W) € C, and return victim .

[0090] In some cases, the victim may not be null if and only if the cache cannot accommodate (addr, val), in which case, the eviction of victim makes room for the insertion.

[0091] Cache Hierarchy

[0092] A memory architecture often contains a cascade of caches, denoted by LI, L2, L3. The LI cache is connected to a core and memory accesses of the core are directed to the LI cache. TheZ2 cache “serves” one or moreZf caches, and the / .3 “serves” one or more L2 caches. The last cache in the hierarchy (e.g., the L3-cache in this discussion) is connected to the main memory.

[0093] In some cases, one may view the cache hierarchy as a tree rooted in the main memory, where the leaves of the tree are the LI caches.

[0094] By way of example, assuming an Li cache C and an Li+1 cache C, cache C may be considered to be a descendant of cache C'if cache C serves cache C. Similarly, cache C may be considered to be the parent of cache C.

[0095] In some cases, to reduce path lengths, one may consider adding cross busses (or horizontal busses) between caches. For example, a bus that connects all Z7-caches.

[0096] Set-Associative Cache Organization Principles

[0097] In a PIPT setting (i.e. Physically-Indexed-Physically-Tagged), cache operations may be given physical addresses.

[0098] Assuming a cache that can store at most N records (addr, val, mt), namely, there are N slots (i.e. cells) for storing records, to facilitate lookups, the cells may be partitioned into sets, such that s denote the number sets. Both N and s may be assumed to be powers of 2, so that in standard cache design, every set may contain k = N / s cells. The parameter k is referred to as the degree of associativity.

[0099] The term set-mapping (in short, mapping) refers to how keys (i.e., cacheline addresses) are mapped into sets. Let map : t / — > [s] denote a function that maps cacheline addresses into sets. In a k-associative cache, a record with address addr can be stored in any of the k cells of the set map(<x / t / / '). The mapping function can be, for example, (a) a pseudo-random hash function, or (b) simply based on projection (i.e., map(<x / t / / ') is simply a log2S-bit substring of addr).

[0100] Though not listed above, it is to be of note that other types of mapping functions known in the art may also be applicable.

[0101] In relation to (a), it is to be of note that the main advantage of using a hash function map(<x / t / / j to compute the set-index is that it helps improve the balancing of occupancies of sets. That said, the disadvantages may be: (1) extra computation that consumes time, space, and power; and (2) one needs to store a tag of length W - logzB per cacheline (i.e., additional log2S bits).

[0102] In relation to (b), it is to be of note that projection may achieve good balancing of set occupancies in certain memory access patterns such as linear scanning of memory.

[0103] As illustrated in Fig. 1, in a set-associative cache with projection-based mapping, the binary string addr may be split into two substrings: a set-index substring and a tag substring, where set-index may serve as map(<x / t / / j (i.e., set-index is log2S bits long).

[0104] Since the value of set-index equals the set’s index, there is no need to store it explicitly, hence, a cell may store a shortened record (tag, val, mt), referred to as a tagrecord, as the tag serves as a unique key.

[0105] Support of operations (e.g., queries and insertions) requires one to find a tagrecord that matches the tag in set map(<x / t / / j. Two main approaches for finding a match may include, for example:

[0106] (1) Sequentially compare the k tags in the set with tag; and,

[0107] (2) Perform k comparisons in parallel.

[0108] Though not listed above, it is to be of note that other types of approaches for finding a match may also be applicable.

[0109] It is to be further of note that an insertion to a fully occupied set may require determining a victim to be evicted. It is to be further of note that implementation of an eviction policy (e.g., pseudoLRU) often uses additional metadata that may be stored in the sets.

[0110] Way-Prediction vs. Way-Restriction

[0111] Way-prediction and way-restriction are both techniques for filtering out cells in a set whose tag does not match the tag of a queried (or inserted) tag-record. To this end, a micro-tag ( / ztag) may be attached to every tag by employing, for example, a hash function / z : { 0, 1 }W"log2s-iog2B {o, The j(jeajsfhnt £may £>e smaperthan the length of a tag (e.g., £ G [4, 8]), hence, reading the micro-tags of a set and comparing them with the / / tag of the queried tag-record is cheaper than reading and comparing (full) tags.

[0112] In some cases, one may project the micro-tag directly from the tag, namely, split the binary string "tag" into two sub-strings: a micro-tag / ztag, and a remaining sub-tag. This approach may save memory and simplify the tag comparison, but may create many micro-tag collisions if the accesses cachelines are not uniformly distributed.

[0113] Notation. Consider a set S that stores k tag-records, namely, S = {ri, . . . , n}, where r, = (tag / vab, mt,). The cache stores, in addition, a vector of micro-tags, denoted by / / Set = ( / zi, . . . , gk), where gi = / z(tagi). Let vt equal a bit in mt; that indicates whether tag record r, is valid.

[0114] Let addr denote the (physical) address given as an operand of a basic cache operation. Let tag denote the tag corresponding to addr. Define the bit ct by ej = |> tag)

[0115] Definition (micro-tag collision), the event that C / =l is called a micro-tag collision.

[0116] In way-prediction, the search for a matching tag in the set S may be restricted to at most one (valid) tag-record n (typically the first micro-tag collision, i.e., i = min{j | Cj=l }). If tagi 4 tag, then the remaining tags are accessed and compared with tag. This comparison is typically sequential, thus incurring a variable latency of at most k cycles.

[0117] In way-restriction, the search for a matching tag in the set S may be restricted to the micro-tag collisions, i.e., {tagi | ci=l } .

[0118] It is to be of note that failure of way-prediction (i.e. tagi 4 tag) may result with comparing all tags in the set with tag. Way restriction, on the other hand, attempts to reduce the number of tags in the set that need to be compared with tag. Duplicate vs. Distinct Micro-tags

[0119] In some implementations of way-restriction, the vector pSet of micro-tags of a set must contain distinct micro-tags (i.e., duplicates are prohibited). This invariant is summarized below.

[0120] Invariant (distinct micro-tags). For every set S (i.e., “row ” in the cache), the entries in the vector ^Set of micro-tags that correspond to valid tag-records in S are distinct.

[0121] Maintaining the invariant above may require modifying the implementation of cache operations, as described in the following non-limiting example relating to INSERT operation (it is to be of note that other operations may be modified in a similar fashion).

[0122] By way of a non-limiting example, the invariant above affects the processing of operation as follows (recall that it is assumed that before r is inserted it is not in the set):

[0123] 1. Let S denote the set whose index equals the set — index of r.

[0124] 1. If there exists an index i such that = 1, then is evicted from the cache, and r is inserted in place of jy. This maintains the above Invariant.

[0125] 2. Otherwise, if there exists an invalid tag-record in S, then insert r in place of

[0126] 3. Otherwise, the eviction policy of the cache (e.g., pseudo-LRU) chooses a victim rj;to be evicted from the cache, and then r is inserted in place of rit.

[0127] It is to be of note that the example above may justify a distinction between two cases of a cache miss, an easy-miss and a hard miss.

[0128] Definition (easy-miss and hard miss) consider a QUERY operation with tag value of tag. (1) An easy-miss occurs if there does not exist an index i such that ct = 1. (2) A hard-miss occurs if, for every i such that Ct = 1, it holds that tag f tag .

[0129] It is to be of note that when an easy-miss occurs, there is no need to read and compare tags.

[0130] BASIC CACHE OPERATIONS

[0131] In memory systems, cache operations may be issued by a “higher level” (e.g., a core issuing a read to an LI cache), a “lower level” (e.g., an L2 cache returning a missed cacheline to be stored in the LI cache), and a cache coherence protocol. The disclosed subject matter proposes designing a cache controller that supports these functionalities by issuing a restricted set of “local cache operations”.

[0132] Presented below are four suggested “local cache operations” that can be classified as basic cache operations in the sense that they can implement every standard operation over caches (e.g., inclusive / exclusive cache policies, write policies, cache coherence protocols, etc.).

[0133] It is to be of note that the terminology of key-value pairs, used below, was adapted from dictionaries and refer to the address addr as the key and to the cacheline val as the value. It is to be further of note that the set of records stored in the cache at time t (i.e., cache contents) is denoted by D(t).

[0134] The basic cache operations are as follows:

[0135] 1. QUERY(Wt / / j (i.e., lookup). If the cache contents D(t) contains a record r' = (addr', val', mt) with addr' = addr, then return hit <— 1 with val' and mt'; otherwise, return miss (i.e., hit <— 0). The cache contents remain unchanged (i.e., D(t +1) <— D(t)).

[0136] 2. DELETE(<x / t / / '). Remove the record from D with an address that equals addr (assuming that the record is in D).

[0137] 3. UPDATE(r), where r = (addr, val, mt). If there exists a record r' G D(t) whose address equals addr, then update the value and metadata of r' to val and mt (else, return “error” or “miss”).

[0138] In some cases, the UPDATE operation may consider two restricted updates:

[0139] (a) Value-update VAL-UPDATE (addr, val). Update the value of the record r and set the dirty -bit (which is part of the metadata) to 1.

[0140] (b) Metadata-update MT-UPDATE(<x / t / / ', mt). Only update the metadata of the record r (such an update may change only part of the metadata, for example, set or reset a single bit in the metadata).

[0141] 4. INSERT(r), where r = (addr, val, mt). Insert the record r to D(t + 1) (assuming r g D(t)). If the set of r is not full before the insertion, then D(t +1) <— D(t ) U {r}. If the set of r is full before the insertion, then the cache selects a victim r' from the set of r. Hence, r' is deleted and r is inserted, i.e., D(t +1) <— D(t)U{r}\{r The operation returns a flag evicted and a record r'. The evicted flag that indicates if the insertion required an eviction. The output r' equals the victim if evicted = 1.

[0142] It is to be of note that in some cases, said QUERY operation may be a partial QUERY, which can have an additional parameter specifying which part of the value should be returned on a hit, e.g., which word in the cacheline.

[0143] It is to be further of note that in some cases, said VAL-UPDATE operation may be a partial VAL-UPDATE, which can have an additional parameter specifying which part of the value should be updated, e.g., which word in the cacheline.

[0144] It is to be further of note in some cases, additional operations may be added to the list of basic cache operations to improve performance. In one non-limiting example, the basic cache operations may include an EXTRACT(addr) operation (i.e., deleting query), combining a query operation and a delete operation, as follows: Perform a QUERY(addr). On a hit, the queried record r' is output and deleted (i.e., D(t + 1) — D(t) \ {r'}). In another non-limiting example, additionally or alternatively to the above, the basic cache operations may include an UPSERT(r) operation, combining an INSERT operation and an UPDATE operation, as follows: If r £ D(t), then INSERT(r), else UPDATE(r).

[0145] From a functional point of view, an UPSERT(r) operation can be implemented by a QUERY(r) operation followed either by an INSERT(r) operation or an UPDATE(r) operation.

[0146] WAY-RESTRICTION WITH BOUNDED COLLISIONS (WRBC-CACHE)

[0147] By way of introduction, the following segment introduces an invariant limiting the number of repetitions of micro-tags per set in a micro-tag based cache.

[0148] As mentioned hereinbefore, the number of repetitions of micro-tags in way- restricted based caches, also known as WRBC-caches, may be limited by a parameter p. Definition. The multiplicity of x in a sequence a = {xiji^i is the cardinality of the set {xt | Xi = x}. The maximum multiplicity of a sequence cr = {xj / e / , is the maximum over all X; of the multiplicity of X; in cr.

[0149] Consider a set S of k tags. Let tag denote an arbitrary tag. Assume that the function / z is a random function. For every tag; e S, let / ztag / = / z(tagz) denote the micro-tag of tag / . Let £ denote the number of bits per micro-tag.

[0150] If tag $ S (i.e., a miss), then the expected number of micro-tag collisions with / z(tag) is k / 2i. If tag e S (i.e., a hit), then the expected number of micro-tag collisions is at most 1 + (k - 1 ) / 2f.

[0151] The following invariant uses a parameter p to bound the maximum multiplicity of micro-tags per set.

[0152] Invariant ( / ? micro-tag collisions). Let S denote a set of distinct valid tag-records, and let / / Set denote the sequence of micro-tags that correspond to the tags in S. The set S satisfies the P micro-tag collisions invariant if the multiplicity of every micro-tag in / / Set is at most p.

[0153] By way of definition, a bounded collision cache with a parameter / ? is a cache that satisfies the invariant above, while a WRBC-cache is a / 7-bounded collision cache that employs way-restriction.

[0154] The following theorem is useful to view the random variable Xi as the micro-tag that is attached to a cacheline address i.

[0155] Theorem (Birthday paradox). Let Xi, . . . ,Xk denote k uniformly distributed 3- independent random variables attaining values in the range [2€], If k = , then the probability that there exist a collision of at least 3 random variables (i.e., three distinct indexes ii, ii, is such that Xu = Xn = X,3 ) is bounded by l / 6k.

[0156] Proof. Let Bl4,_ denote the indicator random variable of the event Xi = / z. Uniform distribution implies that Pr\Bi^ = 1] = 2_(. The probability of a triple collision is bounded Since k2= 2lit follows that the probability of a triple collision is bounded by — I < — . For small values of k, this '3--;>v upper bound is close the precise probability. Corollary. Consider a WRBC-cache with > = 2 in which the cacheline addresses are uniformly distributed and 3-independent. Let t be the length in bits of the micro-tags and k the associativity of the cache. If

[0157] Remark (Comparison of analytic bound and benchmark). Consider the setting of k = 16, £ = 8, and p = 2, the corollary above implies an upper bound of l / (6k) ~ 1% on the rate of / ?-evictions. In Table 2 below, the rate of / ^-evictions for this setting is 0.20%.

[0158] In accordance with the description presented thus far, a WRBC-cache component may be configured to perform at least one cache operation of: (i) a query operation for determining if a given record is included in said cache component, (ii) a delete operation for removing a given record from said cache component, (iii) an update operation for updating a given record of said cache component, or (iv) an insert operation for inserting a new record into said cache component.

[0159] Initially, attention is directed to the steps involved in executing the query operation.

[0160] Fig. 2 presents a flowchart illustrating one example of operations carried out by a WRBC-cache component to perform a query operation 200, in accordance with the presently disclosed subject matter. The WRBC-cache component may comprise a plurality of sets partitioned into a plurality of cells, each capable of storing a record composed of (i) a cacheline, and (ii) at least part of a cacheline address. Of said sets, at least one set of said sets may be associated with a vector of micro-tags, each of which may be related to a respective record of said set, such that said vector may be capable of including one or more duplicate micro-tags, where, for every duplicate micro-tag, the number of duplicate micro-tags may be at least two but less than the number of cells of said at least one set.

[0161] For this purpose, the WRBC-cache component obtains a query record cacheline address (block 202), and generates a query record micro-tag from the tag portion of the obtained query record cacheline address (block 204). Next, said WRBC-cache component compares the generated query record microtag to the micro-tags of said vector of micro-tags associated with the set corresponding to the set-index of said query record cacheline address (block 206).

[0162] Upon said comparison yielding no match, said WRBC-cache component provides an indication that said query record cacheline address is not within said cache component (block 208)

[0163] Upon said comparison of block 208 yielding at least one match, said WRBC- cache component compares the tag of said query record cacheline address to the tags of the one or more records corresponding to the micro-tags of said vector found to match said query record micro-tag (block 210). upon said comparison of block 210 yielding a match, said WRBC-cache component provides an indication that said query record cacheline address is within said cache component, otherwise said WRBC-cache component provides an indication that said query record cacheline address is not within said cache component (block 212).

[0164] In some cases, the comparison in block 210 may be performed sequentially until a matching tag is found or until all records corresponding to the matched micro-tags of said vector have been compared.

[0165] In some cases, upon said comparison in block 210 yielding a match, in addition to said indication that said query record cacheline address is within said cache component, said query operation may include at least one of: (i) updating the metadata of the record matching said query record, (ii) returning at least part of the value of the record matching said query record, or (iii) returning at least part of the metadata of the record matching said query record.

[0166] In some cases, said vector may be capable of including one or more pairs of identical micro-tags, and may be incapable of including more than two identical microtags.

[0167] Next, attention is directed to the steps involved in executing the delete operation.

[0168] Fig. 3 presents a flowchart illustrating one example of operations carried out by a WRBC-cache component to perform a delete operation 300, in accordance with the presently disclosed subject matter. As described above in relation to Fig. 2, the WRBC- cache component may comprise a plurality of sets partitioned into a plurality of cells, each capable of storing a record composed of (i) a cacheline, and (ii) at least part of a cacheline address. Of said sets, at least one set of said sets may be associated with a vector of micro-tags, each of which may be related to a respective record of said set, such that said vector is capable of including one or more duplicate micro-tags, where, for every duplicate micro-tag, the number of duplicate micro-tags may be at least two but less than the number of cells of said at least one set.

[0169] For this purpose, the WRBC-cache component obtains a cacheline address of a record to be deleted (block 302), and generates a micro-tag from the tag portion of said cacheline address of the record to be deleted (block 304).

[0170] Next, said WRBC-cache component compares said micro-tag to the micro-tags of the vector of micro-tags associated with the set corresponding to the set-index of said cacheline address of the record to be deleted (block 306).

[0171] Upon said comparison in block 306 yielding at least one match, said WRBC-cache component compares the tag of said record to be deleted to the tags of the one or more records corresponding to the matched micro-tags of said vector (block 308).

[0172] Upon said comparison in block 306 yielding a match, said WRBC-cache component updates the metadata of the record of the matched tag of the at least one matched micro-tag to reflect its deletion from the set corresponding to the set-index of said cacheline address of the record to be deleted (block 310).

[0173] In some cases, the comparison in block 308 may be performed sequentially until a matching tag is found or until all records corresponding to the matched micro-tags of said vector have been compared.

[0174] In some cases, prior to block 310, said delete operation may include additional step of outputting at least part of the record being deleted in said block 310.

[0175] In some cases, said vector may be capable of including one or more pairs of identical micro-tags, and may be incapable of including more than two identical microtags.

[0176] Next, attention is directed to the steps involved in executing the insert operation.

[0177] Fig. 4 presents a flowchart illustrating one example of operations carried out by a WRBC-cache component to perform an insert operation 400, in accordance with the presently disclosed subject matter. As described above in relation to Figs. 2 and 3, the WRBC-cache component may comprise a plurality of sets partitioned into a plurality of cells, each capable of storing a record composed of (i) a cacheline, and (ii) at least part of a cacheline address. Of said sets, at least one set of said sets may be associated with a vector of micro-tags, each of which may be related to a respective record of said set, such that said vector is capable of including one or more duplicate micro-tags, where, for every duplicate micro-tag, the number of duplicate micro-tags may be at least two but less than the number of cells of said at least one set.

[0178] For this purpose, the WRBC-cache component obtains a record to be inserted, including at least (i) a record to be inserted cacheline, and (ii) a record to be inserted cacheline address (block 402), and generates a micro-tag from the tag portion of said record to be inserted cacheline address (block 404).

[0179] Next, said WRBC-cache component compares said micro-tag to the micro-tags of the vector of micro-tags associated with the set corresponding to the set-index of said record to be inserted cacheline address (block 406).

[0180] Upon said comparison in block 406 yielding a number of matches equal to the number of identical micro-tags within said vector, said WRBC-cache component evicts a record of said records associated with said identical micro-tags from said set, and inserting said record to be inserted in place of said evicted record (block 408).

[0181] Upon said comparison in block 406 yielding a number of matches that is less than the number of identical micro-tags within said vector, said WRBC-cache component determines whether the set corresponding to the set-index of said cacheline address of the record to be inserted is full (block 410).

[0182] Upon said determination in block 410 being that said set is not full, said WRBC- cache component inserts said record to be inserted to a vacant cell within said set (block 412)

[0183] Upon said determination in block 410 being that said set is full, said WRBC-cache component evicts a record of said records from said set, and inserts said record to be inserted in place of said evicted record (block 414).

[0184] In some cases, said vector may be capable of including one or more pairs of identical micro-tags, and may be incapable of including more than two identical microtags.

[0185] Finally, attention is directed to the steps involved in executing the update operation.

[0186] Fig. 5 presents a flowchart illustrating one example of operations carried out by a WRBC-cache component to perform an update operation 500, in accordance with the presently disclosed subject matter. As described above in relation to Figs. 2 to 4, the WRBC-cache component may comprise a plurality of sets partitioned into a plurality of cells, each capable of storing a record composed of (i) a cacheline, and (ii) at least part of a cacheline address. Of said sets, at least one set of said sets may be associated with a vector of micro-tags, each of which may be related to a respective record of said set, such that said vector may be capable of including one or more duplicate micro-tags, where, for every duplicate micro-tag, the number of duplicate micro-tags may be at least two but less than the number of cells of said at least one set.

[0187] For this purpose, the WRBC-cache component obtains an update cacheline address and at least one of (i) at least part of an update cacheline, or (ii) at least part of update metadata (block 502), and generates an update micro-tag from the tag portion of said update cacheline address (block 504).

[0188] Next, said WRBC-cache component compares said update micro-tag to the microtags of the vector of micro-tags associated with the set corresponding to the set-index of said update cacheline address (block 506).

[0189] Upon said comparison in block 506 yielding at least one match, said WRBC-cache component compares the tag of said update cacheline address to the tags of the matched micro-tags of said vector (block 508).

[0190] Upon said comparison in block 506 yielding a match, said WRBC-cache component performs at least one of: (i) updating at least part of the metadata of the record associated with the matched micro-tag according to the at least part of update metadata, or (ii) replacing at least part of the cacheline of the record associated with the matched micro-tag with the at least part of the update cacheline (block 510).

[0191] In some cases, said vector may be capable of including one or more pairs of identical micro-tags, and may be incapable of including more than two identical microtags.

[0192] PERFORMANCE EVALUATION

[0193] By way of introduction, the following segment demonstrates the potential benefits of the WRBC-cache. In this segment we propose a benchmark and evaluate the miss rates, easy-miss rates, and distribution of number of tags read from the memory.

[0194] Notation. Let s denote the number of sets, k the number of ways (i.e., cells), p the bound on the number of micro-tag collisions, £ number of bits per micro-tag.

[0195] The number of cachelines in the cache is N = k ■ s. Benchmark. The benchmark considers a random dataset D of N key -value pairs. The number of sets is s = 212. It proceeds by inserting the N pairs to D, and then iterates 103times with queries over D in a round robin-fashion (same order as the insertion order). A missed query involves an eviction and insertion of the queried cacheline. The number of cache misses, easy cache misses, tags read from the memory per query are counted. The results of this benchmark are summarized in Table 2 below.

[0196] Table 2

[0197] Discussion of the Results The results of the benchmark suggest the following conclusions.

[0198] (1) For k = 8, the value of p has only a small effect on the miss rate. Hence, a choice of / ? = 1 is justified thanks to the simpler circuitry.

[0199] (2) For k = 16, almost all of the reduction in the miss rate due to increasing from 1 to 16 is already achieved with (3 = 2. Hence, the choice of / ? = 2 is justified as it achieves almost all the gains of ? = 16 while not incurring either the hardware overhead or the variable latency associated with using / ? = 16.

[0200] (3) For k = 16 and e {2, 16}, most cache misses (i.e. 93%) are easy-misses (no micro-tag collision). This means that a tag is read from memory in less than 7% of the cache misses.

[0201] If ? = 1, then about a third of the cache misses require reading a tag from memory and comparing it with the queried tag. Reduction of reading tags from memory and comparing them with the queried tag help reduce both latency and power consumption.

[0202] (4) For k = 16 and / ? = 2, the distribution of the number of tag reads and comparisons per query is: (i) zero accesses 11% (i.e., almost all the cache misses), (ii) a single access 84%, and (iii) two accesses 5%. Hence, on 84 / 89 = 94% of the cache hits, only a single tag is read and compared. Hence, when compared to / ? = 1, the benchmark with p = 2 indicates only a 4.5% increase in the number of tag read-and-compare operations.

[0203] It is to be of note that we also demonstrate the benefits of a WRBC-cache with a k=32 associative cache, 8-bit micro-tags, and 0=2. In a benchmark with a dataset whose size equals the cache size with random cacheline addresses and round-robin queries, we obtain the following results:

[0204] • the hit rate almost matches the hit rate of a 16-associative cache (i.e., loss of less than 0.5% in the hit rate).

[0205] • Most cache misses do not require reading a tag at all as no micro-tag match is found (i.e., roughly 80%).

[0206] • Read and compare two tags in roughly 9% of the operations.

[0207] • Eviction due to micro-tags collisions exceeding 0 are rare (e.g., roughly 0.8%). Hence, we maintain the preferred eviction policy in all but a negligible fraction of the accesses.

[0208] OBVIATING SNOOP FILTERS

[0209] By way of introduction, snoop filters are key-only dynamic filters that can reduce the forwarding of broadcast operations for cache coherency. A dynamic filter is a data- structure that supports insertions and deletions as well as queries. Responses to queries in a filter are approximate in the sense that they have a one-sided error. In particular, a destination snoop filter for a cache C maintains the set of cacheline addresses stored in the cache C. When a cacheline is inserted to C, its address is added to the destination snoop filter. When a cacheline is evicted or invalidated, its address is removed from the destination snoop filter. The destination snoop filter of cache C receives addresses of cachelines that need to be invalidated if they are stored actually in cache C.

[0210] To reduce processing by cache C, the destination snoop filter performs a local query, and forwards the invalidation operation to cache C only if the query returns an affirmative answer.

[0211] Easy misses in WRBC-caches serve the same purpose as a destination snoop filter. To support queries that replace the destination snoop filter as well as operations from descendant cache (or core in case of an LI -cache), a multi-port extension of a WRBC- cache may be required.

[0212] It is to be of note that, from a conceptual point of view, way restriction in a WRBC-cache is a special type of filter that stores a multi-set of key -value pairs. The keys are the micro-tags (with at most p duplicates allowed) and the values are way indices. Such data structures are referred to in the literature as retrieval data structures or Bloomier filters.

[0213] FUNCTIONAL DESCRIPTION

[0214] By way of introduction, the following segment describes the functionality of the basic cache operations in a WRBC-cache. It is to be of note that the hardware implementation of said basic operations is described in detail below.

[0215] WRBC-cache functionality is depicted in the following algorithms:

[0216] (1) Fig- 6 is an exemplary Julia pseudo-code for a WRBC-cache data- structure. The WRBC-cache employs a Map module that implements an interface to the cache (Fig. 7).

[0217] Let s denote the number of sets in the cache, and let k denote the number of ways per set. The Cache structure contains the following components: (a) parameter which bounds the number of / ztag-collisions, (b) K matrix being a k xs array of tags (i.e., set keys), (c) V matrix being a k * s array of values (i.e., cacheline contents), and (d) Mt being a metadata vector with s components. Each component contains: (1) A k vector of micro-tags, (2) A vector WayMt of k metadata strings, one per way, and (3) Metadata for a set denoted by SetMt (e.g., pseudo-LRU state bits). Also described is a retrieve method that retrieves a tag-record that is stored in a given set and way.

[0218] (2) Fig. 8 is an exemplary Julia pseudo-code for an insert operation. The pseudocode describes the functionality an insertion (not an implementation).

[0219] The vector / Coll indicates micro-tag collisions with valid tag-records in the set. The selection of the victim in case of a / ?-eviction (Line 11) is based on metadata (e.g., metadata of the colliding tag-records or on a state of a counter modulo p stored per set). The selection of the victim in case of a set-eviction (Line 17) implements an eviction policy (e.g., pseudo-LRU, MRU, etc.) that is also based on metadata. The update of metadata takes place in Line 29.

[0220] We distinguish between two types of evictions for insertions:

[0221] (a) A jff-eviction occurs if the / tag of the inserted tag-record collides with micro-tags in the set. In this case, one of the colliding tag-records is evicted to make room for the inserted tag-record.

[0222] (b) A set-eviction occurs if the set is full and a p -eviction does not occur. In this case, one of the tag-records in the set is evicted to make room for the inserted tag-record.

[0223] (3) Fig. 9 is an exemplary Julia pseudo-code for finding a matching tag in a set. The input consists of the cache and the address. The output is the index of the matching way and the set-index of the address. If no matching tag is found, then the first component of the output is “nothing".

[0224] The find procedure is employed by the methods listed in the Julia pseudo-code of Fig. 10 (i.e., QUERY, UPDATE, EXTRACT, and DELETE). These operations may consider two types of misses: (a) An easy-miss occurs if the / tag corresponding to addr does not collide with any / tag in the set, and (b) A hard- miss occurs if an easy-miss does not occur and the tag corresponding to addr does not match any tag in the set.

[0225] MICRO-ARCHITECTURE By way of introduction, the following segment presents the micro-architecture of a WRBC-cache. The description of the micro-architecture is composed of three parts: memory organization, partitioning of the pipeline to three pipeline stages, and the functional blocks.

[0226] Memory Organization

[0227] A WRBC-cache may contain three memories: a tag-memory (K), a value-memory (V), and a metadata memory (Mt). Each memory may be a matrix of k X s elements (the Mt also contains set metadata that is stored in an s vector).

[0228] It is to be of note that the memories above are mere examples, not intended in any way to limit the scope of the presently disclosed subject matter, and that other types of memories, additionally or alternatively, may also be applicable.

[0229] (1) Mt-Memory. A dual-port RAM that supports a concurrent read and write. If the read and write access the same address, then forwarding is employed, namely, the read operation returns the written data (i.e., read-after- write). For simplicity, the Mt is assumed to be set-addressable (i.e., a read or write accesses the metadata of all the ways in the set).

[0230] (2) K-Memory. A multi-port RAM that supports either at most p concurrent reads or a single read-before- write (meaning that a write operation always returns the previously stored data). It is to be of note that the reads access the same set-index and differ only in the way index (reducing the delay and cost of address decoder).

[0231] (3) V-Memory. A single port read-before-write RAM the stores the cacheline contents (i.e., B bytes per cacheline address). It is to be of note that the V-Memory may be split if the bus-width along which the cacheline value is transmitted is smaller than the cacheline length B. For example, suppose that the bus-width is B / 4, then, every cacheline value stored in V-Memory is split to four parts (each of length B / 4).

[0232] Decoder Elision

[0233] A standard memory design is fed an address in binary representation. Access to the correct memory cell is implemented by a decoder that converts the binary representation of the address to one-hot encoding (i.e., the number i is converted to the string 0n 1 1° 1 ° O'). In micro-tag-based caches, the computation of the way -index in one- hot encoding is performed by the micro-tag comparators, therefore, there is no need to decode for the way-index to one-hot encoding.

[0234] For / ? = 2, the outcome of the micro-tag comparators can be split into two one-hot encodings with two priority encoders. For / ? > 2, the bit-mask can be split into p one-hot encodings using parallel prefix computation.

[0235] Pipeline Organization

[0236] By way of introduction the current segment proposes to partition the WRBC- cache to three pipeline stages.

[0237] It is to be of note that additional pipeline stages may be added if the bus-width is smaller than the cacheline width B (For example, if the bus-width is B / 4, then the V is split to four parts, each accessed in a separate pipeline stage).

[0238] For simplicity, EXTRACT, INSERT, and the other basic operations are described separately.

[0239] Notation. Figs. 11-13 depict the pipeline stages. To simplify the diagram, we omit the input set-index to the memories and write the set-index as a subscript. For example, Kset-index denotes the K-memory which is input the set-index as part of the address (the remaining part of the address is the way).

[0240] Pipeline stages for EXTRACT operation.

[0241] Fig. 11 illustrates an exemplary three-stage pipeline block diagram for processing an EXTRACT operation (i.e., a deleting query). Within Fig. 11, inputs are depicted by boxes with rounded corners, outputs by thick boxes with rounded corners, processing by boxes, multiplexer by a trapezoid, and memories by double bordered boxes.

[0242] The inputs (i.e., set-index, tag, and / tag) may be derived from the input addr. The outputs may be hit, val, mt (the output addr may be omitted as it is identical to the input).

[0243] The three-stage pipeline block consists of the following stages:

[0244] (0) Read the metadata (micro-tags of the ways, metadata of the ways, metadata of the set) from the Mt-memory.

[0245] (1) The probe box computes the / ztag-collisions ( / Coll). The tags of the valid ways that collide with the input / tag are read from the K-memory (at most concurrent reads).

[0246] (2) The EQtag box first compares the read tags with the input tag. The EQtag then computes the index match of the matching tag (with respect to the P way-addresses fed to the K-memory). The index match can be computed by a priority encoder. In addition the EQtag computes the hit flag that indicates if matching tag is found.

[0247] The match signal is used to select the way from / Coll by the exact-MUX. The value of the matching tag-record is read from the V-memory (the hit signal enables the memory read). The EDIT(D) box computes the updated metadata string (i.e., invalidate way if a hit occurs) which is stored in the Mt-memory.

[0248] It is to be of note that said diagram can be extended to support QUERY and DELETE operations as follows: (1) To process a QUERY, replace the EDIT(D) box (that implements the deletion) with an EDIT(Q) box that only updates the metadata (e.g., move-to-front in LRU). (2) To process a DELETE, remove the access to the V memory.

[0249] Pipeline stages for INSERT operation.

[0250] Fig. 12 illustrates an exemplary three-stage pipeline block diagram for processing an INSERT operation. Within Fig. 12, inputs are depicted by boxes with rounded corners, outputs by thick boxes with rounded corners, processing by boxes, multiplexer by a trapezoid, and memories by double bordered boxes.

[0251] The inputs (i.e., set-index and tag) may derived from the input addr. The evicted victim may be output with addr' = (set-index, tag'), val' and mt'.

[0252] The three-stage pipeline block consists of the following stages:

[0253] (0) Same as EXTRACT (0).

[0254] (1) The probe box is identical to the probe box for the EXTRACT operation. The candidate victims are computed in parallel as follows: (a) The victim / ? box computes the victim under the assumption of a / ^-collision (i.e., there are already p micro-tags that equalz(tag)). (b) The victimset computes the victim under the assumption that a / ^-collision does not occur (if the set is not full, then it returns a vacant way). The way of the victim is selected between these two victims according to whether micro-tag-collisions occur. The tag of the victim is read from the K-memory and the tag of the inserted tag-record is written to the K-memory (read-before-write). The evicted signal is the logical OR of the set being full and micro-tag-collisions.

[0255] (2) The input val is written to the V-memory (the address is a pair of set-index, way). The EDIT(I) box updates the vector of micro-tags by writing / tag in the component of index way and updates the way metadata and set metadata, as necessary. The revised metadata is written to the Mt-memory. Pipeline stages for a VAL-UPDATE operation.

[0256] Fig. 13 illustrates an exemplary three-stage pipeline block diagram for processing a VAL-UPDATE operation. Within Fig. 13, inputs are depicted by boxes with rounded corners, outputs by thick boxes with rounded corners, processing by boxes, multiplexer by a trapezoid, and memories by double bordered boxes.

[0257] The inputs (i.e., set-index and tag) may be derived from the input addr. The outputs may be hit and mt.

[0258] The three-stage pipeline block consists of the following stages:

[0259] (0) - (1) Same as EXTRACT (0) - (1).

[0260] (2) The EQtag box is identical to the EQtag box for the EXTRACT operation. The exact-MUX is identical to the same MUX for the EXTRACT operation. The new value of val is written into the V-memory at address (set-index, way) (the hit signal enables the memory read and indicates successful update). The EDIT(U) box computes the updated metadata string (e.g. move-to-front on LRU) which is stored in the Mt-memory.

[0261] Pipeline stages for the remaining operations.

[0262] The pipeline stages for the remaining basic cache operations (QUERY and DELETE) are obtained by modifying the 2ndpipeline stage of the EXTRACT operation as follows:

[0263] QUERY : Read the val, but do not invalidate the entry (may update metadata for purposes of eviction policy).

[0264] • DELETE: Do not read the val, only invalidate the way.

[0265] It is to be of note that computation of the micro-tag / tag (i.e., evaluation of a hash function z(at / t / r)) can be performed in the 0thpipeline stage (not depicted in Figs. 11 to 13).

[0266] It to be of further of note that though the focus is on a cache that operates over whole cachelines, one may consider caches that allow accessing part of a cacheline (e.g., read or write a word in a cacheline). The design of a WRBC-cache with partial cacheline access does not affect way-restriction. It is to be further of note that one can optimize the circuit for computing the one hot encoding of the colliding micro-tags as follows:

[0267] (a) Feed the outcome / Coll = (co, . . . , Ck-i) of the micro-tag comparisons to a parallel-prefix computation circuit with a XOR operator (PPC-XOR). The outcome is a Zc-bit “pulse".

[0268] (b) Feed the pulse to two circuits: one locating a rising edge with outcome wayo, and the second locating a falling edge with the outcome wayi.

[0269] (c) Determine the number of micro-tag collisions as follows: (i) If (co, . . , Ck-i) is all zeros, then there are zero micro-tag collisions (i.e., easy-miss). (ii) If the last bit of the PPC-XOR output is 1, then there is a single micro-tag collision (i.e., read only wayo). (iii) Otherwise, there are two micro-tag collisions (i.e., read both wayo and wayi).

[0270] It is to be of note that on a / ?-eviction on insertion, the choice of the victim (i.e., wayo or wayi) is based on the set’s metadata.

[0271] Pipeline Schedule

[0272] By way of introduction, the following segment discusses the pipeline schedule that determines the issue rate and functionally of RAMs. This segment demonstrates that the pipeline can accept a new operation every clock cycle as long as consecutive operations do not access the same set (to avoid data hazards). In addition, this segment justifies the number of concurrent read(s) and writes that each memory should support (as to avoid control hazards).

[0273] Notation. The following notation in reference to RAM accesses is used:

[0274] (1) Rd[t] means a read that serves operation opt , and in general, the index in the bracket refers to the clock cycle in which the operation was issued.

[0275] (2) Wr[t] is a write to the memory.

[0276] (3) A Rd / Wr[t] denotes either Rd[t] or Wr[t] (e.g., a read occurs in EXTRACT, and a write in UPDATE).

[0277] (4) A Rd-b-Wr[t] denotes a read-before-write to the same address (namely, read the old value and write a new value).

[0278] (5) p X Rd[t] denotes at most concurrent reads to distinct ways in the same set. Memory accesses. The pipeline schedule for a sequence of five consecutive operations is listed in Table 3 below with an issue rate of 1 operation per cycle (under the assumption that consecutive operations do not access the same set). Access patterns to the memories may be as follows:

[0279] (1) The K-memory may be accessed only in the 1st pipeline stage either with a single read-before-write (i.e., for an INSERT operation) or with at most p concurrent reads (i.e., for QUERY and EXTRACT operations).

[0280] (2) The Mt-memory may be accessed both in the 0thand 2ndpipeline stages. In the 0thpipeline stage, metadata is read, and in the 2ndpipeline stage metadata is written. Forwarding implements read-after-write if both operations access the same set. Hence the Mt-memory is a dual port RAM: one port for read (0thpipeline stage) and the second port is for read-after- write (2ndpipeline stage).

[0281] (3) The V-memory may be accessed only in the 2ndpipeline stage (either read or write), hence the V-memory is a single port RAM.

[0282] Issue rate. If two consecutive operations access the same set, then there is a risk of a data hazard. For example, consider an INSERT(r) issued at time t and a QUERY(r) issued at time t +1. The response to the query will be miss because the Mt-memory is updated by the insert operation in cycle t + 2, and the query operation reads the metadata in cycle t + 1.

[0283] Table. 3

[0284] The following remark presents a trade-off between a fixed-latency WRBC-cache with / ? = 2 in which the K-memory supports two simultaneous reads vs. a variable latency WRBC-cache with / ? = 2 in which the K-memory is a single port RAM. Remark (Dual port K-memory vs. variable latency.). Table 2 above indicates (with respect to the benchmark) that for k = 16 and / ? = 2, less than 5% of the operations require reading two tags from the K-memory. The fact that two concurrent reads are issued on average once every 20 operations suggests to use a single port memory for the K-memory and repeat the 1stpipeline stage twice if and only if a second tag needs to be read. This modification influences the stall engine because the successive operation needs to be stalled when an operation employs the 1st pipeline stage twice.

[0285] HARDWARE IMPLEMENTATION

[0286] By way of introduction, the following segment describes a hardware implementation of the building blocks of a (pipelined) WRBC-cache.

[0287] Since the different memories were described hereinbefore, the following focuses on the combinational subcircuits of Figs. 12 and 13.

[0288] The probe Circuit

[0289] The probe circuit resides in the 1stpipeline stage deals with comparing micro-tags and finding the micro-tag collisions.

[0290] Abstract probe circuit. An abstract probe circuit is specified as follows:

[0291] Input: A micro-tag / tag and metadata info (e.g., the vector of micro-tags / Set of the accessed set and a valid bit per way).

[0292] Output: A vector / / Coll of at most p (colliding) ways and / Valid that equals the number of colliding micro-tags.

[0293] Functionality: The output / Coll equals the sequence {i | ct = 1}. The output / Valid equals the number of way indices in / / Coll.

[0294] Next, an optimized hardware implementation of probe for the case ? = 2 is described. The advantage of this implementation is that the way addresses do not require decoding in the K-memory.

[0295] It is to be of note that the decoding of the set-index does not depend on probe and is on a separate path, thus reducing the combinational delay of the first pipeline stage.

[0296] Specialized probe circuit ( = 2). A specialized probe circuit specification for the case that / ? = 2 is as follows: Input: A micro-tag / / tag and metadata info (e.g., the vector of micro-tags / / Set of the accessed set and a valid bit per way).

[0297] Output: Two (unit) vectors / / Collo, / / Colli G {0, l }fcand / / Valid G {00, 01, 11 }.

[0298] Functionality: Consider the set C of way indices of micro-tag collisions defined by C = {i | Ct = 1 }. The output / / Valid encodes the cardinality of C in unary. Namely, if C is empty, then / / Valid = 00, if C contains a single index, then / / Valid = 01, and iff contains two indices, then / / Valid = 11.

[0299] Suppose that the set C contains two way indices a and b. Then / / Collo encodes a in one-hot encoding (i.e., a unit vector), namely / / Collo = 0k a° 1 ° 0“ Similarly, / / Colli encodes b, namely / / Colli = ()k h° 1 ° 06-1. If C contains a single index a, then / / Collo = Qk-a o | -oa 1and we do not care about the value of / / Colli. If C is empty, then we do not care about the value of / / Collo and / / Colli.

[0300] Remark: The / / Valid output may serve two purposes: (1) Read-enable signals for the ports the K-memory. (2) Signals that indicate which of the tags output by the K- memory is relevant (for the tag comparison in the 2nd pipeline stage).

[0301] The Circuit

[0302] The EQtag circuit resides in the 2ndpipeline stage and deals with comparing tags and reporting a hit / miss and the matching way index in case of a hit.

[0303] Abstract EQtag circuit. An abstract EQtag circuit is specified as follows:

[0304] Input: (1) a tag, (2) a sequence Tags that consists of p tags output by the K- memory, and (3) / / Valid that indicates the colliding micro-tags.

[0305] Output: A hit {0, 1 } and match e {0, . . . , / ? - I } that equals the index of the matching tag in case hit = 1.

[0306] Functionality: If there exists a tag, e Tags such tag, = tag, then hit = 1 and match = j . Otherwise, hit = 0 and match is arbitrary.

[0307] Remark: Tags are unique keys in a set. Hence, there can be at most one matching tag; e Tags.

[0308] Specialized EQtag circuit ( / ?= 2). A specialized EQtagcircuit specification for the case that / ? = 2 is as follows: Input: A tag, a pair tago, tagi output by the K-memory and / / Valid e {0, I }2output by the probe circuit.

[0309] Output: Signals hit, match G {0, 1 }. hit l(tag8= tag) AND pValid|9] i OR l(tag3= tag) AND pValid[l]) match «H- ( tag. = tag) AND pValidfl ] .

[0310] Functionality: The specification of the outputs is as follows:

[0311] The exact-Multiplexer Circuit. The exact-multiplexer is input (at most) ft microtag collision indices ii, . . . , ip and match e {0, . . . , / ? - 1 } and outputs imatch. The output imatch serves as the index of the way to be read.

[0312] It is to be of note that in the specialized setting of ft = 2, / / Coll consists of two unit vectors, and the exact-multiplexer simply selects one of them according to match. The output way in the specialized setting is a unit vector (i.e., one-hot encoding). The one-hot encoding saves the need to decode the way.

[0313] The EDIT(D) Circuit

[0314] The EDIT(D) circuit resides in the 2ndpipeline stage and deals with computing the new metadata in the processing of EXTRACT and DELETE operations. The following revisions need to take place to the metadata:

[0315] (1) If hit = 1, then the way needs to be deleted. Deletion is implemented by invalidating way, namely, the valid bit of way-mt[way] is reset to zero.

[0316] (2) It may be necessary to revise set metadata set-mt to reflect the new state of the eviction policy.

[0317] (3) If the metadata contains the precomputed victim (for a future eviction), then this victim needs to be recomputed and saved in the metadata.

[0318] The victim / ?, victim,,! Circuits and vic-

[0319] The victim / ?, victimset circuits and vic-multiplexer resides in the 1stpipeline stage and deal with computing the victim in the processing of an INSERT operation. The victim / ? circuit computes the victim under the assumption that the inserted micro-tag collides with ft micro-tags in the set. The victimset circuit computes the victim under the assumption that there are less than p micro-tag collisions. The vic-multiplexer selects one of these victims depending on the number of micro-tag collisions.

[0320] Abstract victimset circuit. The specification of the victimset circuit is as follows:

[0321] Input: Metadata info,

[0322] Output: full e {0, 1 } and waysete{0, . . . , k - 1 }.

[0323] Functionality: The metadata info may contain two relevant parts: (1) a valid bit vi for each way, and (2) the state of the eviction policy. The full signal is that logical and of the valid bits, i.e., full <— and(vi, . . . , Vk). The output wayset is chosen according to the 0\ (pLSUfstate) iffW = 1J

[0324] It is to be of note that if full = 0, then wayset can be any index in {i | Vt = 0}.

[0325] It is to be of note that one may replace the pseudo-LRU eviction policy with other policies (e.g., LFU, MRU).

[0326] Victim / ? circuit. The specification of the victim / ? circuit employs a modulo f counter (denoted by robin) that determines which of the / ? micro-tag collisions is chosen as the victim if / ? micro-tags collide. Every set metadata has one robin counter used for this purpose.

[0327] Input: Metadata counter robin e {0, . . . , / ?-!}, / / Valid, and / / Coll.

[0328] Output: way / ? e {0, l }fc.

[0329] Functionality: way / ? / / Coll [robin].

[0330] It is to be of note that a f -eviction may occur if and only if / / Valid = / ?, in which case, the robin counter is incremented modulo / ?, by the EDIT(I) circuit.

[0331] The vic multiplexer. The specification of the vic multiplexer is as follows:

[0332] Input: way set, way / ? and / / Valid.

[0333] Output: way e {0, l }fc. Functionality: ifjuVaZid = ft \ else. /

[0334] The EDIT(I) Circuit

[0335] The EDIT(I) circuit is a combinational circuit that resides in the 2ndpipeline stage and deals with updating metadata to reflect the insertion and the eviction (or usage of a vacant entry) in the processing of an INSERT operation. The following revisions need to take place to the metadata:

[0336] (1) Overwrite the micro-tag of the inserted item, i.e., / zSetfway] <— / / tag.

[0337] (2) The way’s metadata needs to store the metadata of the inserted tag-record, i.e., way-mt[way] <— mt and the valid bit of the way should be set to one.

[0338] (3) Revise set-mt to reflect the new state of the eviction policy.

[0339] (4) If the metadata contains the precomputed victim (for a future eviction), then this victim needs to be recomputed and saved in the metadata.

[0340] It is to be of note that since all the inputs of the EDIT(I) are computed in the previous pipeline stage, part of the EDIT(I) circuit can be placed in the previous pipeline stage to help balance the delay of the critical paths.

[0341] It is to be further of note that update the eviction policy, e.g., update pLRU state or increment the robin counter if a / ^-eviction, may also occur.

[0342] VIPT CACHES

[0343] In VIPT (Virtually Indexed Physically Tagged) caches, the set-index is computed based on the virtual address, and the remaining tag, based on the physical address. Let P be defined by P = log2 (Page Size). The P least significant bits of the virtual address vaddr and physical address addr are equal. In a VIPT LI -cache, the / / tag is a function of the virtual address vaddr. The physical address addr is required only at the beginning of the 2ndpipeline stage, giving ample time for a TLB-lookup. (A TLB- miss that delays the addr signal requires stalling the pipeline.)

[0344] When are VIPT and PIPT micro-i Consider an LI cache that does not employ micro-tags. Recall that the least significant log2(B) bits (recall B is the size of the cacheline in bytes) are reserved for the offset within the cacheline and are not part of the (physical or virtual) cacheline address. If s, i.e. the number of sets, satisfies (For example, For Intel x86_64 on Linux, P = 12 and B = 64, hence Equation 3 holds if s < 26) lag2(s) < P — log2(B), then in projection based mapping, the set-indices in VIPT and PIPT are identical.

[0345] Since the tags in VIPT and PIPT are based on the same physical address, it follows that if the equation above holds, then VIPT and PIPT are equivalent.

[0346] VIPT with micro-tags

[0347] Consider a VIPT cache that employs micro-tags. The physical address is ready only after virtual-to-physical translation is completed (e.g., TLB-lookup).

[0348] To avoid waiting for the virtual-to-physical translation, the / tag in VIPT is a function of the virtual address (rather than the physical address). This approach creates a problem of false-misses due to aliasing (i.e., a query with a virtual address returns “miss” although the cacheline is stored in the cache).

[0349] It is to be of note that treatment of false-negatives may require the L2 cache to return the tag-record together with the micro-tag. A special version of an UPSERT operation may need to check if the tag is already stored in the tag-memory. If so, an update may take place which (only) updates the micro-tag. Otherwise, an insert may take place.

[0350] A condition that extends the equation above so that a VIPT cache with microtags is equivalent to a PIPT cache (thus false-misses are avoided) is proposed.

[0351] The following theorem assumes that the micro-tag is projected from the vaddr.

[0352] Theorem. Let P = log2 (Page Size in bytes). If P > £ + log2S + log2B then a VIPT cache with E-bit micro-tags is equivalent to a PIPT cache and false-misses do not occur.

[0353] Example (A IKiB VIPT 4-associative cache with 64 cachelines without false- misses). Consider the following parameter settings: P = 12 (i.e. page size 4KiB), number of sets s = 16, B = 16 bytes per cacheline and micro-tag length C = 4 bits satisfies the Theorem's equation. The birthday paradox requirement that 2(> k2, implies that the k < 4. We conclude that a 4-associative VIPT WRBC-cache with (3 = 2 (for P = 12, s = 16, B = 16 and E = 4) is equivalent to a PIPT cache (and hence free of false-misses). Table 4 below summarizes the benchmark results for a VIPT WRBC-cache with the above parameters. k $ niiss easy-niiss liai'd-miss evict p-eviet set-evict unused hisiogrsm i 0 ; 3S.Q0 % i

[0354] 4 1 25.99 6 305 6 14.60 209 6 23.44 3 1 i i; 1 : 75.08 3 /

[0355] 4 2 303 6 9.73 6 305 6 209 6

[0356] 4 4 3L2S % 2'7 34 % 31 24 3 0 {Hl % 31 24 3 21 09 3

[0357] Table. 4

[0358] MULTI-PORT WRBC-CACHE

[0359] By way of introduction, a multi-port cache is a cache that can be input multiple operations in each clock cycle. The above description of the WRBC-cache considered a single-port cache that is input a single operation in each clock cycle.

[0360] The following segment discusses how to extend the single-port design to a multiple-port design.

[0361] As motivation, a few examples of sources of operations are listed, the combination of which may require a multi-port cache: (1) A CPU with instruction-level parallelism may issue multiple load / store operations to its LI -cache. (2) An L2-cache may concurrently receive operations from a subset of its descendant LI -caches in the same clock cycle. (3) A parent cache may issue an operation to its descendant. (4) The cache coherency protocol may try to invalidate or flush cachelines modified by other caches, etc.

[0362] Hereinafter, three main approaches for designing a multi-port WRBC-cache that supports p concurrent operations per clock cycle are proposed.

[0363] (1) Replicate the single-port WRBC-cache design p times, where the memories (i.e., K, V, Mt) are common with p-times the number of ports.

[0364] It is to be of note that this approach prohibits concurrent operations that access the same set in the same clock cycle or in consecutive clock cycles. (2) Let p = P1 P2, where pi and pi are positive integers. Partition the WRBC- cache design to pi parts, where each part a multi-port cache with p2 ports that is in charge of s / pi sets.

[0365] It is to be of note that each part can be fed at most P2 operations per clock cycle. To this end, incoming operations are routed to their respective part. Each part manages a queue of pending operations and feeds at P2 operations per clock cycle to its multi-port cache with p2 ports. As in the single-port WRBC-cache design, operations issued to each part may not access the same set in the same clock cycle or in consecutive clock cycles.

[0366] (3) A partial replication approach in which only way restriction is replicated p times. Easy-misses on queries and updates are answered immediately (i.e., “early-exit”). The remaining operations are queued and fed serially (or in bunches of p' < p operations) to the remaining design.

[0367] This option is suited to the case of many queries that return “miss” (e.g., Destination Snoop Filters).

[0368] METADATA OF A RECORD

[0369] Cache records contain metadata that may be employed to support the functionality of the cache. Below is a list a few possibilities for metadata information and brief discussion on how it is used.

[0370] It is to be of note that the list below serves as mere example and that other combinations may also be applicable.

[0371] (1) Dirty-bit dirty. The dirty bit indicates whether the value of a record has been modified. Typically, a record is inserted with dirty = 0 and upon a value update the dirty- bit is set to one.

[0372] (2) Valid bit. The valid bit indicates whether the value of the record is up-to-date. Resetting the valid bit to zero is a cheap and fast way to delete a tag-record.

[0373] (3) Executable bit. A bit that indicates whether the cacheline val contains instructions. If set, the record is read-only (to avoid modification of instructions).

[0374] (4) Read-only bit. A bit that indicates whether the cacheline val may not be modified. In shared memory architectures, setting the read-only bit to one is a simple technique to declare a cacheline immutable, hence rendering synchronization or coherency redundant.

[0375] (5) Inclusive bit vector. Contains a bit per descendant cache C that indicates if the record (with the same address but possibly different value or metadata) is stored in C. Can also contain a bit for the parent cache C to indicate if C contains the record.

[0376] Example (Support of Cache Coherence Protocols). Cache coherence protocols in multi-core processor systems (e.g., MSI, MESI, MOESI, etc.) store a state per cacheline instance. The encoding of the state can be done by a combination of special explicit bits in the metadata or by metadata bits described heretofore.

[0377] EXAMPLES: EXCLUSIVE AND INCLUSIVE CACHES

[0378] By way of introduction, this following segment presents the functionality of two variants of L2 caches with private-data. The reason for focusing on an L2 cache is the assumption that: (1) there is at least one descendant LI cache; and (2) the L2 cache has a parent L3 cache. Hence, we can refer to a descendant cache and to the parent cache.

[0379] Two variants of how data is organized are considered: an exclusive cache and inclusive cache.

[0380] Private-Data Cache

[0381] The term private-data refers to data that belongs to a single core that is not shared with other cores. For simplicity, we focus on a private-data cache.

[0382] It is to be of note that the private data may be shared by different processes or shared by different threads of the same process, as long as they run on the same core.

[0383] A Simple Exclusive Private-Data L2 Cache

[0384] The term exclusive refers to a setting in which the L2 cache contents does not intersect the cache contents of “descendant" LI caches. An exclusive L2 cache may support two external operations: insertions and extractions (i.e. deleting queries).

[0385] An insertion of a record r may occur only when r is deleted from descendant LI cache, in which case exclusivity implies that r is not already stored in the L2 cache (the record retrieved after a cache miss from both the LI and L2 caches is stored in the LI cache but not in the L2 cache). A query that finds a record r (i.e., a cache hit) must delete r from the L2 cache to maintain exclusivity since r is inserted to the descendant LI cache that issued the query.

[0386] As the record r is private, no other LI cache will issue a query for r.

[0387] A Simple Inclusive Private-Data L2 Cache

[0388] The term inclusive refers to a setting in which the L2 cache contents contains the cache contents of descendant LI caches (it is assumed that the LI caches are “write-back” and not “write-though”, meaning that they do not immediately propagate updates to the L2 cache (updates are deferred to deletion from the LI)).

[0389] An inclusive L2 cache supports three external operations: insertion, update and query (It is assumed that the L3 cache is also inclusive with respect to L2).

[0390] (1) If a query for record r in a descendant LI cache returns a miss and the query for r in the L2 cache also returns a miss, then the record r is retrieved (from the L3 cache or below) and inserted both to the descendant LI cache and the L2 cache (with dirty = 0).

[0391] Suppose that the insertion to the L2 cache causes an eviction, then the L2 cache sends a delete operation of the victim r' to the LI cache if the inclusive bit is set. Record r' from the LI cache is deleted from the LI cache and sent to the L3 cache if the dirty bit in the record of r' is set to one in the LI cache. Note that the L3 cache may receive two different records with the same address: one from the L2 cache and one from the LI cache. In this case, the record from the LI cache is up-to-date.

[0392] (2) A deletion of record r from a descendant LI cache leads to an update of r in the L2 cache if dirty = 1 in the LI record of r.

[0393] (3) A query (from the LI cache) that finds a record r in the L2 cache (i.e., a cache hit) simply returns r without any deletion (r is stored in the descendant LI cache with the dirty -bit set to zero).

[0394] Remark. It is common to mix an inclusive cache policy with an exclusive one. For example, (1) the LI contents are a subset of the L2 contents (i.e., L2 is inclusive of LI), and (2) the L2 contents are disjoint from the L3 contents (i.e., L3 is exclusive of L2). In such a mixed setting, queries to the L3 cache are deleting while queries to the L2 cache are not. It is to be noted, with reference to Figs. 2 to 5, that some of the blocks can be integrated into a consolidated block or can be broken down to a few blocks and / or other blocks may be added. It is to be further noted that some of the blocks are optional. It should be also noted that whilst the flow diagram is described also with reference to the system elements that realizes them, this is by no means binding, and the blocks can be performed by elements other than those described herein.

[0395] It is to be understood that the presently disclosed subject matter is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings. The presently disclosed subject matter is capable of other embodiments and of being practiced and carried out in various ways. Hence, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. As such, those skilled in the art will appreciate that the conception upon which this disclosure is based may readily be utilized as a basis for designing other structures, methods, and systems for carrying out the several purposes of the present presently disclosed subject matter.

[0396] It will also be understood that the system according to the presently disclosed subject matter can be implemented, at least partly, as a suitably programmed computer. Likewise, the presently disclosed subject matter contemplates a computer program being readable by a computer for executing the disclosed method. The presently disclosed subject matter further contemplates a machine-readable memory tangibly embodying a program of instructions executable by the machine for executing the disclosed method.

Claims

CLAIMS:

1. A cache component comprising a plurality of sets partitioned into a plurality of cells, each capable of storing a record composed of (i) a cacheline, and (ii) at least part of a cacheline address, wherein at least one set of said sets is associated with a vector of micro-tags, each of which is related to a respective record of said set, such that said vector is capable of including one or more duplicate micro-tags, wherein, for every duplicate micro-tag, the number of duplicate micro-tags is at least two but less than the number of cells of said at least one set.

2. The cache component of claim 1, wherein said micro-tags are each related to the at least part of a cacheline address of their respective record.

3. The cache component of claim 1, wherein said record further includes metadata attached to said cacheline.

4. The cache component of claim 3, wherein said at least part of a cacheline address contains a tag portion configured to represent the at least part of a cacheline address of said record.

5. The cache component of claim 4, wherein at least one micro-tag of said micro-tags is generated by executing a hash function on the tag portion of said at least one micro-tag's respective cacheline address.

6. The cache component of claim 1, wherein said cache component is a hardware component.

7. The cache component of claim 1, wherein said cache component is a cache in a cache hierarchy.

8. The cache component of claim 4, wherein said cache component is configured to perform at least one cache operation of: (i) a query operation for determining if a given record is included in said cache component, (ii) a delete operationfor removing a given record from said cache component, (iii) an update operation for updating a given record of said cache component, or (iv) an insert operation for inserting a new record into said cache component.

9. The cache component of claim 8, wherein said cache component is further configured to perform at least one of: (v) an extract operation, combining said query operation and said delete operation, such that upon identifying that a given record is within said cache component, said given record is being deleted from said cache component, or (vi) an upsert operation, combining said update operation and said insert operation, such that upon said given record is within said cache component, said given record is being updated, otherwise it is being inserted into said cache component.

10. The cache component of claim 8, wherein the query operation includes:(a) obtaining a query record cacheline address;(b) generating a query record micro-tag from the tag portion of the obtained query record cacheline address;(c) comparing the generated query record micro-tag to the micro-tags of said vector of micro-tags associated with the set corresponding to the set-index of said query record cacheline address;(d) upon said comparison of step (c) yielding no match, providing an indication that said query record cacheline address is not within said cache component;(e) upon said comparison of step (c) yielding at least one match, comparing the tag of said query record cacheline address to the tags of the one or more records corresponding to the micro-tags of said vector found to match said query record micro-tag, and,(f) upon said comparison of step (e) yielding a match, providing an indication that said query record cacheline address is within said cache component, otherwise providing an indication that said query record cacheline address is not within said cache component.

11. The cache component of claim 10, wherein the comparison of step (e) is performed sequentially until a matching tag is found or until all records corresponding to the matched micro-tags of said vector have been compared.

12. The cache component of claim 10, wherein upon said comparison of step (e) yielding a match, in addition to said indication that said query record cacheline address is within said cache component, said query operation includes at least one of: (i) updating the metadata of the record matching said query record, (ii) returning at least part of the value of the record matching said query record, or (iii) returning at least part of the metadata of the record matching said query record.

13. The cache component of claim 8, wherein the delete operation includes:(a) obtaining a cacheline address of a record to be deleted;(b) generating a micro-tag from the tag portion of said cacheline address of the record to be deleted;(c) comparing said micro-tag to the micro-tags of the vector of micro-tags associated with the set corresponding to the set-index of said cacheline address of the record to be deleted;(d) upon said comparison of step (c) yielding at least one match, comparing the tag of said record to be deleted to the tags of the one or more records corresponding to the matched micro-tags of said vector, and,(e) upon said comparison of step (d) yielding a match, updating the metadata of the record of the matched tag of the at least one matched micro-tag to reflect its deletion from the set corresponding to the set-index of said cacheline address of the record to be deleted.

14. The cache component of claim 13, wherein the comparison of step (d) is performed sequentially until a matching tag is found or until all records corresponding to the matched micro-tags of said vector have been compared.

15. The cache component of claim 13, wherein prior to step (e), said delete operation includes outputting at least part of the record being deleted in said step (e).

16. The cache component of claim 8, wherein the insert operation includes:(a) obtaining a record to be inserted, including at least (i) a record to be inserted cacheline, and (ii) a record to be inserted cacheline address;(b) generating a micro-tag from the tag portion of said record to be inserted cacheline address;(c) comparing said micro-tag to the micro-tags of the vector of micro-tags associated with the set corresponding to the set-index of said record to be inserted cacheline address;(d) upon said comparison of step (c) yielding a number of matches equal to the number of identical micro-tags within said vector, evicting a record of said records associated with said identical micro-tags from said set, and inserting said record to be inserted in place of said evicted record;(e) upon said comparison of step (c) yielding a number of matches that is less than the number of identical micro-tags within said vector, determining whether the set corresponding to the set-index of said cacheline address of the record to be inserted is full;(f) upon said determination of step (e) being that said set is not full, inserting said record to be inserted to a vacant cell within said set;(g) upon said determination of step (e) being that said set is full, evicting a record of said records from said set, and inserting said record to be inserted in place of said evicted record.

17. The cache component of claim 8, wherein the update operation includes:(a) obtaining an update cacheline address and at least one of (i) at least part of an update cacheline, or (ii) at least part of update metadata;(b) generating an update micro-tag from the tag portion of said update cacheline address;(c) comparing said update micro-tag to the micro-tags of the vector of microtags associated with the set corresponding to the set-index of said update cacheline address;(d) upon said comparison of step (c) yielding at least one match, comparing the tag of said update cacheline address to the tags of the matched micro-tags of said vector, and,(e) upon said comparison of step (d) yielding a match, performing at least one of: (i) updating at least part of the metadata of the record associated with the matched micro-tag according to the at least part of update metadata, or (ii)replacing at least part of the cacheline of the record associated with the matched micro-tag with the at least part of the update cacheline.

18. The cache component of claim 13, wherein said cache component is a hardware component associated with (i) a metadata memory configured to store the metadata of the records stored within the cells of a given set, (ii) a K-memory configured to store the tags of the records stored within the cells of said given set, (iii) a probe box configured to perform step (c) of the delete operation, (iv) an EQtagbox configured to perform step (d) of the delete operation, and (v) an EDIT box configured to perform step (e) of the delete operation.

19. The cache component of claim 18, wherein the output of said probe box is in one-hot encoding and wherein said K-memory is configured to access the cells whose indices are in one-hot encoding, thus eliminating the need to decode said cell indices.

20. The cache component of claim 16, wherein said cache component is a hardware component associated with (i) a metadata memory configured to store the metadata of the records stored within the cells of a given set, (ii) a K-memory configured to store the tags of the records stored within the cells of said given set, (iii) a V-memory configured to store the cachelines of the records stored within the cells of said given set, (iv) a probe box configured to perform step (c) of the insert operation, (v) a victim^ box configured to determine the evicted record of step (d) of the insert operation, (vi) a victimset box configured to determine whether the set has a vacant cell, and if so, output the index of said vacant cell, and if set is full to determine the evicted record of step (g) of the insert operation, (vii) a multiplexer configured to output a cell index selected either from the output of the victim^ box or the output of victimset box, and (viii) an EDIT box configured to perform at least part of any of steps (d), (f), and (g) of the insert operation.

21. The cache component of claim 20, wherein the output of said multiplexer is in one-hot encoding and wherein said K-memory is configured to access the cells whose indices are in one-hot encoding, thus eliminating the need to decode said cell indices.

22. The cache component of claim 17, wherein said cache component is a hardware component associated with (i) a metadata memory configured to store the metadata of the records stored within the cells of a given set, (ii) a K-memory configured to store the tags of the records stored within the cells of said given set, (iii) a V-memory configured to store the cachelines of the records stored within the cells of said given set, (iv) a probe box configured to perform step (c) of the update operation, (vi) an EQtagbox configured to perform step (d) of the update operation, and (vii) an EDIT box configured to perform at least part of step (e) of the update operation.

23. The cache component of claim 22, wherein the output of said probe box is in one-hot encoding and wherein said K-memory is configured to access the cells whose indices are in one-hot encoding, thus eliminating the need to decode said cell indices.

24. The cache component of claim 1, wherein said vector is capable of including one or more pairs of identical micro-tags, and is incapable of including more than two identical micro-tags.

25. The cache component of claim 17, wherein the comparison of step (d) is performed sequentially until a matching tag is found or until all records corresponding to the matched micro-tags of said vector have been compared.

26. The cache component of claim 10, wherein said cache component is a hardware component associated with (i) a metadata memory configured to store the metadata of the records stored within the cells of a given set, (ii) a K-memory configured to store the tags of the records stored within the cells of said given set, (iii) a probe box configured to perform step (c) of the query operation, (iv) an EQtagbox configured to perform step (d) of the query operation, and (v) an EDIT box configured to perform step (e) of the query operation.

27. The cache component of claim 26, wherein the output of said probe box is in one-hot encoding and wherein said K-memory is configured to access the cells whose indices are in one-hot encoding, thus eliminating the need to decode said cell indices.

28. The cache component of claim 17, wherein said update operation further includes obtaining an indication as to which part of the cacheline or metadata is to be updated.

29. The cache component of claim 18, wherein the K-memory is a dual port K-memory that allows simultaneous reading and comparing of at least two tags of the tags stored therein.

30. The cache component of claim 20, wherein the K-memory is a dual port K-memory that allows simultaneous reading and comparing of at least two tags of the tags stored therein.

31. The cache component of claim 22, wherein the K-memory is a dual port K-memory that allows simultaneous reading and comparing of at least two tags of the tags stored therein.

32. The cache component of claim 26, wherein the K-memory is a dual port K-memory that allows simultaneous reading and comparing of at least two tags of the tags stored therein.

Images