System and method for storing and processing efficient compressed cache lines

Abstract

The present disclosure provides a multi-processor system, comprising: a plurality of nodes and at least one memory, wherein each node comprises at least one processor, a first cache dedicated to the node, a second cache at a higher level than the first cache, and a cache location buffer dedicated to the node, wherein, for at least one of the plurality of nodes, at least one of the first cache and the second cache included in the at least one node comprises at least one cache location capable of storing compressed data units of different sizes, and the cache location buffer included in the at least one node is configured to store a plurality of cache location buffer entries, each of the plurality of cache location buffer entries comprising a plurality of location information values. The present disclosure also provides a data management method and a computer-readable storage medium.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] U.S. Provisional Application No. 62 / 588,717, filed with the U.S. Patent and Trademark Office on November 20, 2017, is incorporated herein by reference in its entirety. Technical Field

[0003] At least some example embodiments of the inventive concepts relate generally to methods and systems for accessing data in a computer memory device, and more particularly, to mechanisms and techniques for representing and processing compressed data. Background Art

[0004] Today's processors are often equipped with caches that can store copies of data and copies of instructions stored in some high-capacity memory. A popular example of such high-capacity memory today is dynamic random access memory (DRAM). In this article, the term "memory" will be used to collectively refer to all existing and future memory implementations. Cache memory, or simply "cache", is usually built with a much smaller and faster memory than other memory implementations, and can then only keep a copy of a small portion of the data stored in the main memory or auxiliary storage device at any given time. Today, caches are often implemented using SRAM, and large caches can be implemented using DRAM. The caches described herein can be implemented using any existing and future memory technology.

[0005] Typically, the memory system of a computer system includes a series of caches, with larger and slower caches (referred to herein as higher-level caches) close to the main memory and smaller and faster caches (referred to herein as lower-level caches) close to the processor. This configuration is often referred to as a cache hierarchy, memory hierarchy, or memory system. Each level in the cache hierarchy is called a cache level. Figure 1 is a block diagram showing a portion of a computer system. Figure 1 In the example shown, the computer system is a multiprocessor system in which each CPU 101 is assigned its own first level private cache 102 (L1 cache). Figure 2A more detailed example of CPU 101 and L1 cache 102 is described. A private cache is a cache in which space for data cells can only respond to activity of the CPU local to the cache (or a CPU located in the same node as the cache, where the cache is private to that node). In this example, the second level cache 103 is shared by all CPUs 101 and can contain data and instructions accessed by all CPUs 101. Because space for data cells can be made responsive to activity of any CPU 101, cache 103 is a global cache. Furthermore, cache 103 is a shared cache in that each data cell can only exist in one shared copy (whereas each private cache can have its own replica). Memory 105 also stores instructions and data accessed by all CPUs 101. Physical addresses (or PA for short) are typically used to access (main) memory 105, while addresses generated by CPU 101 are typically virtual addresses. For example Figure 1 The system shown, a computer system with multiple CPUs and multiple dedicated caches, requires an efficient mechanism to find a requested data unit in one of the caches and to keep multiple copies of the data stored in different nodes consistent.

[0006] In addition to the main memory and cache memory, such a computer system typically includes one or more auxiliary storage devices. These auxiliary storage devices may include one or more of a hard disk drive, an optical drive, a flash drive, etc., collectively represented here by disk 104. Disk or auxiliary storage 104 can store several orders of magnitude more data than memory 105, but cannot be directly accessed using physical addresses. If CPU 101 wants to access data stored on disk 104, the virtual memory system ( Figure 1 105) moves large blocks of data (often called pages) from disk 104 to memory 105 and creates translations from the virtual addresses corresponding to the pages to the physical addresses. Special types of translation caches ( Figure 1 The virtual memory system (not shown) (commonly referred to as a "translation look-aside buffer" or TLB for short) caches translation mappings from virtual pages to physical pages. The virtual memory system can be viewed as a cache system that stores a portion of the data located on disk 104 in memory 105.

[0007] Figure 2 is a block diagram used to explain an example cache hierarchy for a computer system. Figure 2 Shown from Figure 1201 and a more detailed view of the cache organization, the CPU 204 is shown to include multiple cache-like structures, such as an instruction TLB (ITLB, 201) and a data TLB (DTLDB, 205), and the CPU core 202. The level 1 cache 102 is shown to include an instruction cache 200 and a data cache 206 inserted between the CPU 204 and the level 2 (and possibly higher level) cache 207. Figure 2 Also shown is a high capacity memory 208 connected to the cache hierarchy. Figure 2 The computer system in is shown as a uniprocessor (ie, single processor) system. Figure 2 The computer system in may be part of a multiprocessor system.

[0008] When CPU 101 requests data that is present in the cache (referred to as a cache hit), the request can be serviced much faster than an access to data that is not present in the cache (referred to as a cache miss). Typically, an application running on CPU 101 that experiences fewer cache misses will execute faster and consume less energy than the same application that experiences more cache misses (assuming the two versions of the application have similar properties). Therefore, considerable effort has been invested in techniques for avoiding cache misses, and implementations have been designed that can make cache hits and cache misses efficient in terms of access latency and the energy consumed by these memory transactions.

[0009] In order to understand how to develop solutions to this cache hit / miss problem, some understanding of cache organization and terminology will be useful. Typically, data is installed into the cache in fixed blocks that are larger than the word size of the processor, called cache lines. Common cache line sizes today are, for example, 32, 64, and 128 bytes, but as will be appreciated by those skilled in the art, there are larger and smaller cache line sizes for various cache implementations. For some cache implementations, the cache line size may also be variable.

[0010] A common way to organize data placement in memory 208 is in a way that each data word is statically mapped to reside in a specific cache line. Each cache typically has an index function that identifies the portion of the cache where each cache line can reside, which is called a group. A group can contain space for holding one or more cache lines at the same time. The number of cache lines that a group can hold is called its associativity. Typically, the associativity of all groups in a cache is the same. Such caches are typically called set associative caches. Associativity can also vary between groups. Each space in a group that can hold a cache line is called a way.

[0011] In order to determine the identity of each cache line stored in each group, the cache lines in the cache each have some identifier associated with them. A common example of such an identifier is an address tag. When looking up a particular cache line in a cache, its address can be used to determine the group of cache in which it may reside. The address tag of the cache line of the corresponding group is compared to the tag portion of the address used to identify the cache line (see, for example, in Figure 3 The ADDR address tag is a tag (depicted in and described below) to determine whether the desired cache line resides in the cache, and if so, which way it resides, that is, which space in the group that can hold the cache line. Typically, the size of this ADDR address tag is quite large and can be in the range of 30-40 bits, which is 6-10% of the typical cache line size.

[0012] Typically, each cache has a built-in strategy to determine which cache lines to keep in a set and which cache lines to evict (also called replaced) to make room for new cache lines to be introduced into the set, which is called a replacement policy. The cache lines that are replaced are called victim cache lines or replaced cache lines. Popular replacement policies used in conjunction with cache memories include (but are not limited to) least recently used (LRU), pseudo-LRU, and random replacement policies.

[0013] An inclusive cache hierarchy requires that a copy of a data block (e.g., a cache line) present in one cache level, such as the L1 cache, also be present in higher cache levels (here, higher refers to cache levels with a number higher than 1), such as in the L2 and L3 caches. An exclusive cache hierarchy only has one copy of a data block (e.g., a cache line) present in the entire cache hierarchy, while a non-inclusive hierarchy may have a mix of the two strategies. In both exclusive and non-inclusive cache hierarchies, a cache line is typically installed in the next higher cache level upon eviction from a given cache level.

[0014] Keeping this background of caches and cache lines in mind, the discussion now continues by describing the methods used to identify caches such as Figure 1 and Figure 2 A conventional technique for specifying a cache location within a cache hierarchy as shown. Figure 3 is a block diagram for explaining a conventional implementation of a two-level cache hierarchy including a set associative cache and a set associative data translation lookaside buffer (TLB). Figure 3An example of the organization of a computer system node including a CPU 301, a DTLB 303, an L1 cache 307, and an L2 cache 311 is shown. The CPU 301 generates a memory request including a virtual address 302 (VADDR), which is used to perform an associative lookup in the DTLB 303. In this example, the virtual address 302 is divided into three parts: P_OFFSET (page offset, e.g., composed of low-order bits), a tag, and an index. The index portion of the virtual address 302 is used to identify a group within the DTLB 303 where the address translation can be stored.

[0015] The DTLB 303 used in this example is shown to have a bidirectional associative organization, which shows two entries of the identified group. Each entry consists of an address tag (ATAG) 304 and a physical page frame (PPF) 305. The tag portion of the virtual address 302 is compared with the ATAG 304 of each entry of the identified group. Logic 318 determines whether there is a match for any entry, and if so, controls MUX 306 to select the corresponding PPF 305. The physical address PADDR 316 is composed by concatenating the PPF 305 selected by MUX 306 with the P_OFFSET portion of the virtual address 302. Alternatively, if none of the entries of the identified group matches the tag portion of the virtual address 302, a TLB fill operation is performed and the required translation entries are introduced into the DTLB 303.

[0016] PADDR 316 is used to perform a lookup in L1 cache 307. More specifically, the index portion of PADDR 316 is used to identify a group that can store a cache line containing the requested data. (It should be noted that the L1 index may contain different bits than the DTLB index). The depicted L1 cache 307 has a bidirectional associative organization with two entries in the identified group. Each entry consists of an address tag (ATAG) 308 and data 309 of the corresponding cache line. The tag portion of PADDR 316 is compared with the ATAG of each entry of the identified group. Logic 319 determines whether there is a match for any entry and controls MUX 310 to select the corresponding data. If none of the entries match (indicated by L1 miss 317), a lookup is required in L2 cache 311. Although the division of PADDR 316 into index and tag portions for L2 cache lookups may be different from the divisions made for L1 cache lookups, the remaining steps for L2 cache lookups are generally similar to the steps performed for lookups in L1 caches. If a miss in the L2 cache is determined, a new lookup in a higher level cache or access to memory may be required. It may be noted that if the index portion of PADDR 316 used by the L1 cache consists entirely of P_OFFSET bits, accesses to the L1 cache 307 may begin before the PPF information from MUX 306 is available. This is often referred to as a virtually indexed physically tagged (VIPT) cache.

[0017] Those skilled in the art will understand that each has Figure 3 Multiple "nodes" of a memory hierarchy similar to the memory hierarchy shown in FIG. 1 can be connected together to form a coherent multiprocessor system. Figure 1 Another example of a multiprocessor is shown in , where each CPU has its own private L1 cache. This further complicates the task of locating where the requested data resides, since it is no longer sufficient to always search for the data in the next higher cache level. A mechanism is needed for searching the caches of other nodes and for keeping multiple copies of datums stored in different nodes consistent.

[0018] As further background information, Figure 4is a block diagram illustrating an example implementation of a tag-less cache. Such a cache relies on location information (LI) (sometimes referred to as a cache line pointer CP) corresponding to a requested cache line of a matching cache table entry (CTE) (sometimes referred to as a region location) to indicate in which cache the requested cache line resides and at which location in that cache. This will cause the computing system to perform a direct cache lookup in the appropriate cache. Figure 4 The example implementation of the sparsely tagged cache shown in FIG. 1 includes a two-level cache hierarchy and a two-level cache location buffer (CLB) hierarchy, wherein the first level is accessed virtually and the second level is accessed physically.

[0019] For example, in Figure 4 In the system shown (which may be a node in a multiprocessor system), the location information may distinguish between the L1 cache and the L2 cache. It may also be possible to distinguish between other caches ( Figure 4 (not shown) to distinguish between. Figure 4 An sparsely tagged node is shown that can be connected to other nodes in a multiprocessor system having a two-level cache hierarchy of set associative caches and a CPU (which can contain one or more traditional caches, described here as L0), where the CPU 401 generates an address (ADDR) 402. Figure 4 Also shown are group-associative CLBs of a two-level CLB hierarchy, where a first-level CLB CLB1 410 and a second-level CLB CLB2 420 are indexed by ADDR. As used in the present disclosure, the term "CLB1" may refer to a first-level CLB, and the term "CLB2" may refer to a second-level CLB. Each entry in CLB1 410 includes at least an address tag (AT) 411 and a cache location table (CLT) 412. It may store additional information, such as region information (RI) 413. CLT 412 stores location information (LI) of cache lines of a corresponding region (also referred to as a micro-page). Figure 4Four LIs per region are shown as an example. Empirical studies by the inventors have shown that 8 to 16 LIs per region (i.e., 8 to 16 data units (cache lines) per region) are cost-effective and efficient. The location information encodes the location of the corresponding cache line that can be found in a multiprocessor system. For example, the location information may include an identification of a cache in which the data unit can be found. The location information may also include the location where the data unit can be found within the cache. Alternatively, the location information may include an identification of a node in which the data unit can be found, or some symbols that provide information about its location, such as the symbol "MEM" indicating that the data unit can be found in a memory. In at least some cases, the symbol "unknown" may indicate that the location of the corresponding data unit is not recorded in the CLT corresponding to the region.

[0020] MUX 416 selects the CLT having AT 411 that matches the tag portion of ADDR 402, and MUX 415 selects the LI 414 corresponding to the requested cache line based on the L-OFFSET portion of ADDR 402. If the cache tag stored in the selected LI corresponds to L2 cache 440, an address 417 for accessing the requested data unit in L2 cache 440 is formed using the index portion of address 402 and a portion of location information 414. In the example of a set associative L2 cache 440, this portion contains information corresponding to the associated way in which the requested data unit is stored. If the cache tag stored in the selected LI corresponds to L1 cache 430, an address 418 for accessing the requested data unit in L1 cache 430 is formed in a similar manner.

[0021] For a cache hierarchy consisting of set-associative caches with the same associativity, the number of bits in each LI depends on the number of ways and the number of caches of the cache hierarchy covered by the CLB. One LI value may be reserved for invalid mode ("MEM"). In this embodiment, the number of LI bits is equal to the base 2 logarithm of the number of ways plus the base 2 logarithm of the number of cache levels covered by the CLB plus 1 (i.e., log2(way)+log2(level)+1). If the number of ways of different cache levels varies, or if caches other than set-associative caches are used, those skilled in the art will appreciate that other LI representations are possible. Typically, the size of the LI is much smaller than the size of a typical address tag. A 3-level cache hierarchy with 16-way associative caches at each level may be encoded using 6 LI bits.

[0022] exist Figure 4In the example shown in , each entry in L1 cache 430 and L2 cache 440 has a back pointer (BP) (432 and 442, respectively) associated with its cache line (DATA) (431 and 441, respectively). In this example, the BP pointers point to their respective associated CTEs in CLB2 420. Each CTE in CLB2 420 contains an address tag (AT) 421, a CLT 422, and a CLB1 pointer (CIP) 423. If a CLB2 entry has a corresponding CLT residing on CLB1 410, its CIP will point to the entry. In this case, CLB1 410 will contain the latest information for the region. In this case, the CLE in CLB1 is called an active CLB entry, while the CLB entry for the region in CLB2 420 contains a passive entry. If CLB1 410 contains an active entry for a region, CLB2 420 will be required to contain a passive entry for the region.

[0023] In one embodiment, when no matching CTE is found in CLB1 410, the corresponding CTE is looked up in CLB2 420 and copied to CLB1, and the old entry in CLB1 is copied to CLB2 420, which is referred to as overflow and fill or simply overflow / fill ( Figure 4 Overflow and fill in the cache). For example, overflow is also performed when a traditional inclusive cache evicts a data unit to the next level to make room for a new data unit, which can be brought in (filled) from the next cache level. Similar overflow / fill activities will be performed between different CLB levels, where CLB entries are moved between CLB levels because the CLB levels form an inclusive hierarchy. The directory DIR can be located at the root of the CLB hierarchy and use the highest level CLB for overflow / fill.

[0024] Access to the requested data unit may be initiated in parallel with the overflow / fill activity.

[0025] The contents of the CLBs of the multi-level CLB implementation may be consistent with the locations of the data units of the multi-level cache system they manage. Data units moved in the cache hierarchy (including but not limited to movements caused by requests for data units from the CPU, evictions of data units, prefetch activities, and coherency activities) result in updates to their corresponding location information in one or more CLBs. For example, in Figure 4In the example, evicting a data unit 441 from cache 440 may include the following steps: following its associated BP 442 pointer to find its corresponding CTE (which is in CLB2 420 in this example) and updating its corresponding location information to point to the new location of the data unit (e.g., storing the MEM symbol). And, if the CIP pointer of the CLE corresponding to CLB2 is valid, following the CIP to find its corresponding CTE in CLB1 410 and updating its location information associated with the cache line. In one embodiment, if there is a valid CIP pointer, the entry in CLB2 is not updated. For example, the CLT may be updated so that only the active CLT is updated.

[0026] As will be appreciated from the foregoing description, use of the above-mentioned sparsely tagged cache system (i.e., a cache system in which data units stored in the cache do not have address tags associated therewith) will result in many different types of transactions occurring when the CPU 401 requests various cache lines located in different storage devices. Figure 4 The node depicted in FIG. 1 may be a subsystem (or node) of a larger multiprocessor system composed of multiple such cache nodes. Keeping the data in the caches of multiple nodes consistent requires a cache consistency protocol, such as implementing snooping or directory-based consistency. The consistency protocol sends a consistency request to the cache. Figure 4 Node depicted (external request). These requests typically go first to CLB2 420, which can determine that the cache system does not have the requested cache line (CLB2 miss or CLB2 hits an entry with an inactive CIP and the location information of the requested cache line maintains a numerical memory mode). If so, no further action is required regarding the data caches L1 and L2 (called consistency filtering), but the corresponding LI in the active CLT still needs to be updated to track the new location associated with the data unit. Otherwise, the CLB2 lookup can provide the requested location information (CLB2 hits and the location information of the requested cache line maintains the location where the cache line is stored), or it can determine that CLB1 410 stores the associated location information (CLB2 hits and a valid CIP). In the latter case, a lookup is required in CLB1 410 to determine the location of the requested cache line or the cache line does not reside in the cache subsystem. (The corresponding location information in CLB1 contains the memory mode). Depending on the nature of the coherence request, such as an invalidation request, a read request, or an exclusive read request, the coherence protocol will perform some operations on the requested data unit and may change its state. It may also be necessary to update the corresponding LI in the active CLT to track the new location associated with the data unit.

[0027] Although Figure 4 The node shown in FIG. 1 is composed of two cache levels, but those skilled in the art will appreciate that other numbers of levels may be used to assemble the node, and that the node may contain more than one CPU. In addition, the number of cache levels in a node and the number of CLB levels do not need to be the same.

[0028] Figure 5 is a block diagram showing a portion of a computer system including two CPUs connected to a two-level cache hierarchy and a two-level cache location buffer (CLB) hierarchy. For example, Figure 5 The computer system in FIG. 5 includes two CPU nodes (580 and 590), each having a CLB (CLB1A 503 and CLB1B 513) and private cache L1 caches (504 and 514) in their own first-level cache hierarchy. The system also includes a globally shared L2 cache 523 (sometimes referred to as a last-level cache LLC) that is shared between the two CPUs 501 and 511.

[0029] The lookup in CLB1 (513 or 503) selects a single LI based on the address generated by their respective CPU (501, 511), for example using techniques similar to CLB1 410. As indicated by the three arrows starting from the two CLT entries in CLB1A 503 and CLB1B 513 shown, the selected LI may identify that the location is in the corresponding L1 cache (504 and 514, respectively), in the L2 cache 523, or in another node (represented by 570 and 571, respectively). In this example, CLB1A 503 identifies that its access data is 505 in the shared L2 cache 523, while CLB1A 503 identifies that its access data is 505 in its L1 514 (shown by the solid arrows).

[0030] Figure 6 is a block diagram illustrating a portion of a computer system including a sparsely tagged cache hierarchy having a single monolithic last level cache. Figure 6 , Figure 6A general description of a tag-less multiprocessor memory system is shown. This example includes N nodes, ranging from node 1 601 to node N 699. Each node has X-level CLBs and Y-level dedicated caches (i.e., caches dedicated to the node). The nodes are connected to each other through an on-chip network (NoC) 650 circuit. NoC 650 also connects the node to a directory (DIR) 660, a global last-level cache (LLC) 670, and a memory 680. DIR 660 is organized similarly to CLB, and DIR 660 has entries consisting of at least an address tag (AT) 661 and a cache location table (CLT) 662. The entry may also contain additional information, such as region information (RI) 663, which will be described in more detail below. The highest-level CLB in the node (CLB-X 630, CLB-X 693) overflows / fills from DIR. DIR also plays a central role in the consistency protocol that keeps the contents of the cache and CLB consistent and coherent.

[0031] The CLB and DIR can be viewed as forming an inclusive "cache hierarchy" that caches metadata with information about the contents of the data cache hierarchy. The data hierarchy containing L-1 to LY and LLC can be viewed as a separate cache hierarchy, on which no inclusive attribute is imposed. For example, it can be non-inclusive, inclusive, or exclusive.

[0032] exist Figure 6 In the example shown, each node has one CPU (CPU-1 600, CPU-1 696), which may contain zero, one or more traditional tag-based caches, which are depicted as LO-D and LO-1. A memory request that cannot be satisfied by any traditional LO cache will generate a lookup in the node's CLB-1 (e.g., 610) to search for the corresponding CLB entry CLT, which can be obtained using a method similar to Figure 4610 in node 1 601. If a corresponding entry is found in CLB-1, location information (LI) corresponding to the requested data unit may be located in its CLT field (612). The LI identifies the location where the requested data unit can be found. The LI in CLB 610 residing in node 1 601 may identify any of the node's own caches (L-1 620 to LY 640), LLC (670), memory 680, or other nodes (node ​​2 690 to node N 699) as the location of the data unit it is tracking. The LI may also identify the location of the data unit in the identified cache. The LI may also indicate whether the data unit resides in any of the traditional caches L0-D or L0-I of the CPU 600 connected to the node.

[0033] If a cache location (or memory) is identified as where the data can be found, the request can be satisfied by reading the cache directly without consulting the directory (DIR) 660. If a node is identified as where the requested data can be found, the request is sent to the CLB-X of that node and is addressed to the node. Figure 4 The external request discussed is similarly satisfied. But there is still no need to consult the directory to find the data. This is different from most traditional directory-based consistency schemes, in which the directory is consulted and updated for accesses outside the node.

[0034] despite this, Figure 6 Although one CPU is shown in each node, those skilled in the art will recognize that a node may contain any number (including zero) of CPUs, GPUs, accelerators, or other devices that can access memory, such as I / O devices. In addition, the configuration and size of CLBs and caches may vary greatly between nodes.

[0035] There can be a strict hierarchical search level by level in an inclusive CLB / DIR hierarchy. If the location information is not found at level CLB-i, a search is performed at the next level CLB-(i+1). If the location information is not found in the highest level (CLB-X) of the node, a search is performed in the DIR. If the location information is found at the CLB level or DIR, a read request can be sent to the data location identified by the corresponding location information of that level, and a new CLE entry corresponding to the requested area is created in CLB-1. The CLB / DIR hierarchy is inclusive. This means that if there is an available CLB entry at level CLB-L, an entry must be installed at level CLB-(L+1). The CLB entry closest to the CPU is in the active state. It can be found in cache level K (and lower caches, i.e. Figure 6) and a certain CLB level C (and all higher CLB levels, i.e. Figure 6 In the embodiment of the present invention, there is a mandatory inclusion between caches L-1 620 and LY 640. For example, in order for data to be available in any cache L-1 620 to LY 640, there must be a corresponding entry in CLB-X 630. If the entry in CLB-X 630 is evicted, all data in the corresponding region should be evicted from caches L-1 620 to LY 640. This is called mandatory eviction. In addition, all CLB entries under the same node to CLB-1 610 need to be evicted.

[0036] It should be noted that there is no inclusion requirement between cache levels within a node (e.g., cache L-1 620 to LY 640) or between LLC 670 and caches in a node (e.g., cache L-1 620 to LY 640). For example, valid data corresponding to a particular address may be present in L1 cache 620, but not in cache LY 640 or LLC cache 670. This enables cache bypass optimizations, where, for example, streaming data only needs to be installed in L1 cache 620 (actually, only in L0 if it is present), and not in other levels. However, corresponding CLB entries (typically 50 times smaller than the corresponding data area) need to be installed in all levels.

[0037] The directory (DIR) 660 contains CLB-like information: address tags (AT) 661, cache location table (CLT) 662, and region information (RI) 663. Although these fields have the same names as the fields of the CLB (e.g., CLB-X 630), the contents of these fields may be different from the corresponding CLB fields. The RI field 663 of the DIR 660 may contain N so-called presence bits (PB) that indicate which nodes are tracking the region, where N corresponds to the number of nodes in the system. If bit K in the PB is set, it means that there is a corresponding entry in CLB-X of node K. This further means that there may be cached data in any of the cache levels L-1 to LY of node K, and there may also be a corresponding entry in CLB-1 in node K, which will enable the CPU in node K to access any cache line of the corresponding region. Although an N-bit implementation involving PB information is described, those skilled in the art will recognize that many scalable techniques for directory implementation (including but not limited to coarse-grained, limited pointers, and linked lists) can be used to achieve similar functionality.

[0038] The PB bit can be used to classify regions. If one of the PB bits of the corresponding entry in the directory 660 is set, the region is classified as a dedicated region (PR). Empirical studies have shown that on average about 80% of all accessed regions are PR regions of widely selected research benchmarks. Regional classification PR means that the data of the region can only exist in one node whose corresponding PB bit is set, and it is also guaranteed that no other node can access the data from the region at this point in time. Node 1 601, whose PB bit is the only set bit in the DIR, can be notified that the corresponding region is a dedicated region (PR), and the regional classification can be recorded in its corresponding region information (RI) in its CLB (e.g., in CLB-1 610 to CLB-X 630). PR regions can be accessed efficiently in many ways. For example, global consistency is not required to add write permissions for a node to a data unit present in the node, while the node currently only has read permissions for the data unit (to upgrade the data unit from state S to state E), because only one node may have cached a copy of the data of the data unit, and no other node can access any data unit of the region. In addition, the movement of data units of the PR region can be efficiently implemented because data can be moved up and down the entire cache hierarchy (e.g., performing an eviction from LX to LLC or moving data from LLC to L-1) without notifying any structure outside the node. The movement also needs to be recorded in the CLB (610 to 630) local to the node. This means that the CLT information of the PR region stored in the CLB (610 to 630) of the node can be different from (e.g., it is updated) the CLT information of the region stored in the DIR 660.

[0039] When the second node (e.g., 690) accesses the PR region (empirical studies have shown that this is a relatively rare event), it will miss in all of its CLBs 691 to 693 and will send a request to DIR 660 to obtain the appropriate location information LI required to access the data and allow its own CLB entries to be created in CLBs 691 to 693. Since this request is for the PR region, DIR 660 may not have the latest information and it needs to obtain the latest location information from Node 1 601, ensure that the CLB of Node 1 601 no longer marks the region as PR, update its own CLT information for the region, set the PB bit corresponding to Node 2 690 and send the information of the appropriate CLB for the region to Node 2 690. Node 2 690 can now create its own local CLB entry managed with the region. Node 1 601 may have to complete all of its ongoing direct data memory requests for the region before sending the latest location information to DIR 660.

[0040] If another node (e.g., node N 699) accesses the region, it will also miss in all of its CLBs and will also send a request to DIR 660, which now has the latest information about the region and can reply to node N 699 with the appropriate information and will also set the PB bit for the region for the requesting node (i.e., node N 699).

[0041] A region with more than one PB bit set is classified as a shared region (SR). This means that multiple nodes have CLBs that are tracking the location information of the region. It also means that nodes with corresponding PB bits set can store data units of the region in any of their caches (e.g., 620 to 640). If the region is classified as SR, requests from the CPU (e.g., 600) to access data units that cannot be satisfied locally within the node (e.g., by caches L-1 620 to LY 640) may need to initiate a global coherence transaction.

[0042] Globally consistent read transactions to SR or PR regions (reads that cannot be satisfied by local caches (e.g., caches L-1 620 to LY 640)) can utilize location information retrieved from its local CLB (e.g., CLB-1 610 to CLB-X 630) to directly access the data without involving the DIR 660 or communicating with any node outside of that node that can be identified by the location information. This is referred to herein as a direct-to-master access. Empirical studies have shown that 75% of all global consistency transactions for SR regions are direct-to-master read transaction accesses. Therefore, if 20% of the global transactions initiated by the CPU are for SR pages, and 75% do not require access to the DIR, then when using at least some of the cache structures and techniques mentioned above, 95% of the global consistency transactions do not require access to the DIR, while for traditional directory-based protocols, all global consistency transactions require access to the DIR.

[0043] Since an SR region has several nodes that track the location of its data units, the movement of their data units needs to be reflected in all nodes that track the region (i.e., those nodes that have active CLB entries for the region) and the location information of the moved data units updated in these nodes. The location information of the CLBs of different nodes that track the data units should be updated in a consistent manner so that the CLBs agree on where the data is located (called its primary location). It may also be highly likely or optionally guaranteed that the data unit location accessed based on the location information retrieved from the CLB contains the correct data (called determinism).

[0044] A common reason for such data unit movement is a global coherent write, i.e., a requesting node that does not currently have write permission for a data unit requests write permission through a global coherent request (e.g., an invalidate request or an exclusive read request). After completing the request, the data unit will reside only in the requesting node.

[0045] A global consistency write may be sent by a requesting node (e.g., 601) to a directory 660, which (locates the corresponding directory entry) retrieves the corresponding PB bit for the region and forwards the request to all nodes (except the requesting node) with the corresponding PB bit set, which are referred to as slave nodes (e.g., nodes 690 and 699). The slave node sends an ACK message when it ensures that any future request for the data unit originating from the slave node will be directed to the requesting node. This may be achieved, for example, by changing the location information of the data unit in their CLB to point to the requesting node (601), which will become the new master node. The ACK message may be sent to the DIR 660 or directly to the requesting node 601, in which case the DIR 660 aggregates the ACK message and sends an ACK to the requesting node once all ACKs are received. Once the requesting node knows that all ACKs have been received, it has obtained exclusive write permission for the cache line and completes the request. After completing the request, a completion message may be sent to the DIR 660.

[0046] Another common reason for data movement is eviction (also called replacement). Node-local evictions (e.g., eviction from L-1 620 to LY 640) are handled locally, tracked by their local CLB entries, and are not visible outside the node. However, the global eviction of a master data unit (e.g., from location 641 in LY 640 to a new location 671 in LLC 670) must be reflected by all nodes tracking the region (i.e., all nodes with CLB entries for the region). Global eviction can be achieved by first copying the value of the data unit stored in its old location (641) to the new location (671) while keeping the data stored in the old location (641) valid. Then, a global consistency eviction request with information about the new location (671) is sent from the requesting node (e.g., 601) to the DIR (660), which forwards the request to the slave nodes (e.g., 690 and 699). Once the slave nodes update their corresponding location information to the new location (671), they send ACK messages. Once the requesting node knows that all ACKs have been sent, it obtains permission to evict the data unit and can reclaim the old position and complete the request. Once the request is completed, a completion message can be sent to the DIR 660.

[0047] The slave node may not send an ACK message until some outstanding requests for the requested cache line (eg, all requests directly to the master) have completed.

[0048] The directory 660 may implement a blocking mechanism that ensures that there is at most one outstanding global request of a particular type per data unit. Examples of such request types may be global consistency writes and global consistency eviction requests. The blocking of a cache line ends after the request is completed, or as otherwise specified by the consistency protocol. The blocking mechanism may be exact, such as implemented by one lock per data unit in the directory, or may be highly approximate, where addresses hash to a limited pool of locks, and a block on a data unit will block access to all other data units that hash to the same lock.

[0049] Blocking can be done at region granularity instead of data unit granularity, and some CLB / directory overflow / fill requests also need to be blocked until their updates have completed. This ensures that no globally consistent write requests are processed while nodes and directories are overflowing / filling CLB information.

[0050] Another useful region classification is a region where no PB bit associated with the region is set in DIR 660. This means that the region data units will not exist in any of the node's caches L-1 to LY, and no node can currently access its data units. This is called a non-tracking region (UR). The data in the UR region can still be cached in the LLC. The data units in the UR region can be evicted from the LLC 670 without any global consistency request. Regions that do not have corresponding entries in DIR 660 are classified as memory regions (MR). The data units in the MR region can neither be cached in the node nor in the LLC. Replacing the CLT entry from DIR 660 means that all data units in the region should be forcefully evicted from the LLC.

[0051] Figure 6 The cache system may be a tag-less cache system, i.e., a cache system in which data units stored in a cache (e.g., caches L-1 through LY and LLC of a node) do not have address tags associated with them. Instead, location information with address tags associated with them identifies the location where the data unit is stored. Figure 6 The traditional caches in (LO-1 and LO-D) may still have address tags associated with their cache data cells.

[0052] Figure 7 is a block diagram illustrating a generalized sparsely tagged cache hierarchy having many slices of last level cache. Figure 7 and Figure 6 The difference is that the LLC has been cut into LLC slices (LLC-1 771, LLC-2 772, and LLC-N 779) and placed on the side of the network on chip (NoC) 750 close to the node. Each slice is configured with a replacement counter (RC) 775, 776, and 777, respectively. These counters count the number of cache replacements in each LLC slice. It should be noted that Figure 7 The example shown in places LLC slices outside of each node (indicated by the dashed boxes). Even though each LLC slice (e.g., 771) is adjacent to each node and can be accessed from the node's CPU (700) with very short latency, they can still be considered part of a large global LLC. Evicting a region entry from CLB-X (e.g., 730) does not necessarily require that the corresponding region data residing in an LLC slice (e.g., 771) close to the node be forcibly evicted. In addition, a node (e.g., 701) can store location information in its CLB (e.g., 710) that directly points to a location in a remote LLC slice adjacent to the remote node (e.g., the location in 772), and access that data without indirect steps (indirections) through DIR 760 or any CLB of that node (790).

[0053] The existing LLC implementation method is reminiscent of Figure 7 (Only insofar as LLC is physically divided into banks that are physically placed close to the nodes, Figure 7 However, current solutions (e.g., server chips from Intel) place data units in LLC banks based on their addresses, which results in 25% of accesses hitting in LLC banks hitting in banks close to the nodes, assuming four nodes. The cache access techniques described according to at least some of the examples discussed above (e.g., based on location information pointers) allow for sufficient flexibility in placing data units and are independent of the addresses of the data units, which may increase the local portion of LLC hits. One option is to replicate all shared data in all LLC slices where the shared data is accessed. However, replicating all shared data in all LLC slices will consume LLC capacity. As previously mentioned, 20% of the area is typically shared. Assuming that all shared areas need to be replicated in all nodes, a four-node system will occupy 80% of the LLC capacity for storing replicated LLC data. This may be suboptimal, prompting the need for smarter LLC allocation / replication decisions.

[0054] When a data unit is first allocated to an LLC slice, it seems beneficial to always place it in an LLC slice adjacent to the CPU accessing it. However, since threads running in different CPUs may have different LLC capacity requirements, it may sometimes be beneficial to place a data unit in a remote LLC slice and thus "steal" some of its local LLC capacity. In at least one example, allocation decisions are made based on LLC slice cache pressure. Each LLC slice has a replacement counter (RC 775, 776, 777) that records the number of replacements that have occurred in the slice. In at least one example, these counters are periodically monitored to determine the cache pressure of each LLC slice, where a higher count indicates a higher pressure. When allocating LLC space for a data unit, if its cache pressure is the lowest, space is always allocated in the local LLC slice. If not, a configurable portion of its allocation will be made in the remote LLC slice that currently has the lowest pressure, and the rest will be allocated in the local LLC slice. In at least one example, the allocation of space in the remote LLC slice will only occur in certain circumstances, for example, if the cache pressure of the local LLC slice is above a certain threshold.

[0055] In a traditional cache topology, a single last-level cache is shared by multiple nodes (e.g. Figure 1 L2 103 or Figure 6 LLC 670), there is usually at most one copy of each data unit residing in the LLC (called a shared cache), while caches dedicated to a node (also called private caches) (e.g. Figure 1 The L1 caches 102) can each store a duplicate copy of the same data unit. Figure 7 An LLC slice may choose to store a single copy of a data unit in one of the LLC slices and have all nodes access it using the same location information, or may choose to store multiple so-called replicated copies of the data unit in multiple LLC slices.

[0056] For example, data unit 774 may be the only LLC copy of the data unit, and both CLT 712 in CLB-1 710 and CLT in CLB-1 791 have associated location information pointing to location 774. This is referred to herein as shared LLC data. After deciding to replicate the data, a copy of the data may be placed in location 773 and the associated location information in CLT 712 may be changed to point to location 773. The data unit stored in 773 is referred to as a local LLC data unit. Local data stored in 773 may only be accessed indirectly by looking up in a CLB local to Node 1 701, while global data stored in location 774 may still be accessed directly by using location information pointing to location 774, for example, through Node N 799.

[0057] In at least one example, Figure 7 The cache system of is a tag-less cache system, i.e., a cache system in which data units stored in the caches (e.g., caches L-1 through LY and LLC of the node) do not have address tags associated with them. Instead, location information with address tags associated with them identifies the location where the data unit is stored. Figure 7 The traditional caches in (LO-1 and LO-D) may still have address tags associated with their cache data cells.

[0058] Summary reference Figures 3 to 7 Based on the discussion of , those skilled in the art will appreciate that current implementations of cache hierarchies may require performing many dependent lookups in the TLB and at different cache levels before the location of the requested cache line can be determined. In addition, each translation entry and cache line of the cache requires an accompanying address tag, and a plurality of these tags need to be compared with portions of the address of each of these levels. In general, this implementation is very costly in terms of the dynamic energy used for each dependent lookup. It also requires a large amount of static energy and memory capacity to store the additional tag information required to perform the comparison. In addition, it also requires a large amount of static energy and memory capacity to store the data of each cache line in the cache hierarchy.

[0059] It would therefore be desirable to provide a system and method that avoids the aforementioned problems and disadvantages associated with cache lookups for data by supporting efficient representation and processing of compressed data in a cache hierarchy, while removing some of the capacity and static power costs associated with cache line storage. Summary of the invention

[0060] According to at least one example embodiment of the present invention, data units are stored in dedicated caches in nodes of a multiprocessor system, each node including at least one processor (CPU), at least one cache dedicated to the node, and at least one cache location buffer (CLB) dedicated to the node. Location information values ​​are stored in each CLB, each location information value indicating a location associated with a corresponding data unit, wherein each location information value stored in a given CLB indicates that the location is a location within a dedicated cache in the same node as the given CLB, a location in one of the other nodes, or a location in a main memory. Compressing cache lines is a method of using cache capacity more efficiently. The ability to identify the locations of uncompressed cache lines and compressed cache lines can be used to expand CLB location information while adding only a logarithmic cost (e.g., adding two bits for each location information) compared to the traditional linear cost (adding four times the address tags). The CLB location information may mark a cache line with a "0" containing a special ZERO symbol, thereby avoiding allocation of a cache line containing "0" in the cache hierarchy, avoiding movement of such cache lines and avoiding repeated copying of the value "0" to a newly allocated memory area.

[0061] According to at least one example embodiment of the inventive concept, a multiprocessor system includes: a plurality of nodes and at least one memory, wherein each node includes at least one processor, a first cache dedicated to the node, a second cache at a higher level than the first cache, and a cache location buffer (CLB) dedicated to the node, wherein, for at least one of the plurality of nodes, at least one of the first cache and the second cache included in the at least one node includes at least one cache location capable of storing compressed data units of different sizes, the CLB included in the at least one node is configured to store a plurality of CLB entries, each of the plurality of CLB entries includes a plurality of location information values, each location information value points to the start of a location associated with a corresponding data unit, each location information value stored in the CLB included in the at least one node indicates the following location to which the location information value points, including a location within the first cache in the at least one node, a location within the second cache in the at least one node, or a location in the at least one memory, and at least one of the plurality of location information values ​​can identify a location corresponding to the corresponding data unit at a finer granularity than the size of an uncompressed data unit.

[0062] According to at least one example embodiment conceived in the present invention, a data management method includes: storing data units in dedicated caches in multiple nodes of a multi-processor system, wherein each node includes at least one processor, at least one cache dedicated to the node, and at least one cache location buffer (CLB) dedicated to the node; and for at least one of the multiple nodes of the multi-processor system, storing multiple location information values ​​in the at least one CLB included in the at least one node, each location information value indicating a location associated with a corresponding data unit, wherein each location information value stored in the at least one CLB indicates that the location associated with the corresponding data unit is a location in the at least one dedicated cache included in the at least one node, a location in one of the multiple nodes other than the at least one node, or a location in a main memory, and encoding at least one location information value among the multiple location information values ​​that has a symbol indicating that the data unit stores a first value.

[0063] According to at least one example embodiment conceived by the present invention, a non-temporary computer-readable storage medium including computer-executable instructions, when a computer executes the instructions, causes the computer to perform operations including the following steps: storing a data unit in a dedicated cache included in multiple nodes of a multi-processor system, in a global cache and in a memory, wherein each node includes at least one processor and at least one cache dedicated to the node, and the multiple nodes are connected via a network; and for at least one of the multiple nodes, storing multiple location information values ​​in a cache location buffer dedicated to the at least one node; and encoding at least one of the multiple location information values ​​using a symbol, wherein the symbol indicates that the data unit stores a first value. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The above and other features and advantages of the exemplary embodiments of the present invention will become more apparent by describing in detail exemplary embodiments of the present invention with reference to the accompanying drawings. The accompanying drawings are intended to describe exemplary embodiments of the present invention and should not be interpreted as limiting the intended scope of the claims. Unless explicitly stated, the accompanying drawings should not be considered to be drawn to scale.

[0065] Figure 1 is a block diagram illustrating a portion of a computer system;

[0066] Figure 2 is a block diagram for explaining an example cache hierarchy of a computer system;

[0067] Figure 3is a block diagram for explaining a conventional implementation of a two-level cache hierarchy including a set-associative cache and a set-associative data translation lookaside buffer (TLB);

[0068] Figure 4 is a block diagram illustrating an example implementation of a tagged-less cache;

[0069] Figure 5 is a block diagram showing a portion of a computer system including two CPUs connected to a two-level cache hierarchy and a two-level cache location buffer (CLB) hierarchy;

[0070] Figure 6 is a block diagram illustrating a portion of a computer system including a tagged-less cache hierarchy including a single monolithic last level cache;

[0071] Figure 7 is a block diagram illustrating a generalized sparsely tagged cache hierarchy having many slices of last level cache;

[0072] Figure 8 is a block diagram illustrating three tag-based cache architectures with various degrees of data compression according to at least one example embodiment of the inventive concepts;

[0073] Fig. 9 is a block diagram illustrating a simplified representation of a little-tagged cache system according to at least one example embodiment of the inventive concepts; and

[0074] Fig.10 is a block diagram illustrating a simplified representation of a tag-less cache system that efficiently supports compressed data storage and processing according to at least one example embodiment of the inventive concepts. DETAILED DESCRIPTION

[0075] As in accordance with the tradition of the field of the present invention, the embodiment is described in terms of functional blocks, units and / or modules and shown in the accompanying drawings. It will be understood by those skilled in the art that these blocks, units and / or modules are physically implemented by electronic (or optical) circuits, such as logic circuits, discrete components, microprocessors, hard-wired circuits, memory elements, wiring connections, etc. formed using semiconductor-based manufacturing techniques or other manufacturing techniques. In the case of blocks, units and / or modules implemented by microprocessors or the like, they can be programmed using software (e.g., microcode) to perform the various functions discussed herein, and can be optionally driven by firmware and / or software. Alternatively, each block, unit and / or module can be implemented by dedicated hardware, or can be implemented as a combination of dedicated hardware that performs certain functions and processors (e.g., one or more programmed microprocessors and related circuits) that perform other functions. Moreover, without departing from the scope of the present invention, each block, unit and / or module of the embodiment can be physically divided into two or more interactive and discrete blocks, units and / or modules. Furthermore, without departing from the scope of the inventive concept, the blocks, units and / or modules of the embodiments may be physically combined into more complex blocks, units and / or modules.

[0076] In the following description, details of functional blocks and example cache location buffers CLBs are provided that include effective alternatives to conventional cache tagging schemes according to at least some example embodiments of the present invention. In addition, although the CLB schemes according to at least some example embodiments of the present invention are described with reference to specific caches and computer architectures, those skilled in the art will recognize that one or more example embodiments of the present invention are applicable to a wide range of memory and system architectures. In some cases, well-known circuits and structures are not described in detail for simplicity.

[0077] According to at least some example embodiments of the inventive concept, unlike the known discarding of useful location information each time a cache line is moved between levels in a hierarchy and between nodes, the location information is stored (e.g., updated) in a cache handling scheme (CHS) and used for subsequent cache accesses. In the present disclosure, the term "location information" is synonymous with "LI" or "location information LI", and may also sometimes be referred to as "LI" or "location information LI". In addition, the value of a particular location information may be referred to as a "location information value". The CLB is organized in an associative manner. Therefore, each CLB entry includes an address tag field and a plurality of LIs associated with the address tag field, but according to at least some example embodiments of the inventive concept, other techniques for storing location information are also possible. Subsequently, the information stored in the CLB can be used to quickly access the location of the requested data without the need for additional associative lookups. More specifically, the CLB provides location information LI from which the correct cache identifier and the correct location within the cache can be determined without accessing any address tags of the cache. The LI can also identify a remote node as the location of the associated data. Therefore, using the technology of at least some example embodiments of the inventive concept, the requested data can be found faster in the cache hierarchy and in a manner that consumes less energy. The described technology also enables more flexible and less hierarchical cache topologies to be effectively implemented.

[0078] Therefore, according to at least some example embodiments of the present inventive concepts, such as Figure 1 and Figure 2 The system shown in can also include a cache handling scheme (CHS) as described herein. The scheme includes a cache location buffer (CLB) or a hierarchy of CLBs that keeps track of the locations of data units stored in the cache hierarchy. Whenever a data unit is moved in the cache hierarchy, its new location is recorded in the CLB.

[0079] The CLB according to at least some example embodiments of the present inventive concept may be implemented as an associative organization storing multiple cache location entries (CLEs). Each CLE contains an address tag for associative lookup and some payload, such as a cache location table (CLT), storing multiple location information LIs. In fact, the CLB itself can be regarded as a cache storing location information.

[0080] Now refer to Figures 8 to 10Describe the features of at least some example embodiments of the inventive concept. For the sake of clarity, those skilled in the art will understand that these figures illustrate the logical implementation of at least some example embodiments of the inventive concept rather than the actual physical implementation. Therefore, those skilled in the art will understand that these figures illustrate the logical expression of the implementation, and the physical implementation may be substantially different in its implementation details.

[0081] There are various proposals for data compression in cache hierarchies. It has been recognized that some data values ​​appear frequently in important applications. Using 64-bit words to represent such high-frequency data values ​​may be redundant and waste cache capacity and static power. Instead, many proposals have proposed more efficient ways to represent high-frequency data values. Although there are various different complex compression algorithms, the most important principle is quite simple. Detect frequently used data values ​​and introduce new and efficient encodings for these values. These encodings can be static or dynamic. As a simple example, assume that three hexadecimal values ​​0x0, 0x1, and 0xFFFFFFFF appear frequently in an application. This allows only two bits of information to be used instead of uncompressed 64 bits of information (8 bytes) to represent a 64-bit word storing any of these values. Now assume that each 64-bit value in a cache line starts with a 2-bit compression tag added. If the compression tag has a value of 1, 2, or 3, it means that the word stores the frequently occurring values ​​0x0, 0x1, and 0xFFFFFFFF, respectively, and no more bits are needed to store the value of the word. These compressed values ​​can be represented by storing only their compression tags. The very next portion of the cache line will store the compression tag for the next 64-bit word. If the compression tag stores a value of 0, it means that the next 64 bits of information after the compression tag stores an uncompressed value. If the 8 64-bit words of a 64-byte cache line contain 4 such compressed values ​​and 4 uncompressed values, the cache line can be represented by approximately half the bits compared to a traditional uncompressed cache line.

[0082] While the compression techniques proposed are generally more elaborate than the examples above, they have the same compression effect: some cache lines can be represented by a smaller number of bits than the normal table representation that uses 512 bits to represent a 64-byte cache line. However, finding a compressed representation of the data values ​​of a cache line does not allow for a more compact representation using conventional tag-based caching alone. The cache structure itself must also be changed to be able to access cache lines of different sizes in order to take full advantage of the compression.

[0083] Figure 8 is a block diagram illustrating three tag-based cache architectures with various degrees of data compression according to at least one example embodiment of the inventive concept. Figure 8 , Figure 8Three different set associative caches are shown. Figure 8 Only a two-way associative cache is shown in the figure, and these cache implementations can be generalized to have a greater or lesser number of way associations. The topmost cache (A) consists of an address tag (AT) array 810 and a data array 815. Based on an access request to the cache, an indexing function will select a group in which the requested cache line may exist. Both address tags 811 and 812 are compared to the address of the request to determine whether the requested cache line is stored in its corresponding cache line (816 and 817) in the data array. If no tag matches the requested address, a cache miss is determined.

[0084] The top cache (A) cannot benefit from a cache line compression scheme, such as the simplified example discussed above, because it can only store two cache lines in each set. Figure 8 The cache in (B) shows a design that can benefit from cache compression. Even though each group currently holds two uncompressed cache lines per group in its data array, its corresponding tag array is designed to hold 8 tags per group. As shown in the middle cache example (B), since each tag is quite large (≈30 to 40 bits) and typically consumes more area per bit, this is a rather expensive solution for applications with few compressed cache lines. Here, two uncompressed cache lines 826 and 827 are shown in the data array 825. Only the address tags 821 and 822 corresponding to the start of each uncompressed cache line in the tag array are initialized to hold their corresponding address tags. The remaining six address tags in the group are marked as invalid. Therefore, cache (B) requires four times the address tag capacity of cache (A) to store the same number of uncompressed cache lines.

[0085] However, if Figure 8 The cache architecture depicted in (B) is used to run an application with many compressible cache lines, and the cache usage shown for the bottom cache (C) can be achieved. Here, it is assumed that a compression scheme with a compression degree of 4 is supported, that is, compressed cache lines can be represented as 25%, 50% or 75% of the size of the uncompressed cache line. (A compression degree of 8 can compress cache lines to 12.5% ​​of their uncompressed size, etc.) If all cache lines are compressed to 25% of the size of the uncompressed cache line, each group can hold 8 compressed cache lines. In that case, each of the address tags of the tag array 830 will be valid and correspond to each compressed cache line.

[0086] In contrast, the bottom cache (C) is shown storing cache lines with mixed degrees of compression: 25%, 50%, or 100%. A total of four cache lines 836, 837, 838, and 839 have been loaded into the group. There is a portion of the cache group that cannot be utilized, as shown by the gap between cache lines 837 and 838, which is called fragmentation. Each of the four cache lines has a corresponding valid address tag 831, 832, 833, and 834 (aligned with its starting point). The remaining address tags are unused and are marked as invalid. This means that even in this rather beneficial compression scheme, which manages to store twice the number of cache lines in the group compared to examples (A) and (B), half of the address tags are still unused and will incur static power and area (and possibly dynamic power as they may be compared to the requested address).

[0087] Fig. 9 is a block diagram illustrating a simplified representation of a little-marked cache system according to at least one example embodiment of the inventive concepts. Fig. 9 Shows the use of Figures 4 to 7 The cache system of some of the less-tagged techniques described in FIG. includes two levels of CLBs 911 and 912. Even if the CLBs are Fig. 9 901, L2 902 or L3 903. As previously mentioned, each LI is encoded by the following number of bits, which corresponds to the logarithm of the number of cache levels plus the logarithm of their associativity plus the logarithm of the space for some "symbols", that is, log2(level)+log2(way)+1, assuming that the number of levels and the number of ways and some more "symbols" (such as "MEM" and "don't know" need to be encoded) are even numbers of powers of two that need to be encoded. Assuming a three-level cache hierarchy with a 16-way associative cache at each level, a 7-bit LI is sufficient to also encode the "symbols" including "MEM" and "don't know".

[0088] For simplicity, Fig. 9 Each of the 3 levels of cache in the hierarchy is shown as a two-way associative cache that stores uncompressed cache lines. Fig. 9 It is shown how three such LI pointers 931, 932 and 933 point to the start of a corresponding cache line in each of cache levels L1 901, L2 902 and L3 903, respectively.

[0089] Fig.10 is a block diagram illustrating a simplified representation of a tag-less cache system that efficiently supports compressed data storage and processing according to at least one example embodiment of the inventive concepts. Fig.10 Shown with Fig. 9 The ability to store uncompressed cache lines (in this example, 2 uncompressed cache lines per group) of the same cache hierarchy. According to at least some example embodiments of the present inventive concepts, compressed cache lines may be stored to a particular degree of compression, such as Fig.10 The compression level shown is 4. Each LI pointer (1031, 1032, 1033) points to the start of a compressed cache line in each of the three cache levels 1001, 1002 and 1003, respectively. This means that the LI encoding should be able to identify four more locations in each cache set. Figure 8 Converting a traditional cache to one that supports compression level 4 has a linear cost: from two address tags per group to eight address tags per group, where each address tag is 30 to 40 bits, but supports Fig.10 The cost of the same degree of compression in is logarithmic, with an increase of 2 bits in the size of the LI, e.g., from 6 to 8. According to at least one example embodiment of the inventive concept, the number of extra bits required to support a compressed cache line with a certain degree of compression D compared to a system that does not support compression is log2(D). In one example, the number of bits encoding each LI changes from log2(way)+log2(level)+1 to log2(way)+log2(level)+log2(compression degree)+1.

[0090] According to at least one example embodiment of the present inventive concept, the LI is further extended to encode the size of the compressed cache line it points to. For systems with the highest compression level D, this will add an additional log2(D) bits. According to at least one example embodiment of the present inventive concept, this will make the LI bit count log2(way)+log2(level)+2×log2(compression level)+1. According to at least one example embodiment of the present inventive concept, the LI is further extended to encode the compression type, that is, which decompression method is required to decompress the cache line. Therefore, it should be understood that at least some example embodiments of the present inventive concept enable the LI to identify data locations at a finer granularity than the size of the uncompressed data unit.

[0091] and Figure 8 In contrast to the system shown in (C), the degree of compression does not have to be built in the same way. Fig.10 There is no hardware similar to Figure 8 The structural changes required to transform system (A) to system (C).

[0092] Fig.10 Another compression feature is also introduced: the symbol “ZERO” encoded by the LI, as shown in 1034. According to at least one example embodiment of the inventive concept, the LI storing the symbol ZERO means that the entire cache line referenced by the LI stores the value ZERO. According to at least one example embodiment of the inventive concept, if the LI contains the symbol ZERO, there is no space allocated for the corresponding cache line in any cache. According to at least one example embodiment of the inventive concept, if the core reads a word from its cache line whose corresponding LI stores the symbol ZERO in CLB-1, the cache hierarchy returns the value “0” to the core, but no cache line is allocated or moved in the hierarchy to achieve this. According to at least one example embodiment of the inventive concept, when a cache line attempts to store a word to its cache line whose corresponding LI stores the symbol ZERO in CLB-1, a zero-filled cache line will be allocated in its L1 and it will eventually get write permission. According to at least one example embodiment of the inventive concept, the LI will be updated to reflect the fact that the primary cache line is stored in its L1 cache.

[0093] In a multiprocessor system, a cache line corresponding to a physical address may have associated LIs in multiple CLBs. If this is the case, then either all of them point to the symbol ZERO or none of them should point to the symbol ZERO. According to at least one example embodiment of the inventive concept, before allowing a core to have its cache line converted from the ZERO LI pointer to a zero-filled and writable cache line in its L1, a reference to the LI pointer will need to be involved. Figure 6 and Figure 7 A similar cache coherence protocol as discussed above is used to ensure that other LI=ZERO pointers associated with the same physical cache line are changed. According to at least one example embodiment of the inventive concept, those other LI=ZERO pointers should be changed to a representation corresponding to an invalid coherence state. According to at least one example embodiment of the inventive concept, the change should follow Figure 6 and Figure 7 and those other LI=ZERO pointers should be changed to point to the L1 location storing the new zero-filled writable cache line.

[0094] In reality, many cache lines contain the value "0". Whenever a new memory region is allocated in the runtime system, the entire memory region is initialized to hold the value "0". This is usually achieved by pointing the newly allocated virtual page to a special physical page (the zero page). In a traditional cache system, the zero page contains only cache lines that store the value "0". A core that reads this page will be returned the value "0". Traditionally, this is achieved by allocating and moving cache lines containing "0" in the hierarchy. The first time a core attempts to write a page mapping to the zero page, a trap is generated, a new physical page is allocated and initialized to store only "0" cache lines, and the mapping of the virtual page is changed to point to the new physical page.

[0095] According to at least one example embodiment of the inventive concept, a zero page is represented by a page that has all LIs storing the value ZERO. A read attempt to a zero page does not result in any cache line allocation or any cache line movement in the cache hierarchy. Instead, the core reading the zero page will be returned the value "0". According to at least one example embodiment of the inventive concept, when a write is attempted to a zero page, a trap is generated, and the CLB region associated with the accessed virtual page (mapped to the zero page) will be associated with its new physical address instead. This can be achieved, for example, by changing a per-region physical address field (also referred to as a region) associated with each CLE, such as RI 413, 633, or 733. This scheme will use the cache capacity of the hierarchy in a more efficient manner (no cache lines are allocated for LI=ZERO cache lines), will consume less bandwidth in the system (less movement of cache lines containing the value "0") and avoid unnecessary CPU work (copying the value zero to a new cache line).

[0096] According to at least one example embodiment of the inventive concept, LI can encode other predefined values ​​besides "0", such as a value "1" and a value storing a binary value "1" in all bits of a cache line. Those skilled in the art will appreciate that any value can be selected to be represented in this manner, and the selected value can be dynamically selected during execution.

[0097] In conjunction with the accompanying drawings (such as Figures 8 to 10 ) The steps described in the foregoing may be performed in various orders, and some steps may be performed in parallel or sequentially. At least some example embodiments of the inventive concept may provide various beneficial results. For example, detailed simulation studies performed according to at least some example embodiments of the inventive concept have shown that cache implementations according to at least some example embodiments of the inventive concept may save a significant amount of energy compared to conventional cache systems, and may also result in caches having significantly shorter access times.

[0098] For the sake of clarity, most of the description herein generally describes the technique of how to locate a cache line and return it to the requesting CPU. This description does not describe in detail the various ways of selecting the request word contained in the cache line and returning it to the CPU. However, various methods for selecting the request word contained in the cache line and returning the requested cache line to the CPU are known to those skilled in the art.

[0099] For clarity, most of the descriptions of the processing of data in the cache hierarchy described herein describe an exclusive cache hierarchy. Those skilled in the art will appreciate that one or more of the exemplary embodiments of the inventive concept can be extended to also cover inclusive memory hierarchies and non-exclusive memory hierarchies.

[0100] Although one or more exemplary embodiments of the inventive concept described above may be related to a single processor system and a multi-processor system, for example, Figure 1 and Figure 2 The system shown and described, but one or more exemplary embodiments of the inventive concept are mainly shown in conjunction with a single processor system. However, those skilled in the art will appreciate that one or more exemplary embodiments of the inventive concept shown in conjunction with a single processor system are not limited to such implementation.

[0101] Although described above in the context of certain example computer architectures, caches exist in Figures 6 to 8 The example computer system shown in and many other settings outside, and those skilled in the art will appreciate that at least some of the example embodiments of the above-described inventive concept in the context of a computer system may also be applied to such other contexts. An example of such use is a virtual memory system that caches data from a slow, high-capacity memory (e.g., a disk or FLASH memory) into a faster and smaller high-capacity memory that can be implemented using dynamic RAM. Other examples of caching in a computer system include (but are not limited to) a disk cache, a web cache, and a name cache. The organization and caching mechanism of such a cache may be different from the cache discussed above, for example, there are variations in the size of the group, the implementation of the group, and the associativity. Regardless of the implementation of the caching mechanism itself, at least some of the example embodiments of the present invention concept are equally applicable to implementing various caching schemes.

[0102] The methods or flow charts provided in the present application may be implemented in a computer program, software or firmware clearly embodied in a computer-readable storage medium for execution by a general-purpose computer or processor.

[0103] Thus having described the exemplary embodiments of the inventive concept, it will be apparent that the exemplary embodiments of the inventive concept may be modified in a variety of ways. These changes should not be regarded as departing from the intended spirit and scope of the exemplary embodiments of the inventive concept, and all such modifications apparent to those skilled in the art are intended to be included within the scope of the appended claims.

Claims

1. A multiprocessor system comprising: a plurality of nodes and at least one memory, wherein each node comprises at least one processor, a first cache dedicated to the node, a second cache at a higher level than the first cache, and a cache location buffer dedicated to the node, Wherein, for at least one node among the plurality of nodes, at least one of the first cache and the second cache included in the at least one node includes at least one cache location capable of storing compressed data units of different sizes, The cache location buffer included in the at least one node is configured to store a plurality of cache location buffer entries, each of the plurality of cache location buffer entries comprising a plurality of location information values, the plurality of location information values ​​comprising one or more first location information values ​​and one or more second location information values, each first location information value pointing to a start of a location associated with a corresponding data unit, Each first location information value stored in the cache location buffer included in the at least one node indicates the following location to which the first location information value points, comprising a location within said first cache in said at least one node, comprising a location within said second cache in said at least one node, or the at least one memory location, and At least one of the one or more first position information values ​​is capable of identifying a position corresponding to the respective data unit at a finer granularity than a size of an uncompressed data unit, and At least one second position information value of the one or more second position information values ​​encodes a symbol indicating that the data unit stores the first value.

2. The multiprocessor system according to claim 1, wherein: At least one position information value of the one or more first position information values ​​encodes a size of a compressed data unit pointed to by the at least one position information value.

3. The multiprocessor system according to claim 1, wherein: At least one position information value of the one or more first position information values ​​encodes a compression type of a compressed data unit pointed to by the at least one position information value.

4. The multiprocessor system according to claim 1, wherein: The first value is dynamically selected.

5. The multiprocessor system according to claim 1, wherein: In response to at least one processor included in the at least one node performing a read operation on a data unit pointed to by the at least one second position information value encoding the symbol, the first value is returned to the at least one processor. 6 . The multiprocessor system of claim 5 , wherein in response to the read operation, a location of the first value is not to be allocated or moved in the first cache and the second cache included in the at least one node.

7. The multiprocessor system according to claim 1, wherein the first value is "0".

8. The multiprocessor system according to claim 1, wherein the first value is "1".

9. A data management method, comprising: storing the data unit in dedicated caches in a plurality of nodes of a multiprocessor system, wherein each node comprises at least one processor, at least one cache dedicated to the node, and at least one cache location buffer dedicated to the node; as well as For at least one node among the plurality of nodes of the multiprocessor system, A plurality of location information values ​​are stored in the at least one cache location buffer included in the at least one node, the plurality of location information values ​​comprising one or more first location information values ​​and one or more second location information values, each first location information value indicating a location associated with a corresponding data unit, wherein each first location information value stored in the at least one cache location buffer indicates a location associated with the corresponding data unit of, comprising a location in said at least one dedicated cache in said at least one node, a position in a node of the plurality of nodes other than the at least one node, or location in main memory, and At least one of the one or more second position information values ​​is encoded with a symbol indicating that a data unit associated with the at least one second position information value stores a first value.

10. The data management method according to claim 9, wherein: The steps of storing the data unit in dedicated caches in a plurality of nodes of the multiprocessor system include: storing at least one of the compressed data units and the uncompressed data units of different sizes in at least one cache location provided in the dedicated cache, and Wherein, the first value is dynamically selected.

11. The data management method according to claim 10, further comprising: Based on at least one of the one or more first location information values ​​stored in the at least one cache location buffer included in the at least one node, a location corresponding to the corresponding data unit is identified at a granularity finer than a size of an uncompressed data unit.

12. The data management method according to claim 10, further comprising: At least one of the one or more first position information values ​​is encoded with a size of the compressed data unit.

13. The data management method according to claim 10, further comprising: At least one first position information value of the one or more first position information values ​​is encoded with a compression type of the compressed data unit.

14. The data management method according to claim 9, further comprising: performing, by the at least one processor included in the at least one node, a read operation on a data unit corresponding to the at least one second position information value encoded with the symbol; as well as The first value is returned to the processor.

15. The data management method according to claim 14, in, Returning the first value to the at least one processor included in the at least one node includes returning the first value without allocating a location for the first value in the at least one dedicated cache included in the at least one node.

16. The data management method according to claim 9, wherein the first value is "0".

17. A non-transitory computer-readable storage medium comprising computer-executable instructions, which, when executed by a computer, cause the computer to perform operations comprising: storing the data unit in dedicated caches, in a global cache, and in a memory included in a plurality of nodes of a multiprocessor system, wherein each node includes at least one processor and at least one cache dedicated to the node, The plurality of nodes are connected via a network; and For at least one node among the plurality of nodes, storing a plurality of location information values ​​in a cache location buffer dedicated to the at least one node, the plurality of location information values ​​comprising one or more first location information values ​​and one or more second location information values, each first location information value pointing to a location associated with a corresponding data unit; and At least one of the one or more second position information values ​​is encoded with a symbol, wherein the symbol indicates that the data unit stores the first value.

18. The non-transitory computer-readable storage medium of claim 17, wherein: The operations also include: performing a read operation on a data unit indicated by the at least one second position information value encoded using the symbol; and returning the first value to the at least one processor included in the at least one node without allocating a location for the first value in the dedicated cache, and Wherein, the first value is dynamically selected.

19. The non-transitory computer-readable storage medium of claim 18, wherein: The operations also include: storing at least one of the compressed data units and the uncompressed data units of different sizes in at least one cache location provided in the dedicated cache included in the at least one node; and Based on at least one of the one or more first position information values, a position corresponding to the respective data unit is identified at a finer granularity than a size of the uncompressed data unit.

20. The non-transitory computer-readable storage medium of claim 19, wherein: The operations also include: At least one of the one or more first position information values ​​is encoded with a size of the compressed data unit.

Images