Hierarchical Memory Caching

Abstract

A computer-implemented method is disclosed that includes locating, from processor storage, a partial tag corresponding to a memory request for a line stored in a memory having a hierarchical memory cache, and locating, in response to a partial tag hit for the memory request, a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag. The method also includes, in response to a full tag hit, fetching the requested line from the partition of the hierarchical memory cache. Various other methods, systems, and computer-readable media are also disclosed.

Description

[Background technology]

[0001] Processors read and write data and / or instructions from memory. Increases in memory size and processing speed have created bottlenecks when accessing memory. A cache is a faster but smaller memory for storing repeatedly accessed data to reduce memory accesses. Processors often use multiple caches in a cache hierarchy. Additionally, efforts to increase memory size include using cheaper and slower memory. Effective use of caches can mitigate some of these negative effects.

[0002] The accompanying drawings illustrate several exemplary embodiments and are a part of this specification and, together with the following description, demonstrate and explain various principles of the present disclosure. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram of an exemplary system for a hierarchical memory cache. [Figure 2] FIG. 1 is a simplified block diagram of an exemplary cache hierarchy. [Figure 3] FIG. 1 is a simplified block diagram of a hierarchical memory cache. [Figure 4] FIG. 1 is a flow diagram of a partial tag miss in a hierarchical memory cache. [Figure 5A] FIG. 1 is a flow diagram of a partial tag hit in a hierarchical memory cache. [Figure 5B] FIG. 1 is a flow diagram of a partial tag hit in a hierarchical memory cache. [Figure 6] FIG. 1 is a flow diagram for updating dirty pages in a hierarchical memory cache. [Figure 7] FIG. 1 is a flow diagram of an exemplary method for implementing a hierarchical memory cache. DETAILED DESCRIPTION OF THE INVENTION

[0004] Throughout the drawings, like reference numerals and descriptions indicate similar, but not necessarily identical, elements. While the exemplary embodiments described herein are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and are described in detail herein. However, the exemplary embodiments described herein are not intended to be limited to the particular forms disclosed. Rather, the present disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims.

[0005] The present disclosure is generally directed to hierarchical memory caching. As discussed in more detail below, embodiments of the present disclosure provide a hierarchical memory cache that utilizes partial tags for an initial lookup, followed by a full tag lookup. The systems and methods described herein can improve the performance of a computer through more efficient utilization of hierarchical memory caches, which reduces the overhead for looking up entries. For example, embodiments of the present disclosure can reduce the number of times memory is accessed for tag lookups.

[0006] As described in more detail below, this disclosure describes various systems and methods for implementing a hierarchical memory cache. A partial tag stored in processor storage or a cache is first used for tag lookup. In response to a partial tag hit, a corresponding full tag can be looked up, and in response to a full tag hit, the requested line can be retrieved from the hierarchical memory cache.

[0007] In one example, a device for implementing a hierarchical memory cache includes a physical memory, a hierarchical memory cache associated with the physical memory, and a controller, wherein the controller is configured to locate, from processor storage, a partial tag corresponding to a memory request for a line stored in the memory, responsive to a partial tag hit for the memory request, locate a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag, and process the requested line from the partition of the hierarchical memory cache in accordance with the memory request based on locating the full tag.

[0008] In some examples, the controller is further configured to fetch the requested line from memory in response to a full tag miss. In some examples, the controller is further configured to fetch the requested line from memory in response to a partial tag miss. In some examples, the controller is further configured to evict a page from the hierarchical memory cache and replace, in processor storage, a partial tag corresponding to the evicted page with a partial tag corresponding to the missed partial tag. In some examples, the controller is further configured to write the evicted page to memory if the evicted page is dirty.

[0009] In some examples, the full tag is stored using one or more error correction code (ECC) bits in the hierarchical memory cache. In some examples, the partial tag corresponds to a page. In some examples, the partial tag includes a subset of the corresponding full tag. In some examples, the hierarchical memory cache includes a set associative cache for the memory.

[0010] In one embodiment, a system for a hierarchical memory cache includes a physical memory, a hierarchical memory cache faster than the physical memory, at least one physical processor with processor storage, and a controller configured to: locate from the processor storage a partial tag corresponding to a memory request for a line stored in the physical memory; responsive to a partial tag hit for the memory request, locate a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag; responsive to a full tag miss, fetch the requested line from the physical memory; and process the requested line according to the memory request.

[0011] In some examples, the controller is further configured to, in response to a partial tag miss, fetch the requested line from physical memory, evict the page from the hierarchical memory cache, and replace in processor storage the partial tag corresponding to the evicted page with the partial tag corresponding to the missed partial tag. In some examples, the controller is further configured to write the evicted page to memory if the evicted page is dirty.

[0012] In some examples, the full tag is stored using one or more error correcting code (ECC) bits in the hierarchical memory cache. In some examples, the partial tag corresponds to a page, and the partial tag includes a subset of the corresponding full tag. In some examples, the hierarchical memory cache includes a set associative cache for the physical memory.

[0013] In one embodiment, a method for a hierarchical memory cache includes locating, from processor storage of at least one physical processor, a partial tag corresponding to a memory request for a line stored in physical memory; fetching the requested line from the physical memory in response to a partial tag miss; evicting a page from the hierarchical memory cache, including a set associative cache for the physical memory; and replacing, in the processor storage, the partial tag corresponding to the evicted page with the partial tag corresponding to the missed partial tag.

[0014] In some examples, the method further includes, if the evicted page is dirty, writing the evicted page to memory. In some examples, the method further includes, in response to a partial tag hit for the memory request, locating a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag, and in response to a full tag miss, fetching the requested line from physical memory, and processing the requested line from the partition of the hierarchical memory cache according to the memory request based on locating the full tag within the partition of the hierarchical memory cache.

[0015] In some examples, the full tag is stored using one or more error correcting code (ECC) bits in a hierarchical memory cache. In some examples, the partial tag corresponds to a page, and the partial tag includes a subset of the corresponding full tag.

[0016] Features from any of the embodiments described herein may be used in combination with each other, in accordance with the general principles described herein. These and other embodiments, features, and advantages will be more fully understood from the following detailed description, taken in conjunction with the accompanying drawings and claims.

[0017] A detailed description of a hierarchical memory cache is provided below with reference to Figures 1-7. A detailed description of an exemplary system for a hierarchical memory cache is provided in relation to Figures 1-3. A detailed description of an exemplary partial tag miss is provided in relation to Figure 4. A detailed description of an exemplary partial tag hit is provided in relation to Figures 5A-5B. A detailed description of an exemplary dirty page update is provided in relation to Figure 6. A detailed description of a corresponding computer-implemented method is also provided in relation to Figure 7.

[0018] FIG. 1 is a block diagram of an exemplary system 100 for a hierarchical memory cache. System 100 corresponds to a computing device, such as a desktop computer, a laptop computer, a server, a tablet device, a mobile device, a smartphone, a wearable device, an augmented reality device, a virtual reality device, a network device, and / or an electronic device. As shown in FIG. 1, system 100 includes one or more memory devices, such as memory 120. Memory 120 generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. Examples of memory 120 include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache, any variation or combination of one or more of these, and / or any other suitable storage memory.

[0019] 1, exemplary system 100 includes one or more physical processors, such as processor 110. Processor 110 generally represents any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In some examples, processor 110 accesses and / or modifies data and / or instructions stored in memory 120. Examples of processor 110 include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA) implementing a soft-core processor, an application-specific integrated circuit (ASIC), a system on a chip (SoC), a digital signal processor (DSP), a neural network engine (NNE), an accelerator, a graphics processing unit (GPU), one or more portions thereof, one or more variations or combinations thereof, and / or any other suitable physical processor.

[0020] As further shown in FIG. 1 , the processor 110 includes a core 112, a cache 114, and a controller 142. The processor 110 is coupled to a hierarchical memory cache 130 of the memory 120. The core 112 corresponds to a processor core, but in other examples, corresponds to a chiplet, such as a chiplet of an accelerator. The cache 114 corresponds to a cache used by the processor 110 (e.g., a client-side cache, such as a low-level cache or L1 cache). In some examples, the cache 114 corresponds to and / or includes another cache, such as a memory-side cache. The hierarchical memory cache 130 corresponds to a last level cache of the memory 120, as described further herein. The controller 142 corresponds to control circuitry that can interface with the hierarchical memory cache 130, such as by fetching tags to coordinate memory requests with the hierarchical memory cache 130. In some examples, the controller 142 also controls aspects of the cache 114.

[0021] Processor 110 reads and operates on instructions and / or data stored in memory 120. Because memory 120 is often slower than processor 110, memory access time creates a bottleneck for processor 110. To alleviate this problem, processor 110 includes cache 114, which is typically a high-speed memory that has an access time that is shorter than that of memory 120, due in part to its physical location within processor 110.

[0022] Cache 114 stores data and / or instructions retrieved from memory 120. Processor 110 (and / or core 112) first makes a memory request to cache 114. If cache 114 stores the requested data (e.g., a cache hit), processor 110 retrieves the data from cache 114, avoiding memory access time to memory 120. If cache 114 does not store the requested data (e.g., a cache miss), processor 110 retrieves the data from memory 120, incurring memory access time. While a larger cache size can reduce cache misses, considerations such as die size and power consumption limit the size of cache 114. Therefore, to further reduce the need to access memory 120 on a cache miss, processor 110 incorporates another cache in the cache hierarchy (e.g., hierarchical memory cache 130 and, in some examples, intervening levels of caches) that is larger but slower than cache 114.

[0023] As will be explained further below, hierarchical memory cache 130 is a last level cache that corresponds to memory 120. Because hierarchical memory cache 130 is smaller than memory 120, hierarchical memory cache 130 is mapped onto a portion of memory 120, as will be explained further below. Because hierarchical memory cache 130 is a last level cache, memory requests will exhibit cache misses on lower level caches before reaching hierarchical memory cache 130.

[0024] 2 illustrates an exemplary cache hierarchy within system 200 corresponding to system 100. System 200 includes one or more processors 210 corresponding to processor 110 and one or more accelerators 211 corresponding to processor 110. As shown in FIG. 2, processor 210 includes core 212A corresponding to core 112, core 212B corresponding to core 112, L1 cache 214A corresponding to cache 114, L1 cache 214B corresponding to cache 114, L2 cache 216A, which may correspond to cache 114, L2 cache 216B, which may correspond to cache 114, and L3 cache 218, which may correspond to cache 114.

[0025] In the cache hierarchy of FIG. 2, level 1 (L1) corresponds to the lowest level of the hierarchy. L1 caches, such as L1 cache 214A and L1 cache 214B, can be implemented using high-speed memory, such as static random-access memory (SRAM). To further prioritize speed, L1 caches can also be integrated with processor 210, for example, within core 212A and core 212B, respectively, which can improve latency and throughput. In some examples, as shown in FIG. 2, processor 210 includes multiple L1 caches.

[0026] An L2 cache, such as L2 cache 216A and L2 cache 216B, is the next level in the cache hierarchy after the L1 cache and may be larger and slower than the L1 cache. Although integrated with processor 210, the L2 cache may, in some examples, be located outside the chip core, but may also be located on the same chip core package. An L3 cache, such as L3 cache 218, may be larger but slower than the L2 cache. In some examples, the L3 cache may function as a bridge to main memory (e.g., memory 220). Thus, the L3 cache may be faster than the main memory. In some examples, multiple processors and / or cores may share an L3 cache, which may be located on the same chip core package or outside the package.

[0027] Memory 220, corresponding to memory 120, stores instructions and / or data for retrieval and use by processor 210. Memory 220 may be implemented as dynamic random-access memory (DRAM). As shown in Figure 2, the cache hierarchy further includes a hierarchical memory cache 230 (e.g., a memory-side cache) corresponding to hierarchical memory cache 130, and a data fabric 240 corresponding to various structures, connections, and control circuits for transmitting data between the memory and cache structures.

[0028] System 200 also includes one or more accelerators with a similar cache hierarchy. Accelerator 211 includes chiplet 213A corresponding to core 112, chiplet 213B corresponding to core 112, chiplet 213C corresponding to core 112, chiplet 213D corresponding to core 112, and L2 cache 217 corresponding to cache 114 shared by the chiplets.

[0029] In some examples, hierarchical memory cache 230 is a last level cache, similar to a level 4 (L4) cache for processor 210 and / or an L3 cache for accelerator 211. Thus, hierarchical memory cache 230 is closest to memory 220 and, in some examples, is partitioned from the same or nearby memory structure, as further described in connection with FIG.

[0030] 3 shows a memory device 300 corresponding to memory 120 and / or memory 220. Memory device 300 represents a memory structure having a size (e.g., 32 GB) partitioned into memory 320 (e.g., main memory usable by the operating system, corresponding to memory 120 and / or memory 220) and hierarchical memory cache 330 corresponding to hierarchical memory cache 130 and / or hierarchical memory cache 230. Hierarchical memory cache 330 is an associative set of memory 320. More specifically, in the example shown in FIG. 3, hierarchical memory cache 330 is a four-way associative set having four ways, namely, way 332A, way 332B, way 332C, and way 332D.

[0031] The size of each of way 332A, way 332B, way 332C, and way 332D can be selected based on the size of memory 320. For example, if memory 320 is 30 GB (out of a total of 32 GB), each way can be 512 MB, such that hierarchical memory cache 330 is 2 GB. In other examples, memory 320 can have a different size and / or include a different percentage of memory device 300, and hierarchical memory cache 330 can have fewer or more ways, with each way having a different and / or dissimilar size. Other mapping schemes can be used instead of associative sets. For example, direct mapping can be used, although it is not as scalable as associative sets. Because direct mapping correlates addresses in the hierarchical memory cache one-to-one to main memory, only a small portion of main memory can be represented by the hierarchical memory cache. Associative sets can be used to hash memory addresses so that more of the main memory can be represented. Therefore, in some instances, using associative sets also requires an updated addressing scheme, as described further herein.

[0032] 2, when processor 210 makes a memory request, the lowest level cache is checked first, and if needed, proceeds to higher level caches on a cache miss until accessing memory 220. For example, core 212A processes a memory request by first checking L1 cache 214A. In the event of an L1 cache miss (e.g., L1 cache 214A does not have the requested data), L2 cache 216A is checked, and in the event of an L2 cache miss (e.g., L2 cache 216A does not have the requested data), L3 cache 218 is checked. In the event of an L3 cache miss (e.g., L3 cache 218 does not have the requested data), hierarchical memory cache 230 is checked.

[0033] 2, hierarchical memory cache 230 is off-chip (e.g., separate from processor 210 and / or accelerator 211) and is accessed across data fabric 240. Checking hierarchical memory cache 230 involves looking up a full tag corresponding to the address of the requested line or other data segment. However, looking up the full tag incurs overhead for accessing hierarchical memory cache 230 across data fabric 240. In some examples, the overhead for such lookups may reduce the performance benefit of using hierarchical memory cache 230.

[0034] To reduce the overhead of full tag lookups in the hierarchical memory cache 230, embodiments described herein store tags on the die. In some examples, local processor storage (e.g., cache 114, L1 caches 214A-214B, L2 caches 216A-216B, L2 cache 217, L3 cache 218, other on-die caches, and / or other processor storage) may store tags for the hierarchical memory cache 230. More specifically, in some examples, partial tags (rather than full tags) may be stored on the die to reduce metadata storage requirements. For example, partial tags correspond to pages or other data segments that are larger than lines, or that, in some examples, require fewer bits to represent, are requested in a memory request. If a corresponding page is not found in the partial tag, it is guaranteed that the requested line is not in the hierarchical memory cache 230. Thus, a partial tag miss can avoid an expensive lookup in the hierarchical memory cache 230 that would result in a miss.

[0035] Figure 4 shows an example flow 400 for a partial tag miss. Figure 4 shows a processor 410 (corresponding to processor 110, processor 210, and / or accelerator 211), processor storage 414 (corresponding to cache 114, L1 caches 214A-214B, L2 caches 216A-216B, L2 cache 217, L3 cache 218, and / or other on-die storage), a hierarchical memory cache 430 (corresponding to hierarchical memory cache 130, hierarchical memory cache 230, and / or hierarchical memory cache 330), and a memory 420 (corresponding to memory 120, memory 220, and / or memory 320).

[0036] When processor 410 and / or its cores / chiplets process a memory request for hierarchical memory cache 430 (e.g., after indicating a cache miss for a lower-level cache), processor 410 and / or its controller first locates the partial tag corresponding to the memory request, at 452. In some examples, partial tags use an addressing scheme that associates a page with the line contained therein. Thus, processor 410 requests the partial tag of the page associated with the line requested in the original memory request.

[0037] If the partial tag for the requested page is unavailable in processor storage 414, a partial tag miss is indicated at 454. A partial tag miss indicates that the requested page, and therefore the requested line, is unavailable in hierarchical memory cache 430, which corresponds to a cache hit. Because a full tag lookup is not required, the memory request proceeds to memory 420 at 456. Memory 420 returns the requested line at 460, which may also be written to hierarchical memory cache 430 at 458 to update hierarchical memory cache 430 in response to the cache miss (which may include a page eviction). Alternatively, in the case of a write request, a write is performed to the requested line in memory 420 at 456, which may be communicated to hierarchical memory cache 430 (e.g., by updating hierarchical memory cache 430) at 458 and acknowledged at 460. Additionally, the corresponding partial and full tags may be written to appropriate storage.

[0038] 4, a partial tag miss avoids an access to the hierarchical memory cache 430. However, in some instances, a partial tag hit does not guarantee a cache hit and requires a full tag lookup.

[0039] 5A and 5B respectively show an example flow 500 and an example flow 501 in the case of a partial tag hit. Figures 5A-5B show a processor 510 (corresponding to processor 110, processor 210, and / or accelerator 211), processor storage 514 (corresponding to cache 114, L1 caches 214A-214B, L2 caches 216A-216B, L2 cache 217, L3 cache 218, and / or other on-die storage), a hierarchical memory cache 530 (corresponding to hierarchical memory cache 130, hierarchical memory cache 230, and / or hierarchical memory cache 330), and a memory 520 (corresponding to memory 120, memory 220, and / or memory 320).

[0040] 5A , when processor 510 and / or its cores / chiplets process a memory request to hierarchical memory cache 530 (e.g., after indicating a cache miss to a lower-level cache), processor 510 and / or its controller first locates a partial tag corresponding to the memory request at 552. As described herein, in some examples, processor 510 requests the partial tag of a page associated with a line requested in the original memory request.

[0041] If a partial tag for the requested page is available in processor storage 514, a partial tag hit is indicated at 554. However, the partial tag hit indicates that the requested page is potentially available in hierarchical memory cache 530. In some examples, the partial tag further indicates which way (e.g., any of ways 332A-332D in FIG. 3 ), such that the partial tag hit indicates that the requested page is potentially located in the indicated way. Thus, in response to the partial tag hit, a full tag is located from the indicated way at 554. Because the partial tag indicates a potential way that may have the requested page / line, a full tag lookup for all ways can be avoided. In some embodiments, locating the full tag includes fetching the line requested in the memory request from the indicated way. In such embodiments, the full tag is stored along with the line of data (e.g., as ECC bits).

[0042] If the requested line is available in the hierarchical memory cache 530 (e.g., in the indicated way) based on the full tag lookup, a full tag hit is indicated at 556. The requested line is fetched from the hierarchical memory cache 530 and returned to the processor 510 for a read request. Alternatively, for a write request, the requested line is updated in the hierarchical memory cache 530 and acknowledged at 556. In some embodiments, the line and tag are fetched from the hierarchical memory cache 530 in a single fetch operation, such as for a full tag lookup. For example, the line indicated by the partial tag (e.g., the indicated way) and the memory request (e.g., the address of the line) is fetched first, and the line's tag (which may be stored in ECC bits) is checked for a full tag hit. If the fetched line is the requested line (e.g., a full tag hit), the fetched line is returned to complete the memory request. If the fetched line is not the requested line (eg, a complete tag miss), the requested line is fetched from memory 520 as described herein.

[0043] As described herein, using a partial tag lookup that also indicates potential ways for a full tag lookup reduces the number of full tag lookups (e.g., one for each way in the hierarchical memory cache 530). Additionally, to reduce the metadata storage requirements for the tag and to avoid requiring additional data structures to store the full tag, in some examples, the full tag is stored along with other reused bits. For example, the full tag may be stored using one or more error correcting code (ECC) bits.

[0044] 5B , in contrast to the full tag hit at 554 in FIG. 5A , the hierarchical memory cache 530 indicates a full tag miss at 554. For example, the full tag is not found in the indicated way, corresponding to a cache miss. The memory request proceeds to memory 520 at 558. Memory 520 returns the requested line at 560, which may be written to hierarchical memory cache 530 at 559 to update hierarchical memory cache 530 in response to the cache miss. Alternatively, in the case of a write request, the requested line is updated in memory 520 at 558, which may be communicated to hierarchical memory cache 530 (e.g., by updating hierarchical memory cache 530) at 559 and acknowledged at 560. Additionally, the corresponding partial and full tags may be written to appropriate storage.

[0045] Figure 6 shows an example flow 600 for dirty pages. Figure 6 shows a processor 610 (corresponding to processor 110, processor 210, and / or accelerator 211), processor storage 614 (corresponding to cache 114, L1 caches 214A-214B, L2 caches 216A-216B, L2 cache 217, L3 cache 218, and / or other on-die storage), a hierarchical memory cache 630 (corresponding to hierarchical memory cache 130, hierarchical memory cache 230, and / or hierarchical memory cache 330), and a memory 620 (corresponding to memory 120, memory 220, and / or memory 320).

[0046] 6, a dirty page in hierarchical memory cache 630 is written to memory 620 at 662. The dirty page may be selected, for example, in response to a partial tag hit for the page (e.g., if the memory request is a write request). A page may be dirty (e.g., modified) in response to a previous or current write request.

[0047] In other examples, the dirty page is written in response to a partial tag miss of another page (e.g., as part of a page eviction process in which a page is removed from the hierarchical memory cache 630 to provide space for the missed page). In some examples, memory 620 may acknowledge the write at 664 so that the hierarchical memory cache 630 can safely evict the dirty page.

[0048] Figure 7 is a flow diagram of an exemplary computer-implemented method 700 for implementing a hierarchical memory cache. The steps illustrated in Figure 7 may be performed by any suitable computer-executable code and / or computing system, including the systems illustrated in Figures 1, 2, and / or 3. In one example, each of the steps illustrated in Figure 7 represents an algorithm whose structure includes and / or is represented by multiple sub-steps, examples of which are provided in more detail below.

[0049] 7, in step 702, one or more of the systems described herein locates from processor storage a partial tag corresponding to a memory request for a line stored in a memory having a hierarchical memory cache. For example, processor 110 locates from processor storage (e.g., cache 114 and / or another on-die storage) a partial tag corresponding to a memory request for a line stored in hierarchical memory cache 130.

[0050] The systems described herein can perform step 702 in a variety of ways. In one example, hierarchical memory cache 130 is a set-associative cache for memory 120. In some examples, partial tags correspond to pages such that the partial tags include a subset of the corresponding full tag.

[0051] In some examples, processor 110 fails to locate the partial tag, indicating a partial tag miss. In response to the partial tag miss, processor 110 fetches the requested line from memory 120. Further, in some examples, in response to the partial tag miss, a controller (e.g., controller 142) of hierarchical memory cache 130 evicts the page from the hierarchical memory cache and replaces, in processor storage, the partial tag corresponding to the evicted page with the partial tag corresponding to the missed partial tag. In some examples, the controller writes the evicted page to memory if the evicted page is dirty.

[0052] In step 704, one or more of the systems described herein, in response to a partial tag hit for the memory request, locates a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag. For example, a controller (e.g., controller 142) of the hierarchical memory cache 130 locates the full tag for the line indicated by the memory request. In some embodiments, the full tag is stored using one or more error correction code (ECC) bits in the hierarchical memory cache 130. For example, the full tag is stored along with the data in the ECC bits of the corresponding line itself. In such embodiments, locating the full tag includes fetching the line requested by the memory request from a partition (e.g., the indicated way) of the hierarchical memory cache.

[0053] In some instances, the hierarchical memory cache 130 fails to locate the perfect tag, indicating a perfect tag miss. In response to a perfect tag miss, the controller fetches the requested line from the memory 120.

[0054] In step 706, one or more of the systems described herein process the requested line from a partition of the hierarchical memory cache according to the memory request based on locating the full tag. For example, the hierarchical memory cache 130 fetches and returns the requested line to the processor 110 in response to a read request. In some examples, the requested line was previously fetched in response to a partial tag hit on a full tag lookup. The previously fetched line can be returned to the partition of the hierarchical memory cache 130 in the case of a read request, or updated and written back to the partition of the hierarchical memory cache 130 in the case of a write request.

[0055] As described herein, the present disclosure is directed to improving hierarchical memory caches. Implementing a set-associative cache that divides a hierarchical memory cache into sets of ways provides a scalable hierarchical memory cache. In addition, the systems and methods provided herein can reduce the overhead for accessing a hierarchical memory cache. In particular, the systems and methods provided herein reduce the number of lookups to find a line in a hierarchical memory cache by using partial page tags and full tags.

[0056] Partial page tags are stored on-die (e.g., in processor storage) for fast lookup. Partial page tags have reduced metadata storage requirements than full tags, allowing them to be stored on-die. Partial page tags indicate which page is located in which way, so that a partial page tag miss indicates that the requested line is not in the hierarchical memory cache. Thus, a partial page miss avoids the tag lookup process, which is more expensive than in a hierarchical memory cache.

[0057] A partial page hit indicates that the requested line may be in the hierarchical memory cache. To avoid having to perform an expensive full tag lookup for each way, the partial page tag points to a specific way so that only a single full tag lookup is required. In the case of a full tag hit, the hierarchical memory cache returns the requested line or performs the requested write on the line. In the case of a full tag miss, memory is accessed and the hierarchical memory cache (and the partial page tag and full tag) are updated accordingly. Thus, the systems and methods herein can reduce overhead by reducing accesses to the hierarchical memory cache and memory.

[0058] As noted above, the computing devices and systems described and / or illustrated herein broadly represent any type or form of computing device or system capable of executing computer-readable instructions, such as those contained within the modules described herein. In their most basic configurations, these computing devices each include at least one memory device and at least one physical processor.

[0059] In some examples, the term "memory device" generally refers to any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, a memory device stores, loads, and / or maintains one or more of the modules and / or circuits described herein. Examples of memory devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drives (HDDs), solid-state drives (SSDs), optical disk drives, caches, variations or combinations of one or more of these, or any other suitable storage memory.

[0060] In some examples, the term "physical processor" generally refers to any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In one example, a physical processor accesses and / or modifies one or more modules stored in the memory devices described above. Examples of physical processors include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a field programmable gate array (FPGA) implementing a soft-core processor, an application-specific integrated circuit (ASIC), a system-on-chip (SoC), a digital signal processor (DSP), a neural network engine (NNE), an accelerator, a graphics processing unit (GPU), one or more portions thereof, one or more variations or combinations thereof, or any other suitable physical processor.

[0061] In some embodiments, the term "computer-readable medium" generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission-type media such as carrier waves, and non-transitory-type media such as magnetic storage media (e.g., hard disk drives, tape drives, and floppy disks), optical storage media (e.g., compact disks (CDs), digital video disks (DVDs), and BLU-RAY disks), electronic storage media (e.g., solid-state drives and flash media), and other distribution systems.

[0062] The process parameters and order of steps described and / or illustrated herein are given by way of example only and can be changed as desired. For example, although the steps illustrated and / or described herein are illustrated or described in a particular order, these steps do not necessarily have to be performed in the order illustrated or described. The various exemplary methods described and / or illustrated herein may omit one or more of the steps described or illustrated herein or may include additional steps in addition to those disclosed.

[0063] The above description is provided to enable those skilled in the art to best utilize various aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or to be limited to any precise form disclosed. Many modifications and variations are possible without departing from the spirit and scope of the present disclosure. The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. In determining the scope of the present disclosure, reference should be made to the appended claims and their equivalents.

[0064] Unless otherwise specified, the terms "connected to" and "coupled to" (and their derivatives) as used in this specification and claims should be interpreted as allowing both direct and indirect connections (i.e., via other elements or components). Additionally, the terms "a" or "an" as used in this specification and claims should be interpreted as meaning "at least one of." Finally, for ease of use, the terms "including" and "having" (and their derivatives) as used in this specification and claims are interchangeable with the term "comprising," and have the same meaning.

Claims

1. A device, Physical memory and a hierarchical memory cache associated with the physical memory; a controller; The controller Locating from processor storage a partial tag corresponding to a memory request for a line stored in said physical memory; responsive to a partial tag hit for the memory request, locating a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag; processing the requested line from the partition of the hierarchical memory cache according to the memory request based on locating the full tag; configured to: device.

2. the controller is configured to fetch the requested line from the memory in response to a full tag miss. The device of claim 1.

3. the controller is configured to fetch the requested line from the memory in response to a partial tag miss. The device of claim 1.

4. The controller Evicting pages from the hierarchical memory cache; replacing, in the processor storage, the partial tag corresponding to the evicted page with the partial tag corresponding to the missed partial tag; configured to: The device of claim 3.

5. the controller is configured to write the evicted page to the memory if the evicted page is dirty; The device of claim 4.

6. the full tag is stored using one or more error correcting code (ECC) bits in the hierarchical memory cache; The device of claim 1.

7. The partial tag corresponds to a page. The device of claim 1.

8. the partial tag comprises a subset of the corresponding full tag; The device of claim 1.

9. the hierarchical memory cache includes a set associative cache for the physical memory; The device of claim 1.

10. 1. A system comprising: Physical memory and a hierarchical memory cache that is faster than the physical memory; at least one physical processor with processor storage; a controller; The controller Locating from the processor storage a partial tag corresponding to a memory request for a line stored in the physical memory; responsive to a partial tag hit for the memory request, locating a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag; fetching the requested line from the physical memory in response to a full tag miss; processing the requested line in accordance with the memory request; configured to: system.

11. The controller fetching the requested line from the physical memory in response to a partial tag miss; Evicting pages from the hierarchical memory cache; replacing, in the processor storage, the partial tag corresponding to the evicted page with the partial tag corresponding to the missed partial tag; configured to: The system of claim 10.

12. the controller is configured to write the evicted page to the memory if the evicted page is dirty; The system of claim 11.

13. the full tag is stored using one or more error correcting code (ECC) bits in the hierarchical memory cache; The system of claim 10.

14. the partial tag corresponds to a page, the partial tag comprising a subset of the corresponding full tag; The system of claim 10.

15. the hierarchical memory cache includes a set associative cache for the physical memory; The system of claim 10.

16. 1. A method comprising: Locating, from processor storage of at least one physical processor, a partial tag corresponding to a memory request for a line stored in physical memory; fetching the requested line from the physical memory in response to a partial tag miss; evict pages from a hierarchical memory cache, including a set associative cache for said physical memory; and replacing, in the processor storage, a partial tag corresponding to the evicted page with a partial tag corresponding to the missed partial tag. method.

17. writing the evicted page to the memory if the evicted page is dirty.

17. The method of claim 16.

18. responsive to a partial tag hit for the memory request, locating a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag; fetching the requested line from the physical memory in response to a full tag miss; and processing the requested line from a partition of the hierarchical memory cache according to the memory request based on locating the full tag within the partition of the hierarchical memory cache.

17. The method of claim 16.

19. the full tag is stored using one or more error correcting code (ECC) bits in the hierarchical memory cache; 20. The method of claim 18.

20. the partial tag corresponds to a page, the partial tag comprising a subset of the corresponding full tag; 20. The method of claim 18.

Images