Hierarchical memory cache

Abstract

A disclosed computer-implemented method includes positioning, from a processor storage device, a partial tag corresponding to a memory request for a row stored in a memory having a hierarchical memory cache; and in response to a partial tag hit for the memory request, locating a complete tag for the row from a partition of the hierarchical memory cache indicated by the partial tag. The method also includes retrieving the requested row from the partition of the tiered memory cache in response to the complete tag hit. Various other methods, systems, and computer readable media are also disclosed.

Description

Background Art

[0001] Processors read and write data and / or instructions from memory. Increasing memory size and processing speed have created bottlenecks in memory access. Caches are faster, but smaller, memories used to store repeatedly accessed data, reducing memory accesses. Processors typically use multiple caches in a cache hierarchy. Furthermore, efforts to increase memory size have included the use of cheaper and slower memory. Effective use of caches can mitigate some of these negative effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The accompanying drawings illustrate several exemplary implementations and are a part of the specification. Together with the following description, these drawings illustrate and explain the various principles of the present disclosure.

[0003] Figure 1 is a block diagram of an exemplary system for hierarchical memory caching.

[0004] Figure 2 is a simplified block diagram of an exemplary cache hierarchy.

[0005] Figure 3 is a simplified block diagram of a hierarchical memory cache.

[0006] Figure 4 is a flow chart of a partial tag miss in a hierarchical memory cache.

[0007] Figures 5A to 5B is a flow chart of a partial tag hit in a hierarchical memory cache.

[0008] Figure 6 is a flow chart for updating dirty pages of a hierarchical memory cache.

[0009] Figure 7 is a flow chart of an exemplary method for implementing a hierarchical memory cache.

[0010] Throughout the drawings, the same reference numerals and descriptions indicate similar, but not necessarily identical, elements. Although the exemplary embodiments described herein are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and will be described in detail herein. However, the exemplary embodiments described herein are not intended to be limited to the specific forms disclosed. On the contrary, this disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims. DETAILED DESCRIPTION

[0011] The present disclosure generally relates to hierarchical memory caches. As explained in greater detail below, embodiments of the present disclosure provide a hierarchical memory cache that utilizes partial tags for initial lookups followed by full tag lookups. The systems and methods described herein can improve computer operation by more efficiently utilizing a hierarchical memory cache that reduces the overhead of looking up entries. For example, embodiments of the present disclosure can reduce the number of memory accesses required for tag lookups.

[0012] As will be described in greater detail below, the present disclosure describes various systems and methods for implementing a hierarchical memory cache. Partial tags stored in a processor storage device or cache are initially used for a tag lookup. In response to a partial tag hit, the corresponding full tag can be looked up, and in response to a full tag hit, the requested line can be retrieved from the hierarchical memory cache.

[0013] In one example, an apparatus for implementing a hierarchical memory cache includes: a physical memory; a hierarchical memory cache associated with the physical memory; and a controller configured to: locate, from processor storage, a partial tag corresponding to a memory request for a line stored in the memory; in response to a partial tag hit for the memory request, locate a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag; and based on locating the full tag, process the requested line from the partition of the hierarchical memory cache according to the memory request.

[0014] In some examples, the controller is further configured to, in response to a full tag miss, retrieve the requested line from the memory. In some examples, the controller is further configured to, in response to a partial tag miss, retrieve the requested line from the memory. In some examples, the controller is further configured to: evict a page from the hierarchical memory cache; and replace, in the processor storage, a partial tag corresponding to the evicted page with a partial tag corresponding to the missed partial tag. In some examples, the controller is further configured to write the evicted page to the memory when the evicted page is dirty.

[0015] In some examples, the full tag is stored in the hierarchical memory cache using one or more error correction code (ECC) bits. In some examples, the partial tag corresponds to a page. In some examples, the partial tag includes a subset of the corresponding full tag. In some examples, the hierarchical memory cache includes a set associative cache for memory.

[0016] In one embodiment, a system for a hierarchical memory cache includes: a physical memory; a hierarchical memory cache that is faster than the physical memory; at least one physical processor including processor storage; and a controller. The controller is configured to: locate a partial tag corresponding to a memory request for a line stored in the physical memory from the processor storage; in response to a partial tag hit for the memory request, locate a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag; in response to a full tag miss, retrieve the requested line from the physical memory; and process the requested line according to the memory request.

[0017] In some examples, the controller is further configured to: in response to a partial tag miss, fetch the requested line from the physical memory; evict a page from the hierarchical memory cache; and replace, in the processor memory device, a partial tag corresponding to the evicted page with a partial tag corresponding to the missed partial tag. In some examples, the controller is further configured to write the evicted page to the memory when the evicted page is dirty.

[0018] In some examples, the full tag is stored in the hierarchical memory cache using one or more error correction code (ECC) bits. In some examples, the partial tag corresponds to a page, and the partial tag includes a subset of the corresponding full tag. In some examples, the hierarchical memory cache includes a set associative cache for physical memory.

[0019] In one embodiment, a method for a hierarchical memory cache includes locating, from processor storage of at least one physical processor, a partial tag corresponding to a memory request for a line stored in physical memory; in response to a partial tag miss, retrieving the requested line from the physical memory; evicting a page from a hierarchical memory cache including a set associative cache for the physical memory; and replacing, in the processor storage, a partial tag corresponding to the evicted page with a partial tag corresponding to the missed partial tag.

[0020] In some examples, the method further includes writing the evicted page to the memory when the evicted page is dirty. In some examples, the method further includes: in response to a partial tag hit for the memory request, locating a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag; in response to a full tag miss, fetching the requested line from the physical memory; and processing the requested line from the partition of the hierarchical memory cache according to the memory request based on locating the full tag in the partition of the hierarchical memory cache.

[0021] In some examples, the full tag is stored in the hierarchical memory cache using one or more error correction code (ECC) bits. In some examples, the partial tag corresponds to a page, and the partial tag includes a subset of the corresponding full tag.

[0022] According to the general principles described herein, the features of any specific implementation described herein can be used in combination with each other. These and other specific implementations, features and advantages will be more fully understood after reading the following detailed description in conjunction with the accompanying drawings and claims.

[0023] refer to Figures 1 to 7 , a detailed description of the hierarchical memory cache will be provided below. Figures 1 to 3 A detailed description of an example system for a hierarchical memory cache is provided. Figure 4 Provides a detailed description of the example partial label misses. Figures 5A to 5B Provides a detailed description of the sample part tag hits. Figure 6 Provides a detailed description of an example of updating a dirty page. Figure 7 A detailed description of the corresponding computer-implemented method is provided.

[0024] Figure 1 is a block diagram of an example system 100 for hierarchical memory caching. System 100 corresponds to a computing device such as a desktop computer, a laptop computer, a server, a tablet device, a mobile device, a smartphone, a wearable device, an augmented reality device, a virtual reality device, a network device, and / or an electronic device. Figure 1 As shown, system 100 includes one or more memory devices, such as memory 120. Memory 120 generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. Examples of memory 120 include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache, variations or combinations of one or more of the foregoing, and / or any other suitable memory.

[0025] like Figure 1As shown, exemplary system 100 includes one or more physical processors, such as processor 110. Processor 110 generally represents any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In some examples, processor 110 accesses and / or modifies data and / or instructions stored in memory 120. Examples of processor 110 include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA) implementing a soft-core processor, an application-specific integrated circuit (ASIC), a system on a chip (SoC), a digital signal processor (DSP), a neural network engine (NNE), an accelerator, a graphics processing unit (GPU), one or more portions of the foregoing, one or more variations or combinations of the foregoing, and / or any other suitable physical processor.

[0026] like Figure 1 As further illustrated, processor 110 includes core 112, cache 114, and controller 142. Processor 110 is coupled to a hierarchical memory cache 130 of memory 120. Core 112 corresponds to a processor core, but in other examples corresponds to a chiplet (such as an accelerator chiplet). Cache 114 corresponds to a cache used by processor 110 (e.g., a client-side cache, such as a low-level cache or L1 cache). In some examples, cache 114 corresponds to and / or includes other caches (such as memory-side caches). As will be discussed further herein, hierarchical memory cache 130 corresponds to a last-level cache of memory 120. Controller 142 corresponds to control circuitry that can coordinate with hierarchical memory cache 130, such as by fetching tags to coordinate memory requests to hierarchical memory cache 130. In some examples, controller 142 also controls various aspects of cache 114.

[0027] Processor 110 reads and operates on instructions and / or data stored in memory 120. Because memory 120 is typically slower than processor 110, memory access time creates a bottleneck for processor 110. To alleviate this problem, processor 110 includes cache 114, which is typically a fast memory with an access time that is less than that of memory 120, in part because the cache is physically located within processor 110.

[0028] Cache 114 holds data and / or instructions read from memory 120. Processor 110 (and / or core 112) first makes a memory request to cache 114. If cache 114 holds the requested data (e.g., a cache hit), processor 110 reads the data from cache 114 and avoids the memory access time of memory 120. If cache 114 does not hold the requested data (e.g., a cache miss), processor 110 retrieves the data from memory 120, thereby incurring a memory access time. Although a larger cache size can reduce cache misses, considerations such as die size and power consumption limit the size of cache 114. Therefore, to further reduce the need to access memory 120 in the event of a cache miss, processor 110 incorporates another cache in the cache hierarchy that is larger but slower than cache 114 (e.g., hierarchical memory cache 130 and, in some examples, a mid-level cache).

[0029] As will be described further below, the hierarchical memory cache 130 is a last-level cache corresponding to the memory 120. Because the hierarchical memory cache 130 is smaller than the memory 120, the hierarchical memory cache 130 is mapped to a portion of the memory 120, as will be described further below. Because the hierarchical memory cache 130 is a last-level cache, memory requests experience cache misses at lower-level caches before reaching the hierarchical memory cache 130.

[0030] Figure 2 1 illustrates an example cache hierarchy in a system 200 corresponding to the system 100. The system 200 includes one or more processors 210 corresponding to the processor 110 and one or more accelerators 211 corresponding to the processor 110. Figure 2 As illustrated, processor 210 includes core 212A corresponding to core 112, core 212B corresponding to core 112, L1 cache 214A corresponding to cache 114, L1 cache 214B corresponding to cache 114, L2 cache 216A corresponding to cache 114, L2 cache 216B corresponding to cache 114, and L3 cache 218 corresponding to cache 114.

[0031] exist Figure 2In the cache hierarchy of FIG, level 1 (L1) corresponds to the lowest level of the hierarchy. L1 caches, such as L1 cache 214A and L1 cache 214B, may be implemented using fast memory, such as static random access memory (SRAM). To further prioritize speed, L1 caches may also be integrated with processor 210, such as within core 212A and core 212B, respectively, which may improve latency and throughput. In some examples, such as Figure 2 As shown, processor 210 includes multiple L1 caches.

[0032] The L2 cache (such as L2 cache 216A and L2 cache 216B) is the next level in the cache hierarchy after the L1 cache and can be larger and slower than the L1 cache. Although integrated with processor 210, in some examples, the L2 cache can be located outside the chip core, but can also be located on the same chip core package. The L3 cache (such as L3 cache 218) can be larger than the L2 cache, but can also be slower. In some examples, the L3 cache can act as a bridge to main memory (e.g., memory 220). Therefore, the L3 cache can be faster than the main memory. In some examples, multiple processors and / or cores can share an L3 cache, which can be located on the same chip core package or outside the package.

[0033] The memory 220 corresponding to the memory 120 stores instructions and / or data for the processor 210 to read and use. The memory 220 may be implemented by a dynamic random access memory (DRAM). Figure 2 As shown, the cache hierarchy also includes a hierarchical memory cache 230 (e.g., a memory-side cache) corresponding to the hierarchical memory cache 130, and a data fabric 240 corresponding to various structures, connections, and control circuitry for transferring data between the memory and cache structures.

[0034] System 200 also includes one or more accelerators with a similar cache hierarchy. Accelerator 211 includes chiplet 213A corresponding to core 112, chiplet 213B corresponding to core 112, chiplet 213C corresponding to core 112, chiplet 213D corresponding to core 112, and L2 cache 217 corresponding to cache 114 shared by the chiplets.

[0035] In some examples, the hierarchical memory cache 230 is a last level cache, similar to the level 4 (L4) cache for the processor 210 and / or the L3 cache for the accelerator 211. Thus, the hierarchical memory cache 230 is closest to the memory 220 and, in some examples, is partitioned from the same or nearby memory structure, as will be seen in conjunction with Figure 3 Further explanation.

[0036] Figure 3 A memory device 300 corresponding to the memory 120 and / or the memory 220 is illustrated. The memory device 300 represents a memory structure having a size (e.g., 32 GB) divided into the memory 320 (e.g., main memory corresponding to the memory 120 and / or the memory 220, such as that usable by the operating system) and the hierarchical memory cache 330 corresponding to the hierarchical memory cache 130 and / or the hierarchical memory cache 230. The hierarchical memory cache 330 is an associative group of the memory 320. More specifically, in Figure 3 In the illustrated example, the hierarchical memory cache 330 is a 4-way associative set having four ways, namely, way 332A, way 332B, way 332C, and way 332D.

[0037] The size of each of ways 332A, 332B, 332C, and 332D can be selected based on the size of memory 320. For example, for memory 320 of 30 GB (out of a total of 32 GB), each way can be 512 MB, making hierarchical memory cache 330 2 GB. In other examples, memory 320 can be of a different size and / or comprise a different percentage of memory device 300, hierarchical memory cache 330 can have fewer or more ways, and each way can be of a different and / or dissimilar size. Other mapping schemes can be used instead of associative groups. For example, direct mapping can be used, although direct mapping is not as scalable as associative groups. Because direct mapping associates addresses in the hierarchical memory cache with main memory on a 1-to-1 basis, only a small portion of main memory can be represented by the hierarchical memory cache. Using associative groups, memory addresses can be hashed, allowing more main memory to be represented. Therefore, in some examples, using associative groups also requires an updated addressing scheme, as further described herein.

[0038] return Figure 2When processor 210 makes a memory request, it first checks the lowest level cache, and in the event of a cache miss, it proceeds to higher level caches until it accesses memory 220 when needed. For example, core 212A processes a memory request by first checking L1 cache 214A. In the event of an L1 cache miss (e.g., L1 cache 214A does not have the requested data), it checks L2 cache 216A. In the event of an L2 cache miss (e.g., L2 cache 216A does not have the requested data), it checks L3 cache 218. In the event of an L3 cache miss (e.g., L3 cache 218 does not have the requested data), it checks hierarchical memory cache 230.

[0039] like Figure 2 As illustrated, the hierarchical memory cache 230 is off-chip (e.g., separate from the processor 210 and / or accelerator 211) and is accessed across the data fabric 240. Checking the hierarchical memory cache 230 includes looking up the full tag corresponding to the address of the requested line or other data segment. However, looking up the full tag incurs overhead for accessing the hierarchical memory cache 230 across the data fabric 240. In some examples, the overhead for such a lookup can reduce the performance benefits of using the hierarchical memory cache 230.

[0040] To reduce the overhead of full tag lookups in the hierarchical memory cache 230, implementations described herein store tags on-die. In some examples, local processor storage (e.g., cache 114, L1 cache 214A-B, L2 cache 216A-B, L2 cache 217, L3 cache 218, other on-die caches, and / or other processor storage) may store tags for the hierarchical memory cache 230. More specifically, in some examples, partial tags (rather than full tags) may be stored on-die to reduce metadata storage requirements. For example, a partial tag corresponds to a page or other data segment that is larger than the line or content requested in a memory request, which in some examples requires fewer bits to represent. If a corresponding page is not found in the partial tag, the requested line is guaranteed not to be in the hierarchical memory cache 230. Thus, a partial tag miss can avoid an expensive lookup in the hierarchical memory cache 230 that would otherwise result in a miss.

[0041] Figure 4 An example flow 400 for a partial tag miss is illustrated. Figure 4Illustrated are a processor 410 (corresponding to processor 110, processor 210, and / or accelerator 211), processor storage 414 (corresponding to cache 114, L1 cache 214A-B, L2 cache 216A-B, L2 cache 217, L3 cache 218, and / or other on-die storage), a hierarchical memory cache 430 (corresponding to hierarchical memory cache 130, hierarchical memory cache 230, and / or hierarchical memory cache 330), and a memory 420 (corresponding to memory 120, memory 220, and / or memory 320).

[0042] When the processor 410 and / or its core / chiplet processes a memory request to the hierarchical memory cache 430 (e.g., after a cache miss is received to a lower level cache), the processor 410 and / or its controller first locates a partial tag corresponding to the memory request at 452. In some examples, the partial tag uses an addressing scheme that relates a page to the line contained therein. Thus, the processor 410 requests a partial tag for the page associated with the line requested in the original memory request.

[0043] When a partial tag for a requested page is not available in processor memory 414, a partial tag miss occurs at 454. A partial tag miss indicates that the requested page, and therefore the requested line, is not available in hierarchical memory cache 430, which corresponds to a cache hit. No full tag lookup is required, so the memory request proceeds to memory 420 at 456. Memory 420 returns the requested line at 460, which may also be written to hierarchical memory cache 430 at 458 to update hierarchical memory cache 430 in response to the cache miss (which may include a page eviction). Alternatively, for a write request, a write is performed on the requested line in memory 420 at 456, which may be propagated to hierarchical memory cache 430 at 458 (e.g., by updating hierarchical memory cache 430) and acknowledged at 460. In addition, the corresponding partial tags and full tags may be written to the appropriate storage devices.

[0044] like Figure 4 As illustrated, a partial tag miss avoids accessing the hierarchical memory cache 430. However, in some examples, a partial tag hit does not guarantee a cache hit, requiring a full tag lookup.

[0045] Figure 5A and Figure 5B An example process 500 and an example process 501 of partial tag hits are respectively illustrated. Figures 5A to 5BIllustrated are a processor 510 (corresponding to processor 110, processor 210, and / or accelerator 211), processor storage 514 (corresponding to cache 114, L1 cache 214A-B, L2 cache 216A-B, L2 cache 217, L3 cache 218, and / or other on-die storage), a hierarchical memory cache 530 (corresponding to hierarchical memory cache 130, hierarchical memory cache 230, and / or hierarchical memory cache 330), and a memory 520 (corresponding to memory 120, memory 220, and / or memory 320).

[0046] exist Figure 5A , when the processor 510 and / or its core / chiplet processes a memory request to the hierarchical memory cache 530 (e.g., after representing a cache miss to a lower level cache), the processor 510 and / or its controller first locates a partial tag corresponding to the memory request at 552. As described herein, in some examples, the processor 510 requests a partial tag for a page associated with a line requested in the original memory request.

[0047] A partial tag hit is indicated at 554 when a partial tag for the requested page is available in processor storage 514. However, a partial tag hit indicates that the requested page is potentially available in hierarchical memory cache 530. In some examples, the partial tag further indicates which way (e.g., Figure 3 332A-D in the memory request), such that a partial tag hit indicates that the requested page is potentially in the indicated way. Therefore, in response to the partial tag hit, a full tag is located from the indicated way at 554. Because the partial tag indicates a potential way with the requested page / row, a full tag lookup on each way can be avoided. In some implementations, locating the full tag includes obtaining the requested row in the memory request from the indicated way. In such implementations, the full tag is stored with the data row (e.g., as ECC bits).

[0048] When the requested line based on the full tag lookup is available in the hierarchical memory cache 530 (e.g., in the manner indicated), a full tag hit is indicated at 556. The requested line is retrieved from the hierarchical memory cache 530 and returned to the processor 510 for the read request. Alternatively, for a write request, the requested line is updated in the hierarchical memory cache 530 and acknowledged at 556. In some implementations, the line and tag are retrieved from the hierarchical memory cache 530 in a single fetch operation, such as for a full tag lookup. For example, the line indicated by the partial tag (e.g., the indicated way) and the memory request (e.g., the address of the line) is first fetched, and the line's tag (which may be stored in the ECC bits) is checked for a full tag hit. If the fetched line is the requested line (e.g., a full tag hit), the fetched line is returned to fulfill the memory request. If the fetched line is not the requested line (e.g., a full tag miss), the requested line is fetched from the memory 520 as described herein.

[0049] As described herein, using a partial tag lookup (which also indicates potential ways for a full tag lookup) reduces the number of full tag lookups (e.g., once for each way in the hierarchical memory cache 530). Furthermore, to reduce metadata storage requirements for tags and avoid the need for additional data structures for storing full tags, in some examples, full tags are stored along with other repurposed bits. For example, one or more error correction code (ECC) bits can be used to store the full tag.

[0050] exist Figure 5B In, with Figure 5A In contrast to a full tag hit at 554 in the hierarchical memory cache 530, a full tag miss at 554 is indicated. For example, a full tag is not found in the indicated way, which corresponds to a cache miss. At 558, the memory request proceeds to the memory 520. The memory 520 returns the requested line at 560, which may also be written to the hierarchical memory cache 530 at 559 to update the hierarchical memory cache 530 in response to the cache miss. Alternatively, for a write request, the requested line is updated in the memory 520 at 558, which may be propagated to the hierarchical memory cache 530 at 559 (e.g., by updating the hierarchical memory cache 530) and acknowledged at 560. In addition, the corresponding partial and full tags may be written to the appropriate memory.

[0051] Figure 6 An example process 600 for dirty pages is illustrated. Figure 6Illustrated are a processor 610 (corresponding to processor 110, processor 210, and / or accelerator 211), processor storage 614 (corresponding to cache 114, L1 cache 214A-B, L2 cache 216A-B, L2 cache 217, L3 cache 218, and / or other on-die storage), a hierarchical memory cache 630 (corresponding to hierarchical memory cache 130, hierarchical memory cache 230, and / or hierarchical memory cache 330), and a memory 620 (corresponding to memory 120, memory 220, and / or memory 320).

[0052] exist Figure 6 , dirty pages in the hierarchical memory cache 630 are written to the memory 620 at 662. For example, a dirty page may be selected in response to a partial tag hit for the page (e.g., if the memory request is a write request). The page may be dirty (e.g., modified) in response to a previous or current write request.

[0053] In other examples, a dirty page is written in response to a partial tag miss for another page (e.g., as part of a page eviction process where a page is removed from the hierarchical memory cache 630 to make room for the missed page). In some examples, the memory 620 may acknowledge the write at 664 so that the hierarchical memory cache 630 can safely evict the dirty page.

[0054] Figure 7 is a flow chart of an exemplary computer-implemented method 700 for implementing a hierarchical memory cache. Figure 7 The steps shown in can be performed by any suitable computer executable code and / or computing system, including Figure 1 、 Figure 2 and / or Figure 3 In one example, Figure 7 Each of the steps shown in represents an algorithm whose structure includes and / or is represented by a plurality of sub-steps, examples of which are provided in more detail below.

[0055] like Figure 7 As illustrated, at step 702, one or more of the systems described herein locates a partial tag from processor storage corresponding to a memory request for a line stored in a memory having a hierarchical memory cache. For example, the processor 110 locates a partial tag from processor storage (e.g., cache 114 and / or another on-die storage) corresponding to a memory request for a line stored in the hierarchical memory cache 130.

[0056] The systems described herein can perform step 702 in various ways. In one example, hierarchical memory cache 130 is a set associative cache for memory 120. In some examples, the partial tag corresponds to a page such that the partial tag includes a subset of the corresponding full tag.

[0057] In some examples, processor 110 fails to locate the partial tag, indicating a partial tag miss. In response to the partial tag miss, processor 110 retrieves the requested line from memory 120. Furthermore, in some examples, in response to the partial tag miss, a controller of hierarchical memory cache 130 (e.g., controller 142) evicts the page from the hierarchical memory cache and replaces the partial tag corresponding to the evicted page in processor storage with the partial tag corresponding to the missed partial tag. In some examples, when the evicted page is dirty, the controller writes the evicted page to memory.

[0058] At step 704, one or more of the systems described herein, in response to a partial tag hit for the memory request, locates a full tag for the line from the partition of the hierarchical memory cache indicated by the partial tag. For example, a controller of the hierarchical memory cache 130 (e.g., controller 142) locates the full tag for the line indicated by the memory request. In some implementations, the full tag is stored in the hierarchical memory cache 130 using one or more error correction code (ECC) bits. For example, the full tag is stored with the data in the ECC bits of the corresponding line itself. In such implementations, locating the full tag includes retrieving the line requested by the memory from the partition of the hierarchical memory cache (e.g., the indicated way).

[0059] In some examples, the hierarchical memory cache 130 fails to locate a full tag, indicating a full tag miss. In response to the full tag miss, the controller fetches the requested line from the memory 120.

[0060] At step 706, one or more of the systems described herein processes the requested line from the partition of the hierarchical memory cache in response to the memory request based on locating the complete tag. For example, the hierarchical memory cache 130 retrieves the requested line for a read request and returns it to the processor 110. In some examples, the requested line was previously retrieved in response to a partial tag hit for the complete tag lookup. The previously retrieved line can be returned for a read request or updated and written back to the partition of the hierarchical memory cache 130 for a write request.

[0061] As described herein, the present disclosure relates to improvements to hierarchical memory caches. Implementing a set-associative cache that partitions a hierarchical memory cache into a set of multiple ways provides a scalable hierarchical memory cache. Furthermore, the systems and methods provided herein can reduce the overhead of accessing the hierarchical memory cache. Specifically, the systems and methods provided herein reduce the number of lookups required to locate a line in the hierarchical memory cache by using partial page tags and full tags.

[0062] Partial page tags are stored on the die (e.g., in processor storage) for fast lookups. Compared to full tags, partial page tags have reduced metadata storage requirements, allowing partial page tags to be stored on the die. Partial page tags indicate which pages are located in which ways, so that a partial page tag miss indicates that the requested line is not in the hierarchical memory cache. Thus, a partial page miss avoids the more expensive tag lookup process in the hierarchical memory cache.

[0063] A partial page hit indicates that the requested line may be in the hierarchical memory cache. To avoid having to perform an expensive full tag lookup for each way, the partial page tag also points to a specific way so that only a single full tag lookup is required. On a full tag hit, the hierarchical memory cache returns the requested line or performs the requested write to the line. On a fill tag miss, memory is accessed and the hierarchical memory cache (as well as the partial page tag and full tag) is updated accordingly. Thus, the systems and methods herein can reduce overhead by reducing accesses to the hierarchical memory cache and memory.

[0064] As described above, the computing devices and systems described and / or illustrated herein broadly represent any type or form of computing device or system capable of executing computer-readable instructions (such as those contained in the modules described herein). In its most basic configuration, these computing devices each include at least one storage device and at least one physical processor.

[0065] In some examples, the term "memory device" generally refers to any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, a memory device stores, loads, and / or maintains one or more of the modules and / or circuits described herein. Examples of storage devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache, variations or combinations of one or more of the foregoing, or any other suitable memory.

[0066] In some examples, the term "physical processor" generally refers to any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In one example, the physical processor accesses and / or modifies one or more modules stored in the above-mentioned memory device. Examples of physical processors include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a field programmable gate array (FPGA) implementing a soft-core processor, an application-specific integrated circuit (ASIC), a system on a chip (SoC), a digital signal processor (DSP), a neural network engine (NNE), an accelerator, a graphics processing unit (GPU), one or more portions of the foregoing, one or more variations or combinations of the foregoing, or any other suitable physical processor.

[0067] In some implementations, the term "computer-readable medium" generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission-type media such as carrier waves, and non-transitory media such as magnetic storage media (e.g., hard drives, tape drives, and floppy disks), optical storage media (e.g., compact disks (CDs), digital video disks (DVDs), and Blu-ray disks), electronic storage media (e.g., solid-state drives and flash memory media), and other distribution systems.

[0068] The order of process parameters and steps described and / or illustrated herein is provided by way of example only and may be varied as needed. For example, although the steps illustrated and / or described herein are shown or discussed in a particular order, these steps do not necessarily need to be performed in the order illustrated or discussed. The various exemplary methods described and / or illustrated herein may also omit one or more steps described or illustrated herein, or include additional steps in addition to those disclosed.

[0069] The foregoing description has been provided to enable others skilled in the art to best utilize the various aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or limited to any precise form disclosed. Many modifications and variations are possible without departing from the spirit and scope of the present disclosure. The embodiments disclosed herein should be considered in all respects as illustrative and not restrictive. In determining the scope of the present disclosure, reference should be made to the appended claims and their equivalents.

[0070] Unless otherwise indicated, the terms "connected to" and "coupled to" (and their derivatives) as used in the specification and claims will be construed to allow both direct and indirect (i.e., via other elements or components) connections. Additionally, the terms "a" or "an" as used in the specification and claims will be construed to mean "at least one." Finally, for ease of use, the terms "including" and "having" (and their derivatives) as used in the specification and claims are interchangeable with the word "comprising" and have the same meaning.

Claims

1. A device, comprising: physical memory; a hierarchical memory cache associated with the physical memory; and A controller configured to: locating, from a processor storage device, a partial tag corresponding to a memory request for a line stored in the physical memory; responsive to a partial tag hit for the memory request, locating a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag; as well as Based on locating the complete tag, a requested line from the partition of the hierarchical memory cache is processed according to the memory request. 2 . The apparatus of claim 1 , wherein the controller is further configured to fetch the requested row from the memory in response to a full tag miss. 3 . The apparatus of claim 1 , wherein the controller is further configured to fetch the requested row from the memory in response to a partial tag miss.

4. The apparatus of claim 3, wherein the controller is further configured to: evicting pages from the hierarchical memory cache; and In the processor storage device, a partial tag corresponding to the evicted page is replaced with a partial tag corresponding to the missed partial tag. 5 . The apparatus of claim 4 , wherein the controller is further configured to write the evicted page to the memory when the evicted page is dirty.

6. The apparatus of claim 1, wherein the full tag is stored in the hierarchical memory cache using one or more error correction code (ECC) bits. The apparatus according to claim 1 , wherein the partial tag corresponds to a page. The apparatus of claim 1 , wherein the partial tag comprises a subset of a corresponding full tag.

9. The apparatus of claim 1, wherein the hierarchical memory cache comprises a set associative cache for the physical memory.

10. A system comprising: physical memory; a hierarchical memory cache, the hierarchical memory cache being faster than the physical memory; at least one physical processor, the at least one physical processor including processor storage; and A controller configured to: locating, from the processor storage device, a partial tag corresponding to a memory request for a line stored in the physical memory; responsive to a partial tag hit for the memory request, locating a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag; In response to a full tag miss, fetching the requested line from the physical memory; and The requested row is processed according to the memory request.

11. The system of claim 10, wherein the controller is further configured to: In response to a partial tag miss, retrieving the requested line from the physical memory; evicting pages from the hierarchical memory cache; and In the processor storage device, a partial tag corresponding to the evicted page is replaced with a partial tag corresponding to the missed partial tag. 12 . The system of claim 11 , wherein the controller is further configured to write the evicted page to the memory when the evicted page is dirty.

13. The system of claim 10, wherein the full tag is stored in the hierarchical memory cache using one or more error correction code (ECC) bits.

14. The system of claim 10, wherein the partial tag corresponds to a page, and the partial tag comprises a subset of a corresponding full tag.

15. The system of claim 10, wherein the hierarchical memory cache comprises a set associative cache for the physical memory.

16. A method comprising: locating, from processor storage of at least one physical processor, a partial tag corresponding to a memory request for a line stored in physical memory; In response to a partial tag miss, retrieving the requested line from the physical memory; evicting a page from a hierarchical memory cache comprising a set associative cache for the physical memory; as well as In the processor storage device, a partial tag corresponding to the evicted page is replaced with a partial tag corresponding to the missed partial tag. 17 . The method of claim 16 , further comprising writing the evicted page to the memory when the evicted page is dirty.

18. The method according to claim 16, further comprising: responsive to a partial tag hit for the memory request, locating a full tag for the line from a partition of the hierarchical memory cache indicated by the partial tag; In response to a full tag miss, fetching the requested line from the physical memory; as well as Based on locating the full tag in the partition of the hierarchical memory cache, a requested line from the partition of the hierarchical memory cache is processed according to the memory request.

19. The method of claim 18, wherein the full tag is stored in the hierarchical memory cache using one or more error correction code (ECC) bits.

20. The method of claim 18, wherein the partial tag corresponds to a page, and the partial tag comprises a subset of a corresponding full tag.

Images