Integrated Flexible Cache

Abstract

A computer-implemented method is disclosed that includes partitioning a cache structure into multiple cache partitions designated by multiple cache types, forwarding a memory request to a cache partition corresponding to the target cache type of the memory request, and executing the memory request using the cache partition. Various other methods, systems, and computer-readable media are also disclosed.

Description

[Background technology]

[0001] Current processor architectures often include various processing cores and / or chiplets with various cache structures on the die. The cache structures may be client-side caches (e.g., caches used by the processor) or memory-side caches (e.g., caches representing memory devices that may be off-die). Cache structures are designed for specific purposes and lack the ability to be reused.

[0002] The accompanying drawings illustrate several exemplary embodiments and are a part of this specification and, together with the following description, demonstrate and explain various principles of the present disclosure. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram of an exemplary system for a unified flexible cache. [Figure 2] FIG. 1 is a simplified block diagram of an exemplary cache hierarchy. [Figure 3] FIG. 1 is a simplified block diagram of an exemplary cache hierarchy including a flexible cache. [Figure 4] FIG. 2 is a simplified block diagram of the cache structure of a flexible cache. [Figure 5] FIG. 1 is a simplified diagram of routing of memory requests. [Figure 6] FIG. 1 is a flow diagram of an exemplary method for implementing a flexible cache. DETAILED DESCRIPTION OF THE INVENTION

[0004] Throughout the drawings, like reference numerals and descriptions indicate similar, but not necessarily identical, elements. While the exemplary embodiments described herein are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and are described in detail herein. However, the exemplary embodiments described herein are not intended to be limited to the particular forms disclosed. Rather, the present disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims.

[0005] The present disclosure is generally directed to a unified flexible or flex cache. As described in more detail below, embodiments of the present disclosure can configure cache structures for multiple purposes or cache types and route memory requests to the cache structures accordingly. Implementing a flex cache as described herein can improve the functionality of the computer itself by more efficiently utilizing the cache structure, reducing signal latency, and improving cache performance.

[0006] As described in more detail below, this disclosure describes various systems and methods for configuring and using a unified flex cache, where a cache structure is partitioned into cache partitions of various cache types, and memory requests are forwarded to a target cache partition based on the target cache type of the memory request, so that the target cache partition can fulfill the memory request.

[0007] In one example, a device for flex caching includes a cache structure and a cache controller configured to partition the cache structure into a plurality of cache partitions designated by a plurality of cache types, forward memory requests to a target cache partition corresponding to a target cache type of the memory request, and execute the memory request using the target cache partition.

[0008] In some examples, forwarding the memory request is performed based on an addressing scheme that incorporates a cache type. In some examples, the addressing scheme includes one or more bits for identifying a target cache partition. In some examples, the one or more bits correspond to a port coupled to the target cache partition.

[0009] In some examples, partitioning the cache structure includes partitioning the cache structure based on physical delineations of the cache structure, where the physical delineations correspond to at least one of a bank, a way, an index, or a macro.

[0010] In some examples, the plurality of cache types includes at least one of a processor cache, an accelerator cache, a memory cache, or a probe filter. In some examples, partitioning the cache structure further includes partitioning the cache structure at boot time. In some examples, partitioning the cache structure further includes dynamically partitioning the cache structure based on workload.

[0011] In one embodiment, a system for flex caching includes at least one physical processor, a physical memory, a cache structure including a plurality of ports, and a cache controller configured to partition the cache structure into a plurality of cache partitions specified by a plurality of cache types, each cache partition coupled to at least one of the plurality of ports, forward memory requests along any of the plurality of ports to a target cache partition for the memory request, and execute the memory request using the target cache partition.

[0012] In some examples, forwarding the memory request is performed based on an addressing scheme that includes one or more bits to identify a port coupled to the target cache partition. In some examples, partitioning the cache structure includes partitioning the cache structure based on at least one of a bank, a way, an index, or a macro.

[0013] In some examples, the plurality of cache types includes at least one of a processor cache, an accelerator cache, a memory cache, or a probe filter. In some examples, partitioning the cache structure further includes partitioning the cache structure at boot time of the device. In some examples, partitioning the cache structure further includes dynamically partitioning the cache structure based on a workload of the device.

[0014] In one embodiment, a method for flex caching includes partitioning a cache structure during system boot into multiple cache partitions specified by multiple cache types, forwarding a memory request to a cache partition corresponding to a target cache partition of the memory request, and executing the memory request using the cache partition.

[0015] In some examples, forwarding the memory request is performed based on an addressing scheme that includes one or more bits to identify a target cache type. In some examples, partitioning the cache structure includes partitioning the cache structure based on at least one of a bank, a way, an index, or a macro. In some examples, the multiple cache types include at least one of a processor cache, an accelerator cache, a memory cache, or a probe filter. In some examples, partitioning the cache structure further includes dynamically partitioning the cache structure based on system workload.

[0016] Features from any of the embodiments described herein may be used in combination with each other in accordance with the general principles described herein. These and other embodiments, features, and advantages will be more fully understood from the following detailed description read in conjunction with the accompanying drawings and claims.

[0017] The following provides a detailed description of a unified flexible cache and related systems and methods with reference to Figures 1-6. A detailed description of an exemplary system is provided in connection with Figure 1. A detailed description of an exemplary cache architecture is provided in connection with Figures 2-4. A detailed description of an exemplary memory request routing is provided in connection with Figure 5. A detailed description of a corresponding computer-implemented method is also provided in connection with Figure 6.

[0018] FIG. 1 is a block diagram of an exemplary system 100 for Flex Cache. System 100 corresponds to a computing device, such as a desktop computer, a laptop computer, a server, a tablet device, a mobile device, a smartphone, a wearable device, an augmented reality device, a virtual reality device, a network device, and / or an electronic device. As shown in FIG. 1, system 100 includes one or more memory devices, such as memory 120. Memory 120 generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. Examples of memory 120 include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache, any variation or combination of one or more of these, and / or any other suitable storage memory.

[0019] 1, exemplary system 100 includes one or more physical processors, such as processor 110. Processor 110 generally represents any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In some examples, processor 110 accesses and / or modifies data and / or instructions stored in memory 120. Examples of processor 110 include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA) implementing a soft-core processor, an application-specific integrated circuit (ASIC), a system on a chip (SoC), a digital signal processor (DSP), a neural network engine (NNE), an accelerator, a graphics processing unit (GPU), one or more portions thereof, one or more variations or combinations thereof, and / or any other suitable physical processor.

[0020] As further shown in FIG. 1 , processor 110 includes cores 112, cache 114, flex cache 130, and controller 142. Cores 112 correspond to processor cores, but in other examples, correspond to chiplets, such as accelerator chiplets. Cache 114 corresponds to a cache used by processor 110 (e.g., a client-side cache, such as a low-level cache or L1 cache). In some examples, cache 114 corresponds to and / or includes other caches, such as memory-side caches. Flex cache 130 corresponds to a cache structure that can be flexibly used for various purposes, as described further herein. Controller 142 corresponds to control circuitry that can configure and control flex cache 130, such as by regulating memory requests to flex cache 130. In some examples, controller 142 also controls aspects of cache 114.

[0021] Processor 110 reads and operates on instructions and / or data stored in memory 120. Because memory 120 is often slower than processor 110, memory access time creates a bottleneck for processor 110. To alleviate this problem, processor 110 includes cache 114, which is typically a high-speed memory that has an access time that is shorter than that of memory 120, due in part to its physical location within processor 110.

[0022] The cache 114 holds data and / or instructions read from the memory 120. The processor 110 (and / or the core 112) first makes a memory request to the cache 114. If the cache 114 holds the requested data (e.g., a cache hit), the processor 110 reads the data from the cache 114, avoiding the memory access time of the memory 120. If the cache 114 does not hold the requested data (e.g., a cache miss), the processor 110 retrieves the data from the memory 120, incurring the memory access time. While a larger cache size can reduce cache misses, considerations such as die size and power consumption limit the size of the cache 114. Therefore, to further reduce the need to access the memory 120 on a cache miss, the processor 110 incorporates another cache in the cache hierarchy that is larger but slower than the cache 114.

[0023] As described further below, flex cache 130 may be used for (and therefore replace) various types of cache that would normally require a separate physical cache structure occupying die space. For example, flex cache 130 may be configured as one or more of a processor cache, an accelerator cache, a memory cache, or a probe cache. In some examples, when system 100 boots, controller 142 may configure flex cache 130 by partitioning flex cache 130 into various cache partitions corresponding to various cache types. For example, the BIOS of system 100 may include a configuration specifying the cache type and cache size, and controller 142 may partition flex cache 130 according to the configuration. Furthermore, in some examples, controller 142 may dynamically partition flex cache 130 based on the workload of system 100. For example, controller 142 and / or other circuitry in processor 110 may analyze the workload of system 100 (e.g., how the cache is used, how memory 120 is accessed, the type of data being processed, etc.) to determine more efficient use of flex cache 130 (e.g., the type of cache and the size of each type) and reconfigure flex cache 130 accordingly.

[0024] In some examples, flex cache 130 may be configured as one or more levels of a cache hierarchy. Figure 2 shows an exemplary cache hierarchy in system 200 corresponding to system 100. System 200 includes one or more processors 210 corresponding to processor 110 and one or more accelerators 211 corresponding to processor 110. As shown in Figure 2, processor 210 includes core 212A corresponding to core 112, core 212B corresponding to core 112, L1 cache 214A corresponding to cache 114, L1 cache 214B corresponding to cache 114, L2 cache 216A, which may correspond to cache 114, L2 cache 216B, which may correspond to cache 114, and L3 cache 218, which may correspond to cache 114 and / or flex cache 130.

[0025] In the cache hierarchy of FIG. 2, level 1 (L1) corresponds to the lowest level of the hierarchy. L1 caches, such as L1 cache 214A and L1 cache 214B, may be implemented using high-speed memory, such as static random access memory (SRAM). To further prioritize speed, the L1 caches may also be integrated with processor 210, for example, within core 212A and core 212B, respectively, to improve latency and throughput. In some examples, as shown in FIG. 2, processor 210 includes multiple L1 caches.

[0026] An L2 cache, such as L2 cache 216A and L2 cache 216B, is the next level in the cache hierarchy after the L1 cache and may be larger and slower than an L1 cache. Although integrated with processor 210, the L2 cache may in some examples be located outside the chip core, but may also be located on the same chip core package. An L3 cache, such as L3 cache 218, may be larger but slower than an L2 cache. The L3 cache may act as a bridge to main memory (e.g., memory 220). Thus, the L3 cache may be faster than the main memory. In some examples, multiple processors and / or cores may share an L3 cache, which may be located on the same chip core package or outside the package.

[0027] Memory 220, corresponding to memory 120, stores instructions and / or data for retrieval and use by processor 210. Memory 220 may be implemented as dynamic random-access memory (DRAM). As shown in Figure 2, the cache hierarchy further includes a memory cache 222 (e.g., a memory-side cache), which in some examples corresponds to cache 114, and a data fabric 240, which corresponds to various structures, connections, and control circuits for transmitting data between the memory and cache structures.

[0028] Additionally, system 200 includes one or more accelerators with similar cache hierarchies. Accelerator 211 includes chiplet 213A corresponding to core 112, chiplet 213B corresponding to core 112, chiplet 213C corresponding to core 112, chiplet 213D corresponding to core 112, and L2 cache 217 corresponding to cache 114 shared by the chiplets.

[0029] 3 illustrates another exemplary cache hierarchy for system 300 corresponding to system 200. In FIG. 3, various cache structures are replaced by a unified flexible cache. System 300 includes one or more processors 310 corresponding to processor 110 and one or more accelerators 311 corresponding to processor 110. As shown in FIG. 3, processor 310 includes core 312A corresponding to core 112, core 312B corresponding to core 112, L1 cache 314A corresponding to cache 114, L1 cache 314B corresponding to cache 114, L2 cache 316A, which may correspond to cache 114, L2 cache 316B, which may correspond to cache 114, and flex cache 330 corresponding to flex cache 130.

[0030] Additionally, system 300 includes one or more accelerators that also use flex cache 330. Accelerator 311 includes chiplet 313A corresponding to core 112, chiplet 313B corresponding to core 112, chiplet 313C corresponding to core 112, and chiplet 313D corresponding to core 112.

[0031] In some examples, system 300 further includes a memory cache 322 corresponding to cache 114 , a memory 320 corresponding to memory 120 , and a data fabric 340 .

[0032] 3, different cache structures, namely, L3 cache 218 and L2 cache 217, are replaced with flex cache 330. More specifically, flex cache 330 is partitioned into an L3 cache for processor 310 and an L2 cache for accelerator 311. Unlike the static structures of L3 cache 218 and L2 cache 217, which have predetermined sizes, flex cache 330 can be configured to provide different sizes and numbers of L3 and / or L2 caches as needed.

[0033] 3, flex cache 330 may be configured as other types of caches and thus may replace cache structures such as memory caches (e.g., memory cache 322), probe filters, and other processor and / or accelerator caches corresponding to other levels of the cache hierarchy. For example, flex cache 330 may be a single cache structure or a series of cache structures shared among processor 310, accelerator 311, and / or other processors / devices. In other examples, processor 310 and accelerator 311 may each have their own flex cache 330.

[0034] Figure 4 illustrates a device 400 corresponding to system 100. Device 400 includes a flex cache 430 corresponding to flex cache 130 and / or flex cache 330. Figure 4 illustrates a simplified cache structure of flex cache 430. Flex cache 430 includes various ports 436, various banks 432 organized into macros 434, and a controller 442 corresponding to controller 142.

[0035] In some examples, the controller 442 may partition the flex cache 430 based on physical divisions such as banks 432, macros 434, indices (e.g., identifiers of physical structures), ports 436, ways (e.g., subsets of structures), etc. For example, based on partition size, the controller 442 may partition the flex cache 430 by designating particular banks 432 (e.g., banks similarly indexed across macros 434) or by selecting macros 434 as partitions.

[0036] In some examples, after partitioning flex cache 430, controller 442 can forward subsequent memory requests to the appropriate cache partition. In some examples, controller 442 forwards memory requests based on an addressing scheme that identifies the target cache partition. For example, one or more bits of the address can identify a port coupled to the target cache partition. Figure 5 further illustrates how memory requests can be forwarded.

[0037] FIG. 5 illustrates a device 500 corresponding to system 100. Device 500 includes a cache fabric 530 corresponding to flex cache 130, and more specifically, a cache structure for flex cache 130. Cache fabric 530 includes various nodes or vertices, such as cache node 538A and cache node 538B, that correspond to the interconnected physical structure of the cache structure. Cache node 538A includes a controller 542A that corresponds to controller 142 and a cache partition 552A. Cache node 538B includes a controller 542B that corresponds to controller 142 and a cache partition 552B. FIG. 5 also illustrates a cache 554 and control circuitry 544 that corresponds to controller 142.

[0038] When cache fabric 530 receives a memory request, various controllers (e.g., controller 542A, controller 542B, and / or control circuitry 544) can forward the memory request to the appropriate cache partition (e.g., cache partition 552A, cache partition 552B, and / or cache 554). In some examples, forwarding the memory request can also include forwarding the memory request from one cache node to another, from one cache partition to another cache partition or controller, etc., as needed.

[0039] In one example, cache node 538A receives a memory request targeted for cache 554. Based on the addressing scheme, controller 542A can map the memory request to be targeted for a different cache node (and / or cache partition) and forward the memory request to cache node 538B accordingly. Controller 542B (and / or cache partition 552B in some examples) can map the memory request to be targeted for a different cache partition and forward the memory request to control circuitry 544.

[0040] The control circuitry 544 can map the memory request to be targeted to the cache 554 and forward the memory request accordingly. In another example, the control circuitry 544 can forward the memory request to the cache partition 552B, which can forward the memory request to the cache 554 based on a cache miss.

[0041] Figure 6 is a flow diagram of an exemplary computer-implemented method 600 for implementing a unified Flex Cache. The steps illustrated in Figure 6 may be performed by any suitable computer-executable code and / or computing system, including the systems illustrated in Figures 1, 2, 3, 4, and / or 5. In one example, each of the steps illustrated in Figure 6 represents an algorithm, the structure of which includes and / or is represented by multiple sub-steps, examples of which are provided in more detail below.

[0042] 6, one or more of the systems described herein partition a cache structure into multiple cache partitions designated by multiple cache types, in step 602. For example, controller 142 partitions flex cache 130 into different partitions designated as different cache types.

[0043] The systems described herein may perform step 602 in a variety of ways. In one example, the cache type includes a processor cache, an accelerator cache, a memory cache, or a probe filter, as described herein.

[0044] In some examples, partitioning the cache structure includes partitioning the cache structure based on physical divisions of the cache structure, which may correspond to at least one of a bank, a way, an index, or a macro, as described herein.

[0045] In some examples, partitioning the cache structure includes partitioning the cache structure at boot time. In some examples, partitioning the cache structure includes dynamically partitioning the cache structure based on workload.

[0046] In step 604, one or more of the systems described herein forward the memory request to a cache partition corresponding to the target cache type of the memory request. For example, controller 142 forwards the memory request to the appropriate cache partition of flex cache 130 corresponding to the target cache type to satisfy the memory request.

[0047] The systems described herein may perform step 604 in a variety of ways. In one example, forwarding the memory request is performed based on an addressing scheme that incorporates the cache type. For example, the addressing scheme may include one or more bits to identify the target cache partition. In some examples, the one or more bits correspond to a port coupled to the target cache partition. In some embodiments, the bits may be reused bits of the address. In other embodiments, additional bits may be added to the address.

[0048] In step 606, one or more of the systems described herein use a cache partition to fulfill the memory request. For example, a cache partition of flex cache 130 fulfills the memory request, such as reading or writing data.

[0049] As described herein, a unified flexible cache may be a large cache structure that can replace a variety of smaller cache structures, simplifying design and fabrication and improving yield during manufacturing. Note that a unified flexible cache may be used for various types of cache, such as various levels of processor and / or accelerator caches, as well as other cache structures for managing cache hierarchies, such as probe filters. Because a flex cache can be partitioned into partitions of various sizes, cache types are not limited to a particular size (e.g., limited by a physical structure). Thus, a flex cache may be reconfigured to provide more efficient cache utilization based on the needs of the system.

[0050] As noted above, the computing devices and systems described and / or illustrated herein broadly represent any type or form of computing device or system capable of executing computer-readable instructions, such as those contained within the modules described herein. In their most basic configurations, these computing devices each include at least one memory device and at least one physical processor.

[0051] In some examples, the term "memory device" generally refers to any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, a memory device stores, loads, and / or maintains one or more of the modules and / or circuits described herein. Examples of memory devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drives (HDDs), solid-state drives (SSDs), optical disk drives, caches, variations or combinations of one or more of these, or any other suitable storage memory.

[0052] In some examples, the term "physical processor" generally refers to any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In one example, a physical processor accesses and / or modifies one or more modules stored in the memory devices described above. Examples of physical processors include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a field programmable gate array (FPGA) implementing a soft-core processor, an application-specific integrated circuit (ASIC), a system-on-chip (SoC), a digital signal processor (DSP), a neural network engine (NNE), an accelerator, a graphics processing unit (GPU), one or more portions thereof, one or more variations or combinations thereof, or any other suitable physical processor.

[0053] The process parameters and order of steps described and / or illustrated herein are given by way of example only and can be changed as desired. For example, although the steps illustrated and / or described herein are shown or described in a particular order, these steps do not necessarily have to be performed in the order illustrated or described. The various exemplary methods described and / or illustrated herein may omit one or more of the steps described or illustrated herein or may include additional steps in addition to those disclosed.

[0054] The above description is provided to enable those skilled in the art to best utilize various aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or to be limited to any precise form disclosed. Many changes and modifications are possible without departing from the spirit and scope of the present disclosure. The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. In determining the scope of the present disclosure, reference should be made to the appended claims and their equivalents.

[0055] Unless otherwise specified, the terms "connected to" and "coupled to" (and their derivatives) as used in this specification and claims should be interpreted as allowing both direct and indirect connections (i.e., via other elements or components). Additionally, the terms "a" or "an" as used in this specification and claims should be interpreted as meaning "at least one of." Finally, for ease of use, the terms "including" and "having" (and their derivatives) as used in this specification and claims are interchangeable with the term "comprising," and have the same meaning.

Claims

1. A device, a cache structure; a cache controller; The cache controller Partitioning the cache structure into a plurality of cache partitions designated by a plurality of cache types; forwarding the memory request to a target cache partition corresponding to a target cache type of the memory request; servicing the memory request using the target cache partition; and configured to: device.

2. forwarding the memory request is based on an addressing scheme incorporating a cache type; The device of claim 1.

3. the addressing scheme includes one or more bits for identifying the target cache partition; The device of claim 2.

4. the one or more bits correspond to a port coupled to the target cache partition; The device of claim 3.

5. Partitioning the cache structure includes partitioning the cache structure based on physical divisions of the cache structure. The device of claim 1.

6. the physical division corresponds to at least one of a bank, a way, an index, or a macro; The device of claim 5.

7. the plurality of cache types include at least one of a processor cache, an accelerator cache, a memory cache, or a probe filter; The device of claim 1.

8. partitioning the cache structure includes partitioning the cache structure at boot time of the device. The device of claim 1.

9. partitioning the cache structure includes dynamically partitioning the cache structure based on a workload of the device. The device of claim 1.

10. 1. A system comprising: at least one physical processor; Physical memory and a cache structure including a plurality of ports; a cache controller; The cache controller partitioning the cache structure into a plurality of cache partitions designated by a plurality of cache types, each cache partition being coupled to at least one of the plurality of ports; forwarding the memory request along one of the plurality of ports to a target cache partition of the memory request; servicing the memory request using the target cache partition; and configured to: system.

11. forwarding the memory request based on an addressing scheme that includes one or more bits to identify a port coupled to the target cache partition; The system of claim 10.

12. partitioning the cache structure includes partitioning the cache structure based on at least one of a bank, a way, an index, or a macro. The system of claim 10.

13. the plurality of cache types include at least one of a processor cache, an accelerator cache, a memory cache, or a probe filter; The system of claim 10.

14. Partitioning the cache structure includes partitioning the cache structure at boot time of the system. The system of claim 10.

15. partitioning the cache structure includes dynamically partitioning the cache structure based on workload of the system. The system of claim 10.

16. 1. A method comprising: partitioning a cache structure into multiple cache partitions designated by multiple cache types during system boot; forwarding the memory request to a cache partition corresponding to a target cache partition of the memory request; and executing the memory request using the cache partition. method.

17. forwarding the memory request based on an addressing scheme that includes one or more bits for identifying a target cache type; 17. The method of claim 16.

18. partitioning the cache structure includes partitioning the cache structure based on at least one of a bank, a way, an index, or a macro.

17. The method of claim 16.

19. the plurality of cache types include at least one of a processor cache, an accelerator cache, a memory cache, or a probe filter; 17. The method of claim 16.

20. partitioning the cache structure includes dynamically partitioning the cache structure based on workload of the system.

17. The method of claim 16.

Images