Dynamic control of cache mode

Abstract

A method, system, and apparatus for dynamically selecting a cache mode, including a computer program encoded on a computer storage medium. One method includes operating a system-level cache (SLC) in victim cache mode by implementing a portion of the SLC as a victim cache partition for the primary cache. The method further includes monitoring performance metrics of the victim cache partition. When the performance metrics meet one or more switching criteria, the primary cache and the SLC operate in normal cache mode.

Description

Background Art

[0001] This specification relates to caching.

[0002] A system-on-chip (SoC) is an integrated circuit that integrates various components of a mobile computing device, including a central processing unit (CPU), memory, input / output ports, cellular radio, and secondary storage. In contrast to a conventional motherboard-based PC architecture that houses and connects removable or replaceable components, an SoC integrates all of these components into a single integrated circuit. SoCs are typically used in mobile computing, edge computing, and embedded systems, such as smartphones, tablet computers, WiFi (registered trademark) routers, Internet of Things (IoT) devices, and the like.

[0003] A memory hierarchy can enable faster access to data in memory by storing recently used data in an intermediate-level cache. A victim cache can improve the performance of the memory hierarchy by storing data evicted from higher-level caches in the hierarchy than the victim cache. For example, in a memory hierarchy with a non-victim cache, in the event of a cache miss in a higher-level cache than the non-victim cache, the requested block of data is taken into the higher-level cache, and the block evicted from the higher-level cache is discarded. In a memory hierarchy with a victim cache, in the event of a cache miss in a higher-level cache than the victim cache, the requested block of data is taken into the higher-level cache, and the block evicted from the higher-level cache to make space for the requested block of data is stored in the victim cache. If the evicted block of data is needed in a future memory request, the evicted block can be found in the victim cache. In some embodiments, after the evicted block has been taken into the higher-level cache, the evicted block may be discarded from the victim cache. Accessing the victim cache from a higher-level cache is faster than accessing a lower-level memory hierarchy, resulting in power and performance advantages.

[0004] However, in some cases, the use of a victim cache can negatively impact power and performance. The overhead of maintaining a victim cache is significant because blocks of data are sent from higher-level caches to the victim cache. For example, a cache shared among client devices can adjust the size allocated to each client device's partition. If a victim cache partition is not large enough to store the evicted blocks of data from the higher-level cache, the evicted blocks are discarded by the victim cache partition. Since evicted blocks are not stored by the victim cache partition, there is no benefit in sending evicted blocks of data to the victim cache partition. [Overview of the project]

[0005] This specification describes a system capable of dynamically selecting a cache mode. The system operates the system-level cache (SLC) in victim cache mode by implementing a portion of the SLC as a victim cache partition for the client device cache. The system monitors the performance metrics of the victim cache partition, and if the performance metrics meet one or more switching criteria, the system operates the client device cache and the SLC in normal cache mode. When the system is in normal cache mode, the system monitors the performance metrics of the victim cache partition, and if the performance metrics meet one or more switching criteria, the system operates the client device cache and the SLC in victim cache mode.

[0006] Certain embodiments of the subject matter described herein can be implemented to achieve one or more of the following advantages:

[0007] The system can improve runtime performance by switching cache modes to enhance system performance. For example, the system can switch from victim cache mode to normal cache mode. The system can decide to switch cache modes according to any appropriate performance metric, for example, if a metric measuring the overhead of maintaining a victim cache partition meets a threshold. For example, in applications such as streaming video or audio, if the hit rate of a victim cache partition is close to zero, or if the victim cache partition is not large enough to store more evicted lines, the higher-level cache can evict clean cache lines and send them to the victim cache partition without gaining any benefit from it. The system can reduce overhead by disabling victim cache mode at runtime and thus monitoring performance metrics such as the size of the SLC available by the victim cache partition or the hit rate of the victim cache partition, and determining whether the performance metrics meet one or more switching criteria. Additionally, disabling victim cache mode can reduce power consumption when sending cache lines to victim cache partitions, which can be particularly important for mobile devices.

[0008] The system can improve runtime performance and power consumption by re-enabling victim cache mode when the system determines that a metric measuring the overhead of maintaining victim cache partitions meets a threshold.

[0009] The system can maintain data coherence and limit the impact on system traffic while switching from victim cache mode to normal cache mode and from normal cache mode to victim cache mode. For example, after the system determines that a metric measuring the overhead of maintaining a victim cache partition meets a threshold and therefore decides to switch modes, there may still be outstanding memory requests to the SLC sent before the decision to switch, which are still propagating through the memory hierarchy and should be executed in the original cache mode. The system can use trigger signals and a count of outstanding memory requests to ensure that outstanding memory requests are completed in the mode intended by the sender of the memory requests. Thus, the system can switch between modes in a coordinated manner.

[0010] A system can maintain the same level of performance as a system with a larger cache that does not switch cache modes by using a smaller cache that switches cache modes. A system can also achieve better performance by using a cache that switches cache modes than a system with the same-sized cache that does not. For example, a system can improve or maintain performance by switching cache modes at runtime.

[0011] Details of one or more embodiments of the subject matter of this specification are described in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram of an exemplary system for dynamically controlling the cache mode. [Figure 2]This is an illustrative flowchart of the process for switching from victim cache mode to normal cache mode. [Figure 3A] This is an exemplary process flowchart for switching the cache mode from victim cache mode to normal cache mode. [Figure 3B] This is an exemplary process flowchart for switching the cache mode from victim cache mode to normal cache mode. [Figure 4] This is an exemplary process flowchart for switching the cache mode from normal cache mode to victim cache mode. [Modes for carrying out the invention]

[0013] Similar reference numbers and symbols in various drawings refer to the same elements.

[0014] Figure 1 is a block diagram of an exemplary system 100 for dynamically controlling the cache mode. System 100 includes various components such as client devices 102a to 102n (collectively referred to as "client devices 102"), a primary cache 104, a system-level cache (SLC) 106, and a memory device 110.

[0015] A system-level cache (SLC) is any suitable device that caches data retrieved from or stored in memory for multiple different hardware devices within a system. In other words, the various cache lines of an SLC can store data belonging to different hardware devices, such as different processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), microcontrollers, machine learning accelerators, or image processors, to name just a few. As shown in Figure 1, an SLC can be part of a larger cache hierarchy. For some hardware devices, there may be one or more additional cache levels between the hardware device and the SLC. Other hardware devices may use the SLC directly without any intervening cache levels.

[0016] Caches higher up in the cache hierarchy are closer to various hardware devices, meaning there is less cache level between the cache and hardware devices than caches lower down in the cache hierarchy. Caches lower down in the cache hierarchy are closer to memory, meaning there is less cache level between the cache and memory than caches higher up in the cache hierarchy.

[0017] The client devices among client devices 102a to 102n may be any type of hardware device configured to provide memory requests 103 to the memory device 110 via the SLC 106, such as a central processing unit (CPU), a digital signal processor (DSP), a neural processing unit (NPU), or a cache.

[0018] Each client device 102a to 102n may have a cache higher in the cache hierarchy than the SLC 106. In the exemplary system 100, client 1 102a has client cache 1 104.

[0019] SLC106 can be any cache configured to cache memory requests on memory device 110, and can operate in victim cache mode by implementing a portion of it as a victim cache partition 108 while also supplying memory requests for other client devices.

[0020] In the exemplary system 100, the SLC 106 includes a victim cache partition 108 and partitions for clients 2-N 102b-102n, and client partitions 2-N 109b-109n. The SLC is shared across all client devices 102 and can adjust the partition sizes allocated to each client device within client partitions 109b-109n and the victim cache partition 108.

[0021] In some embodiments, the SLC106 can implement multiple parts as victim cache partitions for multiple client devices. In some embodiments, the SLC106 can include a status monitor, counters, and an SLC cache mode selector for each victim cache partition. For example, each victim cache partition can be in its own cache mode independently of other victim cache partitions. Each client and their corresponding client partition can determine its cache mode, whether it is victim cache mode or normal cache mode. For example, client partition 2 109b can be the victim cache partition for client 2 102b, and client cache 2 (not shown) for client 2 102b, regardless of whether victim cache partition 108 is actually configured to be in victim cache mode or normal cache mode. Each victim cache partition, for example, victim cache partition 108, is considered an extension of its corresponding client's internal cache, for example, client cache 1, and visibility and data coherence may conform to the theory of operation as if the clean-extracted victim cache were located in the client's internal cache.

[0022] The memory device 110 may be any type of computing device that stores data in response to memory requests, such as a cache, random access memory (RAM), or dynamic random access memory (DRAM).

[0023] Client cache 1 104 includes a cache mode selector 114 that can be used by the system to control the cache mode of client cache 1 104. The cache mode selector 114 is a software or hardware entity that can control the cache mode of client cache 1 104, for example, by enabling or disabling the victim cache mode of client cache 1 104. Possible cache modes can include, for example, a victim cache mode (where the victim cache mode is enabled) or a normal cache mode (where the victim cache mode is disabled).

[0024] In the case of a cache miss in client cache 1 104, data can be retrieved from a lower level of the memory hierarchy, such as SLC 106 or memory device 110. The data retrieved from the lower level of the memory hierarchy replaces the data in client cache 1 104, the victim line. In the normal cache mode, the system silently drops the data onto a clean victim line. In the victim cache mode, the system stores the data from the victim line in a lower-level cache, i.e., the victim cache.

[0025] The system can implement a part of SLC106 as the victim cache partition 108 of the client cache 1 104. The system can use the SLC cache mode selector 115 to enable or disable the victim cache mode of the victim cache partition 108. The SLC cache mode selector 115 is a software or hardware entity that can control the cache mode of the victim cache partition 108, for example, by enabling or disabling the victim cache mode of the victim cache partition 108. Possible cache modes can include, for example, the victim cache mode (where the victim cache mode is enabled) or the normal cache mode (where the victim cache mode is disabled).

[0026] In the case of a cache miss in the client cache 1 104, data can be retrieved from a lower level of the memory hierarchy such as the SLC106 or the memory device 110. The data retrieved from the lower level of the memory hierarchy replaces a victim line in the client cache 1 104. In the normal cache mode, the system silently drops the data onto a clean victim line. In the victim cache mode, the system stores the data from the victim line in the victim cache partition 108.

[0027] The system can use the status monitor 112 to monitor performance metrics of the victim cache partition 108. For example, performance metrics may include the size of the SLC 106 available to the victim cache partition 108. In some embodiments, performance metrics may include the hit rate for the victim cache partition 108. In some embodiments, performance metrics may also measure the intensity of traffic on the victim cache partition 108, the read / write ratio of the victim cache partition 108, or the active use cases of the victim cache partition 108.

[0028] If SLC106 is in victim cache mode, and the system determines that one or more performance metrics meet the switching criteria, the system can operate client cache 104 and SLC106 in normal cache mode instead of victim cache mode. For example, the switching criteria could be a threshold victim cache partition size or a threshold hit rate for a victim cache partition. The system can use status monitor 112 to send a trigger signal 116 to trigger the process of switching from victim cache mode to normal cache mode. To maintain data coherence when switching from victim cache mode to normal cache mode, the system can use counter 118 to maintain a count of outstanding memory requests made to the SLC before the trigger signal was issued but not yet supplied by the SLC. This switching process is described in more detail below with reference to Figures 2 and 3.

[0029] When SLC106 is operating in normal cache mode, the system can store cache data from clients 2~N 102b~102n in victim cache partition 108 (the portion of SLC106 that functioned as an exclusive victim cache partition for client cache 1 104 when SLC106 was operating in victim cache mode).

[0030] If SLC106 is in normal cache mode, and the system determines that one or more performance metrics meet the switching criteria, the system can operate client cache 104 and SLC106 in victim cache mode instead of normal cache mode. For example, the switching criteria could be a threshold victim cache partition size or a threshold hit rate for a victim cache partition. The system can use status monitor 112 to send a trigger signal 116 to trigger the process of switching from normal cache mode to victim cache mode. To maintain data coherence when switching from normal cache mode to victim cache mode, the system can use counter 118 to maintain a count of outstanding memory requests made to the SLC before the trigger signal was issued but not yet supplied by the SLC. This switching process is described in more detail below with reference to Figure 4.

[0031] Figure 2 is a flowchart of an exemplary process 200 for switching from victim cache mode to normal cache mode. Process 200 can be performed by any suitable system, for example, system 100 in Figure 1.

[0032] The system can operate the primary cache, for example, client cache 1 104 in Figure 1, in victim cache mode (205). For example, the system can enable victim cache mode in the primary cache using the primary cache mode selector, for example, cache mode selector 114 in Figure 1.

[0033] The system can operate the SLC in victim cache mode by implementing a portion of the SLC as a victim cache partition for the primary cache (210). The SLC can also supply memory requests for other client devices, such as clients 2-N 102b-102n in Figure 1. An SLC operating in victim cache mode can store data from victim lines of cache misses in the primary cache in the victim cache partition.

[0034] The system can monitor performance metrics for the victim cache partition (220). In some embodiments, the performance metrics may include the size of the SLC available to the victim cache partition. In some embodiments, the performance metrics may include the hit rate for the victim cache partition.

[0035] When the system determines that the performance metrics meet one or more switching criteria, it can allow the primary cache and SLC to operate in normal cache mode (230). An SLC operating in normal cache mode can drop data from the victim line of a cache miss in the primary cache.

[0036] In embodiments where the performance metric includes the size of the SLC available to the victim partition, determining that the performance metric meets one or more switching criteria may include comparing the size of the SLC available to the victim cache partition to a threshold victim cache partition size. In embodiments where the performance metric includes the hit rate of the victim cache partition, determining that the performance metric meets one or more switching criteria may include comparing the hit rate of the victim cache partition to a threshold hit rate of the victim cache partition.

[0037] In some embodiments, the switching criterion may be a software-programmable parameter. In some embodiments, the switching criterion may be defined at design time.

[0038] The system can operate the primary cache and SLC in normal cache mode by using trigger signals. The trigger signals are described in more detail below with reference to Figure 3B.

[0039] Figures 3A-3B are flowcharts of an exemplary process 300 for switching the cache mode from victim cache mode to normal cache mode. Process 300 can be performed by any suitable system, for example, system 100 in Figure 1.

[0040] Figure 3A shows how the system can be initially configured to allow dynamic switching of the cache mode from victim cache mode to normal cache mode. For example, upon startup, the system may be configured in victim cache mode. The system can then dynamically switch the cache mode from victim cache mode to normal cache mode by performing the steps in Figure 3B. In some embodiments, upon startup, the system is configured in normal cache mode, and the system can then dynamically switch the cache mode from normal cache mode to victim cache mode by performing the steps in Figure 4.

[0041] After the system has started up (310), the system configures the SLC with victim cache support (312). Configuring the SLC with victim cache support may include implementing a portion of the SLC as a victim cache partition of a client device, for example, client cache 1 104 in Figure 1. It is important that both the SLC and the client device caches match in terms of cache mode. In some embodiments, configuring the SLC with victim cache support may include, for example, reading and writing values ​​to registers, or other interactions between components of the SLC.

[0042] The system enables victim cache mode on the client device cache and SLC, and enables a partition on the SLC as a victim cache partition (314). An SLC operating in victim cache mode can store data from victim lines of cache misses in the client device cache in the victim cache partition. The system can enable victim cache mode on the client device cache using the client device cache mode selector, for example, the cache mode selector 114 in Figure 1. For example, the system can enable the write-back of extracted clean cache lines. The system can enable a partition on the SLC as a victim cache partition using the SLC cache mode selector, for example, the SLC cache mode selector 115 in Figure 1. For example, clean cache lines written back by the client device cache can be stored in the victim cache partition. The system can also invalidate cache lines read from the victim cache partition by the client device cache.

[0043] The system sets a trigger threshold (316). For example, the trigger threshold could be the size of the threshold victim cache partition or the threshold hit rate of the victim cache partition. The system can use the trigger threshold to define the switching criteria.

[0044] Here, the system can operate in victim cache mode (317). An SLC operating in victim cache mode can store data from the victim line of a cache miss in the client device's cache in the victim cache partition. The system can dynamically switch the cache mode from victim cache mode to normal cache mode by performing the steps in Figure 3B.

[0045] Figure 3B illustrates how the system can dynamically switch the cache mode from victim cache mode to normal cache mode. The system is in victim cache mode (317) as described above with reference to Figure 3A.

[0046] The system monitors the status of the victim cache partition (318). The system may monitor performance metrics that describe the status of the victim cache partition. For example, the system may monitor the size of the SLC available by the victim cache partition, or the hit rate of the victim cache partition. In some embodiments, the system may monitor the intensity of traffic on the victim cache partition, the read / write ratio of the victim cache partition, or the active use cases of the victim cache partition.

[0047] If the system determines that the trigger threshold is not met (320), no trigger signal is generated (322), and the system continues to monitor the status of the victim cache partition (318). For example, if the system compares the size of SLC available to the victim cache partition with the threshold victim cache partition size and determines that the size of SLC available to the victim cache partition is greater than the threshold victim cache partition size, no trigger signal is generated.

[0048] A trigger signal is generated (324) when the system determines that the trigger threshold is met (320). For example, if the system compares the size of SLC available to the victim cache partition with the threshold victim cache partition size and determines that the size of SLC available to the victim cache partition is less than or equal to the threshold victim cache partition size, a trigger signal is generated.

[0049] The system disables victim cache mode in the client device cache and SLC (326). The system can disable victim cache mode in the client device cache using the client device cache mode selector, for example, the cache mode selector 114 in Figure 1. For example, the system can disable the write-back of evicted clean cache lines. The system can disable a partition on the SLC as a victim cache partition using the SLC cache mode selector, for example, the SLC cache mode selector 115 in Figure 1. The system assigns new memory requests to normal cache mode (326).

[0050] The system completes outstanding memory requests in victim cache mode (327). For example, the system can maintain a count of outstanding memory requests made to the SLC before the trigger signal was issued but not yet supplied by the SLC. When the system completes each outstanding memory request in victim cache mode, it can subtract that number from the count of outstanding memory requests.

[0051] When the system determines that the count of pending memory requests is zero, it can send a ready signal to the SLC (328). The ready signal indicates that the system has completed the pending memory requests in victim cache mode.

[0052] In some embodiments, the system invalidates clean cache lines within a victim cache partition (330). The system may also make cache lines within a victim cache partition available to other client devices. That is, the system configures the SLC to store cache data from other client devices in the portion of the SLC that acted as a dedicated victim cache partition for client device caches when victim cache mode was enabled in the SLC. In some embodiments, for example, the SLC removes the indication that clean evicted lines can only be hit by transactions within the coherence domain. In some embodiments, the system may use protocols such as snooping to maintain data coherence.

[0053] Regarding dirty cache lines from the client device cache, the system can maintain data coherence. The system can remove the indication that the dirty line belongs to a particular victim cache partition.

[0054] Here, the system can operate in normal cache mode (332). An SLC operating in normal cache mode can drop data from the victim line of a cache miss in the client device's cache. The system can dynamically switch the cache mode from normal cache mode to victim cache mode by performing the steps in Figure 4.

[0055] Figure 4 is a flowchart of an exemplary process 400 for switching the cache mode from normal cache mode to victim cache mode. Process 300 can be performed by any suitable system, for example, system 100 in Figure 1.

[0056] The system is in normal cache mode (332) as described above with reference to Figure 3B. The client device cache and SLC victim cache modes are disabled, but the victim cache partition is enabled. The victim cache partition is not being used as a victim cache. The system has already set thresholds as described with reference to step 316 in Figure 3A.

[0057] The system monitors the status of the victim cache partition (410). For example, the system can monitor the size of the SLC available to the victim cache partition, or the hit rate of the victim cache partition. In some embodiments, the system can monitor the traffic intensity of the victim cache partition, the read / write ratio of the victim cache partition, or the active use cases of the victim cache partition.

[0058] If the system determines that the trigger threshold is not met (412), no trigger signal is generated (414), and the system continues to monitor the status of the victim cache partition (410). For example, if the system compares the size of SLC available to the victim cache partition with the threshold victim cache partition size and determines that the size of SLC available to the victim cache partition is less than or equal to the threshold victim cache partition size, no trigger signal is generated.

[0059] A trigger signal is generated (416) if the system determines that the trigger threshold is met (412). For example, a trigger signal is generated if the system compares the size of the SLC available to the victim cache partition with the threshold victim cache partition size and determines that the size of the SLC available to the victim cache partition is greater than the threshold victim cache partition size.

[0060] The system enables victim cache mode on the client device cache and the SLC (418). The system can enable victim cache mode on the client device cache using the cache mode selector for the client device cache, for example, the cache mode selector 114 in Figure 1. For example, the system can enable the write-back of evicted clean cache lines. The system can enable a partition on the SLC as a victim cache partition using the SLC cache mode selector, for example, the SLC cache mode selector 115 in Figure 1. The system assigns new memory requests to victim cache mode (418).

[0061] The system typically completes pending memory requests in cache mode (419). For example, the system can maintain a count of pending memory requests made to the SLC before a trigger signal was issued but not yet supplied by the SLC. Once the system has completed each pending memory request, it can subtract that number from the count of pending memory requests.

[0062] When the system determines that the count of pending memory requests is zero, it can send a ready signal to the SLC (420). The ready signal indicates that the system has completed the pending memory requests in normal cache mode.

[0063] In some embodiments, the system invalidates clean cache lines within the victim cache partition (422). By invalidating clean cache lines, the system can use the victim cache partition to store other data.

[0064] Here, the system can operate in victim cache mode (424). An SLC operating in victim cache mode can store data from the victim line of a cache miss in the client device's cache in the victim cache partition. The system can dynamically switch the cache mode from victim cache mode to normal cache mode by performing steps 318-332 in Figure 3B.

[0065] The subject matter and functional embodiments described herein can be implemented in digital electronic circuits, tangibly embodied computer software or firmware, or computer hardware, including the structures disclosed herein and their structural equivalents, or one or more combinations thereof. Embodiments of the subject matter described herein may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded in a tangible, non-temporary storage medium, which is executed by or controls the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage board, a random-access memory device or a serial-access memory device, or one or more combinations thereof. Alternatively, or in addition, the program instructions may be encoded in artificially generated transmitted signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0066] The term "data processing device" refers to data processing hardware and encompasses all kinds of devices, machines, and equipment for data processing, including, for example, programmable processors, computers, or multiple processors or multiple computers. A device may be, or further include, a special-purpose logic circuit, such as an FPGA (Field-Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit). Optionally, in addition to hardware, a device may include code that generates an execution environment for computer programs (e.g., code that constitutes processor firmware, protocol stacks, database management systems, operating systems, or one or more combinations thereof).

[0067] When one or more computer systems are configured to perform a particular operation or action, it means that the systems have installed software, firmware, hardware, or a combination thereof that causes the operating systems to perform that operation or action. When one or more computer programs are configured to perform a particular operation or action, it means that one or more programs, when executed by a data processing device, contain instructions that cause the device to perform that operation or action.

[0068] The processes and logic flows described herein can be performed by one or more programmable computers executing one or more computer programs to act on input data and produce outputs, thereby performing their functions. Alternatively, the processes and logic flows can be performed by special-purpose logic circuits, such as FPGAs or ASICs, or by a combination of special-purpose logic circuits and one or more programmed computers.

[0069] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices including EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0070] In addition to the embodiments described above, the following embodiments are also innovative.

[0071] Embodiment 1 is a system, Memory devices and, A system-level cache (SLC) configured to cache memory requests to the aforementioned memory device, The system includes a plurality of client devices configured to provide memory requests to the memory device via the SLC, wherein the plurality of client devices have a primary cache that is higher in the cache hierarchy than the SLC, and the system is configured to operate the primary cache in victim cache mode. The system is configured to operate the SLC in victim cache mode by implementing a portion of the SLC as a victim cache partition for the primary cache, while also supplying memory requests for other client devices among the plurality of other client devices. The system is configured to monitor the performance metrics of the victim cache partition and, when it determines that the performance metrics meet one or more switching criteria, to operate the primary cache and the SLC in normal cache mode instead of victim cache mode. It is a system.

[0072] Embodiment 2 is the system according to claim 1, wherein operating the SLC in victim cache mode includes storing data from the victim line of a cache miss in the primary cache in the victim cache partition of the SLC.

[0073] Embodiment 3 is the system according to any one of Embodiments 1 to 2, wherein operating the SLC in the normal cache mode includes dropping data from the victim line of a cache miss in the primary cache.

[0074] Embodiment 4 is a system according to any one of Embodiments 1 to 3, wherein operating the SLC in the normal cache mode includes storing cache data from other client devices in a portion of the SLC that functions as an exclusive victim cache partition for the primary cache when the SLC is operating in the victim cache mode.

[0075] Embodiment 5 is a system according to any one of Embodiments 1 to 4, wherein the performance metric includes the size of the SLC available to the victim cache partition, and determining that the performance metric satisfies one or more switching criteria includes comparing the size of the SLC available to the victim cache partition with a threshold victim cache partition size.

[0076] Embodiment 6 is a system according to any one of Embodiments 1 to 5, wherein the performance metric includes the hit rate of the victim cache partition, and determining that the performance metric satisfies one or more switching criteria includes comparing the hit rate of the victim cache partition with a threshold hit rate of the victim cache partition.

[0077] Embodiment 7 is the system according to any one of Embodiments 1 to 6, wherein the system is configured to use a trigger signal to cause the primary cache and the SLC to operate in the normal cache mode instead of the victim cache mode.

[0078] Embodiment 8 is the system according to Embodiment 7, wherein the system is configured to maintain a count of outstanding memory requests in the victim cache mode, and outstanding memory requests are memory requests made to the SLC before the trigger signal but not yet processed by the SLC.

[0079] Embodiment 9 is the system described in Embodiment 8, wherein when the system completes an outstanding memory request, it subtracts that amount from the count of outstanding memory requests, and when it determines that the count is zero, it operates the primary cache and the SLC in the normal cache mode.

[0080] Embodiment 10 is the system according to Embodiment 9, wherein when the count is determined to be zero, the system is configured to clear the clean cache lines in the victim cache partition.

[0081] Embodiment 11 is configured such that the system operates the SLC and the primary cache in normal cache mode. The system is configured to monitor the performance metrics of the victim cache partition and, when it determines that the performance metrics meet one or more switching criteria, to operate the primary cache and the SLC in the victim cache mode instead of the normal cache mode. The system is described in any one of Embodiments 1 to 6.

[0082] Embodiment 12 is the system described in Embodiment 11, wherein the system is configured to use a trigger signal to cause the primary cache and the SLC to operate in the victim cache mode instead of the normal cache mode.

[0083] Embodiment 13 is the system according to Embodiment 12, wherein the system is configured to maintain a count of outstanding memory requests in the normal cache mode, and outstanding memory requests are memory requests made to the SLC before the trigger signal but not yet processed by the SLC.

[0084] Embodiment 14 is the system described in Embodiment 13, wherein when the system completes an outstanding memory request, it subtracts that amount from the count of outstanding memory requests, and when it determines that the count is zero, it causes the primary cache and the SLC to operate in the victim cache mode.

[0085] Embodiment 15 is the system according to Embodiment 14, wherein when the count is determined to be zero, the system is configured to clear clean cache lines in the victim cache partition.

[0086] Embodiment 16 is, Memory devices and, A system-level cache (SLC) configured to cache memory requests to the aforementioned memory device, A plurality of client devices configured to provide memory requests to the memory device via the SLC, A method performed by a system including, The aforementioned method, By implementing a portion of the SLC as a victim cache partition for the primary cache, while also supplying memory requests for other client devices among the aforementioned multiple client devices, the SLC is operated in victim cache mode. Monitoring the performance metrics of the aforementioned victim cache partition, When it is determined that the performance metric satisfies one or more switching criteria, the primary cache and the SLC are made to operate in normal cache mode instead of victim cache mode, This method includes [something].

[0087] Embodiment 17 is the method of Embodiment 16, wherein operating the SLC in victim cache mode includes storing data from the victim line of a cache miss in the primary cache in the victim cache partition of the SLC.

[0088] Embodiment 18 is a method according to any one of Embodiments 16 to 17, wherein operating the SLC in the normal cache mode includes dropping data from the victim line of a cache miss in the primary cache.

[0089] Embodiment 19 is a method according to any one of Embodiments 16 to 18, wherein operating the SLC in the normal cache mode includes storing cache data from other client devices in a portion of the SLC that functions as an exclusive victim cache partition for the primary cache when the SLC is operating in the victim cache mode.

[0090] Embodiment 20 is a method according to any one of Embodiments 16 to 19, wherein the performance metric includes the size of the SLC available to the victim cache partition, and determining that the performance metric satisfies one or more switching criteria includes comparing the size of the SLC available to the victim cache partition with a threshold victim cache partition size.

[0091] Embodiment 21 is a method according to any one of Embodiments 16 to 20, wherein the performance metric includes the hit rate of the victim cache partition, and determining that the performance metric satisfies one or more switching criteria includes comparing the hit rate of the victim cache partition with a threshold hit rate of the victim cache partition.

[0092] Embodiment 22 is a method according to any one of Embodiments 16 to 21, wherein the primary cache and the SLC are operated in the normal cache mode instead of the victim cache mode by using a trigger signal.

[0093] Embodiment 23 is an embodiment in which the above method is The further includes maintaining a count of outstanding memory requests in the victim cache mode, where outstanding memory requests are memory requests made to the SLC before the trigger signal but not yet processed by the SLC. This is the method described in Embodiment 22.

[0094] Embodiment 24 is an embodiment in which the above method is When an outstanding memory request is completed, it is deducted from the aforementioned count of outstanding memory requests, When it is determined that the count is zero, the primary cache and the SLC are made to operate in the normal cache mode. This is the method of Embodiment 23, which further includes the following:

[0095] Embodiment 25 is the method of Embodiment 24, further comprising clearing a clean cache line in the victim cache partition when it is determined that the count is zero.

[0096] Embodiment 26 is an embodiment in which the above method is The SLC and the primary cache are operated in normal cache mode, Monitoring the performance metrics of the aforementioned victim cache partition, When it is determined that the performance metric satisfies one or more switching criteria, the primary cache and the SLC are made to operate in victim cache mode instead of normal cache mode, The method according to any one of embodiments 16 to 21, further including the above.

[0097] Embodiment 27 is the method of Embodiment 26, wherein the primary cache and the SLC are operated in the victim cache mode instead of the normal cache mode by using a trigger signal.

[0098] Embodiment 28 is an embodiment in which the above method is The method further includes maintaining a count of outstanding memory requests in the normal cache mode, where outstanding memory requests are memory requests made to the SLC before the trigger signal but not yet processed by the SLC. This is the method described in Embodiment 27.

[0099] Embodiment 29 is an embodiment in which the above method is When an outstanding memory request is completed, it is deducted from the aforementioned count of outstanding memory requests, When it is determined that the count is zero, the primary cache and the SLC are made to operate in victim cache mode, The method described in Embodiment 28 further includes the following:

[0100] Embodiment 30 is the method of Embodiment 29, further comprising, when the count is determined to be zero, clearing a clean cache line in the victim cache partition.

[0101] This specification includes details of many specific embodiments, but these should not be construed as limiting the scope of any invention or claimable content, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Certain features described herein as separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described as a single embodiment may also be implemented in multiple embodiments, individually or in any preferred subcombination. Furthermore, features may be described above as functioning in a particular combination, and even if initially claimed as such, one or more features from the claimed combination may be removed from the combination, and the claimed combination may cover a subcombination or a variation of a subcombination.

[0102] Similarly, while the drawings show operations in a specific order, this should not be understood as requiring that such operations be performed in a specific or sequential order shown, or that all shown operations be performed, in order to obtain the desired results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described can generally be integrated into a single software product or packaged into multiple software products.

[0103] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions described in the claims may be performed in a different order, and this may still yield desirable results. As an example, the process shown in the accompanying drawings does not necessarily require to be performed in the specific order or sequence shown to obtain the desired results. In certain embodiments, multitasking and parallel processing may be advantageous.

Claims

1. It is a system, Memory devices and, A system-level cache (SLC) configured to cache memory requests to the aforementioned memory device, The system includes a plurality of client devices configured to provide memory requests to the memory device via the SLC, wherein the plurality of client devices have a primary cache that is higher in the cache hierarchy than the SLC, and the system is configured to operate the primary cache in victim cache mode. The system is configured to operate the SLC in victim cache mode by implementing a portion of the SLC as a victim cache partition for the primary cache, while also supplying memory requests for other client devices among the plurality of other client devices. The system is configured to monitor the performance metrics of the victim cache partition and, when it determines that the performance metrics meet one or more switching criteria, to operate the primary cache and the SLC in normal cache mode instead of victim cache mode. system.

2. The system according to claim 1, wherein operating the SLC in the victim cache mode includes storing data from the victim line of a cache miss in the primary cache in the victim cache partition of the SLC.

3. The system according to any one of claims 1 to 2, wherein operating the SLC in the normal cache mode includes dropping data from the victim line of a cache miss in the primary cache.

4. The system according to any one of claims 1 to 3, wherein operating the SLC in the normal cache mode includes storing cache data from other client devices in a portion of the SLC that functions as an exclusive victim cache partition for the primary cache when the SLC is operating in the victim cache mode.

5. The performance metric includes the size of the SLC available to the victim cache partition, The system according to any one of claims 1 to 4, wherein determining that the performance metric satisfies one or more switching criteria includes comparing the size of the SLC available by the victim cache partition with a threshold victim cache partition size.

6. The performance metric includes the hit rate of the victim cache partition, The system according to any one of claims 1 to 5, wherein determining that the performance metric satisfies one or more switching criteria includes comparing the hit rate of the victim cache partition with the threshold hit rate of the victim cache partition.

7. The system according to any one of claims 1 to 6, wherein the system is configured to use a trigger signal to cause the primary cache and the SLC to operate in the normal cache mode instead of the victim cache mode.

8. The system is configured to maintain a count of outstanding memory requests in the victim cache mode. The system according to claim 7, wherein an unprocessed memory request is a memory request made to the SLC before the trigger signal but has not yet been processed by the SLC.

9. The system according to claim 8, wherein the system is configured to subtract from the count of outstanding memory requests when an outstanding memory request is completed, and to operate the primary cache and the SLC in the normal cache mode when it is determined that the count is zero.

10. The system according to claim 9, wherein the system is configured to clear a clean cache line in the victim cache partition when it determines that the count is zero.

11. The system is configured to operate the SLC and the primary cache in normal cache mode. The system is configured to monitor the performance metrics of the victim cache partition and, when it determines that the performance metrics meet one or more switching criteria, to operate the primary cache and the SLC in the victim cache mode instead of the normal cache mode. The system according to any one of claims 1 to 6.

12. The system according to claim 11, wherein the system is configured to use a trigger signal to cause the primary cache and the SLC to operate in the victim cache mode instead of the normal cache mode.

13. The system is configured to maintain a count of outstanding memory requests in the normal cache mode. The system according to claim 12, wherein an unprocessed memory request is a memory request made to the SLC before the trigger signal but has not yet been processed by the SLC.

14. The system according to claim 13, wherein the system is configured to subtract from the count of outstanding memory requests when an outstanding memory request is completed, and to operate the primary cache and the SLC in victim cache mode when it is determined that the count is zero.

15. The system according to claim 14, wherein the system is configured to clear clean cache lines in the victim cache partition when it determines that the count is zero.

16. Memory devices and, A system-level cache (SLC) configured to cache memory requests to the aforementioned memory device, A plurality of client devices configured to provide memory requests to the memory device via the SLC, A method performed by a system including, The aforementioned method, By implementing a portion of the SLC as a victim cache partition for the primary cache, while also supplying memory requests for other client devices among the aforementioned multiple client devices, the SLC is operated in victim cache mode. Monitoring the performance metrics of the aforementioned victim cache partition, When it is determined that the performance metric satisfies one or more switching criteria, the primary cache and the SLC are made to operate in normal cache mode instead of victim cache mode. Methods that include...

17. The method according to claim 16, wherein operating the SLC in the victim cache mode includes storing data from the victim line of a cache miss in the primary cache in the victim cache partition of the SLC.

18. The method according to any one of claims 16 to 17, wherein operating the SLC in the normal cache mode includes dropping data from the victim line of a cache miss in the primary cache.

19. The method according to any one of claims 16 to 18, wherein operating the SLC in the normal cache mode includes storing cache data from other client devices in a portion of the SLC that functions as an exclusive victim cache partition for the primary cache when the SLC is operating in the victim cache mode.

20. The performance metric includes the size of the SLC available to the victim cache partition, The method according to any one of claims 16 to 19, wherein determining that the performance metric satisfies one or more switching criteria includes comparing the size of the SLC available by the victim cache partition with a threshold victim cache partition size.

21. The performance metric includes the hit rate of the victim cache partition, The method according to any one of claims 16 to 20, wherein determining that the performance metric satisfies one or more switching criteria includes comparing the hit rate of the victim cache partition with the threshold hit rate of the victim cache partition.

22. The method according to any one of claims 16 to 21, wherein operating the primary cache and the SLC in the normal cache mode instead of the victim cache mode includes using a trigger signal.

23. The aforementioned method, The method further includes maintaining a count of outstanding memory requests in the victim cache mode, An unprocessed memory request is a memory request that was made to the SLC before the trigger signal but has not yet been processed by the SLC. The method according to claim 22.

24. The aforementioned method, When an outstanding memory request is completed, it is deducted from the aforementioned count of outstanding memory requests, When it is determined that the count is zero, the primary cache and the SLC are made to operate in the normal cache mode. The method according to claim 23, further comprising:

25. The method according to claim 24, further comprising clearing a clean cache line in the victim cache partition when it is determined that the count is zero.

26. The aforementioned method, The SLC and the primary cache are operated in normal cache mode, Monitoring the performance metrics of the aforementioned victim cache partition, When it is determined that the performance metric satisfies one or more switching criteria, the primary cache and the SLC are made to operate in victim cache mode instead of normal cache mode, The method according to any one of claims 16 to 21, further comprising:

27. The method according to claim 26, wherein operating the primary cache and the SLC in the victim cache mode instead of the normal cache mode includes using a trigger signal.

28. The aforementioned method, The method further includes maintaining a count of outstanding memory requests in the normal cache mode, An unprocessed memory request is a memory request that was made to the SLC before the trigger signal but has not yet been processed by the SLC. The method according to claim 27.

29. The aforementioned method, When an outstanding memory request is completed, it is deducted from the aforementioned count of outstanding memory requests, When it is determined that the count is zero, the primary cache and the SLC are made to operate in the victim cache mode, The method according to claim 28, further comprising:

30. The method according to claim 29, further comprising clearing a clean cache line in the victim cache partition when it is determined that the count is zero.

Images