Dynamic control of cache mode

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for dynamically selecting a cache mode. One of the methods includes operating a system level cache (SLC) in a victim cache mode by implementing a portion of the SLC as a victim cache partition of a first cache. The method further includes monitoring a performance indicator of the victim cache partition. The first cache and the SLC are operated in a normal cache mode when the performance metric satisfies one or more handover criteria.

Description

Background Art

[0001] This specification refers to cache.

[0002] A system-on-chip (SoC) is an integrated circuit that integrates the different components of a mobile computing device, including a central processing unit (CPU), memory, input / output ports, a cellular radio, and secondary storage. Compared to traditional motherboard-based PC architectures (where the motherboard houses and connects removable or replaceable components), an SoC integrates all of these components into a single integrated circuit. SoCs are commonly used in mobile computing, edge computing, and embedded systems such as smartphones, tablets, WiFi routers, and Internet of Things (IoT) devices.

[0003] A memory hierarchy can allow faster access to data in memory by storing recently used data in intermediate levels of cache. A victim cache can improve the performance of a memory hierarchy by storing data that has been evicted from a cache at a higher level in the hierarchy than the victim cache. For example, in a memory hierarchy with a non-victim cache, if a cache miss occurs at a cache at a higher level than the non-victim cache, the requested data block is brought into the higher level cache, and the evicted block from the higher level cache is discarded. In a memory hierarchy with a victim cache, if a cache miss occurs at a cache at a higher level than the victim cache, the requested data block is brought into the higher level cache, and blocks evicted from the higher level cache to make room for the requested data block are stored in the victim cache. If the evicted data block is needed in a future memory request, it can be found in the victim cache. In some implementations, the evicted block can be discarded from the victim cache after being brought into the higher level cache. Accessing the victim cache from a higher level cache is faster than accessing lower levels of the memory hierarchy, which is beneficial for power and performance.

[0004] However, in some cases, using a victim cache can negatively impact power and performance. When data blocks are sent from a higher-level cache to a victim cache, the overhead of maintaining the victim cache is non-negligible. For example, a cache shared between client devices can adjust the size of the partition allocated to each client device. If the partition used for the victim cache is not large enough to store an evicted data block from a higher-level cache, the evicted block will be discarded by the victim cache partition. Since evicted blocks are not stored by the victim cache partition, there is no benefit in sending evicted data blocks to the victim cache partition. Summary of the Invention

[0005] This specification describes a system that can dynamically select a cache mode. The system operates a system-level cache (SLC) in victim cache mode by implementing a portion of the SLC as a victim cache partition of a client device cache. The system monitors performance metrics of the victim cache partition, and if the performance metrics meet one or more switching criteria, the system operates the client device cache and the SLC in normal cache mode. When the system is in normal cache mode, the system monitors performance metrics of the victim cache partition, and if the performance metrics meet one or more switching criteria, the system operates the client device cache and the SLC in victim cache mode.

[0006] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.

[0007] The system can improve runtime performance by switching cache modes to improve system performance. For example, the system can switch from victim cache mode to normal cache mode. The system can determine to switch cache modes based on any appropriate performance metric (e.g., when a metric measuring the overhead of maintaining the victim cache partition meets a threshold). For example, in applications such as streaming video or audio, where the hit rate of the victim cache partition can be close to zero, or where the victim cache partition is not large enough to store more evicted lines, higher levels of cache can evict clean cache lines and send them to the victim cache partition without gaining any benefit from the victim cache partition. The system can disable victim cache mode at runtime and thereby reduce overhead by monitoring performance metrics such as the size of the SLC available to the victim cache partition or the hit rate of the victim cache partition and determining whether the performance metric meets one or more switching criteria. Disabling victim cache mode can also reduce the amount of power consumed by sending cache lines to the victim cache partition, which is particularly important for mobile devices.

[0008] When the system determines that a metric measuring the overhead of maintaining a victim cache partition meets a threshold, the system can improve runtime performance and power by re-enabling victim cache mode.

[0009] The system can maintain data consistency and limit the impact on system traffic when switching from victim cache mode to normal cache mode and vice versa. For example, after the system determines that a metric measuring the overhead of maintaining a victim cache partition meets a threshold and therefore determines to switch modes, there may still be outstanding memory requests destined for the SLC that were sent before the switch was determined but are still propagating through the memory hierarchy and should therefore be executed in the original cache mode. The system can use trigger signals and a count of outstanding memory requests to ensure that outstanding memory requests are completed in the mode expected by the sender of the memory request. The system can thus switch between modes in a coordinated manner.

[0010] A system using a smaller cache with switched cache modes can maintain the same level of performance as a system with a larger cache that does not switch cache modes. A system using a cache with switched cache modes can also achieve better performance than a system with a cache of the same size that does not switch cache modes. For example, a system can improve performance or maintain performance by switching cache modes at runtime.

[0011] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a block diagram of an example system for dynamically controlling cache modes.

[0013] Figure 2 is a flow chart of an example process for switching from victim cache mode to normal cache mode.

[0014] Figures 3A to 3B is a flow chart of an example process for dynamically switching a cache mode from a victim cache mode to a normal cache mode.

[0015] Figure 4 is a flow chart of an example process for dynamically switching a cache mode from a normal cache mode to a victim cache mode.

[0016] Like reference numbers and names in the various drawings indicate like elements. DETAILED DESCRIPTION

[0017] Figure 1 is a block diagram of an example system 100 for dynamically controlling a cache mode. System 100 includes various components such as client devices 102a through 102n (collectively, “client devices 102”), a first cache 104, a system level cache (SLC) 106, and a memory device 110.

[0018] A system-level cache (SLC) is any suitable device for caching data retrieved from or to be stored in memory for multiple different hardware devices in a system. In other words, different cache lines of the SLC can store data belonging to different hardware devices, such as different processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), microcontrollers, machine learning accelerators, or image processors, to name a few. Figure 1 As shown, the SLC can be part of a larger cache hierarchy. For some hardware devices, one or more additional cache levels may exist between the hardware device and the SLC. Other hardware devices can use the SLC directly without intermediate cache levels.

[0019] A cache higher in the cache hierarchy is closer to a different hardware device, i.e., has fewer cache levels between it and the hardware device than a cache lower in the cache hierarchy. A cache lower in the cache hierarchy is closer to memory, i.e., has fewer cache levels between it and memory than a cache higher in the cache hierarchy.

[0020] The client devices 102a to 102n may be any type of hardware device configured to provide memory request 103 to memory device 110 via SLC 106, such as a central processing unit (CPU), digital signal processor (DSP), neural processing unit (NPU), or cache.

[0021] Each client device 102a-102n may have a cache higher in the cache hierarchy than the SLC 106. In the example system 100, client 1 102a has client cache 1 104.

[0022] SLC 106 may be any cache configured to cache memory requests to memory device 110 and may operate in a victim cache mode by implementing a portion as victim cache partition 108 while also servicing memory requests of other client devices.

[0023] In the example system 100, the SLC 106 includes a victim cache partition 108 and partitions for client 2 102b through client N 102n, namely, client partition 2 109b through client partition N 109n. The SLC is shared among the client devices 102, and the size of the partitions allocated to each of the client partitions 109b through 109n and the victim cache partition 108 can be adjusted.

[0024] In some implementations, the SLC 106 may implement multiple portions as victim cache partitions for multiple client devices. In some implementations, the SLC 106 may include a status monitor, counters, and an SLC cache mode selector for each victim cache partition. For example, each victim cache partition may be in its own cache mode independent of other victim cache partitions. Each client and its corresponding client partition may determine the cache mode, whether victim cache mode or normal cache mode. For example, client partition 2 109b may be the victim cache partition of client 2 102b and client cache 2 (not shown) of client 2 102b, regardless of whether victim cache partition 108 is actually configured in victim cache mode or normal cache mode. Each victim cache partition (e.g., victim cache partition 108) is treated as an extension of its corresponding client's internal cache (e.g., client cache 1), and visibility and data consistency may conform to the theory of operation as if clean, evicted victim cache data were in the client's internal cache.

[0025] Memory device 110 may be any type of computing device that stores data for memory requests, such as cache, random access memory (RAM), or dynamic random access memory (DRAM).

[0026] The client cache 1 104 includes a cache mode selector 114 that the system can use to control the cache mode of the client cache 1 104. The cache mode selector 114 is a software or hardware entity that can control the cache mode of the client cache 1 104, for example, by enabling or disabling a victim cache mode of the client cache 1 104. Possible cache modes can include, for example, a victim cache mode (victim cache mode is enabled) or a normal cache mode (victim cache mode is disabled).

[0027] In the event of a cache miss in client cache 1 104, data can be retrieved from a lower level of the memory hierarchy, such as SLC 106 or memory device 110. The data retrieved from the lower level of the memory hierarchy replaces the data in client cache 1 104 (the victim line). In normal cache mode, the system silently discards the data in the clean victim line. In victim cache mode, the system stores the data from the victim line in a lower level cache or victim cache.

[0028] The system can implement a portion of the SLC 106 as a victim cache partition 108 of the client cache 1 104. The system can use an SLC cache mode selector 115 to enable or disable a victim cache mode for the victim cache partition 108. The SLC cache mode selector 115 is a software or hardware entity that can control the cache mode of the victim cache partition 108, for example, by enabling or disabling the victim cache mode for the victim cache partition 108. Possible cache modes can include, for example, a victim cache mode (victim cache mode is enabled) or a normal cache mode (victim cache mode is disabled).

[0029] In the event of a cache miss in client cache 1 104, data can be retrieved from a lower level of the memory hierarchy, such as SLC 106 or memory device 110. The data retrieved from the lower level of the memory hierarchy replaces the victim line in client cache 1 104. In normal cache mode, the system silently discards the data in the clean victim line. In victim cache mode, the system stores the data from the victim line in victim cache partition 108.

[0030] The system can use the status monitor 112 to monitor performance metrics of the victim cache partition 108. For example, the performance metric can include the size of the SLC 106 available for use by the victim cache partition 108. In some implementations, the performance metric can include a hit rate for the victim cache partition 108. In some implementations, the performance metric can also measure the intensity of traffic to the victim cache partition 108, the read / write ratio of the victim cache partition 108, or the active use case of the victim cache partition 108.

[0031] If the SLC 106 is in victim cache mode, the system may operate the client cache 1 104 and the SLC 106 in normal cache mode instead of victim cache mode upon determining that the performance metric satisfies one or more switching criteria. For example, the switching criterion may be a threshold victim cache partition size or a threshold hit rate for the victim cache partition. The system may use a status monitor 112 to send a trigger signal 116 to trigger the process of switching from victim cache mode to normal cache mode. To maintain data consistency when switching from victim cache mode to normal cache mode, the system may use a counter 118 to maintain a count of outstanding memory requests that were initiated to the SLC but not yet serviced by the SLC before the trigger signal was issued. This will be referenced below. Figure 2 and FIG3 describe the switching process in more detail.

[0032] If the SLC 106 is operating in normal cache mode, the system may store cache data from Client 2 102b through Client N 102n in the victim cache partition 108 (the portion of the SLC 106 that serves as an exclusive victim cache partition of Client Cache 1 104 when operating the SLC 106 in victim cache mode).

[0033] If the SLC 106 is in normal cache mode, the system may operate the client cache 1 104 and the SLC 106 in victim cache mode instead of normal cache mode upon determining that the performance metric satisfies one or more switching criteria. For example, the switching criterion may be a threshold victim cache partition size or a threshold hit rate for the victim cache partition. The system may use a status monitor 112 to send a trigger signal 116 to trigger the process of switching from normal cache mode to victim cache mode. In order to maintain data consistency when switching from normal cache mode to victim cache mode, the system may use a counter 118 to maintain a count of outstanding memory requests that were initiated to the SLC but not yet serviced by the SLC before the trigger signal was issued. This will be referred to below. Figure 4 The handover process is described in more detail.

[0034] Figure 2 is a flow chart of an example process 200 for switching from victim cache mode to normal cache mode. Process 200 may be performed by any suitable system, such as Figure 1 The system 100 is executed.

[0035] The system may operate the first cache in a victim cache mode, e.g. Figure 1 For example, the system may use a cache mode selector of the first cache, such as Figure 1 The cache mode selector 114 is used to enable the victim cache mode on the first cache.

[0036] The system can operate the SLC in a victim cache mode by implementing a portion of the SLC as a victim cache partition of the first cache (210). The SLC can also serve other client devices, such as Figure 1 The SLC operating in the victim cache mode may store data of a victim line of a cache miss from the first cache in a victim cache partition.

[0037] The system can monitor a performance metric of the victim cache partition (220). In some implementations, the performance metric can include the size of the SLC available to the victim cache partition. In some implementations, the performance metric can include a hit rate of the victim cache partition.

[0038] When it is determined that the performance metric satisfies one or more switching criteria, the system can operate the first cache and the SLC in a normal cache mode (230). The SLC operating in the normal cache mode can discard data from a victim line of a cache miss in the first cache.

[0039] In implementations where the performance metric includes a size of an SLC available for use by the victim cache partition, determining that the performance metric satisfies one or more switching criteria may include comparing the size of the SLC available for use by the victim cache partition to a threshold victim cache partition size. In implementations where the performance metric includes a hit rate of the victim cache partition, determining that the performance metric satisfies one or more switching criteria may include comparing the hit rate of the victim cache partition to a threshold hit rate of the victim cache partition.

[0040] In some implementations, the switching criteria may be a software programmable parameter. In some implementations, the switching criteria may be defined at design time.

[0041] The system can operate the first cache and the SLC in a normal cache mode by using a trigger signal. Figure 3B The trigger signal is described in more detail.

[0042] Figures 3A to 3B is a flow chart of an example process 300 for dynamically switching a cache mode from a victim cache mode to a normal cache mode. The process 300 may be performed by any suitable system, such as Figure 1 system 100 to execute.

[0043] Figure 3A The following illustrates how the system can be initially configured to enable dynamic switching of the cache mode from victim cache mode to normal cache mode. For example, when the system boots up, the system can be configured to be in victim cache mode. The system can perform Figure 3B In some implementations, when the system boots up, the system can be configured to be in normal cache mode and perform Figure 4 to dynamically switch the cache mode from normal cache mode to victim cache mode.

[0044] After the system boots up (310), the system configures victim cache support for the SLC (312). Configuring victim cache support for the SLC may include implementing a portion of the SLC as a client device cache, e.g. Figure 1104 in the victim cache partition of the client cache 1. Importantly, both the SLC and the client device cache should be consistent in cache mode. In some implementations, for example, configuring victim cache support for the SLC may include reading and writing values ​​to registers, or other interactions between components of the SLC.

[0045] The system enables victim cache mode on the client device cache and the SLC, and enables a partition on the SLC as a victim cache partition (314). The SLC operating in victim cache mode can store data from a victim line of a cache miss from the client device cache in the victim cache partition. The system can use a cache mode selector of the client device cache, such as Figure 1 The system can use the SLC cache mode selector 114 to enable victim cache mode on the client device cache. For example, the system can enable write-back of clean cache lines that are evicted. The system can use the SLC cache mode selector, such as Figure 1 The SLC cache mode selector 115 in the SLC cache mode selector 115 can be used to enable a partition on the SLC as a victim cache partition. For example, a clean cache line written back by the client device cache can be stored in the victim cache partition. The system can also invalidate cache lines read from the victim cache partition by the client device cache.

[0046] The system configures a trigger threshold (316). For example, the trigger threshold can be a threshold victim cache partition size or a threshold hit rate of the victim cache partition. The system can use the trigger threshold to define the switching criteria.

[0047] The system can now operate in victim cache mode (317). The SLC operating in victim cache mode can store data from a victim line of a cache miss from a client device cache in a victim cache partition. The system can perform Figure 3B to dynamically switch the cache mode from victim cache mode to normal cache mode.

[0048] Figure 3B It shows how the system can dynamically switch the cache mode from victim cache mode to normal cache mode. Figure 3A The victim cache mode (317).

[0049] The system monitors the status of the victim cache partition (318). The system can monitor a performance indicator that describes the status of the victim cache partition. For example, the system can monitor the size of the SLC available to the victim cache partition or the hit rate of the victim cache partition. In some implementations, the system can monitor the intensity of the victim cache partition's traffic, the victim cache partition's read / write ratio, or the victim cache partition's active use cases.

[0050] If the system determines that the trigger threshold is not met (320), then the trigger signal is not issued (322), and the system continues to monitor the status of the victim cache partition (318). For example, if the system compares the size of the SLC available for use by the victim cache partition to the threshold victim cache partition size and determines that the size of the SLC available for use by the victim cache partition is greater than the threshold victim cache partition size, then the trigger signal will not be issued.

[0051] If the system determines that the trigger threshold is met (320), a trigger signal is issued (324). For example, if the system compares the size of the SLC available for use by the victim cache partition to a threshold victim cache partition size and determines that the size of the SLC available for use by the victim cache partition is less than or equal to the threshold victim cache partition size, a trigger signal is issued.

[0052] The system disables the victim cache mode on the client device cache and the SLC (326). The system can use the cache mode selector of the client device cache, such as Figure 1 Cache mode selector 114 to disable the victim cache mode on the client device cache. For example, the system can disable writeback of clean cache lines that are evicted. The system can use the SLC cache mode selector, such as Figure 1 The SLC cache mode selector 115 in the IO_CONNECT_SLC is used to disable the partition on the SLC as a victim cache partition. The system assigns the new memory request to the normal cache mode (326).

[0053] The system completes outstanding memory requests in victim cache mode (327). For example, the system can maintain a count of outstanding memory requests initiated to the SLC but not yet serviced by the SLC before the trigger signal is issued. Each time an outstanding memory request is completed in victim cache mode, the system can subtract from the count of outstanding memory requests.

[0054] When determining that the count of outstanding memory requests is zero, the system may send a ready signal to the SLC (328). The ready signal indicates that the system has completed outstanding memory requests in victim cache mode.

[0055] In some implementations, the system invalidates clean cache lines in the victim cache partition (330). The system can also make cache lines in the victim cache partition available to other client devices. That is, the system configures the SLC to store cache data from other client devices in a portion of the SLC that serves as an exclusive victim cache partition for client device cache when victim cache mode is enabled on the SLC. In some implementations, for example, the SLC removes an indication that clean evicted lines can only be hit by transactions within the coherency domain. In some implementations, the system can use a protocol such as snooping to maintain data coherency.

[0056] For dirty cache lines from the client device cache, the system can maintain data consistency.The system can remove the indication that the dirty line belongs to a specific victim cache partition.

[0057] The system can now operate in normal cache mode (332). The SLC operating in normal cache mode can discard data from the victim line of the cache miss from the client device cache. The system can perform Figure 4 to dynamically switch the cache mode from normal cache mode to victim cache mode.

[0058] Figure 4 is a flow chart of an example process 400 for dynamically switching a cache mode from a normal cache mode to a victim cache mode. The process 300 may be implemented by any suitable system, such as Figure 1 system 100 to execute.

[0059] The system is as above reference Figure 3B The normal cache mode (332) is described. The victim cache mode of the client device cache and the SLC is disabled, but the victim cache partition is enabled. The victim cache partition is not used as a victim cache. The system has referenced Figure 3A In step 316, the threshold value as described above is configured.

[0060] The system monitors the status of the victim cache partition (410). For example, the system can monitor the size of the SLC available to the victim cache partition or the hit rate of the victim cache partition. In some implementations, the system can monitor the intensity of the victim cache partition's traffic, the victim cache partition's read / write ratio, or the victim cache partition's active use cases.

[0061] If the system determines that the trigger threshold is not met (412), then the trigger signal is not issued (414), and the system continues to monitor the status of the victim cache partition (410). For example, if the system compares the size of the SLC available for use by the victim cache partition to the threshold victim cache partition size and determines that the size of the SLC available for use by the victim cache partition is less than or equal to the threshold victim cache partition size, then the trigger signal will not be issued.

[0062] If the system determines that the trigger threshold is met (412), a trigger signal is issued (416). For example, if the system compares the size of the SLC available for use by the victim cache partition with a threshold victim cache partition size and determines that the size of the SLC available for use by the victim cache partition is greater than the threshold victim cache partition size, a trigger signal is issued.

[0063] The system enables the client device cache and victim cache mode on the SLC (418). The system can use the cache mode selector of the client device cache, such as Figure 1 Cache mode selector 114 to enable victim cache mode on the client device cache. For example, the system can enable write-back of clean cache lines that are evicted. The system can use the SLC cache mode selector, such as Figure 1 The system assigns the new memory request to the victim cache mode (418).

[0064] The system completes outstanding memory requests in normal cache mode (419). For example, the system may maintain a count of outstanding memory requests initiated to the SLC but not yet serviced by the SLC before the trigger signal is issued. Each time an outstanding memory request is completed, the system may subtract from the count of outstanding memory requests.

[0065] When it is determined that the count of outstanding memory requests is zero, the system may send a ready signal to the SLC (420). The ready signal indicates that the system has completed outstanding memory requests in normal cache mode.

[0066] In some implementations, the system invalidates the clean cache line in the victim cache partition (422). Invalidating the clean cache line allows the system to use the victim cache partition to store other data.

[0067] The system can now operate in victim cache mode (424). The SLC operating in victim cache mode can store data from a victim line of a cache miss from a client device cache in a victim cache partition. The system can perform Figure 3BSteps 318 to 332 are performed to dynamically switch the cache mode from the victim cache mode to the normal cache mode.

[0068] Embodiments of the subject matter and functional operations described in this specification may be implemented in digital electronic circuit systems, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more thereof. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by a data processing device or for controlling the operation of a data processing device. A computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more thereof. Alternatively or in addition, program instructions may be encoded on an artificially generated propagated signal (e.g., a machine-generated electrical, optical, or electromagnetic signal) that is generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0069] The term "data processing equipment" refers to data processing hardware and includes all types of equipment, devices and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. The equipment may also be or further include special-purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the equipment may optionally include code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0070] For a system of one or more computers to be configured to perform specific operations or actions, it is meant that the system has installed thereon software, firmware, hardware, or a combination thereof that, in operation, causes the system to perform those operations or actions. For one or more computer programs to be configured to perform specific operations or actions, it is meant that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform the operations or actions.

[0071] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry (e.g., an FPGA or an ASIC), or by a combination of special purpose logic circuitry and one or more programmed computers.

[0072] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0073] In addition to the embodiments described above, the following embodiments are also innovative:

[0074] Embodiment 1 is a system comprising:

[0075] memory device;

[0076] a system level cache (SLC) configured to cache memory requests to the memory device; and

[0077] a plurality of client devices configured to provide memory requests to the memory device through the SLC, wherein a first client device has a first cache higher in a cache hierarchy than the SLC, wherein the system is configured to operate the first cache in a victim cache mode,

[0078] wherein the system is configured to operate the SLC in a victim cache mode by implementing a portion of the SLC as a victim cache partition of the first cache while also servicing memory requests of other client devices of the plurality of other client devices,

[0079] And wherein the system is configured to monitor a performance metric of the victim cache partition and, upon determining that the performance metric satisfies one or more switching criteria, operate the first cache and the SLC in a normal cache mode rather than the victim cache mode.

[0080] Embodiment 2 is the system of claim 1 , wherein operating the SLC in the victim cache mode comprises storing data of a victim line from a cache miss of the first cache in the victim cache partition of the SLC.

[0081] Embodiment 3 is the system of any one of embodiments 1-2, wherein operating the SLC in the normal cache mode includes discarding data from a victim line of a cache miss of the first cache.

[0082] Embodiment 4 is a system as described in any of embodiments 1 to 3, wherein operating the SLC in the normal cache mode includes storing cache data from other client devices in a portion of the SLC, which portion serves as an exclusive victim cache partition of the first cache when operating the SLC in the victim cache mode.

[0083] Embodiment 5 is a system as described in any of embodiments 1 to 4, wherein the performance metric includes the size of the SLC available for use by the victim cache partition, and wherein determining that the performance metric meets one or more switching criteria includes comparing the size of the SLC available for use by the victim cache partition with a threshold victim cache partition size.

[0084] Embodiment 6 is a system as described in any of embodiments 1 to 5, wherein the performance metric includes a hit rate of the victim cache partition, and wherein determining that the performance metric meets one or more switching criteria includes comparing the hit rate of the victim cache partition with a threshold hit rate of the victim cache partition.

[0085] Embodiment 7 is the system of any one of embodiments 1 to 6, wherein the system is configured to use a trigger signal to operate the first cache and the SLC in the normal cache mode instead of the victim cache mode.

[0086] Embodiment 8 is a system as described in embodiment 7, wherein the system is configured to maintain a count of outstanding memory requests in the victim cache mode, wherein the outstanding memory requests are memory requests initiated to the SLC before the trigger signal but have not yet been serviced by the SLC.

[0087] Embodiment 9 is the system of embodiment 8, wherein the system is configured to subtract from a count of outstanding memory requests when completing the outstanding memory request, and when determining that the count is zero, operate the first cache and the SLC in the normal cache mode.

[0088] Embodiment 10 is the system of embodiment 9, wherein upon determining that the count is zero, the system is configured to flush clean cache lines in the victim cache partition.

[0089] Embodiment 11 is the system of any one of embodiments 1 to 6, wherein the system is configured to operate the SLC and the first cache in a normal cache mode,

[0090] And wherein the system is configured to monitor a performance metric of the victim cache partition and, upon determining that the performance metric meets one or more switching criteria, operate the first cache and the SLC in the victim cache mode rather than the normal cache mode.

[0091] Embodiment 12 is the system of embodiment 11, wherein the system is configured to use a trigger signal to operate the first cache and the SLC in the victim cache mode instead of the normal cache mode.

[0092] Embodiment 13 is a system as described in embodiment 12, wherein the system is configured to maintain a count of outstanding memory requests in the normal cache mode, wherein the outstanding memory requests are memory requests initiated to the SLC before the trigger signal but not yet serviced by the SLC.

[0093] Embodiment 14 is the system of embodiment 13, wherein the system is configured to subtract from a count of outstanding memory requests when completing the outstanding memory request, and when determining that the count is zero, operate the first cache and the SLC in the victim cache mode.

[0094] Embodiment 15 is the system of embodiment 14, wherein upon determining that the count is zero, the system is configured to flush clean cache lines in the victim cache partition.

[0095] Embodiment 16 is a method performed by a system, the system comprising:

[0096] memory device;

[0097] a system level cache (SLC) configured to cache memory requests to the memory device; and

[0098] a plurality of client devices configured to provide memory requests to the memory device via the SLC;

[0099] The method includes:

[0100] operating the SLC in a victim cache mode by implementing a portion of the SLC as a victim cache partition of the first cache while also servicing memory requests of other client devices of the plurality of other client devices;

[0101] monitoring performance metrics of the victim cache partition; and

[0102] Upon determining that the performance indicator satisfies one or more switching criteria, the first cache and the SLC are operated in a normal cache mode instead of the victim cache mode.

[0103] Embodiment 17 is the method of embodiment 16, wherein operating the SLC in the victim cache mode includes storing data of a victim line from a cache miss of the first cache in the victim cache partition of the SLC.

[0104] Embodiment 18 is the method of any one of embodiments 16-17, wherein operating the SLC in the normal cache mode includes discarding data from a victim line of a cache miss of the first cache.

[0105] Embodiment 19 is a method as described in any of embodiments 16 to 18, wherein operating the SLC in the normal cache mode includes storing cache data from other client devices in a portion of the SLC, which serves as an exclusive victim cache partition of the first cache when operating the SLC in the victim cache mode.

[0106] Embodiment 20 is a method as described in any one of embodiments 16 to 19, wherein the performance metric includes the size of the SLC available for use by the victim cache partition, and wherein determining that the performance metric meets one or more switching criteria includes comparing the size of the SLC available for use by the victim cache partition with a threshold victim cache partition size.

[0107] Embodiment 21 is a method as described in any of embodiments 16 to 20, wherein the performance metric includes a hit rate of the victim cache partition, and wherein determining that the performance metric meets one or more switching criteria includes comparing the hit rate of the victim cache partition with a threshold hit rate of the victim cache partition.

[0108] Embodiment 22 is the method of any one of embodiments 16 to 21, wherein operating the first cache and the SLC in the normal cache mode instead of the victim cache mode includes using a trigger signal.

[0109] Embodiment 23 is the method of embodiment 22, wherein the method further comprises:

[0110] A count of outstanding memory requests is maintained in the victim cache mode, wherein an outstanding memory request is a memory request initiated to the SLC before the trigger signal but not yet serviced by the SLC.

[0111] Embodiment 24 is the method of embodiment 23, wherein the method further comprises:

[0112] subtracting from a count of outstanding memory requests upon completion of the outstanding memory request; and

[0113] When it is determined that the count is zero, the first cache and the SLC are operated in the normal cache mode.

[0114] Embodiment 25 is the method of embodiment 24, wherein upon determining that the count is zero, the method further comprises flushing a clean cache line in the victim cache partition.

[0115] Embodiment 26 is the method of any one of embodiments 16 to 21, wherein the method further comprises:

[0116] operating the SLC and the first cache in a normal cache mode;

[0117] monitoring performance metrics of the victim cache partition; and

[0118] Upon determining that the performance indicator satisfies one or more switching criteria, the first cache and the SLC are operated in the victim cache mode instead of the normal cache mode.

[0119] Embodiment 27 is the method of embodiment 26, wherein operating the first cache and the SLC in the victim cache mode instead of the normal cache mode includes using a trigger signal.

[0120] Embodiment 28 is the method of embodiment 27, wherein the method further comprises:

[0121] A count of outstanding memory requests is maintained in the normal cache mode, wherein the outstanding memory requests are memory requests initiated to the SLC before the trigger signal but not yet serviced by the SLC.

[0122] Embodiment 29 is the method of embodiment 28, wherein the method further comprises:

[0123] subtracting from a count of outstanding memory requests upon completion of the outstanding memory request; and

[0124] When it is determined that the count is zero, the first cache and the SLC are operated in the victim cache mode.

[0125] Embodiment 30 is the method of embodiment 29, wherein upon determining that the count is zero, the method further comprises flushing a clean cache line in the victim cache partition.

[0126] Although this specification contains many specific implementation details, these details should not be interpreted as limitations on the scope of any invention or the scope that may be claimed, but should be interpreted as descriptions of features that may be unique to a particular embodiment of a particular invention. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable subcombination. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may involve a subcombination or a variant of the subcombination.

[0127] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all illustrated operations be performed to achieve the desired results. In certain situations, multitasking and parallel processing can be advantageous. Furthermore, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0128] Specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

Claims

1. A system comprising: memory device; a system level cache (SLC) configured to cache memory requests to the memory device; a plurality of client devices configured to provide memory requests to the memory device through the SLC, wherein a first client device has a first cache higher in a cache hierarchy than the SLC, wherein the system is configured to operate the first cache in a victim cache mode, wherein the system is configured to operate the SLC in a victim cache mode by implementing a portion of the SLC as a victim cache partition of the first cache while also servicing memory requests of other client devices of the plurality of other client devices, And wherein the system is configured to monitor performance indicators of the victim cache partition and operate the first cache and the SLC in normal cache mode rather than the victim cache mode when it is determined that the performance indicators meet one or more switching criteria.

2. The system of claim 1, wherein operating the SLC in the victim cache mode comprises storing data of a victim line of a cache miss from the first cache in the victim cache partition of the SLC. 3 . The system of claim 1 , wherein operating the SLC in the normal cache mode comprises discarding data from a victim line of a cache miss of the first cache.

4. The system of any one of claims 1 to 3, wherein operating the SLC in the normal cache mode includes storing cache data from other client devices in a portion of the SLC, the portion serving as an exclusive victim cache partition of the first cache when operating the SLC in the victim cache mode.

5. The system of any one of claims 1 to 4, wherein the performance metric comprises a size of the SLC available for use by the victim cache partition, and wherein determining that the performance metric satisfies one or more switching criteria comprises comparing the size of the SLC available for use by the victim cache partition to a threshold victim cache partition size.

6. The system of any one of claims 1 to 5, wherein the performance metric comprises a hit rate of the victim cache partition, and wherein determining that the performance metric satisfies one or more switching criteria comprises comparing the hit rate of the victim cache partition to a threshold hit rate of the victim cache partition. 7 . The system of claim 1 , wherein the system is configured to use a trigger signal to operate the first cache and the SLC in the normal cache mode instead of the victim cache mode.

8. The system of claim 7, wherein the system is configured to maintain a count of outstanding memory requests in the victim cache mode, wherein an outstanding memory request is a memory request initiated to the SLC before the trigger signal but not yet serviced by the SLC.

9. The system of claim 8, wherein the system is configured to subtract from a count of outstanding memory requests when the outstanding memory requests are completed, and to operate the first cache and the SLC in the normal cache mode when the count is determined to be zero.

10. The system of claim 9, wherein upon determining that the count is zero, the system is configured to flush clean cache lines in the victim cache partition.

11. The system of any one of claims 1 to 6, wherein the system is configured to operate the SLC and the first cache in a normal cache mode, And wherein the system is configured to monitor a performance metric of the victim cache partition and, upon determining that the performance metric meets one or more switching criteria, operate the first cache and the SLC in the victim cache mode rather than the normal cache mode.

12. The system of claim 11, wherein the system is configured to use a trigger signal to operate the first cache and the SLC in the victim cache mode instead of the normal cache mode.

13. The system of claim 12, wherein the system is configured to maintain a count of outstanding memory requests in the normal cache mode, wherein an outstanding memory request is a memory request initiated to the SLC before the trigger signal but not yet serviced by the SLC.

14. The system of claim 13, wherein the system is configured to subtract from a count of outstanding memory requests when the outstanding memory requests are completed, and to operate the first cache and the SLC in the victim cache mode when the count is determined to be zero.

15. The system of claim 14, wherein upon determining that the count is zero, the system is configured to flush clean cache lines in the victim cache partition.

16. A method performed by a system, the system comprising: memory device; a system level cache (SLC) configured to cache memory requests to the memory device; as well as a plurality of client devices configured to provide memory requests to the memory device via the SLC; The method comprises: operating the SLC in a victim cache mode by implementing a portion of the SLC as a victim cache partition of the first cache while also servicing memory requests of other client devices of the plurality of other client devices; monitoring performance metrics of the victim cache partition; and Upon determining that the performance indicator satisfies one or more switching criteria, the first cache and the SLC are operated in a normal cache mode instead of the victim cache mode. 17 . The method of claim 16 , wherein operating the SLC in the victim cache mode comprises storing data of a victim line of a cache miss from the first cache in the victim cache partition of the SLC.

18. The method of any one of claims 16 to 17, wherein operating the SLC in the normal cache mode comprises discarding data from a victim line of a cache miss of the first cache.

19. The method of any one of claims 16 to 18, wherein operating the SLC in the normal cache mode includes storing cache data from other client devices in a portion of the SLC, the portion serving as an exclusive victim cache partition of the first cache when operating the SLC in the victim cache mode.

20. The method of any one of claims 16 to 19, wherein the performance metric comprises a size of the SLC available for use by the victim cache partition, and wherein determining that the performance metric satisfies one or more switching criteria comprises comparing the size of the SLC available for use by the victim cache partition to a threshold victim cache partition size.

21. A method as described in any one of claims 16 to 20, wherein the performance metric includes a hit rate of the victim cache partition, and wherein determining that the performance metric meets one or more switching criteria includes comparing the hit rate of the victim cache partition with a threshold hit rate of the victim cache partition.

22. The method of any one of claims 16 to 21, wherein operating the first cache and the SLC in the normal cache mode rather than the victim cache mode comprises using a trigger signal.

23. The method of claim 22, wherein the method further comprises: A count of outstanding memory requests is maintained in the victim cache mode, wherein an outstanding memory request is a memory request initiated to the SLC before the trigger signal but not yet serviced by the SLC.

24. The method of claim 23, wherein the method further comprises: subtracting from a count of outstanding memory requests upon completion of the outstanding memory request; and When it is determined that the count is zero, the first cache and the SLC are operated in the normal cache mode.

25. The method of claim 24, wherein upon determining that the count is zero, the method further comprises flushing clean cache lines in the victim cache partition.

26. The method of any one of claims 16 to 21, wherein the method further comprises: operating the SLC and the first cache in a normal cache mode; monitoring performance metrics of the victim cache partition; as well as Upon determining that the performance indicator satisfies one or more switching criteria, the first cache and the SLC are operated in the victim cache mode instead of the normal cache mode.

27. The method of claim 26, wherein operating the first cache and the SLC in the victim cache mode instead of the normal cache mode comprises using a trigger signal.

28. The method of claim 27, wherein the method further comprises: A count of outstanding memory requests is maintained in the normal cache mode, wherein an outstanding memory request is a memory request initiated to the SLC before the trigger signal but not yet serviced by the SLC.

29. The method of claim 28, wherein the method further comprises: subtracting from a count of outstanding memory requests upon completion of the outstanding memory request; and When it is determined that the count is zero, the first cache and the SLC are operated in the victim cache mode.

30. The method of claim 29, wherein upon determining that the count is zero, the method further comprises flushing clean cache lines in the victim cache partition.

Images