Running average cache hit rate

Abstract

The disclosed apparatus includes a first register that stores a cumulative delta value and a second register that stores an average cache hit rate. The apparatus also includes control circuitry that calculates a cache hit rate and updates the cumulative delta value based on the cache hit rate and the average cache hit rate. The control circuitry also updates an average cache hit rate based on the updated cumulative delta value, and may update a cache allocation policy based on the updated average cache hit rate. Various other methods, systems, and computer readable media are also disclosed.

Description

BACKGROUND

[0001] Managing a cache involves managing a cache allocation policy (e.g., reserving portions of the cache) to improve performance. Cache thrashing (e.g., consecutive cache misses that require reads from memory) decreases performance and is a factor for determining a cache allocation policy. Detecting cache thrashing involves tracking a running average cache hit rate, which further involves tracking a history of cache hit rates. However, tracking this history of cache hit rates can require sufficient processor storage, which can be expensive. BRIEF DESCRIPTION OF DRAWINGS

[0002] The accompanying drawings illustrate a number of example implementations and are a part of the detailed description. Together with the following description, these drawings demonstrate and explain various principles of the present disclosure.

[0003] Figure 1 is a block diagram of an example system for a running average cache hit rate.

[0004] Figure 2 is an illustration of a running average cache hit rate without a history of cache hit rates.

[0005] Figure 3 is a flowchart of an example method for a running average cache hit rate.

[0006] Figure 4 is a flowchart of another example method for a running average cache hit rate.

[0007] In all of the drawings, like reference characters and descriptions indicate similar, but not necessarily identical, elements. While the example implementations described herein are easily susceptible to various modifications and alternative forms, specific implementations have been shown by way of example in the drawings and will be described in detail herein. However, the example implementations described herein are not intended to be limited to the particular forms disclosed. Rather, the disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims. DETAILED DESCRIPTION

[0008] The present disclosure generally relates to a running average cache hit rate that can be implemented without tracking a history of cache hit rates. As will be explained in greater detail below, implementations of the present disclosure track a cumulative delta value from a current cache hit, as well as an average cache hit rate; and update the average cache hit rate based on the delta. A cache allocation policy can be updated according to the updated average cache hit rate.

[0009] In one embodiment, a device for maintaining an average cache hit rate includes: a first register configured to store a cumulative increment value; a second register configured to store the average cache hit rate; and control circuitry. The control circuitry can be configured to: (i) calculate the cache hit rate; (ii) update the cumulative increment value based on the cache hit rate and the average cache hit rate; (iii) update the average cache hit rate based on the updated cumulative increment value; and (iv) update a cache allocation policy based on the updated average cache hit rate.

[0010] In some examples, the control circuitry is configured to update the cumulative increment value by determining the difference between the cache hit rate and the average cache hit rate and adding that difference to the cumulative increment value.

[0011] In some examples, the control circuitry is configured to update the average cache hit rate based on determining that the updated cumulative increment value exceeds the increment range. In some examples, the control circuitry is configured to determine that the updated cumulative increment value exceeds the increment range by determining that the updated cumulative increment value exceeds an upper increment threshold. In some examples, the control circuitry is configured to update the average cache hit rate by incrementing the average cache hit rate in response to the updated cumulative increment value exceeding the upper increment threshold. In some examples, the control circuitry is configured to increment the average cache hit rate using an increment factor. In some examples, the increment factor is based on the amount by which the updated cumulative increment value exceeds the upper increment threshold.

[0012] In some examples, the control circuitry is configured to determine that the updated cumulative increment value exceeds the increment range by determining that the updated cumulative increment value is below a lower limit increment threshold. In some examples, the control circuitry is configured to update the average cache hit rate by decreasing the average cache hit rate in response to the updated cumulative increment value being below the lower limit increment threshold. In some examples, the control circuitry is configured to decrease the average cache hit rate using a decrement factor. In some examples, the decrement factor is based on the amount by which the updated cumulative increment value is below the lower limit increment threshold.

[0013] In some examples, the control circuitry is also configured to reset the cumulative increment value in response to an update to the average cache hit rate. In some examples, the control circuitry is configured to reset the cumulative increment value using the updated average cache hit rate.

[0014] In one embodiment, a system for maintaining an average cache hit rate includes: physical memory; at least one physical processor including a cache; a first register configured to store a cumulative increment value; a second register configured to store an average cache hit rate for the cache; and control circuitry. The control circuitry may be configured to: (i) calculate a cache hit rate for the cache; (ii) update the cumulative increment value in the first register based on the cache hit rate and the average cache hit rate; (iii) determine that the updated cumulative increment value exceeds an increment range; (iv) update the average cache hit rate in the second register in response to determining that the updated cumulative increment value exceeds an increment range; and (v) update a cache allocation policy for the cache based on the updated average cache hit rate.

[0015] In some examples, the control circuitry is configured to update the cumulative increment value by determining the difference between the cache hit rate and the average cache hit rate and adding that difference to the cumulative increment value.

[0016] In some examples, the control circuitry is configured to determine that the updated cumulative increment value exceeds the increment range by determining that the updated cumulative increment value exceeds an upper increment threshold. In some examples, the control circuitry is configured to update the average cache hit rate in response to the updated cumulative increment value exceeding the upper increment threshold by incrementing the average cache hit rate using an increment factor based on the amount by which the updated cumulative increment value exceeds the upper increment threshold.

[0017] In some examples, the control circuitry is configured to determine that the updated cumulative increment value exceeds the increment range by determining that the updated cumulative increment value is below a lower limit increment threshold. In some examples, the control circuitry is configured to update the average cache hit rate in response to the updated cumulative increment value being below the lower limit increment threshold by decreasing the average cache hit rate using a decrement factor based on the amount by which the updated cumulative increment value is below the lower limit increment threshold.

[0018] In some examples, the control circuitry is also configured to reset the cumulative increment value using the updated average cache hit rate.

[0019] In one implementation, a method for maintaining an average cache hit rate includes: (i) calculating a cache hit rate for the cache; (ii) determining a difference between the cache hit rate for the cache and the average cache hit rate; (iii) updating a cumulative increment value by adding the difference to a cumulative increment value; (iv) determining that the updated cumulative increment value exceeds an increment range; (v) updating the average cache hit rate in response to determining that the updated cumulative increment value exceeds an increment range; (vi) resetting the cumulative increment value in response to updating the average cache hit rate; and (vii) updating a cache allocation policy for the cache based on the updated average cache hit rate.

[0020] In some examples, determining that the updated cumulative increment value exceeds the increment range includes determining that the updated cumulative increment value exceeds an upper increment threshold. In some examples, updating the average cache hit rate includes updating the average cache hit rate by incrementing the average cache hit rate using an increment factor based on the amount by which the updated cumulative increment value exceeds the upper increment threshold in response to the updated cumulative increment value exceeding the upper increment threshold.

[0021] In some examples, determining that the updated cumulative increment value exceeds the increment range includes determining that the updated cumulative increment value exceeds the increment range by determining that the updated cumulative increment value is below a lower increment threshold. In some examples, updating the average cache hit rate includes updating the average cache hit rate by decreasing the average cache hit rate using a decrement factor based on the amount by which the updated cumulative increment value is below the lower increment threshold in response to the updated cumulative increment value being below the lower increment threshold.

[0022] In some examples, resetting the cumulative increment value involves resetting the cumulative increment value using the updated average cache hit rate.

[0023] Based on the general principles described herein, the features of any specific embodiments described herein can be used in combination with each other. These and other specific embodiments, features, and advantages will be more fully understood upon reading the following detailed description in conjunction with the accompanying drawings and claims.

[0024] The following will refer to Figures 1 to 4 Provides a detailed description of the managed runtime average cache hit rate. This will be combined with... Figure 1 Provide a detailed description of the example system. This will be combined with... Figure 2 Provides a detailed description of the running average and increments. It will also incorporate... Figure 3 and Figure 4 Provide a detailed description of the corresponding method.

[0025] Figure 1This is a block diagram of an example system 100 used to run average cache hit rate. System 100 corresponds to computing devices such as desktop computers, laptop computers, servers, tablets, mobile devices, smartphones, wearable devices, augmented reality devices, virtual reality devices, network devices, and / or electronic devices. Figure 1 As illustrated, system 100 includes one or more memory devices, such as memory 120. Memory 120 generally refers to any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. Examples of memory 120 include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache, variations or combinations of one or more of the above components, and / or any other suitable memory.

[0026] like Figure 1 As illustrated, example system 100 includes one or more physical processors, such as processor 110. Processor 110 generally represents a processing unit of any type or form of hardware implementation capable of interpreting and / or executing computer-readable instructions. In some examples, processor 110 accesses and / or modifies data and / or instructions stored in memory 120. Examples of processor 110 include, but are not limited to, chiplets (e.g., smaller and more specialized processing units that can be coordinated into a single chip), microprocessors, microcontrollers, central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs) implementing soft-core processors, application-specific integrated circuits (ASICs), system-on-a-chip (SoCs), digital signal processors (DSPs), neural network engines (NNEs), accelerators, graphics processing units (GPUs), one or more of the above, one or more variations or combinations of the above, and / or any other suitable physical processor.

[0027] like Figure 1As further illustrated, processor 110 includes control circuitry 112, cache 114, register 116, and register 118. Control circuitry 112 corresponds to one or more controllers, including circuitry and / or instructions for maintaining the average cache hit rate of cache 114, and in some examples may also correspond to or be coupled to a cache controller for cache 114 (e.g., for applying cache allocation strategies). Cache 114 corresponds to a local storage device of processor 110 for processing data without causing latency in accessing data from memory 120, and in some examples may correspond to multiple caches (e.g., a cache hierarchy). Registers 116 and 118 each correspond to memory elements for storing values ​​in processor 110, and in some examples may store cumulative increment values ​​and / or average cache hit rates as described herein.

[0028] In some examples, the cache allocation policy for cache 114 can determine whether to allocate data to cache 114. When cache 114 includes data required by processor 110 such that access to memory 120 can be avoided, it is desirable to set the cache allocation policy to allocate (e.g., allocate or reserve a portion of cache 114) so ​​as not to evict cached data. When cache 114 does not include data required by processor 110 such that processor 110 is accessing memory 120, it is desirable to set the cache allocation policy to not allocate data so as to allow data to be cached. Cache hit rate (e.g., the ratio or percentage of cache hits or the finding of requested data in the cache) can indicate whether cache 114 tends to include the required data. More specifically, the running average cache hit rate can indicate the historical performance of cache 114 used to determine the desired cache allocation policy.

[0029] The runtime average cache hit rate can be calculated by storing cache hit rates and averaging them. However, storing a sufficient number of cache hit rates may be prohibitive, for example, requiring a large storage structure. The system and method described in this paper provide a runtime average cache hit rate that avoids storing historical cache hit rates.

[0030] Figure 2Illustration 200 illustrates a graphical representation of the running average cache hit rate 230 (which may be stored in register 118) calculated using a cumulative increment value 240 (which may be stored in register 116). Control circuitry 112 may calculate a new cache hit rate 232 (e.g., using a counter to count cache hits within a predetermined number of cycles). Increment 242 corresponds to the difference between the new cache hit rate 232 and the running average cache hit rate 230. Although Figure 2 The increment 242 is exemplified as positive (e.g., the new cache hit rate 232 is greater than the running average cache hit rate 230), but in other specific implementations, the increment 242 can be zero or negative.

[0031] The cumulative increment value 240 can be updated using the difference between the new cache hit rate 232 and the running average cache hit rate 230 (e.g., increment 242), and stored (e.g., in register 116) as the updated cumulative increment value 244.

[0032] Control circuit 112 can determine whether the updated cumulative increment value 244 exceeds the increment range. For example, in Figure 2 In this process, the updated cumulative increment value 244 exceeds the upper increment threshold 246. In some examples, the upper increment threshold 246 may correspond to the running average cache hit rate 230, but in other examples, the upper increment threshold 246 may correspond to other values, such as a predetermined value, a value relative to the running average cache hit rate 230 (e.g., an offset from and / or a percentage of the running average cache hit rate), a configurable parameter, etc. In response, the control circuitry 112 may update the running average cache hit rate 230. For example, the running average cache hit rate 234 may be updated by adding the increment factor 236 to the updated average cache hit rate 230, which may be stored in register 118. In some examples, the increment factor 236 may be a predetermined value (e.g., 1), but in other examples it may be based on the amount by which the updated cumulative increment value 244 exceeds the upper increment threshold 246 (e.g., factor scaling), all of which may also be configurable parameters.

[0033] although Figure 2The example illustrates an update where the cumulative increment value 244 exceeds the upper increment threshold 246; however, in other examples, the updated cumulative increment value 244 may be below the lower increment threshold. Similar to the upper increment threshold 246, the lower increment threshold may be a predetermined value (e.g., 0), a value relative to the running average cache hit rate 230, and / or a configurable parameter. Control circuitry 112 can update the running average cache hit rate 230 by decreasing it using a decrement factor similar to the increment factor 236, which is a predetermined value (e.g., -1), but in other examples may be based on the amount by which the updated cumulative increment value 244 falls below the lower increment threshold; all of these can also be configurable parameters.

[0034] In some examples, after updating the runtime average cache hit rate 230, control circuitry 112 can reset the updated cumulative increment value 244, for example, by setting the cumulative increment value 240 to a new value stored in register 116. For example, the cumulative increment value 240 can be reset to a predetermined value (e.g., 0), a value relative to the updated runtime average cache hit rate 234, or a value relative to the updated cumulative increment value 244; all of these can be configurable parameters.

[0035] Furthermore, in response to updating the runtime average cache hit rate, the control circuit 112 can correspondingly update the cache allocation strategy for cache 114 based on the updated average cache hit rate 234.

[0036] Figure 3 This is a flowchart of an exemplary computer implementation of a method 300 for tracking the average cache hit rate. Figure 3 The steps shown can be performed by any suitable circuit and / or computing system (including...) Figure 1 The system illustrated is used to execute this. In one example, Figure 3 Each step shown represents an algorithm whose structure includes and / or is represented by multiple sub-steps, examples of which will be provided in more detail below.

[0037] like Figure 3 As illustrated, at step 302, one or more systems in the system described herein calculate the cache hit rate. For example, control circuitry 112 may calculate a new cache hit rate.

[0038] At step 304, one or more systems in the system described herein update the cumulative increment value based on the cache hit rate and the average cache hit rate. For example, control circuitry 112 may update the cumulative increment value stored in register 116 based on the new cache hit rate and the average cache hit rate stored in register 118.

[0039] The system described herein can perform step 304 in a variety of ways. In one example, control circuitry 112 can update the cumulative increment value by determining the difference between the cache hit rate and the average cache hit rate and adding that difference to the cumulative increment value, which can be stored in register 116.

[0040] At step 306, one or more systems in the system described herein update the average cache hit rate based on the updated cumulative increment value. For example, control circuitry 112 may update the average cache hit rate stored in register 118 based on the updated cumulative increment value stored in register 116.

[0041] The system described herein can perform step 306 in several ways. In one example, control circuitry 112 can update the average cache hit rate based on determining that the updated cumulative increment value exceeds an increment range. For example, control circuitry 112 can determine that the updated cumulative increment value exceeds an upper increment threshold, and in response to the updated cumulative increment value exceeding the upper increment threshold, update the average cache hit rate accordingly by incrementing the average cache hit rate. Control circuitry 112 can use an increment factor to increment the average cache hit rate, which in some examples is based on the amount by which the updated cumulative increment value exceeds the upper increment threshold.

[0042] In another example, control circuitry 112 can determine that the updated cumulative increment value exceeds the increment range by determining that the updated cumulative increment value is below a lower increment threshold. Control circuitry 112 can update the average cache hit rate by decreasing the average cache hit rate in response to the updated cumulative increment value being below the lower increment threshold. Control circuitry 112 can use a decrement factor to decrease the average cache hit rate, which in some examples is based on the amount by which the updated cumulative increment value is below the lower increment threshold.

[0043] In some examples, control circuitry 112 may reset the cumulative increment value in response to an update of the average cache hit rate. In some examples, control circuitry 112 may use the updated average cache hit rate to reset the cumulative increment value.

[0044] At step 308, one or more systems in the system described herein update the cache allocation policy based on the updated average cache hit rate. For example, control circuitry 112 may update the cache allocation policy for cache 114 based on the updated average cache hit rate.

[0045] Figure 4This is a flowchart of an exemplary computer implementation of a method 400 for tracking the average cache hit rate. Figure 4 The steps shown can be performed by any suitable circuit and / or computing system (including...) Figure 1 The system illustrated is used to execute this. In one example, Figure 4 Each step shown represents an algorithm whose structure includes and / or is represented by multiple sub-steps, examples of which will be provided in more detail below.

[0046] like Figure 4 As illustrated, at step 402, one or more systems in the system described herein calculate a cache hit rate for the cache. For example, control circuitry 112 may calculate a new cache hit rate for cache 114.

[0047] At step 404, one or more systems in the system described herein determine the difference between the cache hit rate and the average cache hit rate for the cache. For example, control circuitry 112 may determine the difference between the new cache hit rate and the average cache hit rate for cache 114 stored in register 118.

[0048] At step 406, one or more systems in the system described herein update the cumulative increment value by adding the difference to the cumulative increment value. For example, control circuit 112 can update the cumulative increment value stored in register 116 by adding the difference to the cumulative increment value.

[0049] At step 408, one or more systems in the system described herein determine that the updated cumulative increment value exceeds the increment range. For example, control circuitry 112 may determine that the updated cumulative increment value stored in register 116 exceeds the increment range (e.g., above the upper increment threshold or below the lower increment threshold).

[0050] The system described herein can perform step 408 in several ways. In one example, determining that the cumulative increment value of the update exceeds the increment range includes determining that the cumulative increment value of the update exceeds an upper limit increment threshold (e.g., see...). Figure 2 In another example, determining that the cumulative increment value of the update exceeds the increment range includes determining that the cumulative increment value of the update exceeds the increment range by determining that the cumulative increment value of the update is below the lower limit increment threshold.

[0051] At step 410, one or more systems in the system described herein update the average cache hit rate in response to determining that the updated cumulative increment value exceeds the increment range. For example, control circuitry 112 may update the average cache hit rate stored in register 118 in response to determining that the updated cumulative increment value stored in register 116 exceeds the increment range.

[0052] The system described herein can perform step 410 in several ways. In one example, updating the average cache hit rate includes updating the average cache hit rate by incrementing the average cache hit rate using an increment factor in response to the cumulative increment value exceeding an upper limit increment threshold (see, for example, see [link to documentation]). Figure 2 In some examples, the increment factor can be based on the amount by which the updated cumulative increment value exceeds the upper limit increment threshold.

[0053] In other examples, updating the average cache hit rate involves reducing the average cache hit rate by a decrementing factor in response to the cumulative increment value falling below a lower limit increment threshold. In some examples, the decrementing factor may be based on the amount by which the updated cumulative increment value falls below the lower limit increment threshold.

[0054] At step 412, one or more systems in the system described herein reset the cumulative increment value in response to updating the average cache hit rate. For example, control circuitry 112 may reset the cumulative increment value stored in register 116 in response to updating the average cache hit rate stored in register 118.

[0055] The system described herein can perform step 412 in several ways. In one example, resetting the cumulative increment value involves resetting the cumulative increment value using the updated average cache hit rate. In other examples, other values ​​can be used to reset the cumulative increment value.

[0056] At step 414, one or more systems in the system described herein update the cache allocation policy for the cache based on the updated average cache hit rate. For example, control circuitry 112 may update the cache allocation policy for cache 114 based on the updated average cache hit rate stored in register 118.

[0057] The system described herein can perform step 414 in several ways. In one example, control circuitry 112 can compare the updated average cache hit rate with various cache hit rate thresholds corresponding to different cache allocation strategies.

[0058] As detailed above, the system and method presented herein relate to a technique for maintaining an average hit rate in a register without preserving hit rate history. This technique involves having a current average and maintaining an increment to the average. For each new hit rate, the increment is adjusted based on the difference between the latest hit rate and the average, either by addition or subtraction. In other words, increment = increment + (new hit rate - average). If the increment exceeds the average, the average is adjusted. If the increment is positive and exceeds the average, the average can be subtracted from the increment, and the average is incremented (e.g., increment increment / average, and to address the case where the average is small and the increment grows rapidly, the increment could be increment%average). If the increment is negative, the average is subtracted by one, and the increment is adjusted. In this way, if the new hit rate is consistently higher than the average, the average will increase, but if the new hit rate is consistently lower than the average, the average will decrease. A more significant and consistent deviation from the average can result in faster adjustments to the average, while hovering close to the average can either keep the average the same or require several iterations to update it.

[0059] More specifically, for set sampling, it's necessary to detect consecutive cache thrashing (misses) and automatically switch the cache allocation policy to no allocation. This can be tracked by maintaining a large history of hit rates. The running average can be stable when hit rates are consistent and tends to zero as hit rates decrease. The running average can also cover cache warm-up cycles where allocation is expected despite the existence of miss cycles. If the running average slowly decreases to zero, it can cover the warm-up cycles. Thrashing can correspond to long periods of zero hits, in which case the average will tend to and remain at zero.

[0060] Compared to tracking cache hit history, the average cache hit rate technique described in this paper advantageously requires less chip area, consumes less power, and considers a longer history. Similar performance using a FIFO queue based on cache hit rate would require a very deep and wide structure, as well as a large number of logic gates to sum entries. In contrast, the system and method described in this paper can be implemented using two counters (e.g., stored in two registers), and can provide more flexibility and configuration parameters to adjust settings per workload. For example, configuration parameters allow modulation of the average sample / response size, selection between incremental scaling or monotonic scaling, and adjustment of allocation and / or no-allocation thresholds.

[0061] The average sample / response size corresponds to the number of responses to be averaged based on the number of sets observed in the sample set. At the front end, the tag lookup response hit counter can accumulate a fixed number of responses, from which the corresponding subset can be used to calculate a new cache hit rate.

[0062] The incremental scaling threshold corresponds to the scaling factor (e.g., + register / - register) when the incremental difference exceeds the threshold, resulting in a response curve that can respond more quickly when the hit rate drops rapidly or spikes. While monotonic scaling (+1 / -1) can be the default, this scaling can typically be slow to respond to changes in the slope. However, given a large number of samples, monotonic scaling can keep up well. For example, for the average of 64 responses, if the average is reset to the full hit rate (64), the average will take approximately 64 missed (0) responses accumulated to gradually decrease to 0 (e.g., 64*64=4096 missed responses).

[0063] The systems and methods presented in this paper can be advantageously implemented without requiring data structures for actively tracking historical cache hit values, and also allow for tracking longer histories. For example, using a FIFO to track a similar number of cache hits as the systems and methods described above would likely require deep and wide structures (e.g., for storing each value), as well as logic gates for summing entries. Instead, the systems and methods described in this paper can be implemented using two counters (e.g., cumulative increment values ​​and average cache hit rate), where configurable parameters provide further flexibility (e.g., modulating the average sample size, increment value scaling or monotonic scaling, allocated or unallocated thresholds, etc.). Advantageously, the systems and methods presented in this paper can be implemented with less area (e.g., requiring less space in the architecture for component consumption), thereby further reducing power consumption.

[0064] As detailed above, the circuits and systems described and / or illustrated herein broadly represent any type or form of computing device or system capable of executing computer-readable instructions. In their most basic configuration, each of these computing devices includes at least one memory device and at least one physical processor.

[0065] In some examples, the term "memory device" generally refers to any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, a memory device stores, loads, and / or retains one or more of the modules and / or circuits described herein. Examples of storage devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache, variations or combinations of one or more of the above components, or any other suitable memory.

[0066] In some examples, the term "physical processor" generally refers to a processing unit of any type or form of hardware implementation capable of interpreting and / or executing computer-readable instructions. In one example, a physical processor accesses and / or modifies one or more modules stored in the aforementioned memory devices. Examples of physical processors include, but are not limited to, microprocessors, microcontrollers, central processing units (CPUs), field-programmable gate arrays (FPGAs) implementing soft-core processors, application-specific integrated circuits (ASICs), system-on-a-chip (SoCs), digital signal processors (DSPs), neural network engines (NNEs), accelerators, graphics processing units (GPUs), one or more of the above, one or more variations or combinations of the above, or any other suitable physical processor.

[0067] In some implementations, the term "computer-readable medium" generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transport media such as carrier waves, and non-transitory media such as magnetic storage media (e.g., hard disk drives, magnetic tape drives, and floppy disks), optical storage media (e.g., optical discs (CDs), digital video discs (DVDs), and Blu-ray discs), electronic storage media (e.g., solid-state drives and flash memory media), and other distribution systems.

[0068] The order of process parameters and steps described and / or illustrated herein is given by way of example only and may be varied as needed. For example, although steps illustrated and / or described herein are shown or discussed in a particular order, these steps do not necessarily need to be performed in the illustrated or discussed order. The various exemplary methods described and / or illustrated herein may also omit one or more steps described or illustrated herein, or include additional steps in addition to those disclosed.

[0069] The foregoing description has been provided to enable others skilled in the art to best utilize various aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or limited to any precise form disclosed. Many modifications and variations are possible without departing from the spirit and scope of this disclosure. The specific embodiments disclosed herein should be considered illustrative rather than restrictive in all respects. In determining the scope of this disclosure, reference should be made to the appended claims and their equivalents.

[0070] Unless otherwise stated, the terms “connected to” and “coupled to” (and their derivatives) as used in the specification and claims will be considered to allow both direct and indirect connections (i.e., via other elements or components). Additionally, the terms “a” or “an” as used in the specification and claims will be considered to mean “at least one”. Finally, for ease of use, the terms “comprising” and “having” (and their derivatives) as used in the specification and claims are interchangeable with the word “including” and have the same meaning.

Claims

1. An apparatus comprising: a first register configured to store a cumulative delta value; a second register configured to store an average cache hit rate; and control circuitry configured to: calculate a cache hit rate; update the cumulative delta value based on the cache hit rate and the average cache hit rate; update the average cache hit rate based on the updated cumulative delta value; and update a cache allocation policy based on the updated average cache hit rate.

2. The apparatus of claim 1, wherein the control circuitry is configured to update the cumulative delta value by: determining a difference between the cache hit rate and the average cache hit rate; and adding the difference to the cumulative delta value.

3. The apparatus of claim 1, wherein the control circuitry is configured to update the average cache hit rate based on a determination that the updated cumulative delta value exceeds a delta range.

4. The apparatus of claim 3, wherein the control circuitry is configured to determine that the updated cumulative delta value exceeds the delta range by determining that the updated cumulative delta value exceeds an upper delta threshold.

5. The apparatus of claim 4, wherein the control circuitry is configured to update the average cache hit rate by incrementing the average cache hit rate in response to the updated cumulative delta value exceeding the upper delta threshold.

6. The apparatus of claim 5, wherein the control circuitry is configured to increment the average cache hit rate using an increment factor.

7. The apparatus of claim 6, wherein the increment factor is based on a magnitude by which the updated cumulative delta value exceeds the upper delta threshold.

8. The apparatus of claim 3, wherein the control circuitry is configured to determine that the updated cumulative delta value exceeds the delta range by determining that the updated cumulative delta value is below a lower delta threshold.

9. The apparatus of claim 8, wherein the control circuitry is configured to update the average cache hit rate by decrementing the average cache hit rate in response to the updated cumulative delta value being below the lower delta threshold.

10. The apparatus of claim 9, wherein the control circuitry is configured to decrement the average cache hit rate using a decrement factor.

11. The apparatus of claim 10, wherein the decrement factor is based on a magnitude by which the updated cumulative delta value is below the lower delta threshold.

12. The apparatus of claim 1, wherein the control circuitry is further configured to reset the cumulative delta value in response to updating the average cache hit rate.

13. The apparatus of claim 12, wherein the control circuitry is configured to reset the cumulative delta value using the updated average cache hit rate.

14. A system comprising: a physical memory; at least one physical processor comprising a cache; ​ a first register configured to store a cumulative delta value; a second register configured to store an average cache hit rate for the cache; and control circuitry configured to: calculate a cache hit rate for the cache; update the cumulative delta value in the first register based on the cache hit rate and the average cache hit rate; determine that the updated cumulative delta value exceeds a delta range; update the average cache hit rate in the second register in response to determining that the updated cumulative delta value exceeds the delta range; and update a cache allocation policy for the cache based on the updated average cache hit rate.

15. The system of claim 14, wherein the control circuitry is configured to update the cumulative delta value by: determining a difference between the cache hit rate and the average cache hit rate; and adding the difference to the cumulative delta value.

16. The system of claim 14, wherein: the control circuitry is configured to determine that the updated cumulative delta value exceeds the delta range by determining that the updated cumulative delta value exceeds an upper delta threshold; and the control circuitry is configured to update the average cache hit rate in response to the updated cumulative delta value exceeding the upper delta threshold by incrementing the average cache hit rate using an increment factor based on a magnitude by which the updated cumulative delta value exceeds the upper delta threshold.

17. The system of claim 14, wherein: the control circuitry is configured to determine that the updated cumulative delta value exceeds the delta range by determining that the updated cumulative delta value is below a lower delta threshold; and the control circuitry is configured to update the average cache hit rate in response to the updated cumulative delta value being below the lower delta threshold by decrementing the average cache hit rate using a decrement factor based on a magnitude by which the updated cumulative delta value is below the lower delta threshold.

18. The system of claim 14, wherein the control circuitry is further configured to reset the cumulative delta value using the updated average cache hit rate.

19. A method comprising: calculating a cache hit rate for a cache; determining a difference between the cache hit rate and an average cache hit rate for the cache; updating a cumulative delta value by adding the difference to the cumulative delta value; determining that the updated cumulative delta value exceeds a delta range; updating the average cache hit rate in response to determining that the updated cumulative delta value exceeds the delta range; resetting the cumulative delta value in response to updating the average cache hit rate; and updating a cache allocation policy for the cache based on the updated average cache hit rate.

20. The method of claim 19, wherein: ​ determining that the updated cumulative delta value exceeds the delta range includes determining that the updated cumulative delta value exceeds an upper delta threshold; updating the average cache hit rate includes updating the average cache hit rate in response to the updated cumulative delta value exceeding the upper delta threshold by incrementing the average cache hit rate using a delta factor based on a magnitude by which the updated cumulative delta value exceeds the upper delta threshold; determining that the updated cumulative delta value exceeds the delta range includes determining that the updated cumulative delta value exceeds the delta range by determining that the updated cumulative delta value is below a lower delta threshold; updating the average cache hit rate includes updating the average cache hit rate in response to the updated cumulative delta value being below the lower delta threshold by decrementing the average cache hit rate using a decrement factor based on a magnitude by which the updated cumulative delta value is below the lower delta threshold; and resetting the cumulative delta value includes resetting the cumulative delta value using the updated average cache hit rate. determining that the updated cumulative delta value exceeds the delta range includes determining that the updated cumulative delta value exceeds an upper delta threshold; updating the average cache hit rate includes updating the average cache hit rate in response to the updated cumulative delta value exceeding the upper delta threshold by incrementing the average cache hit rate using a delta factor based on a magnitude by which the updated cumulative delta value exceeds the upper delta threshold; determining that the updated cumulative delta value exceeds the delta range includes determining that the updated cumulative delta value exceeds the delta range by determining that the updated cumulative delta value is below a lower delta threshold; updating the average cache hit rate includes updating the average cache hit rate in response to the updated cumulative delta value being below the lower delta threshold by decrementing the average cache hit rate using a decrement factor based on a magnitude by which the updated cumulative delta value is below the lower delta threshold; and resetting the cumulative delta value includes resetting the cumulative delta value using the updated average cache hit rate.

Images