Fast cache tracking with aggressive prefetching

Abstract

Bloom filters are used to track the contents of the cache. The system checks the Bloom filter before deciding whether to prefetch an address (by hashing the address and checking the value of the Bloom filter at the index based on the hash). This allows the system to utilize a more aggressive prefetching scheme by reducing the risk of wasted redundant prefetch operations.

Description

Background Art

[0001] Almost all modern computing systems utilize cache in some form or another. Cache essentially serves as a temporary local copy of information. Accessing cache is significantly more efficient in terms of time and power than reading data from long-term and / or larger storage devices (such as hard drives and / or main memory).

[0002] Prefetching is an increasingly common way to try to reduce the number of time-consuming storage accesses required by an application. If the desired line (referring to a data instruction or data segment) is not stored in the cache, it may be necessary to read the line from long-term storage, which will be accompanied by a significant time penalty. Prefetching takes advantage of a concept known as "temporal locality", which recognizes that data / instructions are frequently called within small groups. For example, if the data at address 0003 is called, temporal locality suggests that the data at addresses 0004, 0005, and 0006 may be called in the near future. Therefore, after detecting a call for the data at address 0003, many prefetch systems can preemptively read addresses 0004-0006 into the cache. In this way, if addresses 0004-0006 are indeed called next, these calls will result in a cache hit. Summary of the Invention

[0003] Some embodiments of the present disclosure may be exemplified as a method. The method includes predicting a candidate object address. The method also includes calculating a hash of the candidate object address. The method also includes checking a Bloom filter based on the hash. The method also includes (based on the checking) determining to prefetch information stored at the candidate object address.

[0004] Some embodiments of the present disclosure may also be exemplified as a computer program product comprising a computer-readable storage medium including program instructions, which may be executed by a computer to cause the computer to perform the above-discussed method.

[0005] Some embodiments of the present disclosure may be exemplified as a system. The system may include a memory and a processing unit, such as a controller or a central processing unit (CPU). The processing unit may be configured to execute instructions to perform the above method.

[0006] The above summary is not intended to describe each illustrated embodiment or every implementation of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings included in this application are incorporated into and form a part of the specification. They illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. The accompanying drawings illustrate certain embodiments only and do not limit the present disclosure. Features and advantages of various embodiments of the claimed subject matter will become apparent as the following detailed description proceeds and upon reference to the accompanying drawings, in which like reference numerals indicate like parts, and wherein:

[0008] Figure 1 An advanced cache monitoring method including checking Bloom filter contents before deciding whether to prefetch information is shown according to several embodiments of the present disclosure.

[0009] Figure 2 A high-level flow chart of a Bloom filter update method according to several embodiments of the present disclosure is shown.

[0010] Figure 3 An example Bloom filter implementation is shown in accordance with several embodiments of the present disclosure.

[0011] Figure 4 A high-level block diagram of an example computer system is shown that can be used to implement embodiments of the present disclosure.

[0012] Although the present invention is susceptible to various modifications and alternative forms, details thereof have been shown by way of example in the accompanying drawings and will be described in detail. However, it should be understood that it is not intended to limit the present invention to the particular embodiments described. On the contrary, the present invention covers all modifications, equivalents, and alternatives that fall within the spirit and scope of the present invention. DETAILED DESCRIPTION

[0013] Aspects of the present disclosure relate to systems and methods for monitoring cache contents. More specific aspects relate to a system for detecting cache hits, determining addresses of prefetch candidates, determining whether to prefetch the candidates based on a Bloom filter, and maintaining the Bloom filter based on cache writes.

[0014] In this disclosure, references are made to "data," "instructions," and "information." Many systems implement separate data caches (Dcache) and instruction caches (Icache). Systems and methods consistent with this disclosure can be implemented to improve the performance of either or both, as well as systems that do not distinguish between data and instructions. Therefore, to avoid confusion, the term "information" is used as a generalization for both "data" and "instructions." A cache is described herein as storing information; this can refer to a Dcache storing data or an Icache storing instructions, as well as other memory management techniques such as paging and single-level storage.

[0015] In this disclosure, reference is made to "prefetching," and determining whether to prefetch. As used herein, prefetching refers to "preemptively" reading information from memory into a cache (reading information from memory into a cache before it is needed). When determining whether to prefetch information, the information is referred to herein as a "prefetch candidate," and the memory address where the information is stored is referred to as a "candidate address."

[0016] In this disclosure, references are made to "addresses" and "indexes". As used herein, an "address" refers to an address in memory (which may refer to volatile memory such as random access memory (RAM) or non-volatile memory such as a hard disk drive (HDD)). Information is stored at a specific address in memory. Information may also be stored at a specific cache address in a cache. However, for the purposes of this disclosure, an "address" refers to a specific location in memory. The present invention refers to one or more arrays and / or vectors for tracking the state of a cache. The "address" of the information is hashed (or otherwise converted) to generate a corresponding "index" to a data structure (such as a vector). While the two are related, they are not interchangeable. The value of the vector stored at a given index is used to inform whether the information stored at the corresponding address is also currently stored in the cache.

[0017] In this disclosure, reference is made to Bloom filters. As used herein, a Bloom filter refers to a probabilistic data structure (e.g., a 1-dimensional matrix or vector). Bloom filters are updated to monitor / track the contents of a cache. Typically, a Bloom filter will have a value of "0" or "1" (which is referred to herein as a "binary" Bloom filter). When a cache write occurs, this value can be updated to reflect that particular information is cached. As an illustrative example, in some embodiments, whenever information at an address stored in memory is written to the cache, the value of the Bloom filter at the index is set to "1" to reflect this. The index depends on the memory address of the cached information. In an example embodiment, the address of the information written to the cache is hashed, and the result of the hash is used as an index into the Bloom filter. Thus, the system is able to check whether information at a given address is cached by hashing the address and checking whether the value of the Bloom filter (at the index of the Bloom filter corresponding to the hash of the address) is "1" or "0."

[0018] It is worth noting that the hashing algorithm used on the address does not necessarily provide a unique output; this enables the use of a Bloom filter with fewer indices than the size of the address space, at the expense of introducing the possibility of "false positives." For example, if address A and address B both hash to index X, then the value of the Bloom filter at index X will be updated when either A or B is cached. For example, caching the information stored at address A will cause X to be set to "1." A later check to determine whether address B is cached will detect that index X (which is "1" due to A) indicates that address B is cached, regardless of whether it is actually cached. Significant performance improvements can be achieved by appropriately fine-tuning / sizing the hashing algorithm / Bloom filter. A simple index calculation algorithm will simply use the necessary number of least significant bits of the cache line address (the address after discarding the cache line offset bits). For example, to access a 1024-entry Bloom filter, only the 10 least significant bits of the cache line address can be used. More complex hashing algorithms can apply an XOR function to additional bits of the cache line address to compress 20-30 bits of the address into a 10-bit index. This can provide more uniform usage of the Bloom filter. Similarly, the size of the Bloom filter itself can be adjusted based on the size of the cache and the footprint of the typical workload running on the processor. A size of 1024 entries has been found to provide adequate coverage in most cases.

[0019] When an address is called, several subsequent addresses may be called (based on temporal locality), as will be understood by one of ordinary skill in the art. Therefore, when an address is read, prefetching a set of subsequent addresses may result in a series of cache hits. However, if the candidates for prefetch are already in the cache, they will be cache hits even without the prefetch, which will cause prefetch redundancy and waste. The power cost of such "unused" prefetches can be particularly high because the cache directory must be queried to verify whether the prefetch is needed. Such queries consume a lot of power, especially for highly associative caches. Since the concept of temporal locality can also be applied to the contents of the cache, prefetching after a cache hit may be particularly likely to result in wasted work.

[0020] As a clear example, the information could be the instruction "Add 3 to X," which can be stored at address 5350 in memory. When address 5350 is called, the instruction "Add 3 to X" is written to the cache. When this occurs, the address (5350) is hashed. For the purposes of a simple example, the hashing algorithm can simply use the last two digits of the address, so the hash result here is 50 (the last two digits of 5350 are 50). The hash result of the address is used as an index into the Bloom filter. The filter's value at that index is set to "1." Therefore, in this example, when the instruction "Add 3 to X" is written to the cache, the 50th entry in the vector is set to "1" (for simplicity, these values are in base 10). Later, the Bloom filter can be checked to determine whether the information stored at address 5350 (i.e., "Add 3 to X") is cached. To perform this check, the address is hashed again (resulting in 50), and the hash result is used as an index (thus, the 50th value of the Bloom filter is checked). Since the value is "1", this indicates that the information stored at the address in question (5350) is in the cache.

[0021] If the candidate objects are not cached, prefetching them may result in a cache hit instead of a cache miss, thereby saving a lot of time. However, if the candidates are already cached, prefetching them is redundant, thus wasting the power, time, and resources required for prefetching. Therefore, "aggressive" prefetching schemes (e.g., automatically prefetching the 5 subsequent addresses after the called address) end up consuming more resources than they save. Therefore, systems and methods consistent with the present disclosure provide a computationally lightweight way to determine whether candidate objects are already in cache, informing the decision whether to prefetch them. If the candidates are already cached, avoiding prefetching them can save duplicate work and power.

[0022] However, maintaining one or more Bloom filters to track the contents of the cache can advantageously enable an "aggressive" prefetching scheme that might otherwise frequently waste power. Some systems implement multiple caches; an instruction cache (Icache) and a data cache (Dcache). Instructions are typically read in relatively small, localized groups when compared to data from the data cache (Dcache). For example, when reading data, thousands of addresses are typically read sequentially, while instructions are typically read in groups of about three to ten. Therefore, typical aggressive prefetching is significantly more likely to be wasteful in the context of Icache vs. Dcache.

[0023] For example, while prefetching several thousand data addresses may result in at least some wasted work (e.g., the information stored at some of the prefetched addresses may already be cached), the time and power saved by the other prefetches (converting cache misses into cache hits) is typically still a significant performance improvement. However, for smaller groups processed in the Icache, only a small number of wasted prefetches are incurred because the cost of the prefetches outweighs the savings. As a simple example, 50 wasted prefetches (typically data reads) out of 3,000 prefetches is a negligible amount of wasted resources when compared to the benefit of converting 2,950 original cache misses into cache hits. On the other hand, 2 wasted prefetches (typically instruction reads) out of 3 prefetches may cost more than the resulting single cache hit savings. Given this risk, many systems neglect to prefetch aggressively in the context of the Icache because the potential gains are outweighed by the considerable risk. Therefore, systems and methods consistent with the present disclosure are particularly advantageous when applied to an instruction cache (Icache), although they will also result in performance improvements when applied to a Dcache.

[0024] Because Bloom filter values are set to "1" when the cache is written, over time the Bloom filter may become saturated, and if left unchecked, all values will eventually become "1." To address this issue, in some embodiments, Bloom filters are periodically refreshed by setting all their values to "0." In some embodiments, whenever information is evicted from the cache, the corresponding index of the Bloom filter may be set to "0," although this may result in false negatives for other addresses that correspond to the same index when hashed.

[0025] In some embodiments, Bloom filters may be described herein as having "life cycles," and referred to as being "older" or "younger" relative to these life cycles. After a Bloom filter is refreshed, subsequent lookups of the newly refreshed filter are likely to return negatives, regardless of the state of the cache. Over time, as information is written to the cache, the values in the filter will change from "0" to "1," so the newly refreshed filter will "mature" as the cache is written. In order to mitigate possible false negatives returned from the "youngest" (most recently refreshed) filter, many embodiments of the present disclosure utilize at least two different filters. For example, in some embodiments, the lifespan of a filter may be set to 10,000 cycles, meaning that every 10,000 cycles, the filter is refreshed. However, using two filters, the refreshes can be offset by 5,000 cycles.

[0026] As an illustrative example (but ignoring for the moment the possibility of "false positives" explained above for purposes of example), if the data address was cached 7,000 cycles ago (i.e., before the younger filter was flushed), it will only appear in the older filter. In other words, the older filter will have a value of "1" at the index corresponding to the hash of the address, but the younger filter will have a "0" at the same index. However, if the data address was cached 12,000 cycles ago (i.e., before both filters were flushed), it will not appear in either filter. On the other hand, if the data address was cached after the youngest filter was flushed, both filters will indicate that the data address was cached.

[0027] When checking filters to determine if an address has been cached, the system can take the logical OR of two (or more) filters. This mitigates false negatives for the youngest filter. When the cache is written, the corresponding indexes of all Bloom filters can be set to "1." Thus, checking the logical OR of the filters is effectively the same as checking the oldest filter (but taking the OR may be faster and more efficient than tracking and determining which filter has not been refreshed for the longest time).

[0028] In some embodiments, rather than specifically setting the value to "1" as in the "binary" Bloom filter discussed above, the value can be incremented. In this way, a Bloom filter can represent the number of times an address (of information written to the cache) has been hashed to a particular output. In some embodiments, multiple such Bloom filters can be implemented, where checking the filter includes taking the maximum value for each index (e.g., if two filters are [0,0,1,0] and [1,0,3,0], the system checking the filter can check the value from [1,0,3,0]). This is functionally the same as checking the "oldest" filter, as described above with reference to the logical OR of "binary" filters (filters whose values are restricted to "1" or "0"). While the present disclosure focuses on the simpler "binary" filter embodiment, this is not limiting. As will be appreciated by one of ordinary skill in the art, Bloom filters with more than two possible values can be implemented with minimal modification.

[0029] In some embodiments, the decision to prefetch can be further informed by understanding specific properties (or "behavior") of the hashing algorithm being used. For example, different hashing algorithms may have differently sized "output spaces" (the range of possible output values given a range of possible inputs). This can be combined with tracking how many times instructions whose addresses hash to a particular output have been cached (e.g., to more accurately determine the likelihood that a given instruction has already been cached). As a simple example, the hashing algorithm can simply divide the input address by two, rounding down, and the Bloom filter can imply that two previous cache writes store information whose memory address hashed to a given index. If the candidate object address hashes to a given index, the knowledge that there are only two possible memory addresses in the entire address space that can hash to a particular index (index*2 and index*2+1), combined with the Bloom filter's indication that two previous cache writes belong to memory addresses that hash to that index, can strongly imply that the information in question is cached. However, rather than information at two addresses that both hash to the same index being written to the cache once, these two cache writes could still be the result of information at a single address being written to the cache twice. Either case could result in the filter having a value of "2", so false positives are still possible.

[0030] As an illustrative example, the output of a first "hash" algorithm might be the last digit ("more than one") of the input number (thus, an input of "64" would produce an output of "4," an input of "1024" would also produce the same output of "4," and an input of "4096" would produce an output of "6," and so on). The output of a second hash algorithm might be the input divided by 16, rounded down (thus, an input of "64" would produce an output of "4," an input of "1024" would produce an output of "64," an input of "4097" would produce an output of "256," and so on). It is worth noting that the first algorithm only has 10 possible outputs, 0-9, so in a system with more than 10 possible different inputs, duplicate outputs are inevitable. While rounding down the second algorithm would also result in duplicates, a system with more than 159 possible inputs would have more possible outputs, and therefore a larger "output space," than that of the first algorithm.

[0031] Figure 1 An advanced cache monitoring method 100 is shown that includes checking the contents of a Bloom filter before deciding whether to prefetch information according to several embodiments of the present disclosure. The method 100 may be implemented by a computer system (e.g., Figure 4 400) is executed by the system.

[0032] Method 100 includes determining a set of addresses at operation 102. In some embodiments, operation 102 may be performed in response to detecting a cache hit. In some embodiments, operation 102 may be performed periodically in response to a read request (e.g., after every 5 read requests, after each read request, etc.). Operation 102 may include determining the address of a first read request. Operation 102 also includes determining one or more subsequent addresses based on the address of the first read request. The one or more subsequent addresses are used as a set of "expected candidates." In some embodiments, the set of prefetch candidates may include only a single address (e.g., the address immediately following the address of the first read request). Other numbers of prefetch candidates are also possible (e.g., the next 3 addresses, the next 5 addresses, etc.). In embodiments where method 100 is performed in the context of a data cache, the group may include significantly more addresses (e.g., the next 5,000 addresses).

[0033] Method 100 also includes generating a hash of the prefetch candidate address at operation 104. Various hashing methods may be utilized; for example, in some embodiments, operation 104 may include performing one or more mathematical operations on the address (e.g., dividing by a power of 2), clipping the address by a predetermined number of bytes, etc. In some embodiments, operation 104 may include applying one or more known hashing algorithms to the address.

[0034] In some embodiments, operation 104 may be performed for each candidate object address. In some embodiments, operation 104 may be performed only for the first candidate object address in the set of candidate object addresses; in such embodiments, a prefetch decision may be made based on the first candidate object and then applied to the remaining candidate objects.

[0035] Method 100 also includes using the hash to index into the Bloom filter at operation 106. Operation 106 may include, for example, utilizing a hash of the candidate object address as an index into the vector. In some embodiments, multiple Bloom filters may be utilized to reduce the chance of false negative results (as explained in further detail below). In some of these "multiple filter" embodiments, operation 106 may include selecting a single filter and checking the value at the hash-derived index. For example, operation 106 may check the "oldest" Bloom filter (the "oldest" Bloom filter being the Bloom filter that has been refreshed the longest). To reduce the overhead of tracking the "age" of various filters, in some embodiments, operation 106 may instead perform a logical OR of all filters and check the index of the resulting vector. It is worth noting that the "age" of the filters can still be monitored (to determine whether the filters should be refreshed), but the same hardware architecture may not be used for both functions. For example, a first architecture may only track the passage of time and reset the oldest filter, while a second architecture may read all filters and perform an OR operation on them.

[0036] As an illustrative example, if the system is using two Bloom filters and the hash of the candidate object address generated in operation 104 is hexadecimal 10FB, operation 106 may include generating a vector V by taking the OR of the two Bloom filters and checking whether V (4347) is "1" or "0" (note that hexadecimal 10FB is equivalent to decimal 4347).

[0037] Method 100 also includes determining, at operation 108, whether the candidate object address is likely to be a cache hit by analyzing the results of operation 106. For example, in some embodiments, if the value at the index checked at operation 106 is "1", then the corresponding candidate object address is considered likely to be cached (108 "yes"). It is worth noting that the use of a Bloom filter introduces the possibility of false positives; therefore, even if the determination made at operation 108 suggests that information about the candidate object address is likely to be in the cache (meaning a lookup for a candidate object address that is likely to be a cache hit), a cache miss is still possible. However, the resources that can be saved via method 100 can enable a more aggressive prefetching scheme, which generally outweighs this risk in the aggregate.

[0038] The prefetch candidate address determined at operation 102 is not guaranteed to be an accurate prediction; the address evaluated at operations 104-108 may not ultimately be called regardless of the state of the cache. However, this possibility is inherent to the concept of prefetching in general and is well established as an acceptable risk.

[0039] If a cache hit is not expected for the candidate object address ("No" at 108), method 100 proceeds to operation 112. Operation 112 includes pre-fetching information stored at the candidate object address. For example, operation 112 may include accessing memory and writing the information stored at the candidate object address to a cache. Operation 112 may also include causing a cache line to be evicted, consistent with typical cache operations.

[0040] Method 100 also includes updating the Bloom filter at operation 114. Regardless of whether the candidate object is likely to be a cache hit, or even regardless of whether the candidate object is actually called, the Bloom filter is updated to reflect what is actually written to the cache. Operation 114 may include, for example, setting the value of the filter to "1" based on the information written to the cache. For example, in some embodiments, regardless of the next address that is called, the next address is hashed (in a manner similar to operation 104) and the hashed address is used as an index into the Bloom filter. The value of the Bloom filter at that index can be set to "1" regardless of its previous state. Notably, even if the next address that is called is not one of the candidate object addresses, the Bloom filter is updated to reflect that the information stored at the next address is written to the cache. In some embodiments, operation 114 includes waiting for the information to actually be written to the cache before setting the value at the corresponding index to "1."

[0041] If a cache hit is expected for the candidate object address (108 "Yes"), then the method 100 proceeds from operation 108 to operation 114. Notably, operation 114 does not include pre-fetching the candidate object address, as this is expected to waste resources since the information stored at the candidate object address is expected to already be in the cache (although this may not be true due to a false positive). Instead, the method 100 proceeds to updating the Bloom filter at operation 114. Figure 2 The process of maintaining and / or updating a Bloom filter is described in more detail.

[0042] Figure 2 1 shows a high-level flow chart of a Bloom filter update method 200 according to several embodiments of the present disclosure. The method 200 may be performed by a computer system (e.g., a computer system) that implements one or more memory caches. Figure 4 The method 200 may enable a system to maintain and / or update one or more Bloom filters to track cache contents (which in turn advantageously enables a more aggressive prefetching strategy).

[0043] Method 200 includes determining, at operation 202, an address (in memory) of information associated with the request. In accordance with an embodiment, the address determined at operation 202 may correspond to information that was recently written, is being written, or is about to be written from memory to a cache (e.g., the address may be the memory address of an instruction recently called by an application). In some embodiments, the address may correspond to information that is already in the cache. For example, if an application calls an instruction and a subsequent cache lookup results in a cache hit, operation 202 may include determining the address of the instruction even if the instruction has not been recently read from memory.

[0044] Method 200 also includes detecting a cache write and / or hit at operation 204. Operation 204 may include, for example, detecting that the request associated with the address determined at operation 202 is a call or read operation that will result in a cache write. In some embodiments, the cache write detected at operation 204 may not have occurred (e.g., operation 204 may include detecting an operation immediately preceding a cache write, etc.). In some embodiments, operation 204 may include detecting a cache hit.

[0045] Method 200 also includes generating a hash of the address at operation 206. Various hashing methods may be utilized; for example, in some embodiments, operation 206 may include performing one or more mathematical operations on the address (such as dividing by a factor), clipping a preset number of bytes of the address, etc. In some embodiments, operation 206 may include applying one or more known hashing algorithms to the address.

[0046] Method 200 also includes determining whether the lifetime of the Bloom filter has expired at operation 208. Operation 208 may include, for example, reading, calculating, or otherwise determining the number of cycles that have elapsed since the filter was last refreshed and comparing that number to a preset maximum. If the number exceeds the maximum, the lifetime of the Bloom filter has expired (208 "Yes"), and method 200 proceeds to refreshing the filter at operation 210. Operation 210 may include, for example, setting each value of the expired filter to "0." Other metrics besides "cycle count" may be used to track the "age" of the filter and are also contemplated herein, such as elapsed time, the number of writes (e.g., the number of writes to one or more of the filter, cache, memory, etc.), and the like. Refreshing (e.g., operations 208 and 210) may be performed outside of method 200 (in other words, refreshing is not dependent on performing operation 204).

[0047] After refreshing the expired filter at operation 208, or if the lifetime of the Bloom filter has not yet expired (208 "No"), method 200 proceeds to updating the value of the Bloom filter at operation 212. Operation 212 may include, for example, setting the value at a specific index of the Bloom filter to "1." The specific index whose value is modified is determined based on the hash address generated at operation 206. For example, if the address determined at operation 202 is "4050" and the hash output of operation 206 is "50," operation 212 may include setting the 50th value of the Bloom filter to "1."

[0048] In some embodiments, operation 212 may increment the value by 1 rather than strictly setting the value to "1". For example, if the hash of the address is "50" again, but the 50th value of the Bloom filter is already "1", operation 212 may set the 50th value of the Bloom filter to "2". This can effectively enable the Bloom filter to track the number of times the information at the address corresponding to a particular index has been written to the cache. It is worth noting that this is not necessarily the number of times the information at the particular address has been written to the cache, as some address inputs to the hashing algorithm(s) may result in the same output (the same problem causing the possibility of false positives). For example, if three addresses "share" an index (meaning all three addresses hash to the same result), a filter value of 4 may mean that one address has been written to the cache three times, a second address has been written to the cache once, and a third address has not yet been written to the cache (since the last refresh). In an "incremental" embodiment, instead of ORing the filters, a MAX may be used.

[0049] Method 200 may be performed on all filters. In embodiments including multiple filters, aspects of method 200 (such as operation 212) may be performed serially or in parallel on each filter. Method 200 ends at 212.

[0050] Figure 3 Example Bloom filter implementations 300 , 310 , and 320 are shown, consistent with several embodiments of the present disclosure. Figure 3 Examples include checking a series of Bloom filters at a given index (300), setting a given index of the Bloom filter (310), and refreshing the Bloom filter (320). To simplify the illustration, these examples include a Bloom filter with 8 values, where the value being checked and / or updated is the 4th (e.g., the address may have been hashed to produce index 4). These examples are separated by dashed lines 301 and 303.

[0051] Checking the bloom filters (300) may include taking a logical OR of each bloom filter. Figure 3, depicts three Bloom filters (302, 304, and 306) being checked. By performing a logical OR on filters 302, 304, and 306, a result vector 308 is generated. If the fourth value (starting from the left) of result vector 308 is "1," this may indicate that the information whose address is hashed to index "4" is in the cache. Since this result vector is effectively the same as the "oldest" Bloom filter, the result vector itself can be considered another Bloom filter.

[0052] In some embodiments, a result vector may be generated every cycle. This can advantageously reduce the time required to check whether an address is likely to result in a cache hit, since the result vector is generated in advance. However, this can result in additional power consumption, since only a single value of the result vector may be checked before the entire vector is replaced. Therefore, in some embodiments, the result vector is not generated until the index in question is known, at which point the value of each Bloom filter at that index is checked (wherein the check ends immediately upon locating a "1"). This can result in power savings, but may be slower than the result vector approach, since it requires checking multiple values and only begins after the index is determined.

[0053] If information whose address hashes to index "4" is written to the cache, then all bloom filters are updated by setting the 4th value of the bloom filter to "1" as shown at 310. Figure 3 , filters 312, 314, and 316 represent updated versions of filters 302, 304, and 306, respectively. Notably, when the fourth values of filters 302 and 304 are "0," filters 312 and 314 set their fourth values to "1." In some embodiments, when the fourth value of filter 306 is already "0," filter 316 can still undergo a write operation (overwriting "1" with another "1"). While this may be redundant, it can also reduce system complexity. In some embodiments, the value is checked and only overwritten if the final result is different (in other words, the check can determine whether the fourth value of filter 306 is "1," and if so, it can be left as is to save resources). In some embodiments, rather than setting a specific value to "1," the value can be incremented. For example, the fourth value of filter 316 can be set to "2" (incrementing the "1" of filter 306).

[0054] The filter may also be periodically refreshed, an example of which is depicted in column 320. Refreshing a filter may include setting all values of the filter to "0". Figure 3As shown, filter 326 (which corresponds to filters 306 and 316) is refreshed. As described above, filters can be refreshed based on time, cycle count, write count, etc. For example, a filter can be refreshed every 10,000 cycles. In an embodiment with multiple filters (e.g., Figure 3 ), the filters can be refreshed at the same frequency, but offset from each other. For example, filters 322, 324, and 326 can be refreshed every 10,000 cycles, but filter 326 can be refreshed at cycle X, filter 322 can be refreshed at cycle (X+3,333), filter 324 can be refreshed at cycle (X+6,667), and then filter 326 can be refreshed again at cycle (X+10,000). Combined with updating the filters simultaneously (as shown in column 310) and logically ORing all filters when checking them (as shown in column 300), this allows result vector 308 to essentially reflect the cache contents based on the last 10,000 cycles (but not affected by any operations that occurred more than 10,000 cycles ago). In this way, result vector 308 can check the contents of the cache in a relatively fast manner without significant overhead. The frequency of refreshes and the number of filters can be adjusted based on the system configuration. For example, if the system's cache is particularly small, such that the cache's contents are typically evicted within 5,000 cycles of being written, the filter may be refreshed more frequently (e.g., every 5,000 cycles instead of every 10,000 cycles) to reduce the chance of a Bloom filter check producing a "false positive."

[0055] Now refer to Figure 4 , shows a high-level block diagram of an example computer system 400 that can be configured to perform various aspects of the present disclosure, including, for example, methods 200 and 300, respectively. According to embodiments of the present disclosure, the example computer system 400 can be used to implement one or more of the methods or modules described herein and any related functions or operations (e.g., using one or more processor circuits of a computer or computer processors). In some embodiments, the main components of the computer system 400 may include one or more CPUs 402, a memory subsystem 408, a terminal interface 416, a storage interface 418, an I / O (input / output) device interface 420, and a network interface 422, all of which may be directly or indirectly communicatively coupled for inter-component communication via a memory bus 406, an I / O bus 414, and an I / O bus interface unit 412.

[0056] Computer system 400 may include one or more general-purpose programmable central processing units (CPUs) 402, some or all of which may include one or more cores 404A, 404B, 404C, and 404D, collectively referred to herein as CPUs 402. In some embodiments, computer system 400 may include multiple processors typical of relatively large systems; however, in other embodiments, computer system 400 may instead be a single-CPU system. Each CPU 402 may execute instructions stored in a memory subsystem 408 on the CPU core 404 and may include one or more levels of onboard cache.

[0057] In some embodiments, memory subsystem 408 may include random access semiconductor memory, storage devices, or storage media (volatile or non-volatile) for storing data and programs. In some embodiments, memory subsystem 408 may represent the entire virtual memory of computer system 400 and may also include the virtual memory of other computer systems coupled to computer system 400 or connected via a network. Memory subsystem 408 may conceptually be a single monolithic entity, but in some embodiments, memory subsystem 408 may be a more complex arrangement, such as a hierarchy of caches and other memory devices. For example, memory may exist in multiple levels of caches, and these caches may be further divided by function, such that one cache stores instructions while another stores non-instruction data used by one or more processors. Memory may also be distributed and associated with different CPUs or groups of CPUs, as is known in any of various so-called non-uniform memory access (NUMA) computer architectures. In some embodiments, main memory or memory subsystem 804 may include elements for controlling and streaming memory used by CPU 402. This may include memory controller 410.

[0058] Although the memory bus 406 Figure 44. In the embodiment of the present invention, the computer system 400 is shown as a single bus structure that provides a direct communication path between the CPU 402, the memory subsystem 408, and the I / O bus interface 412. However, in some embodiments, the memory bus 406 may include multiple different buses or communication paths, which may be arranged in any of a variety of forms, such as hierarchical point-to-point links, star or mesh configurations, multi-layer buses, parallel and redundant paths, or any other suitable type of configuration. In addition, although the I / O bus interface 412 and the I / O bus 414 are shown as single respective units, in some embodiments, the computer system 400 may include multiple I / O bus interface units 412, multiple I / O buses 414, or both. In addition, although multiple I / O interface units are shown that separate the I / O bus 414 from the various communication paths to the various I / O devices, in other embodiments, some or all of the I / O devices may be directly connected to one or more system I / O buses.

[0059] In some embodiments, the computer system 400 can be a multi-user mainframe computer system, a single-user system, or a server computer or similar device having little or no direct user interface but receiving requests from other computer systems (clients). In addition, in some embodiments, the computer system 400 can be implemented as a desktop computer, a portable computer, a laptop or notebook computer, a tablet computer, a pocket computer, a phone, a smart phone, a mobile device, or any other suitable type of electronic device.

[0060] Notice, Figure 4 The following is intended to depict representative major components of the exemplary computer system 400. However, in some embodiments, the various components may have more Figure 4 The greater or lesser complexity represented in , may exist different from Figure 4 Components may be those or components in addition to those shown in , and the number, type, and configuration of these components may vary.

[0061] Throughout this disclosure, reference is made to setting various values of a Bloom filter to "1" (interpreted as implying that information may be present in the cache) and occasionally refreshing the filter by setting all of its values to "0" (interpreted as implying that information may not be present in the cache). As will be understood by those skilled in the art, the specific values used ("1" and "0") are for exemplary purposes only and are not meant to be limiting. Other possible conventions are fully contemplated; for example, in some embodiments, "0" may be interpreted as implying that information may be present in the cache, while the filter may be occasionally "refreshed" by setting all values to "1". For further clarification, in some embodiments, "5" may imply that information may be present in the cache, while "36" may imply that information may not be present in the cache; these values themselves may simply be used as indicators.

[0062] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, the computer-readable program instructions being used to cause a processor to perform various aspects of the present invention.

[0063] A computer-readable storage medium can be a tangible device that can retain and store instructions used by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination thereof. As used herein, a computer-readable storage medium should not be interpreted as a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0064] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0065] The computer-readable program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (ISA) instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data of an integrated circuit, or source code or object code written in any combination of one or more programming languages (including object-oriented programming languages, such as Smalltalk, C++, etc.) and procedural programming languages (such as "C" programming language or similar programming languages). The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider through the Internet). In some embodiments, in order to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute the computer-readable program instructions to personalize the electronic circuit by utilizing the state information of the computer-readable program instructions.

[0066] Aspects of the present invention are described herein with reference to the flowcharts and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the present invention. It will be understood that each block of the flowcharts and / or block diagrams and the combination of blocks in the flowcharts and / or block diagrams can be implemented by computer-readable program instructions.

[0067] These computer-readable program instructions can be provided to a processor of a computer or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can direct the computer, programmable data processing device and / or other equipment to operate in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0068] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0069] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each frame in the flow chart or block diagram can represent a module, segment or part of an instruction, which includes one or more executable instructions for realizing the specified logical function. In some alternative embodiments, the functions noted in the frame may not occur in the order noted in the figure. For example, the two frames shown in succession can actually be implemented as a step, simultaneously, substantially simultaneously, in a manner that overlaps part or all of the time, or these frames can sometimes be performed in reverse order, depending on the functions involved. It will also be noted that each frame of the block diagram and / or flow chart illustration and the combination of the frames in the block diagram and / or flow chart illustration can be implemented by a dedicated hardware-based system that performs a specified function or action or performs a combination of dedicated hardware and computer instructions.

[0070] The description of various embodiments of the present disclosure has been presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to explain the principles of the embodiments, practical applications, or improvements over existing technologies in the market, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for monitoring cache contents, comprising: Predicting candidate object addresses; Calculating a hash of the candidate object address; determining an index based on the hash; checking a first value of a first bloom filter at the index; checking a second value of a second Bloom filter at the index; comparing a first value of the first Bloom filter to a second value of the second Bloom filter; generating a result vector based on the comparison; checking the result vector; determining, based on the examining, to prefetch information stored at the candidate object address; as well as Information stored at the candidate object address is prefetched based on the determination.

2. The method according to claim 1, wherein The forecasts include: Predicting a second candidate address; Calculating a second hash of the second candidate object address; checking the first Bloom filter based on the second hash; and Based on the checking, it is determined that prefetching second information stored at the second candidate address is unnecessary.

3. The method according to claim 1, wherein The forecasts include: determining a first address of a cache hit; and The candidate object address is determined based on the first address.

4. The method according to claim 1, further comprising: Detect cache writes; as well as A first value of the first Bloom filter and a second value of the second Bloom filter are updated based on the hash.

5. The method according to claim 4, wherein: The updating includes incrementing the first value and the second value.

6. The method according to claim 5, further comprising periodically refreshing the first Bloom filter and the second Bloom filter, wherein The refreshing of the first Bloom filter is offset from the refreshing of the second Bloom filter.

7. A system for monitoring cache contents, comprising: Memory; as well as a processing unit coupled to the memory, wherein the processing unit is configured to execute instructions to: determining the address of information written to the cache; Calculating a hash of the address; determining an index based on the hash; updating a first value of a first Bloom filter at the index; updating a second value of a second Bloom filter at the index; comparing a first value of the first Bloom filter and a second value of the second Bloom filter; generating a result vector based on the comparison; checking the result vector; determining to prefetch information stored at the candidate object address based on the examining; and Information stored at the candidate object address is prefetched based on the determination.

8. The system according to claim 7, wherein: The processing unit is further configured to: Detect cache hits; predicting a candidate object address associated with the cache hit; and A decision is made based on the result vector whether to prefetch information stored at the candidate object address.

9. The system according to claim 8, wherein: The forecasts include: determining a first address of a cache hit; and The candidate object address is determined based on the first address.

10. The system according to claim 8, wherein: The decisions include: Calculating a candidate hash of the candidate object address; determining a candidate index based on the candidate hash; and The result vector is checked for a candidate object value at the candidate index.

11. The system according to claim 7, wherein: The processing unit is further configured to periodically refresh the first Bloom filter and the second Bloom filter, wherein refreshing of the first Bloom filter is offset from refreshing of the second Bloom filter.

12. The system according to claim 7, wherein: The updating includes incrementing the first value.

13. The system according to claim 7, wherein: The processing unit is further configured to update the first Bloom filter and the second Bloom filter based on the result vector.

14. A computer program product comprising program instructions executable by a computer to cause the computer to: determining the address of information written to the cache; Calculating a hash of the address; determining an index based on the hash; updating a first value of a first Bloom filter at the index; updating a second value of a second Bloom filter at the index; comparing a first value of the first Bloom filter and a second value of the second Bloom filter; generating a result vector based on the comparison; checking the result vector; determining to prefetch information stored at the candidate object address based on the examining; as well as Information stored at the candidate object address is prefetched based on the determination.

15. The computer program product of claim 14, wherein: The instructions further cause the computer to: Detect cache hits; predicting a candidate object address associated with the cache hit; and A decision is made based on the result vector whether to prefetch information stored at the candidate object address.

16. The computer program product of claim 15, wherein: The decisions include: Calculating a candidate hash of the candidate object address; determining a candidate index based on the candidate hash; and The result vector is checked for a candidate object value at the candidate index.

17. The computer program product of claim 14, wherein: The instructions further cause the computer to periodically refresh the first Bloom filter and the second Bloom filter, wherein refreshing of the first Bloom filter is offset from refreshing of the second Bloom filter.

18. The computer program product of claim 14, wherein: The updating includes incrementing the first value.

Images