Method, system, computer program, and computer-readable storage medium (fast cache tracking to support aggressive prefetching)

Abstract

To provide a system for, a method of, a computer program for, and a computer-readable storage medium for monitoring cache contents.SOLUTION: A Bloom filter is used to trace contents of a cache. A system according to the present invention checks a Bloom filter before determining whether or not an address is subjected to prefetching (by hashing the address to check a value of the Bloom filter based on a hash value). Thus, the system reduces risk of useless, redundant prefetch operation to thereby allow use of more active prefetch scheme.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Aspects of the present disclosure relate to systems and methods for monitoring cache contents, and more particularly to a system for detecting cache hits, determining addresses of prefetch candidates, determining whether to prefetch the prefetch candidates based on a Bloom filter, and maintaining the Bloom filter based on cache writes. [Background technology]

[0002] Nearly all modern computing systems utilize some form of cache, which essentially acts as a temporary local copy of information. Accessing a cache is significantly more time- and energy-efficient than retrieving data from more long-term and / or large-capacity storage (such as a hard drive and / or main memory).

[0003] Prefetching is an increasingly popular method for attempting to reduce the number of time-consuming storage accesses required by applications. If a desired line (whether an instruction or a segment of data) is not stored in the cache, it may need to be read from longer-term storage, resulting in a significant time penalty. Prefetching utilizes a concept known as "temporal locality," which recognizes that data / instructions are frequently called in small groups. For example, if data at address 0003 is called, temporal locality suggests that data at addresses 0004, 0005, and 0006 are likely to be called in the near future. Therefore, upon detecting a call to data at address 0003, many prefetching systems may preemptively read addresses 0004-0006 into the cache. This way, the next time addresses 0004-0006 are actually called, these calls will result in a cache hit. Summary of the Invention [Problem to be solved by the invention]

[0004] A system, method, computer program product, and computer-readable storage medium for monitoring cache content are provided. [Means for solving the problem]

[0005] Some embodiments of the present disclosure can be exemplified as a method. The method includes predicting a candidate address. The method further includes calculating a hash of the candidate address. The method also includes checking a Bloom filter based on the hash. The method also includes determining (based on the checking) to prefetch information stored at the candidate address.

[0006] Some embodiments of the present disclosure may also be exemplified as a computer program product including a computer-readable storage medium having program instructions embodied therein, the program instructions being executable by a computer to cause the computer to perform the methods described above.

[0007] Some embodiments of the present disclosure can be illustrated as a system. The system can include a memory and a processing unit, such as a controller or central processing unit (CPU). The processing unit can be configured to execute instructions to perform the methods described above.

[0008] The above summary is not intended to describe each illustrated embodiment or every implementation of the present disclosure. [Brief explanation of the drawings]

[0009] The drawings are included in this application, are incorporated in and form a part of this specification. The drawings illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure. The drawings are merely illustrative of particular embodiments and are not intended to limit the disclosure. Features and advantages of various embodiments of the claimed subject matter will become apparent as the following detailed description proceeds and by reference to the drawings, in which like numerals indicate like parts.

[0010] [Figure 1] 1 illustrates a high-level cache monitoring method that includes checking the contents of a Bloom filter before deciding whether to prefetch information, consistent with some embodiments of the present disclosure. [Figure 2] 1 illustrates a high-level flow diagram of a Bloom filter update method consistent with some embodiments of the present disclosure. [Figure 3] 1 illustrates an exemplary Bloom filter consistent with certain embodiments of the present disclosure. [Figure 4] 1 shows a high-level block diagram of an exemplary computer system that can be used to implement embodiments of the present disclosure.

[0011] While the invention is susceptible to various modifications and alternative forms, specific features thereof have been shown by way of example in the drawings and will be described in detail. It is to be understood, however, that there is no intention to limit the invention to the particular embodiments described. On the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Aspects of the present disclosure relate to systems and methods for monitoring cache contents, and more particularly to a system for detecting cache hits, determining addresses of prefetch candidates, determining whether to prefetch the prefetch candidates based on a Bloom filter, and maintaining the Bloom filter based on cache writes.

[0013] Throughout this disclosure, references are made to "data," "instructions," and "information." Many systems implement separate data caches (D-cache) and instruction caches (I-cache). Systems and methods consistent with this disclosure can improve performance of either (or both) as well as in systems where there is no distinction between data and instructions. Therefore, to avoid confusion, the term "information" is used as a generalized form of both "data" and "instructions." Although "cache" is described herein as storing information, this can refer to a D-cache that stores data or an I-cache that stores instructions, as well as other memory management techniques such as paging and single-level store.

[0014] Throughout this disclosure, reference is made to "prefetching" and the decision whether to prefetch. As described herein, prefetching refers to "preemptively" reading information from memory into cache (reading information from memory into cache before it is needed). When determining whether to prefetch information, the information is referred to herein as a "prefetch candidate," and the memory address where the information is stored is referred to herein as a "candidate address."

[0015] Throughout this disclosure, reference is made to "addresses" and "indexes." As described herein, an "address" refers to an address in memory (which may refer to volatile memory, such as random access memory (RAM), or non-volatile memory, such as a hard disk drive (HDD)). The information is stored at a specific address in memory. Alternatively, the information may be stored at a specific cache address in a cache. However, for purposes of this disclosure, an "address" refers to a specific location in memory. This disclosure refers to one or more arrays and / or vectors used to track the state of the cache. The "address" of information is hashed (or otherwise transformed) to generate a corresponding "index" in a data structure, such as a vector. The two are related but not interchangeable. The vector value stored at a given index is used to indicate whether the information stored at the corresponding address is still stored in the cache.

[0016] Throughout this disclosure, reference is made to Bloom filters. As described herein, a Bloom filter refers to a probabilistic data structure (e.g., a one-dimensional array or vector). Bloom filters are updated to monitor / track the contents of a cache. Typically, Bloom filters have a value of either “0” or “1” (referred to herein as a “binary” Bloom filter). This value can be updated when a cache write occurs to reflect that particular information has been cached. As an illustrative example, in some embodiments, whenever information stored at an address in memory is written to the cache, the value of a Bloom filter index is set to “1” to reflect this. This index depends on the memory address of the cached information. In an exemplary embodiment, the address of the information written to the cache is hashed, and the hashed result is used as the Bloom filter index. Thus, a system can check whether information at a given address is cached by hashing the address and checking whether the Bloom filter value (at the Bloom filter index corresponding to the hash of the address) is “1” or “0.”

[0017] In particular, the hashing algorithm used on addresses does not necessarily provide a unique output. This allows Bloom filters to be used with indices that are smaller than the size of the address space, but at the cost of the possibility of false positives. For example, if both address A and address B hash to index X, the Bloom filter value for index X is updated when either A or B is cached. For example, caching the information stored at address A sets X to "1." A subsequent determination of whether address B is cached would involve checking index X (which is "1" because of A), indicating that address B is cached, regardless of whether it is actually cached. By properly fine-tuning / adjusting the size of the hashing algorithm / Bloom filter, significant performance improvements can be achieved. A simple indexing algorithm uses as many least significant bits of the cache line address (the address after removing the cache line offset bits) as necessary. For example, to access a 1024-entry Bloom filter, only the 10 least significant bits of the cache line address can be used. More advanced hashing algorithms can apply an XOR function to additional bits of the cache line address, compressing 20-30 bits of the address into a 10-bit index. This can result in more uniform Bloom filter usage. Similarly, the size of the Bloom filter itself can be tuned depending on the size of the cache and the footprint of typical workloads running on the processor. In most cases, a size of 1024 entries has been found to provide sufficient coverage.

[0018] As those skilled in the art will appreciate, when an address is called, it is highly likely (based on temporal locality) that the next few addresses will also be called. Therefore, when an address is read, prefetching the next set of addresses can result in a series of cache hits. However, if the prefetched candidate is already in the cache, the prefetch would have resulted in a cache hit even without the prefetch, making the prefetch redundant and wasteful. The power cost of such "unused" prefetches can be particularly high because the cache directory must be queried to verify whether the prefetch is necessary. Such queries consume a lot of power, especially for highly associative caches. The concept of temporal locality also applies to the contents of the cache, making prefetching after a cache hit particularly prone to wasteful work.

[0019] As an example for clarification, the information could be the instruction "Add 3 to X." This information could be stored in memory at address 5350. When address 5350 is called, the instruction "Add 3 to X" is written to the cache. When this happens, the address (5350) is hashed. As a simple example, the hashing algorithm only needs to take the last two digits of the address, so the hash result here is 50 (the last two digits of 5350 are 50). The hash result of the address is used as an index into a Bloom filter. The value of the filter for this index is set to "1." So, in this example, when the instruction "Add 3 to X" is written to the cache, the 50th entry in the vector is set to "1" (for simplicity, these values ​​are decimal). The Bloom filter can then be checked to determine if the information stored at address 5350 (i.e., "Add 3 to X") is cached. To perform this check, the address is hashed again (to 50 again) and the result of the hash is used as an index (so the 50th value in the Bloom filter is checked). The value is "1", which indicates that the information stored at that address (5350) is in the cache.

[0020] If candidates are not cached, prefetching them can result in a cache hit instead of a cache miss, saving significant time. However, if candidates are already cached, prefetching them is redundant, wasting the power, time, and resources required for prefetching. Thus, an "aggressive" prefetching scheme (e.g., automatically prefetching five subsequent addresses following a called address) may consume more resources than it saves. Therefore, systems and methods consistent with the present disclosure provide a computationally inexpensive way to determine whether candidates are already in the cache to inform the decision whether to prefetch them. Refraining from prefetching candidates if they are already cached can save duplicated work and power.

[0021] However, maintaining one or more Bloom filters to track the contents of a cache can advantageously enable "aggressive" prefetching schemes that might otherwise frequently waste power. Some systems implement multiple caches: an instruction cache (I-cache) and a data cache (D-cache). Instructions are typically read in relatively small, localized groups compared to data from the data cache (D-cache). For example, while it may be common to read thousands of addresses sequentially when reading data, instructions are typically read in groups of roughly 3 to 10. Thus, the potential for aggressive prefetching to be wasted is significantly greater in the context of an I-cache versus a D-cache.

[0022] For example, while prefetching thousands of data addresses may result in at least some wasted work (e.g., the information stored at some of the prefetched addresses is likely already cached), the time and power saved by other prefetches (cache misses turned into cache hits) typically translates to a significant performance gain. However, for the smaller groups handled by an I-cache, only a few wasted prefetches are tolerated for the cost of prefetching to outweigh the savings. As a simple example, 50 wasted prefetches out of 3,000 (typical data reads) is an insignificant waste of resources compared to the benefit of turning 2,950 cache misses into cache hits. On the other hand, 2 wasted prefetches out of 3 (typical instruction reads) may cost more than the resulting single cache hit savings. Given these risks, many systems do not aggressively prefetch in the context of an I-cache because the possible benefits outweigh the corresponding risks. Thus, systems and methods consistent with this disclosure are particularly advantageous when applied to instruction caches (I-caches), but still provide performance improvements when applied to D-caches.

[0023] When the cache is written, the Bloom filter value is set to "1", which over time can cause the Bloom filter to become saturated and, if left unchecked, eventually result in all values ​​being "1". To address this, some embodiments periodically flush the Bloom filter, setting all values ​​to "0". In some embodiments, whenever information is evicted from the cache, the corresponding index in the Bloom filter may be set to "0", which can cause false negatives for other addresses that correspond to the same index when hashed.

[0024] Bloom filters, in some embodiments, may be described herein as having a "life cycle," and are referred to as "old" or "young" relative to these life cycles. After a Bloom filter is flushed, subsequent lookups of the newly flushed filter are likely to return negatives, regardless of the state of the cache. Over time, as information is written to the cache, the filter's value changes from "0" to "1," so the newly flushed filter becomes "mature" once the cache is flushed. To mitigate the likely false negatives returned by the youngest (most recently flushed) filter, many embodiments of the present disclosure utilize at least two different filters. For example, some embodiments set the filter's life to 10,000 cycles, meaning that the filter is flushed every 10,000 cycles. However, using two filters, the flushes can be offset by 5,000 cycles.

[0025] As an illustrative example (but temporarily ignoring the possibility of "false positives" mentioned above for illustrative purposes), if a data address was cached 7,000 cycles ago (i.e., before the younger filter was flushed), the address will only appear in the older filter. In other words, the older filter will have a value of "1" at the index corresponding to the address's hash, while the younger filter will have a value of "0" at the same index. However, if the data address was cached 12,000 cycles ago (i.e., before both filters were flushed), the data address will not appear in either filter. On the other hand, if the data address was cached after the youngest filter was flushed, both filters will indicate that the data address was cached.

[0026] When checking filters to determine if an address is already cached, the system can OR two (or more) filters. This mitigates false negatives from the youngest filter. Every time the cache is written, the corresponding index of all Bloom filters can be set to "1." Thus, checking the OR of filters is effectively the same as checking the oldest filter (although ORing can be faster and more efficient than tracking and determining which filter has not been flushed the longest).

[0027] In some embodiments, rather than specifically setting the value to "1," as in the "binary" Bloom filter described above, the value may be incremented. In this way, the Bloom filter may represent the number of times an address (of cached information) hashed to a particular output. In some embodiments, multiple such Bloom filters may be implemented, and checking the filters may involve taking the maximum value of each index (e.g., if two filters are [0,0,1,0] and [1,0,3,0], a system checking the filters may check for values ​​from [1,0,3,0]). Similar to the logical OR described above with respect to "binary" filters (filters whose values ​​are limited to either "1" or "0"), this is functionally equivalent to checking the "oldest" filter. While this disclosure focuses on the simpler "binary" filter embodiment, this is not intended to be limiting. As will be appreciated by those skilled in the art, Bloom filters with more than two possible values ​​can be implemented with minimal modification.

[0028] In some embodiments, the decision to prefetch can be further informed by knowledge of the specific characteristics (or "behavior") of the hash algorithm used. For example, different hash algorithms can have "output spaces" (the range of possible output values ​​given a range of possible inputs) of various sizes. This can be used in combination with tracking how many times an instruction whose address hashes to a particular output has been cached (e.g., to more accurately determine the likelihood that a given instruction is already cached). As a simple example, a hash algorithm can simply divide the input address by 2 and truncate, and a Bloom filter can suggest that two previous cache writes have stored information about the memory address hashed to a given index. The knowledge that, when a candidate address hashes to a given index, there are only two possible memory addresses in the entire address space that can hash to that particular index (index*2 and index*2+1), combined with the Bloom filter indicating that two previous cache writes relate to the memory address hashed to that index, can strongly suggest that the information is cached. However, these two cache writes could be the result of a single address being written twice to the cache, or two addresses hashing to the same index being written once to the cache. Either way, the filter value could be "2," so there is still the possibility of a false positive.

[0029] As an illustrative example, the output of a first "hash" algorithm may be the last ("ones") digit of the input number (i.e., an input of 64 may result in an output of 4, an input of 1024 may result in the same output of 4, an input of 4096 may result in an output of 6). The output of a second hash algorithm may be the input divided by 16 and truncated (i.e., an input of 64 may result in an output of 4, an input of 1024 may result in an output of 64, an input of 4097 may result in an output of 256, etc.). Notably, because the first algorithm has only 10 possible outputs, i.e., 0 through 9, duplicate outputs are unavoidable in systems where more than 10 distinct inputs are possible. While the truncation of the second algorithm may also result in duplicates, in systems where more than 159 inputs are possible, the "output space" is larger than that of the first algorithm due to the larger number of possible outputs.

[0030] 1 illustrates a high-level cache monitoring method 100 that includes checking the contents of a Bloom filter before determining whether to prefetch information, consistent with some embodiments of the present disclosure. Method 100 can be performed by a computer system that implements one or more memory caches (e.g., system 400 of FIG. 4 ).

[0031] Method 100 includes, in operation 102, determining a set of addresses. In some embodiments, operation 102 may be performed in response to detecting a cache hit. In some embodiments, operation 102 may be performed periodically in response to a read request (e.g., every fifth read request, every read request, etc.). Operation 102 may include determining an address of a first read request. Operation 102 further includes determining one or more subsequent addresses based on the address of the first read request. The one or more subsequent addresses are utilized as a set of “prefetch candidates.” In some embodiments, the set of prefetch candidates may include only a single address (e.g., the address immediately following the address of the first read request). Other numbers of prefetch candidates are also possible (e.g., the next three addresses, the next five addresses, etc.). In embodiments in which method 100 is performed in the context of a data cache, the set may include a significantly larger number of addresses (e.g., the next 5,000 addresses).

[0032] Method 100 includes generating a hash of the prefetch candidate address in operation 104. Various methods of hashing can be utilized; for example, in some embodiments, operation 104 can include performing one or more mathematical operations on the address (e.g., division by a power of two), cropping a predetermined number of bytes of the address, etc. In some embodiments, operation 104 can include performing one or more known hashing algorithms on the address.

[0033] In some embodiments, operation 104 may be performed for each candidate address. In some embodiments, operation 104 may be performed only for the first of a set of candidate addresses, in which case a prefetch decision may be made based on the first candidate and then applied to the remaining ones.

[0034] Method 100 further includes, at operation 106, indexing into the Bloom filter using the hash. Operation 106 may include, for example, utilizing a hash of the candidate address as an index into the vector. In some embodiments, multiple Bloom filters may be utilized to reduce the likelihood of false negative results (as described in more detail below). In some of these “multi-filter” embodiments, operation 106 may include selecting a single filter and checking a value in the hash-derived index. For example, operation 106 may check the “oldest” Bloom filter (where “oldest” is the Bloom filter that has been flushed the longest). To reduce the overhead of tracking the “age” of various filters, in some embodiments, by “checking,” operation 106 may instead OR all of the filters and check the index of the resulting vector. Notably, the “age” of a filter may still be monitored (to determine whether the filter should be flushed), but the same hardware structure may not be used for both functions. For example, the first structure can only keep track of time and reset the oldest filter, while the second structure can read all filters and OR them together.

[0035] As an illustrative example, if the system utilizes two Bloom filters and the hash of the candidate address generated in operation 104 is hexadecimal 10FB, operation 106 may include ORing the two Bloom filters to generate a vector V and checking whether V(4347) is a "1" or a "0" (note that hexadecimal 10FB is equivalent to decimal 4347).

[0036] Method 100 further includes determining whether the candidate address is likely to be a cache hit in operation 108 by analyzing the results of operation 106. For example, in some embodiments, if the value in the index checked in operation 106 is “1,” the corresponding candidate address is considered likely to be cached (“Yes” in 108). Notably, the use of a Bloom filter introduces the possibility of false positives; thus, even if the determination made in operation 108 suggests that the information at the candidate address is likely to be in the cache (meaning that a lookup of the candidate address is likely to be a cache hit), a cache miss is still possible. However, the resources saved by method 100 can enable more aggressive prefetching schemes, which generally outweigh this risk overall.

[0037] The prefetch candidate addresses determined in operation 102 are not guaranteed to be accurate predictions, and regardless of the state of the cache, the addresses evaluated in operations 104-108 may end up uncalled anyway. However, this possibility is generally inherent in the concept of prefetching and has been established as an acceptable risk.

[0038] If a cache hit is not expected for the candidate address ("NO" at 108), method 100 proceeds to operation 112. Operation 112 includes prefetching information stored at the candidate address. For example, operation 112 may include accessing memory and writing the information stored at the candidate address to the cache. Operation 112 may also include evicting the line from the cache, consistent with typical cache operations.

[0039] Method 100 includes, at operation 114, updating the Bloom filter. Regardless of whether the candidate is likely to result in a cache hit or whether the candidate is actually invoked, the Bloom filter is updated to reflect what was actually written to the cache. Operation 114 may include, for example, setting the value of the filter to “1” based on the information written to the cache. For example, in some embodiments, regardless of the next address invoked, the next address is hashed (in a manner similar to operation 104) and the hashed address is used as an index for the Bloom filter. The value of the Bloom filter at this index may be set to “1” regardless of its previous state. Notably, even if the next address invoked is not that of the candidate address, the Bloom filter is updated to reflect that the information stored at the next address will be written to the cache. In some embodiments, operation 114 includes waiting for the information to actually be written to the cache before setting the value at the corresponding index to “1.”

[0040] If a cache hit is expected for the candidate address (“YES” at 108), method 100 proceeds from operation 108 to operation 114. Notably, operation 114 does not involve prefetching the candidate address, as this would be a waste of resources since the information stored at the candidate address is expected to already be in the cache (although this may not be correct as a result of a false positive). Instead, method 100 proceeds to updating the Bloom filter at operation 114. The process of maintaining and / or updating a Bloom filter is described in further detail below with reference to FIG. 2.

[0041] 2 illustrates a high-level flow diagram of a Bloom filter update method 200 consistent with some embodiments of the present disclosure. Method 200 may be performed by a computer system implementing one or more memory caches (e.g., system 400 of FIG. 4 ). Method 200 may enable the system to maintain and / or update one or more Bloom filters to track cache contents (which may advantageously enable a more aggressive prefetching policy).

[0042] Method 200 includes, in operation 202, determining an address (in memory) of information associated with the request. In some embodiments, the address determined in operation 202 may correspond to information that has recently been written, is being written, or is about to be written from memory to the cache (e.g., the address may be the memory address of an instruction recently called by an application). In some embodiments, the address may correspond to information already in the cache. For example, if an application calls an instruction and a subsequent cache lookup results in a cache hit, operation 202 may include determining the address of the instruction even if it was not recently read from memory.

[0043] Method 200 further includes detecting a cache write or a cache hit, or both, at operation 204. Operation 204 may include, for example, detecting that the request associated with the address determined at operation 202 is a call operation or a read operation that will result in a cache write. In some embodiments, a cache write may not have yet occurred at operation 204 (e.g., operation 204 may include detecting an operation immediately prior to the cache write, etc.). In some embodiments, operation 204 may include detecting a cache hit.

[0044] Method 200 further includes generating a hash of the address at operation 206. Various methods of hashing can be utilized; for example, in some embodiments, operation 206 can include performing one or more mathematical operations on the address (such as division by a modulus), cropping a predetermined number of bytes of the address, etc. In some embodiments, operation 206 can include implementing one or more known hashing algorithms on the address.

[0045] Method 200 further includes determining whether the Bloom filter's lifespan has expired at operation 208. Operation 208 may include, for example, reading, calculating, or otherwise determining the number of cycles that have elapsed since the filter was last flushed and comparing that number to a preset maximum value. If the number exceeds the maximum value, the Bloom filter's lifespan has expired ("Yes" at 208), and method 200 proceeds to flush the filter at operation 210. Operation 210 may include, for example, setting all values ​​of the expired filter to "0." Other metrics besides "cycle count" may also be used to track the "age" of a filter, such as elapsed time, number of writes (e.g., to one or more of the filter, cache, memory, etc.), and are contemplated herein. Flushing (e.g., operations 208 and 210) may be performed outside of method 200 (i.e., flushing is not dependent on the execution of operation 204).

[0046] After the expired filter is flushed in operation 208, or if the Bloom filter's lifetime has not yet expired ("NO" at 208), method 200 proceeds to update the Bloom filter's value in operation 212. Operation 212 may include, for example, setting a value at a particular index of the Bloom filter to "1." The particular index at which the value is modified is determined based on the hash address generated in operation 206. For example, if the address determined in operation 202 is "4050" and the hash output of operation 206 is "50," operation 212 may include setting the 50th value of the Bloom filter to "1."

[0047] In some embodiments, rather than setting the value strictly to "1," operation 212 can increment the value by 1. For example, if the hash of the address is again "50," but the 50th value of the Bloom filter is already "1," operation 212 can set the 50th value of the Bloom filter to "2." This effectively allows the Bloom filter to track the number of times information for an address corresponding to a particular index has been written to the cache. This is not necessarily the number of times information for a particular address has been written to the cache, especially since input of several addresses to the hash algorithm can result in the same output (the same problem creates the possibility of false positives). For example, if three addresses "share" an index (meaning all three addresses hash to the same result), a filter value of 4 means that one address has been written to the cache three times, a second address has been written to the cache once, and a third address has not been written to the cache (since the last flush). In an "increment" embodiment, rather than ORing the filters, the filter can be MAXed.

[0048] Method 200 may be performed for all filters. In embodiments including multiple filters, aspects of method 200 (such as operation 212) may be performed for each filter, either serially or in parallel. Method 200 ends at 212.

[0049] FIG. 3 illustrates exemplary Bloom filter implementations 300, 310, and 320 consistent with certain embodiments of the present disclosure. FIG. 3 includes an example of checking a set of Bloom filters at a given index (300), an example of setting a Bloom filter at a given index (310), and an example of flushing a Bloom filter (320). For ease of explanation, these examples include a Bloom filter with eight values, and the value being checked and / or updated is the fourth value (e.g., an address may be hashed to produce an index of 4). These examples are separated by dashed lines 301 and 303.

[0050] Checking the Bloom filters (300) can include ORing each Bloom filter. In Figure 3, three Bloom filters (302, 304, and 306) are shown being checked. A result vector 308 is generated by ORing filters 302, 304, and 306. If the fourth value (from left to right) in result vector 308 is "1," this can indicate that the information at the address hashed to index "4" is in the cache. Because this result vector is effectively the same as the "oldest" Bloom filter, the result vector itself can be thought of as another Bloom filter.

[0051] In some embodiments, this result vector is generated every cycle. This can advantageously reduce the time required to check whether an address is likely to result in a cache hit, because the result vector is generated in advance. However, this can result in additional power consumption, because only a single value in the result vector is likely to be checked before the entire vector is replaced. Therefore, in some embodiments, the result vector is not generated until the index is known, at which point the value of each Bloom filter at that index is checked (checking stops as soon as a "1" is found). This can result in power savings, but can be slower than the result vector approach, because multiple values ​​must be checked and it begins only after the index is determined.

[0052] When the information whose address hashed to index "4" is written to the cache, the Bloom filters are all updated by setting their fourth value to "1," as shown at 310. In FIG. 3, filters 312, 314, and 316 represent updated versions of filters 302, 304, and 306, respectively. Notably, filters 302 and 304 had a fourth value of "0," while filters 312 and 314 have their fourth value set to "1." Although filter 306 already had a fourth value of "0," in some embodiments, filter 316 may still undergo a write operation (overwriting one "1" with another "1"). This may be redundant, but may also reduce system complexity. In some embodiments, the value is checked and overwritten only if the final result is different (in other words, the check determines whether the fourth value of filter 306 is "1," and if so, it may be left as is to save resources). In some embodiments, rather than setting a particular value to "1," the value may instead be incremented. For example, the fourth value of filter 316 can be set to "2" (an increment above the "1" of filter 306).

[0053] Filters can also be flushed periodically, an example of which is shown in column 320. Flushing a filter can include setting all values ​​in the filter to "0." As shown in FIG. 3, filter 326 (corresponding to filters 306 and 316) is flushed. As described above, filters can be flushed based on time, cycle count, write count, etc. For example, a filter can be flushed every 10,000 cycles. In embodiments with multiple filters (such as the example shown in FIG. 3), the filters can be flushed at the same frequency but offset from one another. For example, filters 322, 324, and 326 can be flushed every 10,000 cycles, but filter 326 can be flushed at cycle X, filter 322 can be flushed at cycle (X+3,333), filter 324 can be flushed at cycle (X+6,667), and then filter 326 can be flushed again at cycle (X+10,000). Combined with updating the filters simultaneously (as shown in column 310) and ORing all filters together when they are checked (as shown in column 300), this allows result vector 308 to essentially reflect the cache contents based on the last 10,000 cycles (but unaffected by any operations that occurred more than 10,000 cycles ago). In this way, result vector 308 can be used as a relatively quick means to check the contents of the cache without significant overhead. The frequency of flushing and the number of filters can be adjusted depending on the system configuration. For example, if a system has a particularly small cache and its contents are typically purged within 5,000 cycles of being written, the filters can be flushed more frequently (e.g., every 5,000 cycles instead of every 10,000 cycles) to reduce the chance that a Bloom filter check will result in a "false positive."

[0054] 4, a high-level block diagram of an exemplary computer system 400 is shown that can be configured to perform various aspects of the present disclosure, including, for example, methods 200 and 300, respectively. The exemplary computer system 400 can be used to perform one or more of the methods or modules described herein, and any associated functions or operations, in accordance with embodiments of the present disclosure (e.g., using one or more processor circuits or computer processors of a computer). In some embodiments, the major components of the computer system 400 can include one or more CPUs 402, a memory subsystem 408, a terminal interface 416, a storage interface 418, an I / O (input / output) device interface 420, and a network interface 422, all of which can be communicatively coupled, directly or indirectly, for inter-component communication via a memory bus 406, an I / O bus 414, and an I / O bus interface unit 412.

[0055] Computer system 400 may include one or more general-purpose programmable central processing units (CPUs) 402, some or all of which may include one or more cores 404A, 404B, 404C, and 404D, generally referred to herein as CPUs 402. In some embodiments, computer system 400 may include multiple processors, which is typical of relatively large systems, while in other embodiments, computer system 400 may alternatively be a single CPU system. Each CPU 402 may execute instructions stored in a memory subsystem 408 on a CPU core 404 and may include one or more levels of on-board cache.

[0056] In some embodiments, memory subsystem 408 may include random-access semiconductor memory, storage devices, or storage media (either volatile or nonvolatile) for storing data and programs. In some embodiments, memory subsystem 408 may represent the entire virtual memory of computer system 400 and may also include the virtual memory of other computer systems coupled to computer system 400 or connected via a network. While memory subsystem 408 may conceptually be a single monolithic entity, in some embodiments, memory subsystem 408 may be a more complex configuration, such as a hierarchy of caches and other memory devices. For example, memory may exist in multiple levels of caches, and these caches may be further divided by function, such that some caches hold instructions and other caches hold non-instruction data used by the processor(s). Memory may be further distributed and associated with different CPUs or sets of CPUs, as in any of a variety of so-called non-uniform memory access (NUMA) computer architectures. In some embodiments, main memory or memory subsystem 804 may include elements for control and flow of memory used by CPU 402. This may include memory controller 410.

[0057] 4 as a single bus structure providing a direct communication path between CPU 402, memory subsystem 408, and I / O bus interface 412, memory bus 406, in some embodiments, can include multiple distinct buses or communication paths, which can be configured in any of a variety of forms, such as point-to-point links in a hierarchical, star, or web configuration, multiple hierarchical buses, parallel and redundant paths, or any other suitable type of configuration. Additionally, while I / O bus interface 412 and I / O bus 414 are each shown as single units, computer system 400, in some embodiments, can include multiple I / O bus interface units 412, multiple I / O buses 414, or both. Additionally, although multiple I / O interface units are shown, this separates I / O bus 414 from the various communication paths carried by the various I / O devices; in other embodiments, some or all of the I / O devices can be directly connected to one or more system I / O buses.

[0058] In some embodiments, computer system 400 may be a multi-user mainframe computer system, a single-user system, or a server computer or similar device that has little or no direct user interface but receives requests from other computer systems (clients). Further, in some embodiments, computer system 400 may be implemented as a desktop computer, a portable computer, a laptop or notebook computer, a tablet computer, a pocket computer, a telephone, a smartphone, a mobile device, or any other suitable type of electronic device.

[0059] It should be noted that Figure 4 is intended to illustrate representative major components of an exemplary computer system 400. However, in some embodiments, the individual components may be more or less complex than those depicted in Figure 4, components other than or in addition to those shown in Figure 4 may be present, and the number, type, and configuration of such components may vary.

[0060] Throughout this disclosure, references are made to setting various values ​​of a Bloom filter to "1" (interpreted to mean that the information is likely to be in the cache) and occasionally flushing the filter to set all of its values ​​to "0" (interpreted to mean that the information is likely not in the cache). As will be appreciated by those skilled in the art, the specific values ​​used ("1" and "0") are used for illustrative purposes only and are not meant to be limiting. Other possible conventions are fully contemplated herein; for example, in some embodiments, "0" may be interpreted to mean that the information is likely to be in the cache, but the filter may be occasionally "flushed" by setting all values ​​to "1." For further clarity, in some embodiments, "5" may mean that the information is likely to be in the cache, while "36" may mean that the information is unlikely to be in the cache.

[0061] The present invention may be a system, method, or computer program product, or combination thereof, at any possible level of technical detail of integration. The computer program product may include computer-readable storage medium(s) having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0062] A computer-readable storage medium may be any tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, or semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanical coding devices such as punch cards or raised groove structures having instructions recorded thereon, and any suitable combination of the above. As used herein, computer-readable storage media is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses through fiber optic cable), or electrical signals sent through wires.

[0063] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network can include copper cables, optical fibers, wireless networks, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0064] The computer-readable program instructions for carrying out the operations of the present disclosure may be source or object code written in any combination of assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer readable program instructions to individualize the electronic circuitry by utilizing state information of the computer readable program instructions to implement aspects of the present disclosure.

[0065] Aspects of the present invention will be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0066] These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, whereby the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer program instructions can also be stored in a computer-readable medium that can direct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, whereby the instructions stored in the computer-readable medium can include an article of manufacture containing instructions that implement aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0067] The computer program instructions may be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to generate a computer-implemented process, whereby the instructions running on the computer, other programmable apparatus, or other device perform the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0068] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions shown in the blocks may occur in a different order than that shown in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or a combination of dedicated hardware and computer instructions.

[0069] The descriptions of various embodiments of the present disclosure have been presented for illustrative purposes, but they are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein have been selected to best explain the principles of the embodiments, practical applications, or technical improvements over the art found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method comprising: predicting candidate addresses; computing a hash of the candidate address; ORing the first Bloom filter with the second Bloom filter based on the hash to generate a result vector; checking the result vector; determining, based on the result vector, to prefetch information stored at the candidate addresses; A method comprising:

2. The predicting step comprises: predicting a second candidate address; computing a second hash of the second candidate address; checking the first Bloom filter based on the second hash; determining, based on said checking, that prefetching of second information stored at said second candidate address is unnecessary; The method of claim 1 , comprising:

3. The predicting step comprises: determining a first address of a cache hit; determining the candidate addresses based on the first address; 3. The method of claim 1 or claim 2, comprising:

4. Detecting a cache write; updating values ​​of the first Bloom filter and the second Bloom filter based on the hash; The method of claim 1 further comprising:

5. The method of claim 4 , wherein the updating comprises incrementing the value.

6. 6. The method of claim 5, further comprising periodically flushing the first Bloom filter and the second Bloom filter, wherein the flushing of the first Bloom filter is offset from the flushing of the second Bloom filter.

7. 1. A system comprising: Memory and a processing unit coupled to the memory; and wherein the processing unit comprises: determining the address of the cached information; Compute a hash of said address; determining an index based on the hash; updating values ​​of a first Bloom filter and a second Bloom filter based on the hash; predicting candidate addresses; ORing the first Bloom filter with the second Bloom filter to generate a result vector; Checking the result vector; determining, based on the result vector, whether to prefetch information stored at the candidate address; 20. A system configured to execute instructions for:

8. The processing unit Detect a cache hit, predicting the candidate address associated with the cache hit; determining whether to prefetch information stored at the candidate address based on the result vector; The system of claim 7 further configured to:

9. The predicting step comprises: determining a first address of a cache hit; determining the candidate addresses based on the first address; The system of claim 8 , comprising:

10. The determination is computing a candidate hash of the candidate address; determining a candidate index based on the candidate hashes; checking candidate values ​​at said index of said result vector; The system of claim 8 , comprising:

11. 11. The system of claim 7, wherein the processing unit is further configured to periodically flush the first Bloom filter and the second Bloom filter, the flushing of the first Bloom filter being offset from the flushing of the second Bloom filter.

12. The system of claim 7 , wherein the updating comprises incrementing the value.

13. A computer program comprising program instructions executable by a computer, the program instructions causing the computer to: Determine the address of the information written to the cache; Compute a hash of said address; determining an index based on the hash; updating values ​​of a first Bloom filter and a second Bloom filter based on the hash; Predict candidate addresses, ORing the first Bloom filter with the second Bloom filter to generate a result vector; Checking the result vector; determining, based on the result vector, whether to prefetch information stored at the candidate addresses; Computer program.

14. The program instructions further cause the computer to: Detect a cache hit, predicting the candidate address associated with the cache hit; determining whether to prefetch information stored at the candidate address based on the result vector; 14. A computer program product according to claim 13.

15. The determination is computing a candidate hash of the candidate address; determining a candidate index based on the candidate hashes; checking candidate values ​​at said index of said result vector; 15. The computer program of claim 14, comprising:

16. 16. The computer program product of claim 13, wherein the program instructions further cause the computer to periodically flush the first Bloom filter and the second Bloom filter, the flushing of the first Bloom filter being offset from the flushing of the second Bloom filter.

17. 17. The computer program product of claim 13, wherein the updating comprises incrementing the value.

18. A computer readable storage medium having stored thereon a computer program according to any one of claims 13 to 17.

Images