A system and method that uses a hash table with a set of frequently accessed buckets and a set of less frequently accessed buckets

Abstract

The method and apparatus perform a first hash operation on a first key, the first hash operation biased to map the first key and its associated value to a set of frequently accessed buckets of a hash table. An entry for the first key and its associated value is stored in the set of frequently accessed buckets. A second hash operation is performed on a second key, biased to map the second key and its associated value to a set of less frequently accessed buckets of the hash table. An entry for the second key and its associated value is stored in the set of less frequently accessed buckets. A hash table lookup for the requested key is performed in the set of frequently accessed buckets, and if the requested key is not found, a hash table lookup is performed in the set of less frequently accessed buckets.

Description

Background Art

[0001] A hash table is a data structure that implements associative arrays. In some hash tables, one or more hash functions are used to map a given "key" to one or more indexes that identify buckets in the hash table that can hold the "value" (or pointer to the value) associated with the key. Usually, a bucket contains multiple "slots", and each slot can hold a single key-value pair. Since any slot within a bucket can hold any item mapped to that bucket, the slots within a bucket are fully associative. Hash tables are widely used in many application domains. An ideal hash table attempts to evenly distribute its contents across all buckets. For example, a software library called by an application employs core primitives such as a hash table that provides wide applicability across a wide range of application domains including relational databases, key-value stores, machine learning applications, and other uses.

[0002] However, usually, each bucket contains a mixture of frequently accessed (FA) data and infrequently accessed (IA) data. So, even if the data itself has temporal locality, in the case of a very large hash table, if the contents are evenly distributed across all buckets, cache performance may degrade. For example, a hardware cache is designed to capture frequently accessed data. However, due to the design (i.e., the uniform distribution and placement of items within a bucket), the hash table may mix frequently accessed items and infrequently accessed items within the same cache line, preventing the cache from capturing the hot working set.

[0003] Some conventional high-performance hash tables typically use multiple mapping hash functions (usually two) to map a given key to multiple candidate buckets within the hash table. A simplified example is shown in Figure 1. Generally, when inserting a new data item 10 into a hash table 12, the outputs of hash functions 14, 16 are calculated in parallel, and the item is inserted into the candidate bucket with the slot that is least occupied. In an example of a hash table with eight buckets and four slots per bucket, the occupied slots are shown in gray and the empty slots are shown in white. As shown, the new item (K, V) with key K and value V is inserted using two hash functions F1() and F2() for the identified candidate buckets 5 and 0. If the insertion policy is to insert into the candidate bucket with the least load, then in this example (K, V) is inserted into bucket 5. This makes the reads more evenly distributed among the various buckets of the hash table, enabling uniform reads even at a high load factor. Alternatively, the hash functions can be evaluated sequentially, and the item can be inserted into the first candidate bucket where there is available space. If all candidate buckets are full, an evacuation (such as a cockroach evacuation) is used where one of the existing items is evicted to create space for the new data item. Then, the evicted item is reinserted into one of its alternative candidate buckets. If all candidate buckets are also full, the chain of evacuations continues until the evacuation ends, such as when the last evicted item fits into one of its candidate buckets.

[0004] During the lookup operation, the mapping hash function is applied to the key being searched to locate the candidate bucket. Next, each slot in the candidate bucket is searched and the key of interest is compared to the keys in each slot of the candidate bucket. Since the slots within a bucket are fully associative, all slots must be searched (i.e., the mapping hash function only identifies the bucket and does not identify a specific slot within the bucket). Hash tables typically have poor cache performance for large hash tables because they attempt to evenly distribute data elements across the entire bucket. For example, even if a high percentage of accesses to a data set target only a small portion of the data items, they are distributed across the entire hash table. Additionally, hash buckets typically have a size such that multiple slots of the bucket fit within a single cache line. Thus, when fetching a single frequently used item into the cache, multiple infrequently used items that are part of the same bucket may also be fetched, thereby degrading the cache's ability to capture a useful set of frequently accessed items.

[0005] Therefore, it is highly advantageous to improve hash table operations for cache or other uses.

[0006] This embodiment may be more readily understood in consideration of the following description when accompanied by the following drawings, in which like reference numerals represent like elements.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

DETAILED DESCRIPTION OF THE INVENTION

[0008] Briefly stated, the system and method divide the buckets of a hash table into two relatively prime sets: frequently accessed buckets and less frequently accessed buckets. The terms "less frequently" and "infrequently" are used interchangeably herein. In some embodiments, some of the buckets within the frequently accessed set are selected such that frequently accessed buckets are likely to fit within a hardware cache. In some embodiments, a subset of the mapping hash functions, called frequently accessed (FA) mapping functions, is biased to map primarily to the set of frequently accessed buckets within the hash table, and the remaining hash functions are a subset of the mapping hash functions and are biased to map primarily to the set of less frequently accessed buckets within the hash table. In some embodiments, tracking data is maintained to identify frequently accessed data items and infrequently accessed data items. Based on the tracking data, frequently accessed elements are identified and moved to the frequently accessed buckets of the hash table, while data items with low access frequency remain in the less frequently accessed buckets.

[0009] According to certain embodiments, a method executed by one or more processors performs a first hash operation on a first key, where the first hash operation is biased to map the first key and its associated value to a first candidate bucket within a first set of buckets of a hash table that functions as a set of frequently accessed buckets. The method includes storing an entry for the first key and its associated value in a first candidate bucket within the first set of buckets of the hash table, such as a cache memory having a size adapted to a set of frequently accessed buckets. The method also includes performing a second hash operation on a second key and storing an entry for the second key and its associated value in a second candidate bucket within a second set of buckets of the hash table, where the second hash operation is biased to map the second key and its associated value to a second candidate bucket within the second set of buckets of a hash table that functions as a set of less frequently accessed buckets. In response to a key lookup request, the method includes performing a hash table lookup of the requested key in the first set of frequently accessed buckets, and if the requested key is not found, the method includes performing a hash table lookup of the requested key in the set of less frequently accessed buckets. In other embodiments, the method includes performing the lookup using both hash functions and both sets of buckets in parallel.

[0010] In some examples, the method includes biasing the first and second hash operations such that the first hash operation exclusively maps the first key and its associated value to a first set of buckets of a hash table that functions as a set of frequently accessed buckets, and the second hash operation exclusively maps the second key and its associated value to a second set of buckets of a hash table that functions as a set of less frequently accessed buckets.

[0011] In certain examples, a default priority is given to store entries in buckets that are not accessed very frequently. In some embodiments, all key-value pairs are first inserted into a set of buckets that are not accessed very frequently, and then, if the value is found to be accessed frequently, it is moved to the set of buckets that are accessed frequently. In some examples, this method includes evicting a hash table entry if a second set of buckets of the hash table that function as buckets that are not accessed very frequently is full, and storing the entry of the hashed second key into the second set of buckets. The evicted entry can be selected based on the access frequency such that if there are more frequently accessed entries, they are evicted from the second set of buckets that function as buckets that are not accessed very frequently and moved to the first set of buckets that function as buckets that are accessed frequently.

[0012] In certain examples, this method includes generating tracking data that represents the number of times each slot in the buckets within the first set of frequently accessed buckets of the hash table and the second set of buckets of the hash table that are not accessed very frequently has been accessed. In some examples, this method includes moving entries from the second set of buckets that are not accessed very frequently to the first set of buckets that are accessed frequently, or vice versa, based on the tracking data.

[0013] In some examples, this method includes performing first and second hash operations based on weighting (on a weighted basis) in response to a table entry request. In certain examples, this method includes storing and using per bucket fullness data to determine whether to move a hash table entry between the first and second sets of buckets of the hash table.

[0014] According to certain embodiments, a computing device includes hash table logic that performs a first hash operation on a first key and stores an entry of the first key and its associated value in a first candidate bucket within a first set of buckets of a hash table, the first hash operation being biased to map the first key and its associated value to a first candidate bucket within a first set of buckets of a hash table that functions as a set of frequently accessed buckets. In some embodiments, the hash table logic performs a second hash operation on a second key and stores an entry of the second key and its associated value in a second candidate bucket within a second set of buckets of the hash table, the second hash operation being biased to map the second key and its associated value to a second candidate bucket within a second set of buckets of a hash table that functions as a set of less frequently accessed buckets. In response to a key lookup request, in certain embodiments, the hash table logic performs a hash table lookup of the requested key in the first set of frequently accessed buckets, and if the requested key is not found, performs a hash table lookup of the requested key in the set of less frequently accessed buckets. In other embodiments, both hash functions and both sets of buckets are used in parallel to perform the lookup.

[0015] In some examples, the computing device biases the first and second hash operations such that the first hash operation exclusively maps the first key and its associated value to a first set of buckets of a hash table that functions as a set of frequently accessed buckets, and the second hash operation exclusively maps the second key and its associated value to a second set of buckets of a hash table that functions as a set of less frequently accessed buckets.

[0016] In some examples, when the second set of buckets of the hash table, which functions as buckets that are less frequently accessed, is full, the hash table logic evicts (expels) a hash table entry and stores the entry for the hashed second key in the second set of buckets. The evicted entry can be selected based on access frequency such that it is evicted from the second set of buckets that function as buckets where more frequently accessed entries are less frequently accessed and moved to the first set of buckets that function as frequently accessed buckets.

[0017] In some examples, the hash table logic further includes a slot access counter structure that generates tracking data representing the number of times each slot in the buckets within the first set of frequently accessed buckets and the second set of less frequently accessed buckets of the hash table has been accessed. In some embodiments, the hash table logic moves entries from the second set of less frequently accessed buckets to the first set of frequently accessed buckets, or vice versa, based on the tracking data.

[0018] In a particular example, the hash table logic performs first and second hash operations based on weighting (weighted basis) in response to a table entry request. In some examples, the hash table logic stores and uses fullness data for each bucket to determine whether to move a hash table entry between the first and second sets of buckets of the hash table. In some examples, a computing device includes a memory that includes a hash table and one or more processors that operate as the hash table logic.

[0019] According to some embodiments, when executed by one or more processors, the non-transitory memory medium causes the one or more processors to execute a first hash operation that biases a first key and its associated value to map to a first candidate bucket within a first set of buckets of a hash table that functions as a set of frequently accessed buckets, stores an entry of the first key and its associated value in the first candidate bucket within the first set of buckets of the hash table, executes a second hash operation that biases a second key and its associated value to map to a second candidate bucket within a second set of buckets of a hash table that functions as a set of less frequently accessed buckets, stores an entry of the second key and its associated value in the second candidate bucket within the second set of buckets of the hash table, and in response to a key lookup request, executes a hash table lookup of the requested key in the first set of frequently accessed buckets, and if the requested key is not found, executes a hash table lookup of the requested key in the second set of less frequently accessed buckets. In other embodiments, both hash functions and both sets of buckets are used in parallel to perform the lookup.

[0020] In a particular example, when executed by one or more processors, the non-transitory memory medium causes the one or more processors to bias the first and second hash operations such that the first hash operation exclusively maps a first key and its associated value to a first set of buckets of a hash table that functions as a set of frequently accessed buckets, and the second hash operation exclusively maps a second key and its associated value to a second set of buckets of a hash table that functions as a set of less frequently accessed buckets.

[0021] In certain examples, when executed by one or more processors, the non-transitory storage medium includes executable instructions such that when a second set of buckets of a hash table that function as buckets that are not accessed very frequently is full, the one or more processors evict a hash table entry and store an entry for a second hashed key in the second set of buckets. The evicted entry may be selected based on access frequency such that it is evicted from the second set of buckets that function as buckets that are not accessed very frequently and moved to a first set of buckets that function as buckets that are accessed frequently when there are entries that are accessed more frequently.

[0022] In some examples, when executed by one or more processors, the non-transitory storage medium includes executable instructions such that the one or more processors generate tracking data representing the number of times each slot in buckets within a first set of buckets of a hash table that are accessed frequently and a second set of buckets of the hash table that are not accessed very frequently is accessed, and based on the tracking data, move entries from the second set of buckets that are not accessed very frequently to the first set of buckets that are accessed frequently, or vice versa.

[0023] FIG. 2 is a diagram showing an example of a part of a computing system such as a part of a hardware server, smartphone, wearable, printer, laptop, desktop, or any other suitable computing device using a hash table. In this example, a part of the computing device includes a memory 200 such as a cache or main memory of a processor such as a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), an application specific integrated circuit (ASIC), or other integrated circuits, including a hash table 202 and hash table logic 204. The hash table logic 204 interfaces with the memory 200 to populate and organize the hash table 202. In one example, the hash table logic 204, when executed, is implemented as a programmable processor that executes executable instructions stored in the memory to operate the processor in the manner of the hash table logic described herein. In other embodiments, the hash table logic is implemented as discrete logic, one or more state machines, a field programmable gate array (FPGA), or a suitable combination of a processor that executes instructions and other hardware. In this example, the hash table logic 204 includes a plurality of hash generators 206, 208, a selector 210, control logic 212, a slot counting structure 214, a hash table bucket fullness determiner 216, a bucket mover 218, and a lookup logic 220. It will be recognized that functional blocks are shown and can be combined or separated as needed for various operations. It will also be recognized that not all functional blocks are required in all embodiments.

[0024] During operation, when populating the hash table 202, the hash table logic 204 receives a key (K) 222 and its associated value (V) from an application, a service within a computing system, or another device for input into the hash table 202. After the requested key has been populated (i.e., inserted) into the hash table 202, an application or other entity may request to retrieve cached data using the stored value from a cache that searches the hash table 202. In this way, a hash table lookup request 224 is received and the lookup logic 220 processes a hash table entry request based on the use of the hash table 202, as further described below, to retrieve the cached data.

[0025] Referring also to FIGS. 3 and 4, a method of operation 300 is shown in relation to the block diagram of FIG. 2 and an example of a hash table 202 according to an example shown herein. In some embodiments, the hash generator 206 implements a hash function that exclusively maps to a set of infrequently accessed (IA) buckets (buckets 3 through 7), also referred to as an IA bucket set, and the hash generator 208 implements a hash function that maps only to a set of frequently accessed (FA) buckets (buckets 0 through 2), also referred to as an FA bucket set (see FIG. 4). These functions are shown in the following examples and use the same nomenclature as shown in FIG. 1, but are different hash functions. f1’(x)=mod(f1(x),N) (where x is the key) Equation 1 f2’(x)=N+mod(f2(x),M) Equation 2

[0026] Equation 1 shows an example of the hash function f1’(x) implemented by the hash generator 208, and Equation 2 represents the hash function operation f2’(x) provided by the hash generator 206. Different from the system in FIG. 1, the system described herein divides the buckets of the hash table into a set of frequently accessed buckets (buckets 0 to 2) and a set of buckets with low access frequency (buckets 3 to 7). By doing so, the first N buckets of the hash table correspond to the FA bucket set 228, and the remaining buckets M to N are divided into a bucket set 230 that is not accessed very frequently, where M is the total number of buckets in the hash table. In some embodiments, the FA bucket set is stored in the cache, and the IA buckets are stored in the main memory or other memory.

[0027] Referring back to Equation 1 and Equation 2, the modular operation can be implemented whether N and M are powers of 2 or not powers of 2. Each of the hash generators 206, 208 provides a hash operation biased in such a way that, for example, the hash operation provided by the hash generator 208 is biased to map the key and its associated value to the FA bucket set 228, which is a set of buckets of the hash table that functions as a set of frequently accessed buckets, while the hash generator 206 executes a hash operation on the key and is biased to map the key and its associated value to another different set of buckets (also called the IA bucket set 230) that is assigned as the buckets not accessed very frequently. The hashed key generated by the hash generator 206 is shown as HIA(K), while the hashed key generated by the hash generator 208 is shown as HFA(K).

[0028] As shown in block 302, this method includes performing a hash operation on a key (K) 222 having an associated value (V), and the hash operation biases the key 222 and its associated value to map to a set of FA buckets 228 which is a set of buckets of a hash table that functions as a set of frequently accessed buckets in this example. Thus, in this example, the control logic 212 sends a selection control signal 232 to control the selector 210 to enable the key 222 to be passed to the hash generator 208. As shown in block 304, the hash generator 208 stores an entry of the key 222 in a set of FA buckets 228 corresponding to a set of frequently accessed buckets in the hash table. This may occur, for example, when the set of buckets 230 with low access frequency is full. However, other suitable criteria can also be used.

[0029] As shown in block 306, this method includes performing a different hash operation on a different key provided to hash table logic 204, also referred to as key 222. The different hash operation is biased to map key 222 and its associated value to another bucket set of the hash table that functions as bucket set 230 which is accessed less frequently. Thus, control logic 212 provides a selection control signal 232 to selector 210 to provide key 222 to hash generator 206 which performs a hash operation on the key and stores the hashed key entry within IA bucket set 230 of the hash table as shown in block 308. As shown in block 310, in response to a hash table lookup request 224 from an application or other service, to request a value from the hash table, this method includes performing a hash table lookup of the requested key within frequently accessed bucket set 228. Thus, lookup logic 220 causes hash generator 208 to perform a hash on the received key which is part of hash table lookup request 224 as indicated by hash generation request 238, and lookup logic 220 searches for the key hashed in frequently accessed bucket set 228. If the requested key is not found, lookup logic 220 requests that hash generator 206 generate a hash of the key searched by lookup logic 220 within another bucket set, namely, bucket set 230 with low access frequency, as indicated by hash generation request 241. It will be recognized that the operations described herein can be performed in any suitable order including in parallel, or in any other different order as required. If the key is first found in frequently accessed bucket set 228, lookup logic 220 searches for value 240 and provides a hash table response 242 that includes value 240 searched in response to hash table lookup request 224. The hash table lookup is performed by lookup logic 220 for each lookup as required.

[0030] As described above, for Equations 1 and 2, the hash generators 206, 208 are biased through the hash functions executed by their respective hash generators. For example, one hash operation exclusively maps keys and their associated values to a set of buckets in a hash table that functions as a set of frequently accessed buckets. Another hash generator is configured to exclusively map keys and their associated values to another set of buckets in a hash table that functions as a set of buckets that are not accessed as frequently.

[0031] Many different hash table insertion techniques can be employed. In some embodiments, insertion into the hash table 202 is always performed on the low-access-frequency bucket set 230, except when the candidate low-access-frequency buckets are full. When the candidate IA buckets are full, cuckoo hashing is used to enable insertion into the IA buckets. In other embodiments, when the candidate IA buckets are full, the control logic 212 controls the selector 210 to insert into the FA bucket set 228. In other embodiments, both approaches are used in combination based on heuristics such as the fullness of the entire table or the IA buckets. In some examples, when the candidate buckets are full and the overall load of the IA bucket set 230 is higher than the overall load of the FA bucket set 228 by a set margin, insertion into the FA buckets is performed; otherwise, insertion into the IA buckets is performed using cuckoo hashing.

[0032] In other embodiments, based on a statically or dynamically determined probability distribution, under the control of the control logic 212, insertion is performed into both the IA bucket set 230 and the FA bucket set 228. In some embodiments, a random number generator is used to determine, for each insertion, whether to switch the selector so that the hash generator 206 or the hash generator 208 executes the hash operation and places the hashed key and its associated value in the FA bucket set 228 or the IA bucket set 230.

[0033] In another example, access tracking is performed. In some examples, tracking data 400 representing the number of times each slot in a set of frequently accessed buckets of a hash table and a set of less frequently accessed buckets of the hash table has been accessed is generated by the lookup logic 220 or the slot counting structure 214. In one embodiment, the metadata is used as the tracking data 400 and stored within the hash table 202. In other embodiments, a slot access counter structure 214, such as a slot counter, generates the tracking data 400. In this example, a counter is associated with each slot of the hash table to identify the access frequency of each slot. To reduce the overhead of capacity, the counter is limited to a small number of bits, such as 2 or 3 bits per counter, and can saturate instead of overflowing when reaching the maximum value. Data items, such as slots where the access counter exceeds a threshold, are moved in one example from buckets with a low access frequency in the IA bucket set 230 to buckets in the frequently accessed bucket set 228. This can be performed, for example, using a frequently accessed mapping function. The mapping may in some examples be similar to a new insertion into a frequently accessed bucket set, such as the utilization of the cuckoo shrimp in the frequently accessed set.

[0034] In some embodiments, the counter is reset periodically to ensure that only items accessed frequently enough to reach a threshold between counter resets are identified as frequently accessed. In other embodiments, items that are no longer frequently accessed are remapped to a set of buckets with low access frequency. If it is identified that there are too many or too few frequently accessed slots, i.e., it manifests as a large difference in occupancy between the FA bucket set and the IA bucket set, then either the threshold or the counter reset frequency is changed so that the criteria for what is considered frequently accessed is shifted. This can be performed by the lookup logic 220. When a counter structure is used, another data structure is treated as a hardware or software managed cache, and only the counters for a subset of frequently accessed items are maintained. An item that is not accessed frequently enough for its counter to remain in the counter cache may be considered to have a low access frequency. However, any suitable counting mechanism can be used.

[0035] For data sets where the accessed behavior does not change rapidly over time, tracking of access frequency need only be performed periodically and not all at once. Once sufficient profile information has been collected, further access tracking can be paused to avoid the performance overhead of such tracking. In other examples, access tracking is only performed probabilistically, and the tracking overhead occurs only for a small portion of the lookups.

[0036] In certain embodiments, during lookups, items are always first looked up using a frequently accessed mapping function. If the item is not found in the FA candidate bucket, the IA mapping function is used to look up the IA bucket set. This can increase the number of cache accesses compared to previous systems, but for elements within the IA bucket (e.g., to first check the FA bucket), in this way, the number of main memory accesses for most lookups for a small subset (e.g., a certain form of social network data) of the dataset can be reduced. Also, as described above, two mapping hash functions are used, but any appropriate number can be used. For example, two IA mapping functions can be used, one FA mapping function can be used, or any appropriate number of hash functions can be used.

[0037] In other embodiments, a different lookup approach can be adopted for memory reads of IA buckets, etc., issued in parallel with the lookup of FA buckets, such that the memory access latency of the IA buckets overlaps with the check of the FA buckets. This can improve the latency compared to other embodiments.

[0038] The hash table bucket fullness determiner 216 determines which buckets are full at the level of each bucket and / or, in another example, which bucket sets are full at the level of each bucket set. The control logic 212 notifies the bucket mover 218 to evict a hash table entry, for example, if the IA bucket is full, and then stores the entry in a second set of buckets. The hash table bucket fullness determiner 216 provides bucket fullness data 250 indicating which buckets are full and / or whether a bucket set is full to the control logic. It will be recognized that the described functional blocks can be combined with other functional blocks or otherwise modified as needed.

[0039] Figure 5 is a flowchart showing an example of a method 500 for inserting a hash table entry into a hash table. It will be recognized that the described operations can be performed in any suitable order and other operations can be used as needed. In this example, the hash table is first inserted into the set with the lowest access frequency, except when the bucket with the lowest candidate access frequency is full. When the bucket with the lowest candidate access frequency is full, a hermit crab operation is used to enable insertion into the bucket with the lowest access frequency. In other embodiments, when the bucket with the lowest candidate access frequency is full, insertion into the frequently accessed bucket set is performed. Both approaches are used in combination based on heuristics such as the fullness of the entire hash table or the IA bucket. For example, in some embodiments, when the candidate bucket is full and the overall load of the bucket with the lowest access frequency is a certain margin higher than the overall load of the frequently accessed bucket, insertion into the frequently accessed bucket is performed; otherwise, the hermit crab operation is performed for insertion into the IA bucket.

[0040] As shown in block 501, the method includes performing a hash operation biased to map a key and its associated value to a set of buckets in a hash table that functions as a set of less frequently accessed buckets for the key. As shown in block 502, the method includes determining whether candidate buckets with low access frequency are full. In one example, the control logic 212 uses the bucket fullness data 250 to determine whether to move hash table entries between the IA bucket set 230 and the FA bucket set 228 of the hash table 202. For example, the hash table bucket fullness determiner 216 tracks each of buckets 3 - 7 in the IA bucket set 230. If a candidate bucket in the IA bucket set 230 is full, the method continues to block 504. In one example, the control logic 212 causes the hash generator 208 to perform a hash for the key that is frequently accessed, and as shown in block 506, the hash generator 208 exclusively stores the key and its associated value in the frequently accessed bucket set 228. In other embodiments, as shown in block 508, if a candidate bucket in the low access frequency bucket set 230 is full but the entire low access frequency bucket set is not full, the control logic 212 performs a heron operation within the IA bucket set 230. Alternatively, if all buckets in the low access frequency buckets are full (not shown in FIG. 5), the control logic 212 causes the bucket mover 218 to move one or more entries from the IA bucket set 230 to the frequently accessed bucket set 228. Thus, when a less frequently accessed bucket is full, evict the hash table entry and store the entry of the hashed key in a second bucket set.

[0041] As shown in block 510, this method includes storing, after evicting a hash table entry, an entry for the hashed key and its associated value in a bucket of a hash table that functions as a bucket that is not accessed as frequently when the candidate IA bucket of the hash table is full.

[0042] Thus, in certain embodiments, during a lookup, an item is always first looked up using a frequently accessed mapping function. If the entry is not found in the FA candidate bucket(s), the IA mapping function(s) is used to look up the IA bucket set. This can increase the number of cache accesses compared to certain prior art hash table lookup methods, but this operation can reduce the number of main memory accesses for a dataset where most lookups are performed against a subset of the keys and values of the hash table.

[0043] FIG. 6 is a flowchart showing an example of a method 600 for looking up hash table entries. As shown in block 601, the hash table logic 204 receives a hash table lookup request 224. As shown in block 602, the hash table logic 204 uses a hash generator 208 corresponding to the hash used for the frequently accessed bucket set 228 to perform a hash operation on the key (K) 222 to identify candidate buckets within the FA bucket set 228. Thus, when performing a hash table lookup, the FA bucket set 228 is searched before the IA bucket set 230. For example, in response to the hash table lookup request 224, the lookup logic 220 uses the key received in the hash table lookup request 224 and provides a hash generation request 238 to the hash generator 208 to perform a hash operation on the key. As shown in block 604, the lookup logic 220 searches for the key K in the candidate buckets of the FA bucket set 228. The hash function identifies the candidate bucket to search. Once that bucket is identified, the original key is searched in the bucket. As shown in block 606, if the key K is found in the FA bucket set 228, the method proceeds to block 608, and the value corresponding to the key is returned from the candidate bucket of the FA bucket set 228 from the appropriate slot. As shown in block 610, the slot counter is incremented for the accessed slot.

[0044] Return to block 606. As shown in block 612, if the key is not found in the FA bucket set, use the hash generator 206 to execute a hash operation associated with the bucket with low access frequency, and search the low-access-frequency bucket set 230 to re-hash the key to generate a hashed key (HIA(K)) and identify candidate buckets. In one example, this is controlled by the lookup logic 220 that sends a hash generation request 241 to the hash generator 206 and causes the lookup logic 220 to perform a lookup in the identified candidate buckets of the IA bucket set 230. As shown in block 614, the lookup logic 220 returns a value from the IA candidate bucket and the corresponding slot via the hash table response 242. As shown in block 610, the slot counter is incremented. If the re-hashed key is not found in the IA bucket set 230, the hash table response 242 indicates to the requesting entity that the key was not found. Otherwise, the value is returned to the requesting entity via the hash table response 242. In some embodiments, a simultaneous lookup is performed and the value is returned from the lookup where a hit is found.

[0045] Figure 7 is a flowchart illustrating an example of a method 700 for moving hash table entries. As shown in block 701, control logic 212 determines the slot access frequency. This is done by control logic 212 evaluating a slot counter to determine the number of slots in each slot within hash table 202. As shown in block 702, for example, the number of slot accesses is compared to a threshold by the control logic. If the number of slot accesses exceeds the threshold, hash table logic 204 uses hash generator 208 to move an entry to the frequently accessed bucket set 228. In some embodiments, this movement is only performed for items within the IA bucket set whose access count exceeds the threshold. If already in the FA set, there is no need to move the item. Thus, control logic 212 notifies, for example, bucket mover 218 to delete an entry from the IA bucket set, and the control logic causes hash generator 208 to employ a hash operation that exclusively maps to the FA bucket set. This is shown in block 704. For example, the system uses a copy of the key from the slot or its representation used by another hash function, i.e., the frequently accessed hash function used by hash generator 208. As shown in block 706, this method includes exclusively storing the key and its associated value from the slot in FA bucket set 228. Optionally, cocklebur shrimp operations can also be used within the FA bucket set. As shown in block 708, if the slot access does not exceed the threshold, i.e., the slot is not considered to be frequently accessed, the entry is moved to the IA bucket set using hash generator 206. For example, this is performed when an item is included in the FA bucket set but the access count is below the threshold. Control logic 212 notifies bucket mover 218 to move an entry from one bucket set to another. This can be performed, for example, if a slot within FA bucket set 228 is not accessed frequently enough when moved within the FA bucket set.As shown in block 710, this method includes exclusively storing keys and their associated values from the slots in the IA bucket set 230.

[0046] FIG. 8 is a flowchart showing an example of a method 800 for inserting a hash table entry into a hash table. The hash operation corresponding to the frequently accessed bucket set and the hash operation corresponding to the IA bucket set are executed based on weighting (weight-based) in response to a table entry insertion request. For example, as shown in block 801, in response to receiving a table entry insertion request, the control logic 212 determines the probability that the received key (K) 222 is in the FA bucket set. For example, in one example, the control logic 212 may have pre-stored weight coefficients associated with each hash mapping operation such that the probability that the key is placed in the FA bucket set 228 is 10% and the probability that the key is placed in the IA bucket set 230 is 90%. Thus, 90% of the keys received for insertion are placed in the IA bucket set using the hash generator 206, and 10% are placed in the FA bucket set 228 using the hash generator 208. As shown in block 802, the control logic 212 determines whether it is advantageous to place the key in the FA bucket set 228 with a probability. Otherwise, as shown in block 804, the hash generator 206 executes a hash operation with a low access frequency on the key to determine a candidate bucket. As shown in block 806, the hash table logic 204 exclusively stores the key and its associated value in the IA bucket set 230. However, if the control logic 212 determines that the probability is to place the received key in the FA bucket set 228, as shown in block 808, the hash generator 208 is used to execute a hash operation that is frequently accessed on the key to determine a candidate bucket, and as shown in block 810, the hash table logic 204 exclusively stores the key and its associated value in the FA bucket set 228. Any suitable probability determination can be used, such as using a random number generator to equalize the probability to 50%, or other predetermined probabilities can be used as needed.

[0047] The present invention described above has been found to work well for data structures in which frequently accessed data sets stabilize over a long period of time. If the set is frequently changed, items in the FA bucket can be actively migrated to the IA bucket that has not been recently accessed. This can be performed proactively by migrating such items to the IA bucket periodically, or on demand when new items are moved to the FA bucket and space needs to be freed. Based on access tracking data, items that have not been frequently accessed recently can be determined.

[0048] Some embodiments herein are presented in the context where the FA bucket is the first N buckets and the latter (M - N) buckets are the IA buckets. Alternative partitioning approaches are envisioned and covered by the present invention. These alternatives include approaches where any consecutive N buckets are used as the FA bucket and the rest are used as the IA bucket, or a set of non - consecutive N buckets are used as the FA bucket and the rest are used as the IA bucket. Similarly, the requirements for the FA mapping function are that they map to the appropriate N buckets (i.e., the FA buckets), and are not limited to the specific embodiments above.

[0049] Furthermore, the above embodiments describe the case where the FA mapping function is exclusively mapped to the FA bucket and the IA mapping function is exclusively mapped to the IA bucket. However, alternative embodiments are envisioned where the FA mapping function is mapped to the FA bucket with a higher probability than to the IA bucket (but not exclusively), and the IA mapping function is mapped to the IA bucket with a higher probability than to the FA bucket (but not exclusively).

[0050] Also, in some embodiments, when tracking data is used, the tracking data may be in the context of an absolute count associated with a data item. However, alternative embodiments are envisioned where frequency tracking is performed via an approximate data structure such as a Bloom filter. For example, when an item's key is first accessed, it is marked as accessed in the Bloom filter, and in subsequent accesses, if the item is already found in the Bloom filter, the item may be considered to be frequently accessed. This may result in false positives, but this is acceptable as it is a performance optimization that does not affect accuracy. In this variation, the Bloom filter can be reset periodically to ensure that only accesses repeated within a known limited time interval are considered frequent accesses. Alternatively, a more sophisticated approximate set membership tracking structure such as a counting Bloom filter can be used to maintain an approximate count of accesses (in addition to the fact that the item has been accessed at least once before), and enforce a threshold number of accesses above which the item is considered to be frequently accessed.

[0051] Access tracking data can be provided using other forms of algorithms for determining repeated items. For example, there are prior arts that can provide an approximation of the most frequently seen items within a sequence of items (e.g., in the case of the present invention, the sequence of accessed items). Such an approach can be used to periodically identify the most recently frequently accessed items and remap them to the FA buckets.

[0052] It will be appreciated that the configuration of items within an IA bucket set or an FA bucket set is a performance optimization that does not affect accuracy. Accordingly, any given embodiment can treat any of the optimizations described herein as a best effort. For example, if moving a data item identified as being frequently accessed to the FA set causes a kokko eviction, the embodiment may decide not to perform that move in order to avoid the overhead of the eviction. Alternatively, such a move may cause a kokko eviction, but even if an element is frequently accessed, moving that element to the IA set can end the eviction chain at a certain number of evictions.

[0053] In some embodiments, the present invention is described in the context of a hash table using a kokko hash as an example starting point. However, the present invention is applicable to any hash table that supports multiple mapping functions. Further, the present invention is also applicable to other data structures that rely on using one or more deterministic functions, such as a hash map, to map data items to a set of buckets, and to other set membership data structures, including approximate set membership data structures.

[0054] FIG. 9 is a diagram showing an embodiment of a computing system 900 that uses a hash table with FA bucket sets and IA bucket sets. Generally, the computing system 900 is embodied as any of a number of different types of devices, including but not limited to laptop or desktop computers, mobile devices, servers, network switches or routers, system-on-chips, integrated circuits, multi-package devices, etc. In this example, the computing system 900 includes a number of components 902 - 908 that communicate with each other via a bus 901. In the computing system 900, each of the components 902 - 908 can communicate with any of the other components 902 - 908 either directly via the bus 901 or via one or more of the other components 902 - 908. The components 902 - 908 of the computing system 900 may be included within a single physical enclosure such as a laptop, desktop chassis, or mobile phone casing, or in some embodiments, may be a system-on-chip or other configuration. In an alternative embodiment, some of the components of the computing system 900 are embodied as peripheral devices such that the entire computing system 900 does not exist within a single physical enclosure.

[0055] In some embodiments, the computing system 900 further includes a user interface device for receiving information from or providing information to a user. Specifically, the computing system 900 includes an input device 902 such as a keyboard, mouse, touch screen, or other device for receiving information from the user. The computing system 900 displays information to the user via a display 905 such as a monitor, light-emitting diode (LED) display, liquid crystal display, or other output device. However, it is not necessary to use such devices.

[0056] In certain embodiments, computing system 900 further includes a network adapter 907 for sending and receiving data over a wired or wireless network. Computing system 900 further includes one or more peripheral devices 908. Peripheral devices 908 can include mass storage devices, position detection devices, sensors, input devices, or other types of devices used by computing system 900.

[0057] Computing system 900 includes a processing unit 904. The processing unit 904 receives and executes instructions 909 stored in memory system 906. In one embodiment, the processing unit 904 includes a plurality of processing cores that reside on a common integrated circuit substrate. Memory system 906 includes memory devices used by computing system 900 such as random access memory (RAM) modules, read only memory (ROM) modules, hard disks, and other non-transitory computer-readable media. A portion of the memory device is used as memory 200 for the processing unit 904.

[0058] In certain embodiments, a non-transitory storage medium, such as memory 906, when executed by one or more processors, such as processing unit 904, includes executable instructions that cause the one or more processors to perform a hash operation on a key, where the hash operation is exclusively configured or biased to map the key and its associated value to a set of buckets (also referred to as the FA bucket set) of a hash table that functions as a set of frequently accessed buckets. In some embodiments, the FA bucket set of the hash table has a size that fits within the cache. In some embodiments, the storage medium, when executed by one or more processors, includes executable instructions that cause the one or more processors to store an entry for a key hashed to the FA bucket set within the hash table. In certain embodiments, the storage medium, when executed by one or more processors, includes executable instructions that cause the one or more processors to perform different hash operations on the same or different keys, where the hash operations are biased to map the key and its associated value to a set of buckets of a hash table that functions as a set of less frequently accessed buckets (also referred to as the IA bucket set) and to store an entry for the key hashed to the IA bucket set of the hash table. In response to a key lookup request, the one or more processors perform a hash table lookup of the requested key in the frequently accessed bucket set, and if the requested key is not found, perform a hash table lookup of the requested key in the set of less frequently accessed buckets.

[0059] In certain embodiments, the non-transitory storage medium, when executed by one or more processors, includes executable instructions that cause the one or more processors to bias the hash operations such that one of the first hash operations exclusively maps the key and its associated value to a set of frequently accessed buckets and another hash operation maps the key and its associated value to a set of less frequently accessed buckets. In some embodiments, the storage medium stores executable instructions that cause the one or more processors to operate as described with reference to FIGS. 2-8.

[0060] Some embodiments of computing system 900 may include fewer or more components than the embodiment shown in FIG. 9. For example, certain embodiments are implemented without display 905 or input device 902. Other embodiments have two or more particular components. For example, embodiments of computing system 900 may have multiple processing units 904, bus 901, network adapter 907, memory system 906, and the like.

[0061] Although features and elements have been described above in particular combinations, each feature and element may be used alone without other features and elements or in various combinations with or without other features and elements. The apparatuses described herein in some embodiments are manufactured using a computer program, software, or firmware incorporated in a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-transitory computer-readable storage media include magnetic media such as read only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, internal hard disks, and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVD).

[0062] In the foregoing detailed description of various embodiments, reference is made to the accompanying drawings which form a part hereof and which show by way of illustration specific preferred embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, but it is to be understood that other embodiments may be realized and that logical, mechanical, and electrical changes may be made without departing from the spirit and scope of the invention. To avoid unnecessary detail which would not assist those skilled in the art in practicing the invention, the specification may omit certain information which is well known to those skilled in the art. Further, many other various embodiments incorporating the teachings of the disclosure may be readily constructed by those skilled in the art. Accordingly, the invention is not intended to be limited to the specific forms described herein, but on the contrary, is intended to cover alternatives, modifications, and equivalents as may be reasonably included within the scope of the invention. Accordingly, the foregoing detailed description should not be construed in a limiting sense, and the scope of the invention is defined only by the appended claims. The above detailed description of the embodiments and the examples described therein are presented for purposes of illustration and explanation only and not for purposes of limitation. For example, the described operations may be performed in any suitable order or manner. Accordingly, the invention is considered to embrace any modifications, variations, or equivalents within the scope of the basic underlying principles disclosed above and claimed herein.

[0063] The above detailed description and the examples described therein are presented for purposes of illustration and explanation only and not for purposes of limitation.

Claims

1. A method executed by one or more processors, comprising: Performing a first hash operation on a first key, wherein the first hash operation biases the first key and its associated value to map to a first candidate bucket within a first set of buckets of a hash table that functions as a set of frequently accessed buckets; Storing an entry of the first key and its associated value in the first candidate bucket within the first set of buckets of the hash table; Performing a second hash operation on a second key, wherein the second hash operation biases the second key and its associated value to map to a second candidate bucket within a second set of buckets of the hash table that functions as a set of less frequently accessed buckets; Storing an entry of the second key and its associated value in the second candidate bucket within the second set of buckets of the hash table; In response to a key lookup request, performing a first hash table lookup of the requested key in the first set of buckets of the hash table, and in response to the requested key not being found in the first set of buckets of the hash table, performing a second hash table lookup of the requested key in the second set of buckets of the hash table; Storing and using bucket-by-bucket fullness data to determine whether to move hash table entries between the first set and the second set of buckets of the hash table. A method.

2. The method of claim 1, further comprising biasing the first hash operation and the second hash operation such that the first hash operation exclusively maps the first key and its associated value to the first set of buckets of the hash table that functions as a set of frequently accessed buckets, and the second hash operation exclusively maps the second key and its associated value to the second set of buckets of the hash table that functions as a set of less frequently accessed buckets. The method of claim 1. **Claim 3**: Further comprising, before storing the entry of the second key and its associated value in the second set of buckets of the bucket, evicting a hash table entry from the second set of buckets of the hash table that functions as the set of less frequently accessed buckets when the second set of buckets of the hash table is full, and inserting the evicted entry into the first set of buckets of the hash table that functions as the set of frequently accessed buckets. The method of claim 1. **Claim 4**: Generating tracking data representing the number of times each slot in the buckets of the first set of buckets of the hash table that functions as the set of frequently accessed buckets is accessed and the number of times each slot in the buckets of the second set of buckets of the hash table that functions as the set of less frequently accessed buckets is accessed. Further comprising moving at least one hash table entry between the first set and the second set of buckets of the hash table based on the tracking data. The method of claim 1. **Claim 5** Further comprising executing the first hash operation and the second hash operation based on weighting in response to a table entry insertion request. The method of claim 1. **Claim 6** A computing device comprising: A memory including a hash table; One or more processors, wherein the one or more processors Execute a first hash operation on a first key, the first hash operation being biased to map the first key and its associated value to a first candidate bucket within a first set of buckets of a hash table that functions as a set of frequently accessed buckets; Store the entry of the first key and its associated value in the first candidate bucket within the first set of buckets of the hash table; Execute a second hash operation on a second key, the second hash operation being biased to map the second key and its associated value to a second candidate bucket within a second set of buckets of the hash table that functions as a set of less frequently accessed buckets. Storing the entry of the second key and its associated value in a second candidate bucket within a second set of buckets of the hash table; In response to a key lookup request, performing a first hash table lookup of the requested key in a first set of buckets of the hash table, and in response to the requested key not being found in the first set of buckets of the hash table, performing a second hash table lookup of the requested key in a second set of buckets of the hash table; Storing and using bucket-by-bucket occupancy data to determine whether to move hash table entries between the first set and the second set of buckets of the hash table; configured to perform; a computing device. **Claim 7** The one or more processors are configured to bias the first hash operation and the second hash operation such that the first hash operation exclusively maps the first key and its associated value to a first set of buckets of the hash table that functions as the set of frequently accessed buckets, and the second hash operation exclusively maps the second key and its associated value to a second set of buckets of the hash table that functions as the set of less frequently accessed buckets. The computing device of claim 6. **Claim 8** The one or more processors are configured to evict a hash table entry from a second set of buckets of the hash table that functions as the set of less frequently accessed buckets when the second set of buckets of the hash table is full, and insert the evicted entry into a first set of buckets of the hash table that functions as the set of frequently accessed buckets. The computing device of claim 6.

9. A slot access counter structure configured to generate tracking data representing the number of times each slot in the buckets of a first set of buckets of the hash table that functions as the set of frequently accessed buckets is accessed, and the number of times each slot in the buckets of a second set of buckets of the hash table that functions as the set of less frequently accessed buckets is accessed. The one or more processors are configured to move at least one hash table entry between the first set and the second set of buckets of the hash table based on the tracking data. The computing device of claim 6.

10. The one or more processors are configured to execute the first hash operation and the second hash operation based on weighting in response to a table entry insertion request. The computing device of claim 6.

11. The memory includes a cache memory storing a first set of buckets of the hash table and another memory storing a second set of buckets of the hash table. The computing device of claim 6.

12. A storage medium containing executable instructions, which, when executed by one or more processors, perform a first hash operation on a first key, the first hash operation being biased to map the first key and its associated value to a first candidate bucket within a first set of buckets of a hash table that functions as a set of frequently accessed buckets; store an entry of the first key and its associated value in the first candidate bucket within the first set of buckets of the hash table; perform a second hash operation on a second key, the second hash operation being biased to map the second key and its associated value to a second candidate bucket within a second set of buckets of the hash table that functions as a set of less frequently accessed buckets; store an entry of the second key and its associated value in the second candidate bucket within the second set of buckets of the hash table. In response to a key lookup request, perform a first hash table lookup of the requested key in a first set of buckets of the hash table, and in response to the requested key not being found in the first set of buckets of the hash table, perform a second hash table lookup of the requested key in a second set of buckets of the hash table, store and use per-bucket fullness data to determine whether to move hash table entries between the first and second sets of buckets of the hash table, causing the one or more processors to perform, a storage medium. **Claim 13** The executable instructions further include biasing the first hash operation and the second hash operation such that the first hash operation exclusively maps the first key and its associated value to a first set of buckets of the hash table that functions as the set of frequently accessed buckets, and the second hash operation exclusively maps the first key and its associated value to a second set of buckets of the hash table that functions as the set of less frequently accessed buckets. The storage medium of claim 12. **Claim 14** The executable instructions further include, before storing an entry for the second key and its associated value in the second set of buckets, evicting a hash table entry from the second set of buckets of the hash table that functions as the set of less frequently accessed buckets if the second set of buckets of the hash table is full, and inserting the evicted entry into the first set of buckets of the hash table that functions as the set of frequently accessed buckets. The storage medium of claim 12. **Claim 15** The executable instructions further include generating tracking data representing the number of times each slot within a bucket in the first set of buckets of the hash table that functions as the set of frequently accessed buckets has been accessed, and the number of times each slot within a bucket in the second set of buckets of the hash table that functions as the set of less frequently accessed buckets has been accessed. ​ ​ ​ Based on the tracking data, further comprising moving at least one hash table entry between a first set and a second set of buckets of the hash table. The storage medium of claim 12.

Images