Systems and methods using a hash table with a set of frequently accessed buckets and a set of infrequently accessed buckets

Abstract

A method and apparatus performs a first hash operation on a first key, wherein the first hash operation is biased to map the first key and associated value to a set of frequently accessed buckets of a hash table. Stores an entry and associated value of the first key in the set of frequently accessed buckets. Performs a second hash operation on a second key, wherein the second hash operation is biased to map the second key and associated value to a set of infrequently accessed buckets of the hash table. Stores an entry and associated value of the second key in the set of infrequently accessed buckets. The method and apparatus performs a hash table lookup of a requested key in the set of frequently accessed buckets, and if the requested key is not found, performs a hash table lookup in the set of infrequently accessed buckets.

Description

Background of the Invention

[0002] A hash table is a data structure that implements an associative array. For some hash tables, one or more hash functions are used to map a given "key" to one or more indices that identify buckets within the hash table. These buckets may hold a "value" (or a pointer to that value) associated with the key. Typically, a bucket contains multiple "slots," each of which can hold a single key-value pair. The slots within a bucket are fully associative because any slot in a bucket can hold any item mapped to that bucket. Hash tables are widely used in many application domains. An ideal hash table attempts to distribute its contents evenly across all its buckets. For example, software libraries invoked by applications employ core primitives such as hash tables, which offer broad applicability in a wide range of application domains, including relational databases, key-value stores, machine learning applications, and other uses.

[0003] However, with very large hash tables, uniformly distributing content across all buckets can lead to poor cache performance, even if the data itself exhibits temporal locality, because each bucket typically ends up mixing frequently accessed (FA) and infrequently accessed (IA) data. For example, hardware caches are designed to attempt to capture frequently accessed data. However, by design (i.e., uniformly distributing items and placing them in buckets), hash tables mix frequently accessed items with infrequently accessed items in the same cache line, thus hindering the cache's ability to capture hot-working sets.

[0004] Some traditional high-performance hash tables typically use multiple mapping hash functions (usually two) to map a given key to multiple candidate buckets in the hash table. A simplified example is shown in Figure 1. Typically, when inserting a new data item 10 into hash table 12, the outputs of hash functions 14 and 16 are computed in parallel, and the item is then inserted into the candidate bucket with the most unoccupied slots. In an example hash table with eight buckets and four slots per bucket, occupied slots are shown in gray and empty slots in white. As shown, a new item (K,V) with key K and value V is inserted for the identified candidate buckets 5 and 0 using two hash functions F1() and F2(). If the insertion strategy is to insert into the candidate bucket with the least load, then in this example, (K,V) is inserted into bucket 5. This helps to make the loading more even across the different buckets of the hash table, allowing for uniform loading even under high load factors. Alternatively, the hash functions can be evaluated sequentially, and an item can be inserted into the first candidate bucket with available space. If all candidate buckets are full, eviction (e.g., cuckoo eviction) is used, where one of the existing items is evicted to make room for the new data item. The evicted item is then reinserted into one of its spare candidate buckets. If all candidate buckets are also full, the eviction chain continues until eviction terminates, for example, when the last evicted item can be placed into one of its candidate buckets.

[0005] During a lookup operation, a mapping hash function is applied to the key being looked up to locate a candidate bucket. Then, each slot within the candidate bucket is searched, comparing the key of interest with the keys in each slot of the candidate bucket. Because slots within a bucket are fully associative (i.e., the mapping hash function only identifies the bucket, not a specific slot within it), a search is required across all slots. Large hash tables generally attempt to distribute data elements evenly across buckets, resulting in poor caching performance. For example, even if most accesses to a dataset are for a small subset of data items, these are distributed across the entire hash table. Furthermore, hash bucket sizes are typically designed so that multiple slots within a bucket fit within a single cache line. Therefore, caching a single frequently used item may also caching multiple infrequently used items that are part of the same bucket, reducing the cache's ability to capture a useful set of frequently accessed items.

[0006] Therefore, providing improved hash table operations for caching or other purposes would be highly beneficial. Attached Figure Description

[0007] The implementation will be more readily understood when viewed in conjunction with the following figures and in light of the following description, wherein the same reference numerals denote the same elements, and in the figures:

[0008] Figure 1 is a prior art block diagram illustrating an example of a hash table that uses multiple hash functions to determine the appropriate bucket for inserting hash table entries;

[0009] Figure 2 This is a block diagram illustrating a portion of a computing system according to an embodiment set forth in this disclosure;

[0010] Figure 3 It is shown by Figure 2 A flowchart illustrating an example of a method partially executed by the computing system shown;

[0011] Figure 4 This is a diagram illustrating an example of a hash table according to an example set forth in this disclosure, the hash table employing different groups of buckets separated by frequently accessed buckets and infrequently accessed buckets;

[0012] Figure 5 This is a flowchart illustrating an example of a method for inserting hash table entries into a hash table, according to an example set forth in this disclosure;

[0013] Figure 6 This is a flowchart illustrating an example of a method for looking up a hash table entry, according to an example set forth in this disclosure;

[0014] Figure 7This is a flowchart illustrating an example of a method for moving hash table entries according to an example set forth in this disclosure;

[0015] Figure 8 This is a flowchart illustrating an example of a method for storing hash table entries according to an example set forth in this disclosure; and

[0016] Figure 9 This is a block diagram illustrating an example of a computing system according to an example set forth in this disclosure. Detailed Implementation

[0017] In short, the system and method divide the buckets of the hash table into two disjoint sets: frequently accessed buckets and infrequently accessed buckets. The terms "infrequently" and "rarely" are used interchangeably in this document. In some implementations, multiple buckets from the frequently accessed set are selected such that the frequently accessed buckets may be suitable for hardware caching. In some implementations, the bias may be a subset of the mapped hash functions of the frequently accessed (FA) mapping functions to primarily map to the set of frequently accessed buckets in the hash table, and the remaining hash functions are a subset of the mapped hash functions biased to primarily map to the set of infrequently accessed buckets in the hash table. In some implementations, trace data is maintained to identify which data items are frequently accessed and which are infrequently accessed. Based on the trace data, frequently accessed elements are identified and moved to the frequently accessed buckets of the hash table, while infrequently accessed data items remain in the infrequently accessed buckets.

[0018] According to some implementations, a method executed by one or more processors performs a first hash operation on a first key, wherein the first hash operation is biased to map the first key and associated value to a first candidate bucket within a first set of buckets in a hash table, the first set of buckets serving as a set of frequently accessed buckets. The method includes storing the entry for the first key and associated value in the first candidate bucket within the first set of buckets in the hash table, for example, in a cache memory sized to fit the set of frequently accessed buckets. The method also includes performing a second hash operation on a second key, wherein the second hash operation is biased to map the second key and associated value to a second candidate bucket within a second set of buckets in the hash table, the second set of buckets serving as a set of infrequently accessed buckets; and storing the entry for the second key and associated value in the second candidate bucket within the second set of buckets in the hash table. In response to a lookup request for a key, the method includes performing a hash table lookup of the requested key in the first set of frequently accessed buckets; if the requested key is not found, the method includes performing a hash table lookup of the requested key in the set of infrequently accessed buckets. In other implementations, the method includes performing the lookup in parallel using a hash function and two sets of buckets.

[0019] In some examples, the method includes biasing the first and second hash operations such that the first hash operation exclusively maps the first key and associated value to a first set of buckets in the hash table that serve as the set of frequently accessed buckets, and such that the second hash operation exclusively maps the second key and associated value to a second set of buckets in the hash table that serve as the set of infrequently accessed buckets.

[0020] In some examples, the default priority is given to store entries in infrequently accessed buckets. In some implementations, all key-value pairs are first inserted into the set of infrequently accessed buckets, and then, if the value proves to be frequently accessed, it is moved to the set of frequently accessed buckets. In some examples, the method includes evicting hash table entries when a second set of buckets used as infrequently accessed buckets is full, and then storing entries with a hashed second key in the second set of buckets. The evicted entries can be selected based on their access frequency, such that more frequently accessed entries are evicted from the second set of buckets used as infrequently accessed buckets and moved to the first set of buckets used as frequently accessed buckets.

[0021] In some examples, the method includes generating trace data representing the number of times each slot in a bucket within a first set of frequently accessed buckets and a second set of infrequently accessed buckets of the hash table has been accessed. In some examples, the method includes moving entries from the second set of infrequently accessed buckets to the first set of frequently accessed buckets, or vice versa, based on the trace data.

[0022] In some examples, the method includes performing first and second hash operations on a weighted basis in response to a table entry request. In some examples, the method includes storing and using per-bucket fullness data to determine whether to move a hash table entry between the first and second sets of buckets in the hash table.

[0023] According to some implementations, the computing device includes hash table logic that performs a first hash operation on a first key, wherein the first hash operation is biased to map the first key and associated value to a first candidate bucket within a first set of buckets in the hash table, the first set of buckets serving as a set of frequently accessed buckets; and the entry for the first key and associated value are stored in the first candidate bucket within the first set of buckets in the hash table. In some implementations, the hash table logic performs a second hash operation on a second key, wherein the second hash operation is biased to map the second key and associated value to a second candidate bucket within a second set of buckets in the hash table, the second set of buckets serving as a set of infrequently accessed buckets; and the entry for the second key and associated value are stored in the second candidate bucket within the second set of buckets in the hash table. In response to a lookup request for a key, in some implementations, the hash table logic performs a hash table lookup for the requested key in the first set of frequently accessed buckets; if the requested key is not found, a hash table lookup for the requested key is performed in the set of infrequently accessed buckets. In other implementations, the lookup is performed by using a hash function and two sets of buckets in parallel.

[0024] In some examples, the computing device biases the first and second hash operations such that the first hash operation exclusively maps the first key and associated value to a first set of buckets in the hash table that serve as the set of frequently accessed buckets, and the second hash operation exclusively maps the second key and associated value to a second set of buckets in the hash table that serve as the set of infrequently accessed buckets.

[0025] In some examples, when a second set of buckets used as infrequently accessed buckets of the hash table is full, a hash table entry is evicted, and the entry with the hashed second key is then stored in the second set of buckets. Entry eviction can be based on the frequency of access, such that more frequently accessed entries are evicted from the second set of buckets used as infrequently accessed buckets and moved to the first set of buckets used as frequently accessed buckets.

[0026] In some examples, the hash table logic also includes a slot access counter structure that generates trace data representing the number of times each slot in a bucket within a first set of frequently accessed buckets and a second set of infrequently accessed buckets of the hash table has been accessed. In some implementations, the hash table logic moves entries from the second set of infrequently accessed buckets to the first set of frequently accessed buckets and vice versa, based on the trace data.

[0027] In some examples, the hash table logic performs first and second hash operations on a weighted basis in response to a table entry request. In some examples, the hash table logic stores and uses per-bucket fullness data to determine whether to move a hash table entry between the first and second sets of buckets in the hash table. In some examples, the computing device includes a memory that includes the hash table, and the computing device includes one or more processors as operations of the hash table logic.

[0028] According to some implementations, a non-transitory storage medium includes executable instructions that, when executed by one or more processors, cause one or more processors to: perform a first hash operation on a first key, wherein the first hash operation is biased to map the first key and associated value to a first candidate bucket within a first set of buckets in a hash table, which serves as a set of frequently accessed buckets; store the entry for the first key and associated value in the first candidate bucket within the first set of buckets in the hash table; perform a second hash operation on a second key, wherein the second hash operation is biased to map the second key and associated value to a second candidate bucket within a second set of buckets in a hash table, which serves as a set of infrequently accessed buckets; store the entry for the second key and associated value in the second candidate bucket within the second set of buckets in the hash table; and, in response to a lookup request for a key, perform a hash table lookup of the requested key in the first set of frequently accessed buckets, and if the requested key is not found, perform a hash table lookup of the requested key in the set of infrequently accessed buckets. In other implementations, the lookup is performed by using the hash function and both sets of buckets in parallel.

[0029] In some examples, the non-transitory storage medium includes executable instructions that, when executed by one or more processors, cause one or more processors to bias first and second hash operations such that the first hash operation exclusively maps a first key and associated value to a first set of buckets in the hash table used as the set of frequently accessed buckets, and causes a second hash operation to exclusively map a second key and associated value to a second set of buckets in the hash table used as the set of infrequently accessed buckets.

[0030] In some examples, the non-transitory storage medium includes executable instructions that, when executed by one or more processors, cause one or more processors to evict hash table entries when a second set of buckets (used as infrequently accessed buckets) of the hash table is full, and then store the hashed second-key entry in the second set of buckets. The evictment of entries can be based on their access frequency, such that more frequently accessed entries are evicted from the second set of buckets (used as infrequently accessed buckets) and moved to the first set of buckets (used as frequently accessed buckets).

[0031] In some examples, the non-transitory storage medium includes executable instructions that, when executed by one or more processors, cause one or more processors to: generate trace data representing the number of times each slot in a bucket within a first set of frequently accessed buckets and a second set of infrequently accessed buckets of the hash table has been accessed; and move entries from the second set of infrequently accessed buckets to the first set of frequently accessed buckets, or vice versa, based on the trace data.

[0032] Figure 2This example illustrates a portion of a computing system, such as a hardware server, smartphone, wearable device, printer, laptop computer, desktop computer, or any other suitable computing device that uses a hash table. In this example, the portion of the computing device includes, for example, cache memory 200 and the main memory of a processor such as a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), application-specific integrated circuit (ASIC), or other integrated circuits including hash table 202 and hash table logic 204. Hash table logic 204 interfaces with memory 200 to populate and organize hash table 202. In one example, hash table logic 204 is implemented as a programmable processor that executes executable instructions stored in memory, which, when executed, cause the processor to operate as the hash table logic described herein. In other implementations, hash table logic is implemented as discrete logic, one or more state machines, field-programmable gate arrays (FPGAs), or any suitable combination of a processor and other hardware executing instructions. In this example, hash table logic 204 includes multiple hash generators 206 and 208, a selector 210, control logic 212, a slot counting structure 214, a hash table bucket fullness determiner 216, a bucket mover 218, and lookup logic 220. It will be appreciated that the functional blocks are shown and various operations can be combined or separated as needed. It will also be appreciated that not all functional blocks are required in all implementations.

[0033] In operation, when populating hash table 202, hash table logic 204 receives a key (K) 222 and an associated value (V) from an application, service in a computing system, or other device for inputting into hash table 202. After the requested key is populated (i.e., inserted) into hash table 202, the application or other entity can retrieve cached data from the cache by requesting the stored value using hash table 202. Therefore, a hash table lookup request 224 is received, and lookup logic 220 processes the hash table entry request to retrieve cached data based on the use of hash table 202, as further described below.

[0034] Also refer to Figure 3 and Figure 4 Based on an example described in this article, Figure 2 The block diagram and example of hash table 202 illustrate the method of operation 300. In some implementations, hash generator 206 implements a hash function that exclusively maps to a set of IA buckets (buckets 3 to 7) also referred to as the infrequently accessed (IA) bucket set, and hash generator 208 is only mapped to a set of buckets also referred to as frequently accessed (FA) (see [link to implementation]). Figure 4A set of hash functions for a group of FA buckets (bucket 0 to bucket 2). Examples of these functions are shown below, using a similar naming convention as shown in Figure 1 for different hash functions.

[0035] f1'(x) = mod(f1(x), N) (where x is a bond) Equation 1

[0036] f2'(x)=N+mod(f2(x),M) Equation 2

[0037] Equation 1 shows an example of a hash function f1'(x) implemented by hash generator 208, while Equation 2 represents the hash function operation f2'(x) provided by hash generator 206. Unlike the system in Figure 1, the system described herein divides the hash table buckets into a set of frequently accessed buckets (buckets 0 to 2) and a set of infrequently accessed buckets (buckets 3 to 7), such that the first N buckets of the hash table correspond to the FA bucket set 228 and the remaining buckets MN are divided into the infrequently accessed bucket set 230, where M is the total number of buckets in the hash table. In some implementations, the FA bucket set is stored in a cache, and the IA buckets are stored in main memory or other memory.

[0038] Referring again to equations 1 and 2, modulo operations can be implemented when N and M are powers of 2 or may not be powers of 2. Hash generators 206 and 208 provide hash operations, for example, each hash operation is biased in a certain way such that the hash operation provided by hash generator 208 is biased to map the key and associated value to a set of buckets in the hash table that serve as a frequently accessed set, namely the FA bucket set 228, while hash generator 206 performs a hash operation on the key and is biased to map the key and associated value to another different set of buckets assigned as infrequently accessed buckets 230 (also referred to as the IA bucket set 230). The hash key generated by hash generator 206 is denoted as H. IA (K), while the hash key generated by hash generator 208 is represented as H. FA (K).

[0039] As shown in box 302, the method includes performing a hash operation on a key (K) 222 having an associated value (V), wherein in this example, the hash operation is biased to map the key 222 and the associated value to a set of buckets in the hash table that serve as a frequently accessed set, namely the FA bucket set 228. Therefore, in this example, control logic 212 sends a selection control signal 232 to control selector 210 to allow key 222 to be passed to hash generator 208. As shown in box 304, hash generator 208 stores an entry for key 222 in the FA bucket set 228 corresponding to the frequently accessed bucket set in the hash table. For example, this might occur when the infrequently accessed bucket set 230 is full. However, any other suitable criterion may also be used.

[0040] As shown in box 306, the method includes performing another hash operation on another key (also referred to as key 222) provided to hash table logic 204. The other hash operation is biased to map key 222 and associated values ​​to another set of buckets in the hash table, serving as a set of infrequently accessed buckets 230. Therefore, control logic 212 provides a selection control signal 232 to selector 210 to provide key 222 to hash generator 206, which performs the hash operation on the key and, as shown in box 308, stores an entry for the hash key in the IA bucket set 230 of the hash table. As shown in box 310, in response to a hash table lookup request 224 from an application or other service, to request a value from the hash table, the method includes performing a hash table lookup for the requested key in the frequently accessed bucket set 228. Therefore, lookup logic 220 causes hash generator 208 to perform a hash operation on the received key as part of hash table lookup request 224, as shown in hash generation request 238, and lookup logic 220 searches for the hash key in the frequently accessed bucket set 228. If the requested key is not found, as indicated by hash generation request 241, lookup logic 220 requests hash generator 206 to generate a hash for the key, and then lookup logic 220 searches for said hash in another bucket set, namely, the infrequently accessed bucket set 230. It will be appreciated that the operations described herein can be performed in any suitable order, including in parallel or any other different order as needed. If the key is initially found in the frequently accessed bucket set 228, lookup logic 220 retrieves value 240 and provides a hash table response 242 including the retrieved value 240 in response to hash table lookup request 224. Lookup logic 220 performs hash table retrieval as needed for each lookup.

[0041] As noted above with respect to equations 1 and 2, hash generators 206 and 208 are biased by the hash functions performed by the respective hash generators. For example, one hash operation exclusively maps the key and associated value to a set of buckets in the hash table that serve as a set of frequently accessed buckets. Another hash generator is configured such that the hash operation exclusively maps the key and associated value to another set of buckets in the hash table that serve as a set of infrequently accessed buckets.

[0042] Many different hash table insertion techniques can be employed. In some implementations, insertion into hash table 202 is always performed on the infrequently accessed bucket set 230, unless a candidate infrequently accessed bucket is full. When a candidate IA bucket is full, insertion into the IA bucket is implemented using a cuckoo eviction mechanism. In other implementations, if a candidate IA bucket is full, control logic 212 controls selector 210 to insert into the FA bucket set 228. In other implementations, a hybrid of the two methods is used based on heuristics (e.g., the fullness of the entire table or IA buckets). In some examples, insertion into the FA bucket is performed when a candidate bucket is full and the total load on the IA bucket set 230 is higher than the total load on the FA bucket set 228 by a set margin; otherwise, insertion into the IA bucket is performed using a cuckoo eviction mechanism.

[0043] In other implementations, insertion is performed under the control of control logic 212 based on a statically or dynamically determined probability distribution, into both IA bucket set 230 and FA bucket set 228. In some implementations, a random number generator is used to determine whether to switch the selector for each insertion so that hash generator 206 or hash generator 208 performs a hash operation and places the hash key and associated value into FA bucket set 228 or IA bucket set 230.

[0044] In another example, access tracking is performed. For example, lookup logic 220 or slot counting structure 214 produces tracking data 400, which represents the number of times each slot in the set of frequently accessed buckets and the set of infrequently accessed buckets of the hash table has been accessed. In one implementation, metadata is used as tracking data 400 and stored within hash table 202. In other implementations, for example, a slot access counter structure 214 of the slot counter produces tracking data 400. In this example, the counter is associated with each slot in the hash table to identify the access frequency of each slot. To reduce capacity overhead, the counter can be limited to a small number of bits, such as two or three bits per counter, and saturation is allowed instead of overflow once the maximum value is reached. In one example, data items for slots whose access counters exceed a threshold are moved from infrequently accessed buckets in set IA buckets 230 to buckets in set 228 of frequently accessed buckets. This can be done, for example, using a frequently accessed mapping function. In some examples, the mapping might resemble a new insertion into the frequently accessed bucket set, such as using a cuckoo chase in the frequently accessed set.

[0045] In some implementations, the counter is reset periodically to ensure that only items accessed frequently enough to reach a threshold between counter resets are identified as frequently accessed. In other implementations, items that are no longer frequently accessed are remapped back to a set of infrequently accessed buckets. If too many or not enough slots are identified as frequently accessed—manifested as a large difference in occupancy between the FA and IA bucket sets—the threshold or counter reset frequency is changed to alter the criteria for being considered frequently accessed. This can be performed by lookup logic 220. When using a counter structure, another data structure can be viewed as a hardware or software-managed cache where only a subset of frequently accessed items is kept as a counter. Any item whose access frequency is insufficient to keep its counter in the counter cache can be considered infrequently accessed. However, any suitable counting mechanism can be employed.

[0046] For datasets where access behavior does not change rapidly over time, access frequency tracking only needs to be performed periodically, rather than all at once. Once sufficient profile information has been collected, further access tracking can be paused to avoid the performance overhead of such tracking. In other examples, access tracking is performed probabilistically, where the tracking overhead occurs only on a small subset of lookups.

[0047] In some implementations, during a lookup, the most frequently accessed mapping function is always used first to find the item. If the item is not found in the FA candidate bucket, the IA mapping function is used to look it up in the IA bucket set. Although this increases the number of cache accesses compared to previous systems, for elements in the IA buckets (e.g., since the FA bucket is checked first), this method can reduce the number of main memory accesses to the dataset, where most lookups are for small subsets (e.g., some forms of social network data). Again, as shown above, two mapping hash functions are used, but any suitable number can be used. For example, two IA mapping functions can be used, and one FA mapping function can be used, or any suitable number of hash functions can be used.

[0048] In other implementations, such as memory reads of the IA bucket issued in parallel with the lookup of the FA bucket, different lookup methods can be used, allowing the memory access latency of the IA bucket to overlap with the FA bucket check. This can improve latency compared to other implementations.

[0049] Hash table bucket fullness determiner 216 determines which buckets are full at each bucket level, and in another example, at each bucket set level and / or which bucket set is full. Control logic 212, for example, notifies bucket mover 218 to evict a hash table entry when bucket IA is full, and then stores the entry in a second set of buckets. Hash table bucket fullness determiner 216 provides bucket fullness data 250 to control logic, indicating which buckets are full and / or whether a bucket set is full. It will be appreciated that the described functional block can be combined with other functional blocks or otherwise varied as needed.

[0050] Figure 5 This is a flowchart illustrating an example of method 500 for inserting hash table entries into a hash table. It will be appreciated that the described operations can be performed in any suitable order, and other operations can be employed as needed. In this example, hash table insertion is first performed on the infrequently accessed set, unless a candidate infrequently accessed bucket is full. When a candidate infrequently accessed bucket is full, insertion into an infrequently accessed bucket is implemented using a cuckoo eviction mechanism. In some implementations, insertion into the frequently accessed bucket set is performed if a candidate infrequently accessed bucket is full. A hybrid of the two methods can also be used based on heuristics (e.g., the fullness of the entire hash table or IA buckets). For example, in some implementations, insertion into a frequently accessed bucket is performed when a candidate bucket is full and the total load on infrequently accessed buckets is a margin higher than the total load on frequently accessed buckets; otherwise, cuckoo eviction is performed to insert into an IA bucket.

[0051] As shown in box 501, the method includes performing a hash operation on the key, biasing the hash operation to map the key and associated value to a set of buckets in the hash table that serve as a set of infrequently accessed buckets. As shown in box 502, the method includes determining whether a candidate infrequently accessed bucket is full. In one example, control logic 212 uses bucket fullness data 250 to determine whether to move a hash table entry between the IA bucket set 230 and the FA bucket set 228 of hash table 202. For example, hash table bucket fullness determiner 216 tracks each of buckets 3 through 7 in the IA bucket set 230. If a candidate bucket in the IA bucket set 230 is full, the method continues to box 504, where, in one example, control logic 212 causes hash generator 208 to perform a frequently accessed hash on the key, and hash generator 208 exclusively stores the key and associated value in the frequently accessed bucket set 228, as shown in box 506. In other implementations, as shown in block 508, if candidate buckets in the infrequently accessed bucket set 230 are full, but the entire infrequently accessed bucket set is not full, control logic 212 performs a cuckoo eviction in the IA bucket set 230. Alternatively, when all buckets in the infrequently accessed buckets are full ( Figure 5(Not shown in the diagram), control logic 212 causes bucket mover 218 to move one or more entries from IA bucket set 230 to frequently accessed bucket set 228. Thus, when a less frequently accessed bucket is full, a hash table entry is evicted, and the entry with the hash key is then stored in the second set of buckets.

[0052] As shown in box 510, the method includes storing the hash key entry and associated value in a bucket of the hash table that is used as a bucket that is not frequently accessed, after possibly performing the eviction of a hash table entry when the candidate IA bucket of the hash table is full.

[0053] Therefore, during a lookup, in some implementations, the frequently accessed mapping function is always used first to find the entry. If the entry is not found in the FA candidate bucket, the IA mapping function is used to look up the IA bucket. Although this increases the number of cache accesses compared to some existing hash table lookup methods, this operation reduces the number of main memory accesses to the dataset, where most lookups target a subset of keys and values ​​in the hash table.

[0054] Figure 6 This is a flowchart illustrating an example of a method 600 for looking up a hash table entry. Hash table logic 204 receives a hash table lookup request 224, as shown in box 601. As shown in box 602, hash table logic 204 uses hash generator 208 to perform a hash operation on key (K) 222 to identify candidate buckets in FA bucket set 228, the hash operation corresponding to a hash for frequently accessed bucket set 228. Thus, when performing a hash table lookup, FA bucket set 228 is searched before IA bucket set 230. For example, in response to hash table lookup request 224, lookup logic 220 uses the key received in hash table lookup request 224 and provides hash generation request 238 to hash generator 208 to perform a hash operation on the key. As shown in box 604, lookup logic 220 searches for key K in candidate buckets of FA bucket set 228. A hash function identifies the candidate bucket to be searched. Once the bucket is identified, the original key is searched in that bucket. As shown in box 606, if key K is found in FA bucket set 228, the method continues to box 608, where the value corresponding to the key is returned from the appropriate slot in the candidate buckets of FA bucket set 228. As shown in box 610, for the accessed slot, the slot counter is incremented.

[0055] Returning to box 606 and as shown in box 612, if the key is not found in the FA bucket set, the key is rehashed to produce a hash key (H) by performing a hash operation associated with the infrequently accessed buckets using hash generator 206 and searching the infrequently accessed bucket set 230. IA(K) is used to identify candidate buckets. In one example, this is controlled by lookup logic 220 sending a hash generation request 241 to hash generator 206 and performing a lookup in the identified candidate bucket of IA bucket set 230. As shown in box 614, lookup logic 220 returns values ​​from IA candidate buckets and corresponding slots via hash table response 242. As shown in box 610, a slot counter is incremented. If no rehashed key is found in IA bucket set 230, hash table response 242 indicates to the requesting entity that the key was not found. Otherwise, the value is returned to the requesting entity via hash table response 242. In some implementations, simultaneous lookups are performed, and the value is returned from any lookup that finds a match.

[0056] Figure 7 This is a flowchart illustrating an example of a method 700 for moving hash table entries. As shown in box 701, control logic 212 determines the slot access frequency. This is done by control logic 212 evaluating a slot counter to determine the slot count for each slot in hash table 202. As shown in box 702, for example, the control logic compares the slot access count to a threshold. If the slot access count exceeds the threshold, hash table logic 204 uses hash generator 208 to move the entry to frequently accessed bucket set 228. In some implementations, this move is performed only for items in the IA bucket set where the access count exceeds the threshold. If the item is already in the FA set, there is no need to move it. Therefore, control logic 212, for example, instructs bucket mover 218 to remove the entry from the IA bucket set, and the control logic causes hash generator 208 to perform a hash operation that exclusively maps to the FA bucket set. This is shown in box 704. For example, the system uses a copy or representation of the key from the slot, which is used by another hash function, namely the frequently accessed hash function used by hash generator 208. As shown in box 706, the method includes exclusively storing the key and associated value from the slot in FA bucket set 228. Cuckoo eviction may also be used in the FA bucket set if necessary. As shown in box 708, if slot access does not exceed a threshold, meaning the slot is not considered frequently accessed, the entry is moved to IA bucket set using hash generator 206. This is done, for example, when an item is in the FA bucket set but the access count is below the threshold. Control logic 212 instructs bucket mover 218 to move an entry from one bucket set to another. This can be done, for example, when a slot in the FA bucket set has not been accessed frequently enough after it has already been moved in FA bucket set 228. As shown in box 710, the method includes exclusively storing the key and associated value from the slot in IA bucket set 230.

[0057] Figure 8This is a flowchart illustrating an example of a method 800 for inserting a hash table entry into the hash table. In response to a table entry insertion request, hash operations corresponding to the frequently accessed bucket set and hash operations corresponding to the IA bucket set are performed on a weighted basis. For example, as shown in box 801, in response to receiving a table entry insertion request, control logic 212 determines the probability that the received key (K) 222 is in the FA bucket set. For example, control logic 212 may have a pre-stored weighting factor associated with each hash mapping operation, such that in one example, given a probability of 10% that the key should be placed in the FA bucket set 228 and a probability of 90% that the key should be placed in the IA bucket set 230, hash generator 206 places 90% of the keys received for insertion into the IA bucket set, and hash generator 208 places 10% of the keys into the FA bucket set 228. As shown in box 802, control logic 212 determines whether the probability favors placing the key in the FA bucket set 228. If not, the hash generator 206 performs an infrequently accessed hash on the key to determine candidate buckets, as shown in box 804. As shown in box 806, hash table logic 204 exclusively stores the key and associated value in IA bucket set 230. However, if control logic 212 determines that the probability is to place the received key in FA bucket set 228, as shown in box 808, then the hash generator 208 performs a frequently accessed hash operation on the key to determine candidate buckets, and as shown in box 810, hash table logic 204 exclusively stores the key and associated value in FA bucket set 228. Any suitable probability can be used to determine this, including using a random number generator to achieve a probability equal to 50%, or any other predefined probability can be used as needed.

[0058] It has been found that the described invention is applicable to data structures where frequently accessed datasets are stable over long periods. When the dataset changes frequently, items in the FA bucket can be actively migrated to the IA bucket, which has not been accessed recently. This can be done proactively by periodically migrating such items back to the IA bucket, or on demand when new items are moved to the FA bucket and space needs to be freed up. Items that have not been frequently accessed recently can be identified based on access tracking data.

[0059] Some implementations described herein are presented in the context of the FA bucket being the first N buckets and the last (MN) buckets being the IA buckets. This invention envisions and covers alternative partitioning methods. These alternatives include methods using any consecutive N buckets as FA buckets and the remaining buckets as IA buckets, and methods using a set of non-consecutive N buckets as FA buckets and the remaining buckets as IA buckets. Similarly, the requirement for FA mapping functions is that they map to the appropriate N buckets (i.e., FA buckets) and are not limited to the specific implementations described above.

[0060] Furthermore, the above implementation discusses the case where the FA mapping function exclusively maps to the FA bucket and the IA mapping function exclusively maps to the IA bucket. However, an alternative implementation is envisioned, where the FA mapping function maps to the FA bucket with a higher probability than the IA bucket (but not exclusively), and the IA mapping function maps to the IA bucket with a higher probability than the FA bucket (but not exclusively).

[0061] Furthermore, in some implementations, if tracking data is used, the tracking data can be in the context of an absolute count associated with a data item. However, alternative implementations are conceivable where frequency tracking is accomplished through an approximate data structure, such as a Bloom filter. For example, upon the first access, the item's key might be marked as accessed in the Bloom filter, and in subsequent accesses, if the item has already been found in the Bloom filter, the item might be considered frequently accessed. While this may lead to false positives, it is acceptable because it is a performance optimization and does not compromise correctness. In this variant, the Bloom filter can be periodically reset to ensure that only repeated accesses within a known and bounded time interval are considered frequently accessed. Alternatively, a more complex approximate set membership tracking structure, such as a counting Bloom filter, could be used to maintain an approximate access count (not just the fact that an item has been accessed at least once before) and enforce a threshold number of accesses, exceeding which items are considered frequently accessed.

[0062] Access tracking data can be provided by using other forms of algorithms for determining duplicate items. For example, there are existing techniques that can provide approximations of the most frequent items within a sequence of items (e.g., in the case of this invention, the sequence of accessed items). Such methods can also be used to periodically identify the most recently accessed items and remap them to FA buckets.

[0063] It will be recognized that placing items in either the IA bucket set or the FA bucket set is a performance optimization and does not affect correctness. Therefore, any given implementation can treat any optimization discussed herein as a best-effort approach. For example, if moving a data item identified as frequently accessed to the FA set would result in a cuckoo eviction, the implementation could decide not to perform the move to avoid the overhead of eviction. Alternatively, this move might invoke a cuckoo eviction, but the eviction chain could be terminated at a certain threshold number of evictions by moving the element to the IA set, even if the element is frequently accessed.

[0064] In some implementations, the invention is described in the context of hash tables, with Cuckoo Hash serving as an example starting point. However, the invention is applicable to any hash table that supports multiple mapping functions. Furthermore, the invention is also applicable to other data structures that rely on mapping data items to a set of buckets using one or more deterministic functions, such as hash graphs and other set membership data structures, including approximate set membership data structures.

[0065] Figure 9 An implementation of a computing system 900 using a hash table with a set of FA buckets and a set of IA buckets is shown. Typically, the computing system 900 is embodied in any of many different types of devices, including but not limited to laptop or desktop computers, mobile devices, servers, network switches or routers, system-on-a-chip, integrated circuits, multi-package devices, etc. In this example, the computing system 900 includes a plurality of components 902-908 that communicate with each other via a bus 901. In the computing system 900, each of components 902-908 is capable of communicating directly with any other component 902-908 via the bus 901 or via one or more of the other components 902-908. Components 902-908 in the computing system 900 are contained within a single physical enclosure, such as a laptop or desktop computer rack or mobile phone housing, or in some implementations as a system-on-a-chip or other configuration. In alternative implementations, some components of the computing system 900 are embodied as peripheral devices, such that the entire computing system 900 does not reside within a single physical enclosure.

[0066] In some implementations, the computing system 900 also includes a user interface device for receiving or providing information to the user. Specifically, the computing system 900 includes an input device 902, such as a keyboard, mouse, touchscreen, or other device for receiving information from the user. The computing system 900 displays information to the user via a display 905, such as a monitor, a light-emitting diode (LED) display, a liquid crystal display, or other output device. However, such devices are not required.

[0067] In some implementations, the computing system 900 further includes a network adapter 907 for transmitting and receiving data via wired or wireless networks. The computing system 900 also includes one or more peripheral devices 908. The peripheral devices 908 may include mass storage devices, position detection devices, sensors, input devices, or other types of devices used by the computing system 900.

[0068] The computing system 900 includes a processing unit 904. The processing unit 904 receives and executes instructions 909 stored in main memory 906. In one embodiment, the processing unit 904 includes multiple processing cores residing on a common integrated circuit substrate. The memory system 906 includes memory devices used by the computing system 900, such as random access memory (RAM) modules, read-only memory (ROM) modules, hard disks, and other non-transitory computer-readable media. Some of the memory devices are used as memory 200 of the processing unit 904.

[0069] In some implementations, for example, the non-transitory storage medium of memory 906 includes executable instructions that, when executed by one or more processors, such as processing unit 904, cause the one or more processors to perform a hash operation on a key, wherein the hash operation is exclusively configured or biased to map the key and its associated value to a set of buckets in the hash table that serve as a set of frequently accessed buckets (also referred to as the FA bucket set). In some implementations, the size of the FA bucket set of the hash table is set to be suitable for caching. In some implementations, the storage medium includes executable instructions that, when executed by one or more processors, cause one or more processors to store entries of hash keys in the FA bucket set of the hash table. In some implementations, the storage medium includes executable instructions that, when executed by one or more processors, cause the one or more processors to perform different hash operations on keys, which may be the same or different keys, wherein the hash operation is biased to map the key and its associated value to a set of buckets in the hash table that serve as a set of infrequently accessed buckets (also referred to as the IA bucket set), and entries of hash keys are stored in the IA bucket set of the hash table. In response to a lookup request for a key, one or more processors perform a hash table lookup for the requested key in a frequently accessed bucket set. If the requested key is not found, a hash table lookup for the requested key is performed in a less frequently accessed bucket set.

[0070] In some implementations, a non-transitory storage medium stores executable instructions that, when executed by one or more processors, cause one or more processors to bias hash operations, such that one of the first hash operations exclusively maps keys and associated values ​​to a frequently accessed set of buckets, and another hash operation maps keys and associated values ​​to a set of infrequently accessed buckets. In some implementations, the storage medium stores executable instructions that cause one or more processors to, as referenced... Figures 2 to 8 The operation described.

[0071] Some implementations of the computing system 900 may include more than Figure 9 The embodiments shown have fewer or more components. For example, some embodiments are implemented without any display 905 or input device 902. Other embodiments have more than one particular component; for example, embodiments of computing system 900 may have multiple processing units 904, bus 901, network adapter 907, memory system 906, etc.

[0072] Although features and elements have been described above in specific combinations, each feature or element may be used alone without other features and elements, or in various combinations with or without other features and elements. In some implementations, the apparatus provided herein is manufactured using a computer program, software, or firmware incorporated in a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of computer-readable storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM discs and digital versatile optical discs (DVDs).

[0073] In the above detailed description of various embodiments, reference has been made to the accompanying drawings, which form a part thereof, and specific preferred embodiments in which the invention can be practiced are illustrated by way of illustration. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, and it should be understood that other embodiments can be utilized and logical, mechanical, and electrical changes can be made without departing from the scope of the invention. To avoid details not necessary for those skilled in the art to practice the invention, certain information known to those skilled in the art may be omitted from the description. Furthermore, those skilled in the art can readily construct many other embodiments incorporating the teachings of this disclosure. Therefore, the invention is not intended to be limited to the specific forms set forth herein, but rather, it is intended to cover such alternatives, modifications, and equivalents that can be reasonably included within the scope of the invention. Therefore, the foregoing detailed description is not to be considered limiting, and the scope of the invention is defined only by the appended claims. The above detailed description of the embodiments and examples described therein is presented for illustrative and descriptive purposes only and not for limiting purposes. For example, the described operations are performed in any suitable order or manner. Therefore, the invention is contemplated to cover any and all modifications, variations, or equivalents falling within the scope of the fundamental principles disclosed above and claimed herein.

[0074] The above detailed description and examples described herein are presented for illustrative and descriptive purposes only and not for limiting purposes.

Claims

1. A method executed by one or more processors, the method comprising: Perform a first hash operation on the first key, wherein the first hash operation is biased to map the first key and associated value to a first candidate bucket within a first set of buckets in the hash table, the first set of buckets serving as a set of frequently accessed buckets; The entry and associated value of the first key are stored in the first candidate bucket within the first group of buckets in the hash table; A second hash operation is performed on the second key, wherein the second hash operation is biased to map the second key and associated value to a second candidate bucket within a second set of buckets in the hash table, the second set of buckets serving as a group of infrequently accessed buckets; The entry for the second key and its associated value are stored in the second candidate bucket within the second set of buckets in the hash table; In response to a lookup request for a key, a hash table lookup of the requested key is performed in the first set of frequently accessed buckets. If the requested key is not found, a hash table lookup of the requested key is performed in the first set of infrequently accessed buckets. Generate tracking data, which represents the number of times each slot in the first set of buckets used as the set of frequently accessed buckets of the hash table has been accessed and the number of times each slot in the second set of buckets used as the set of infrequently accessed buckets of the hash table has been accessed. as well as Based on the tracking data, move at least one slot entry between buckets.

2. The method of claim 1, further comprising biasing the first and second hash operations such that the first hash operation is exclusively configured to map the first key and associated value to a first set of buckets of the hash table used as the set of frequently accessed buckets, and such that the second hash operation is exclusively configured to map the second key and associated value to a second set of buckets of the hash table used as the set of infrequently accessed buckets.

3. The method of claim 1, further comprising: Evict hash table entries from the second set of buckets in the hash table that serve as the infrequently accessed buckets, and insert the evicted entries into the first set of buckets in the hash table that serve as the frequently accessed buckets when the second set of buckets in the hash table is full; as well as The entry for the hashed second key is stored in the second bucket.

4. The method of claim 1, further comprising: Based on the tracking data, entries are moved from the second group of infrequently accessed buckets to the first group of frequently accessed buckets.

5. The method of claim 1, further comprising performing the first and second hash operations on a weighted basis in response to a table entry insertion request.

6. The method of claim 1, further comprising storing and using per-bucket fullness data to determine whether to move hash table entries between the first and second sets of buckets in the hash table.

7. A computing device, the computing device comprising: Hash table logic, which is configured as follows: Perform a first hash operation on the first key, wherein the first hash operation is biased to map the first key and associated value to a first candidate bucket within a first set of buckets in the hash table, the first set of buckets serving as a set of frequently accessed buckets; The entry and associated value of the first key are stored in the first candidate bucket within the first group of buckets in the hash table; A second hash operation is performed on the second key, wherein the second hash operation is biased to map the second key and associated value to a second candidate bucket within a second set of buckets in the hash table, the second set of buckets serving as a group of infrequently accessed buckets; The entry for the second key and its associated value are stored in the second candidate bucket within the second set of buckets in the hash table; In response to a lookup request for a key, a hash table lookup of the requested key is performed in the first set of frequently accessed buckets. If the requested key is not found, a hash table lookup of the requested key is performed in the first set of infrequently accessed buckets. A slot access counter structure configured to generate trace data representing the number of times each slot in a bucket within a first group of buckets used as a set of frequently accessed buckets of the hash table has been accessed, and the number of times each slot in a bucket within a second group of buckets used as a set of infrequently accessed buckets of the hash table has been accessed; and The hash table logic is also configured to move at least one slot entry between buckets based on the tracking data.

8. The computing apparatus of claim 7, wherein the hash table logic is further configured to bias the first and second hash operations such that the first hash operation is exclusively configured to map the first key and associated value to a first set of buckets of the hash table used as the set of frequently accessed buckets, and the second hash operation is exclusively configured to map the second key and associated value to a second set of buckets of the hash table used as the set of infrequently accessed buckets.

9. The computing apparatus of claim 7, wherein the hash table logic is further configured to: Evict hash table entries from the second set of buckets used as the infrequently accessed buckets, and insert the evicted entries into the first set of buckets used as the frequently accessed buckets when the second set of buckets is full; and The entry for the second key after hashing is stored in the second bucket.

10. The computing device of claim 7, further comprising: The hash table logic is further configured to move entries from the second group of infrequently accessed buckets to the first group of frequently accessed buckets based on the tracking data.

11. The computing apparatus of claim 7, wherein the hash table logic is further configured to perform the first and second hash operations on a weighted basis in response to a table entry insertion request.

12. The computing apparatus of claim 7, wherein the hash table logic is further configured to store and use per-bucket fullness data to determine whether to move a hash table entry between the first and second sets of buckets in the hash table.

13. The computing device of claim 7, further comprising: The memory includes the hash table; as well as One or more processors configured to perform logical operations on the hash table.

14. A non-transitory storage medium, the non-transitory storage medium comprising executable instructions, the executable instructions causing the one or more processors, when executed by one or more processors: Perform a first hash operation on the first key, wherein the first hash operation is biased to map the first key and associated value to a first candidate bucket within a first set of buckets in the hash table, the first set of buckets serving as a set of frequently accessed buckets; The entry and associated value of the first key are stored in the first candidate bucket within the first group of buckets in the hash table; A second hash operation is performed on the second key, wherein the second hash operation is biased to map the second key and associated value to a second candidate bucket within a second set of buckets in the hash table, the second set of buckets serving as a group of infrequently accessed buckets; The entry for the second key and its associated value are stored in the second candidate bucket within the second set of buckets in the hash table; In response to a lookup request for a key, a hash table lookup of the requested key is performed in the first set of frequently accessed buckets. If the requested key is not found, a hash table lookup of the requested key is performed in the first set of infrequently accessed buckets. Generate tracking data, which represents the number of times each slot in the first set of buckets used as the set of frequently accessed buckets of the hash table has been accessed and the number of times each slot in the second set of buckets used as the set of infrequently accessed buckets of the hash table has been accessed. as well as Based on the tracking data, move at least one slot entry between buckets.

15. The non-transitory storage medium of claim 14, further comprising executable instructions that, when executed by one or more processors, cause the one or more processors to bias the first and second hash operations such that the first hash operation is exclusively configured to map the first key and associated value to a first set of buckets of the hash table used as the set of frequently accessed buckets, and causes the second hash operation to exclusively configure to map the second key and associated value to a second set of buckets of the hash table used as the set of infrequently accessed buckets.

16. The non-transitory storage medium of claim 14, further comprising executable instructions that, when executed by one or more processors, cause the one or more processors to: Evict hash table entries from the second set of buckets used as the infrequently accessed buckets, and insert the evicted entries into the first set of buckets used as the frequently accessed buckets when the second set of buckets is full; and The entry for the second key after hashing is stored in the second bucket.

17. The non-transitory storage medium of claim 14, further comprising executable instructions that, when executed by one or more processors, cause the one or more processors to: Based on the tracking data, entries are moved from the second group of infrequently accessed buckets to the first group of frequently accessed buckets.

Images