Hash table processing method and apparatus, hash table query method and apparatus, and storage medium

Abstract

The present application discloses a hash table processing method and apparatus, a hash table query method and apparatus, and a storage medium. The hash table processing method comprises: in response to a storage request for key-value pairs to be stored, determining a target hash bucket of the key-value pairs to be stored, so as to determine, on the basis of a storage state of the target hash bucket of the key-value pairs to be stored, whether there is a hash collision; when there is the hash collision, determining, on the basis of the location states of key-value pairs recorded by bitmaps and stored in the target hash bucket and / or the storage states of the hash buckets other than the target hash bucket, the storage locations of the key-value pairs to be stored; and storing, into the determined storage locations, the key-value pairs to be stored, using the bitmaps to record the location states of the key-value pairs, and optimizing the storage locations of the key-value pairs to be stored. In the present invention, by combining the bitmaps and the hash buckets, the storage resources of a hash table can be effectively used, and the storage locations of the key-value pairs are reasonably determined without expanding the hash buckets, thereby improving the utilization rate of the storage resources.

Description

Hash table processing method and query method, device, and storage medium

[0001] Cross-reference to Related Applications

[0002] This application claims priority to the Chinese patent application No. 202411380417.5, filed on September 29, 2024, and entitled "Hash table processing method and query method, device, and storage medium", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the technical field of data storage, and in particular to a hash table processing method, a query method, a device, and a storage medium. BACKGROUND

[0004] A hash table is a widely used data structure in computer science and engineering, which has the functions of efficient insertion, deletion, and lookup, and is therefore widely used in fields such as database indexing, cache implementation, and table lookup.

[0005] However, when two or more key-value pairs point to the same index position via hash calculation, a hash collision occurs, which leads to performance degradation. For the processing of hash collisions, the current method usually uses the chain address method or the open address method to solve the problem, but these methods require multiple queries, which leads to low hash query efficiency and other problems, and cannot meet the performance requirements of high-performance table lookup scenarios.

[0006] In addition, in order to reduce hash collisions, the hash table needs to reserve a certain amount of free space or needs to expand the table entries, which in turn leads to low space utilization or inability to implement in a chip.

[0007] Therefore, there is an urgent need for an implementation method to effectively solve the problem of hash collisions. SUMMARY

[0008] The present application aims to at least partially solve one of the above technical problems.

[0009] To achieve the above object, the first aspect of the present application provides a hash table processing method, the hash table comprising a plurality of hash buckets and a bitmap, the processing method comprising: in response to a storage request of a to-be-stored key-value pair, determining a target hash bucket of the to-be-stored key-value pair, to determine whether there is a hash conflict according to a storage state of the target hash bucket of the to-be-stored key-value pair; in the case where the hash conflict exists, determining a storage position of the to-be-stored key-value pair according to a position state of a key-value pair stored in the target hash bucket recorded by the bitmap and / or a storage state of other hash buckets except the target hash bucket; storing the to-be-stored key-value pair into the determined storage position, and recording the position state of the key-value pair by using the bitmap.

[0010] According to an embodiment of the present application, the determining of the storage position of the to-be-stored key-value pair according to the storage state of other hash buckets except the target hash bucket comprises: obtaining the storage state of the other hash buckets, and taking a hash bucket with an idle storage state as a candidate hash bucket for the to-be-stored key-value pair; comparing the load conditions of the candidate hash buckets, and taking a hash bucket with the lightest load as the storage position of the to-be-stored key-value pair.

[0011] According to an embodiment of the present application, the determining of the storage position of the to-be-stored key-value pair according to the position state of the stored key-value pair in the target hash bucket recorded by the bitmap comprises: determining whether there is a backup key in the target hash bucket according to the position state of the stored key-value pair in the target hash bucket recorded by the bitmap; in the case where there is a backup key in the target hash bucket, obtaining the storage state of the other hash buckets except the target hash bucket; and moving the backup key to one of the other hash buckets with an idle storage state, so that the target hash bucket serves as the storage position of the to-be-stored key-value pair.

[0012] According to an embodiment of the present application, the moving of the backup key to the other hash bucket with an idle storage state comprises: determining whether the other hash bucket with an idle storage state includes the target hash bucket of the backup key; if the other hash bucket with an idle storage state includes the target hash bucket of the backup key, moving the backup key to the target hash bucket thereof, otherwise, moving the backup key to the other hash bucket with the lightest load.

[0013] According to an embodiment of the present application, the processing method further comprises: optimizing the storage position of the key-value pair in the hash bucket according to the position state of the key-value pair in the hash bucket recorded by the bitmap.

[0014] According to one embodiment of the present application, the optimizing the storage position of the key-value pair in the hash bucket according to the position state of the key-value pair recorded in the bitmap comprises: traversing the position state of the key-value pair recorded in the bitmap, determining the key-value pair whose position state is not at its target position as a key-value pair to be optimized; recalculating the hash value of the key-value pair to be optimized by using a preset hash function to determine the target hash bucket of the key-value pair to be optimized according to the hash value; and keeping or changing the storage position of the key-value pair to be optimized based on the storage state of the target hash bucket of the key-value pair to be optimized and / or the position state of the stored key-value pair.

[0015] According to one embodiment of the present application, the keeping or changing the storage position of the key-value pair to be optimized based on the storage state of the target hash bucket of the key-value pair to be optimized and / or the position state of the stored key-value pair comprises: if the storage state of the target hash bucket of the key-value pair to be optimized is idle, migrating the key-value pair to be optimized from its current storage position to the target hash bucket; if the storage state of the target hash bucket of the key-value pair to be optimized is full, determining whether there is a backup key in the target hash bucket according to the position state of the stored key-value pair; if there is a backup key in the target hash bucket, migrating the backup key to other hash buckets according to a preset migration strategy to change the storage position of the key-value pair to be migrated to the target hash bucket, otherwise keeping the storage position of the key-value pair to be migrated unchanged.

[0016] According to one embodiment of the present application, the migrating the backup key to other hash buckets according to a preset migration strategy to change the storage position of the key-value pair to be optimized to the target hash bucket if there is a backup key in the target hash bucket comprises: rehashing one of the backup keys to be migrated in the target hash bucket to determine the target hash bucket of the backup key; determining the storage state of the target hash bucket of the backup key, if the storage state of the target hash bucket of the backup key to be migrated is idle, migrating the backup key to be migrated out to the determined target hash bucket and migrating the key-value pair to be optimized to its target hash bucket; or if the storage state of the target hash bucket of the backup key to be migrated is full, continuing to determine whether the target hash bucket of the backup key to be migrated has a backup key based on the bitmap, and repeating the above manner to continue to determine whether the backup key in the target hash bucket of the backup key to be migrated can be migrated out until the migration of the backup key to be migrated is successful and the key-value pair to be optimized is migrated to its corresponding target hash bucket.

[0017] According to one embodiment of the present application, if the target hash bucket has a backup key, a preset migration strategy is used to migrate the backup key to other hash buckets to change the storage location of the key-value pair to be optimized to the target hash bucket, otherwise the storage location of the key-value pair to be optimized remains unchanged, and the method comprises the following steps: obtaining the access attribute of the backup key in the target hash bucket, selecting a backup key to be migrated, and re-performing hash calculation on the backup key to be migrated to re-determine the target hash bucket of the backup key to be migrated; judging the storage state of the target hash bucket of the backup key to be migrated, if the storage state of the backup key to be migrated is idle, moving the backup key to be migrated to the target hash bucket to move the key-value pair to be optimized into the re-determined target hash bucket; or if the target hash bucket of the backup key to be migrated is full, moving the backup key to be migrated to other hash buckets with idle storage units based on the bitmap to move the key-value pair to be optimized into the corresponding target hash bucket.

[0018] To achieve the above object, the second aspect of the present application provides a hash table query method, the hash table comprising a plurality of hash buckets and a bitmap, the query method comprising the following steps: in response to a query request for a key-value pair to be queried, locating a target hash bucket of the key-value pair to be queried; matching the key-value pair to be queried with the key-value pairs stored in the target hash bucket to obtain a matching result for the key-value pair to be queried; if the matching result is a matching success, returning a query success query result; if the matching result is a matching failure, determining a hash bucket with a backup key based on the position state of the key-value pairs in other hash buckets recorded by the bitmap, and matching the key-value pair to be queried with the key-value pairs stored in the hash bucket with the backup key to return a query success or query failure query result.

[0019] To achieve the above object, the third aspect of the present application provides a hash table processing device, the hash table comprising a plurality of hash buckets and a bitmap, the processing device comprising: a judgment unit configured to determine a target hash bucket of a key-value pair to be stored in response to a storage request of the key-value pair to be stored, to determine whether there is a hash conflict according to the storage state of the target hash bucket of the key-value pair to be stored; a determination unit configured to determine a storage location of the key-value pair to be stored according to the position state of the key-value pairs stored in the target hash bucket recorded by the bitmap and / or the storage state of the remaining hash buckets other than the target hash bucket in the case of the hash conflict; and a storage unit configured to store the key-value pair to be stored into the determined storage location and record the position state of the key-value pair by using the bitmap.

[0020] To achieve the above object, the fourth aspect of the present application provides a query device of a hash table, the hash table comprising a plurality of hash buckets and a bitmap, the query device comprising: a positioning unit configured to, in response to a query request for a key-value pair to be queried, position a target hash bucket of the key-value pair to be queried; a matching unit configured to match the key-value pair to be queried with a key-value pair stored in the target hash bucket to obtain a matching result for the key-value pair to be queried; and a query unit configured to, if the matching result is a matching success, return a query result of a query success, or, if the matching result is a matching failure, determine a hash bucket with a backup key based on a position state of a key-value pair in other hash buckets recorded by the bitmap, and match the key-value pair to be queried with a key-value pair stored in the hash bucket with the backup key to return a query result of a query success or a query failure.

[0021] To achieve the above object, the fifth aspect of the present application provides a non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are used to make the computer execute the processing method of the hash table according to any one of the first aspect or the query method of the hash table according to the second aspect.

[0022] Compared with the prior art, the present application has the following advantages:

[0023] 1) The data structure combining the hash bucket and the bitmap is used to improve the accuracy of data storage of the hash table and ensure high table lookup performance;

[0024] 2) The storage state of the hash bucket and the position state of the key-value pair recorded by the bitmap are used to reasonably store the key-value pair, effectively improve the utilization rate of the storage resource in the hash table, and reasonably use the idle storage space;

[0025] 3) The storage position of the backup key is optimized, the key-value pair is reasonably stored at the target position to the greatest extent, and the table lookup performance of the hash table is further improved;

[0026] 4) The hash table does not need to be expanded, and based on the reasonable utilization of the resources of the hash table, the hash table can be easily implemented in a chip.

[0027] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0028] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, taken in conjunction with the following drawings, in which:

[0029] FIG. 1 is a flowchart illustrating a processing method of a hash table according to an exemplary embodiment;

[0030] FIG. 2 illustrates a hash representation of an intent according to an example embodiment;

[0031] FIG. 3 is a flow diagram illustrating a method of determining a storage location of a key-value pair to be stored according to an example embodiment;

[0032] FIG. 4 illustrates another hash representation of an intent according to an example embodiment;

[0033] FIG. 5 is a flow diagram illustrating another method of determining a storage location of a key-value pair to be stored according to an example embodiment.

[0034] FIG. 6 illustrates another hash representation of an intent according to an example embodiment;

[0035] FIG. 7 is a flow diagram illustrating a method of optimizing a location of a key-value pair according to an example embodiment;

[0036] FIG. 8 illustrates yet another hash representation of an intent according to an example embodiment; and

[0037] FIG. 9 is a flow diagram illustrating another method of optimizing a location of a key-value pair according to an example embodiment. DETAILED DESCRIPTION

[0038] Embodiments of the present application are described in detail below with reference to the attached drawings, which are meant to be exemplary and not limiting. The embodiments described below are examples of implementations of the present application and are not intended to limit the present application.

[0039] FIG. 1 is a flow diagram illustrating a method of processing a hash table according to an example embodiment. It should be noted that the method of implementing the embodiments of the present application can be applied to the apparatus for implementing the embodiments of the present application, which can be configured on an electronic device or in a server, where the electronic device can be a PC or a mobile terminal. The embodiments of the present application do not make excessive limitations on this.

[0040] The hash table in the present application adopts a data structure combining hash buckets and bitmaps, as shown in FIG. 1. The method of implementing the hash table includes the following steps:

[0041] In step S110, in response to a storage request of a key-value pair to be stored, a target hash bucket of the key-value pair to be stored is determined, so as to determine whether there is a hash conflict according to a storage state of the target hash bucket of the key-value pair to be stored;

[0042] Step S120, in the case of the hash collision, determining the storage location of the to-be-stored key-value pair according to the position state of the key-value pair stored in the target hash bucket recorded by the bitmap and / or the storage state of other hash buckets except the target hash bucket.

[0043] Step S130, storing the to-be-stored key-value pair into the determined storage location, and recording the position state of the key-value pair by using the bitmap.

[0044] For the purpose of clear understanding of the present application, first of all, the basic structure of the hash table is briefly introduced in combination with FIG. 2. As shown in FIG. 2, the hash table usually includes a plurality of hash buckets, such as Bucketl and Bucket2 in FIG. 2. Each hash bucket includes a plurality of storage units, also called bins, each of which maps different storage addresses. For example, each hash bucket in FIG. 2 includes 8 storage units bins, binl-bin8 of Bucketl map addresses Locationl-Location8 respectively, and binl-bin8 of Bucket2 map addresses Location9-Location16 respectively.

[0045] Therefore, when it is needed to store a key-value pair into the hash table shown in FIG. 2, it is first calculated by a hash function to determine the corresponding hash bucket, which can also be called a target hash bucket or a reasonable hash bucket, which is regarded as the best storage location of the current to-be-stored key-value pair. If there is a free storage unit in the target hash bucket, i.e. there is no hash collision, then a key-value pair is stored in the free storage unit. For example, it is determined by hash calculation that Bucket2 in FIG. 2 is the target hash bucket of the key-value pair. And there are three free storage units in Bucket2. Then one of the three free storage units can be selected to store the key-value pair.

[0046] However, when there is a hash collision, i.e. each storage unit in the target hash bucket stores a key-value, at this time, the key-value pair cannot be stored into its target hash bucket in the above-mentioned manner, at this time, a method is needed to reasonably utilize the storage resources to effectively store the key-value pair in the hash table.

[0047] For the present application, first when responding to the storage request of the key-value pair to be stored, the target hash bucket of the key-value pair is determined by using the pre-set hash function. The specific calculation of the hash function and the specific way of determining the target hash bucket based on the hash function are not limited too much in the present application. Further, whether there is a hash conflict can be determined according to the storage state of the target hash bucket. The storage state in the present application includes two states of full and idle. When the storage state of the hash bucket is full, it means that each storage unit in the current hash bucket stores a key-value pair. At this time, if the new key-value pair is stored in the hash bucket, it means that there is a hash conflict. When the storage state of the hash bucket is idle, it means that the current hash bucket has idle storage units, that is, these idle storage units do not store key-value pairs. At this time, there is no hash conflict, and the new key-value pair can be stored in the idle storage unit in the target hash bucket.

[0048] Further, in the present application, when it is determined that there is a hash conflict, the storage location of the key-value pair to be stored is determined in combination with the record of the bitmap. Specifically, the bitmap of the present application can include a first bitmap and a second bitmap. The first bitmap is used to record the position state of the key-value pair stored in the hash bucket, wherein the position state can include at its target position and not at its target position. For example, the first bitmap can record that the key-value pair is not at its target position with "0", and record that the key-value pair is at its target position with "1". The second bitmap is used to record the storage state of the hash bucket, wherein the storage state can include idle or full. Similarly, for example, the second bitmap can record different storage states with "0" and "1". Further, the present application can reasonably determine the storage location of the key-value pair to be stored based on the position state of the key-value in the hash bucket recorded by the bitmap and / or the storage state of the hash bucket.

[0049] Finally, when the key-value pair is stored in the storage location, the position state can be recorded by using the bitmap, so that the storage of the key-value pair can be effectively optimized and managed subsequently.

[0050] It can be seen that the present application combines the bitmap and the hash bucket, can effectively utilize the storage resources of the hash table, reasonably determine the storage location of the key-value pair without expanding the hash bucket, and improve the utilization rate of the storage resources.

[0051] The specific implementation details of the present application will be further described and explained in combination with the accompanying drawings.

[0052] In a preferred embodiment, as shown in FIG. 3, the storage location of the key-value pair to be stored is determined according to the storage state of the hash bucket other than the target hash bucket, including:

[0053] Step S310, obtaining the storage state of the remaining hash buckets, and taking the hash bucket with an idle storage state as a candidate hash bucket for the to-be-stored key-value pair;

[0054] Step S320, comparing the load conditions of the candidate hash buckets, and taking the hash bucket with the lightest load as the storage location of the to-be-stored key-value pair.

[0055] For example, when it is determined that the target hash bucket has no idle storage unit, it is necessary to first obtain the storage states of other hash buckets, so as to store the key-value pair in other hash buckets to effectively utilize the idle storage resources of other hash buckets. Then, the hash bucket to be stored is selected for the key-value pair according to a pre-configured storage strategy. In this application, the pre-configured storage strategy can include a random strategy, a load strategy, or a configuration strategy based on other conditional factors. The random strategy refers to storing the to-be-stored key-value pair in any idle hash bucket at random. The load strategy refers to storing the to-be-stored key-value pair in an idle hash bucket that meets a pre-set load condition based on the load condition of the idle hash bucket. Other conditional factors can include a location factor based on the hash bucket, for example, placing the to-be-stored key-value pair in the leftmost (or rightmost) idle hash bucket. The embodiment of this application preferably adopts the load strategy, and the pre-set load condition is the lightest load, that is, the key-value pair is stored in the hash bucket with the lightest load by judging the load condition of the hash bucket. As shown in FIG. 4, it is assumed that Bucket3 is the target hash bucket of the to-be-stored key-value pair, but Bucket3 is full and has no idle storage unit. At this time, other hash buckets Bucket 1 and Bucket 2 have idle storage units and can be used as candidate hash systems. Then, the load conditions of Bucket1 and Bucket2 are further compared. For example, the load condition can be judged based on the number of idle storage units in the hash bucket. The more idle storage units, the lighter the load of the hash bucket. As shown in FIG. 4, the number of idle storage units in Bucket1 and Bucket2 is 2 and 3 respectively, so it can be determined that Bucket2 is the hash bucket with the lightest load. Then, the key-value pair is stored in the idle storage unit in Bucket2, and the storage location and storage state of the key-value pair are recorded in the bitmap, so as to track and manage the location of the key-value pair by the bitmap.

[0056] In the embodiment of this application, when there is a hash conflict, the key-value pair is reasonably stored in other hash buckets according to the storage state of the other hash buckets, the idle resources of the other hash buckets are reasonably utilized, and the movement of the key-value pair is reduced. At the same time, the hash bucket for storage is determined based on the load comparison, which can advantageously balance the storage pressure among the hash buckets and improve the storage performance of the hash table.

[0057] In a preferred embodiment, as shown in FIG. 5, the storage location of the to-be-stored key-value pair is determined according to the position state of the stored key-value pairs in the target hash bucket recorded by the bitmap, comprising:

[0058] In step S510, whether there is a backup key in the target hash bucket is determined according to the position state of the stored key-value pairs in the target hash bucket recorded by the bitmap.

[0059] In step S520, in the case that there is a backup key in the target hash bucket, the storage state of the remaining hash buckets other than the target hash bucket is obtained.

[0060] In step S530, the backup key is moved to one of the remaining hash buckets with an empty storage state, and the target hash bucket is made as the storage location for storing the to-be-stored key-value pair.

[0061] For example, compared with the method of storing the key-value pair in other hash bucket with free storage unit in the above embodiment, the embodiment of the present application aims to move the spare key in the target hash bucket to make room for the key-value pair to be stored. Based on the foregoing, it is known that in the process of handling hash conflicts, the target hash bucket corresponding to each key-value pair can be determined by calculating the hash function. In order to store the key-value pair in the corresponding target hash bucket, the embodiment of the present application moves the key-value pair in the target hash bucket based on the position state of the key-value pair in the target hash bucket. Specifically, the spare key mentioned in the present application refers to the key that should be stored in other hash bucket but is stored in the current bucket due to hash conflicts. As mentioned above, the present application uses a bitmap to record the position state of the key-value pair. When the position state of the key-value pair stored in the target hash bucket is "1", it means that the key-value pair is not in the corresponding target hash bucket. The present application determines the key-value pair with the position state of "1" in the bitmap as the spare key. Further, after determining the spare key in the target hash bucket, the spare key can be moved to make room for the storage position of the target hash bucket. Specifically, as shown in FIG. 6, it is assumed that the target hash bucket is Bucket3, and Bucket3 is currently full and has no free storage unit. However, it can be known from the bitmap that Bucket3 contains a spare key that should be stored in other hash bucket, such as the key-value pair stored in Location 19 in FIG. 6. Therefore, the spare key in Location 19 can be moved to other hash bucket with free storage unit. It should be noted that the new position of the key-value pair in Location 19 can be the corresponding target hash bucket or other hash bucket with free storage unit. The embodiment of the present application preferably moves Location 19 to the target hash bucket (when the target hash bucket has free storage unit). After successfully moving the spare key in Location 19, the position of Location 19 is not stored with key-value pair, so the key-value pair to be stored can be stored in Location 19. In addition, if the spare key in Location 19 cannot be successfully moved, the other spare keys in the target hash bucket, such as the spare keys in Location 21 or Location 22, can be moved in turn. The spare keys in Location 21 or Location 22 are moved out using the same moving strategy as Location 19, until the target hash bucket successfully makes room for a storage position and the key-value pair to be stored can be stored in the target hash bucket.

[0062] In the embodiments of the present application, by obtaining the backup key condition of the target hash bucket, the key-value pair to be stored is stored in the corresponding target hash bucket by moving the backup key. Although this method increases the movement of backup keys, it reduces the generation of backup keys when storing new key-value pairs, and enables the key-value pair to be stored in the target hash bucket, effectively improving the lookup performance of the hash table.

[0063] As can be seen from the description of the above embodiments, when facing a hash conflict, whether the key-value pair is stored in a hash bucket with a free storage unit or the backup key is moved to another hash bucket to make room for the current key-value pair to be stored, there will be some key-value pairs (backup keys) that are not stored in their target positions (corresponding target hash buckets). Therefore, in order to improve the subsequent table lookup performance, the present application records the position state of the key-value pair based on the bitmap, and moves the backup key to its target position by reasonable movement of the key-value pair.

[0064] In a preferred embodiment, as shown in FIG. 7, the storage position of the key-value pair in the hash bucket is optimized according to the position state of the key-value pair in the hash bucket recorded by the bitmap, comprising:

[0065] Step S710: The position state of the key-value pair recorded in the bitmap is traversed to determine the key-value pair whose position state is not in its target position as the key-value pair to be optimized;

[0066] Step S720: The hash value of the key-value pair to be optimized is recalculated using a preset hash function to determine the target hash bucket of the key-value pair to be optimized according to the hash value;

[0067] Step S730: Based on the storage state of the target hash bucket of the key-value pair to be optimized and / or the position state of the stored key-value pair, the storage position of the key-value pair to be optimized is maintained or changed.

[0068] For example, as mentioned above, the bitmap can use 0 and 1 to represent whether the key stored in a certain position is in its target position. Therefore, in the embodiments of the present application, the background thread can traverse the positions recorded as "1" in the bitmap to determine the key-value pair to be migrated. After determining the key-value pair to be migrated, the hash value of each key pair to be migrated needs to be recalculated to determine its corresponding target hash bucket. Further, the storage state of the target hash bucket determined by recalculation and / or the position state of the stored key-value pair is obtained to maintain or change the storage position of the key-value pair to be optimized for different strategies of position optimization.

[0069] More preferably, the storage state of the target hash bucket based on the key-value pair to be optimized and / or the position state of the stored key-value pair keeps or changes the storage position of the key-value pair to be optimized, including: if the storage state of the target hash bucket of the key-value pair to be optimized is idle, migrating the key-value pair to be optimized from its current storage position to the target hash bucket; if the storage state of the target hash bucket of the key-value pair to be optimized is full, determining whether there is a backup key in the target hash bucket according to the position state of the stored key-value pair; if there is a backup key in the target hash bucket, migrating the backup key to other hash buckets according to a preset migration strategy to change the storage position of the key-value pair to be migrated to the target hash bucket, otherwise keeping the storage position of the key-value pair to be migrated unchanged.

[0070] For the preset migration strategy, the present application will be described below for two preferred modes respectively.

[0071] In a preferred embodiment, if there is a backup key in the target hash bucket, the backup key is migrated to other hash buckets according to a preset migration strategy to change the storage position of the key-value pair to be optimized to the target hash bucket, including: rehashing one of the backup keys to be migrated in the target hash bucket to determine the target hash bucket of the backup key; determining the storage state of the target hash bucket of the backup key, if the storage state of the target hash bucket of the backup key to be migrated is idle, moving the backup key to be migrated out to the determined target hash bucket, and moving the key-value pair to be optimized into its target hash bucket; or if the storage state of the target hash bucket of the backup key to be migrated is full, determining whether the target hash bucket of the backup key to be migrated has a backup key based on the bitmap, and repeating the above mode to determine whether the backup key in the target hash bucket of the backup key to be migrated can be moved out, until the backup key to be migrated is successfully migrated, and the key-value pair to be optimized is moved into its corresponding target hash bucket.

[0072] For example, as shown in FIG. 8, if the target hash bucket of the key to be migrated is Bucket 3, if the current Bucket 3 has a free storage unit, the key to be migrated can be moved into Bucket 3, and the record of the key-value pair in the bitmap is updated. If the Bucket 3 has no free storage unit, it is further determined whether the Bucket 3 has a backup key. If there is no backup key, it means that all the keys stored in the current Bucket 3 are in their target positions, and the key-value pair to be migrated is not migrated temporarily, and the current storage position is kept unchanged. If the Bucket 3 has a backup key, one of the backup keys can be selected for migration to vacate the position for the key-value pair to be migrated. For example, as shown in FIG. 8, the backup key of Location 19 is tried to migrate, the hash value of the key-value pair corresponding to Location 19 is recalculated, and the target hash bucket is determined to be Bucket 1. The Bucket 1 is full, but contains two backup keys (in Location 4 and Location 5) that should be stored in other hash buckets. Then, the backup key in Location 4 is tried to migrate, the hash value of the key-value pair corresponding to Location 4 is recalculated, and the target hash bucket is determined to be Bucket 2. The storage state of the Bucket 2 is free, and has a free storage unit, for example, Location 10, and thus the key-value pair in Location 4 can be migrated to Location 10. Thus, the key-value pair in Location 19 is migrated to Location 4. It should be noted that if the migration of Location 19 fails, Location 21 is tried to migrate, and if the migration of Location 21 fails, Location 22 is tried to migrate. In addition, if the backup keys in the target hash bucket all fail to migrate, it is considered that there is no backup key in the current target hash bucket to be migrated, and the migration is ended.

[0073] The optimization process of the embodiment is further described below with reference to FIG. 9, and specifically as follows:

[0074] In step S910, the position state of the key-value pair of the hash bucket recorded in the bitmap is traversed to determine whether there is a backup key. For example, whether there is "1" in the bitmap is traversed. If yes, step S920 is performed, otherwise the current position optimization is ended.

[0075] In step S920, when there is a backup key, one of the key-value pairs X is selected for position optimization.

[0076] Step S930, it is judged whether the target hash bucket corresponding to the key-value pair X has an idle storage unit. For example, in the position optimization process, the key-value pair X is re-hashed, the target hash bucket corresponding to the calculated hash value is determined, and it is further judged whether the re-determined target hash bucket has an idle storage unit. If there is no idle storage unit, step S940 is executed, otherwise step S950 is executed.

[0077] Step S940, when there is no idle storage unit in the target hash bucket, it means that the storage of the target hash bucket is full, and it is further needed to query whether there is a backup key stored in the target hash bucket based on the bitmap. If there is, step S960 is executed, otherwise the current position optimization is ended.

[0078] Step S950, when the target hash bucket has an idle storage unit, the key-value pair X is migrated to the target hash bucket, and the current position optimization is ended.

[0079] Step S960, when it is judged that the target hash bucket stores a backup key, all key-value pairs with the record of "1" are obtained based on the bitmap.

[0080] Step S970, one of the key-value pairs is migrated. For example, when one of the key-value pairs is migrated, the hash value of the key-value pair to be migrated is calculated, and the target hash bucket is determined. Then, it is judged whether the target hash bucket has an idle storage unit. If the key-value pair is not successfully migrated, other key-value pairs are tried one by one until an idle position is vacated in the target hash bucket.

[0081] Step S980, when the target hash bucket has an idle storage unit through the migration process in the last step, the key-value pair X is stored in the target hash bucket, and the current position optimization is ended.

[0082] In the embodiments of the present application, by trying to migrate the backup key in the target hash bucket of the key-value pair to be optimized, an idle position is vacated in the target hash bucket, and the key-value pair to be optimized can be reasonably positioned, so that it is stored in the target hash bucket corresponding thereto, the table lookup performance of the hash table is further improved, and a high-performance hash table is realized.

[0083] In another preferred embodiment, if the target hash bucket has a backup key, a preset migration strategy is adopted to migrate the backup key to other hash buckets to change the storage location of the key-value pair to be optimized to the target hash bucket, otherwise the storage location of the key-value pair to be optimized remains unchanged, including: selecting a backup key to be migrated, and re-performing hash calculation on the backup key to be migrated to re-determine the target hash bucket of the backup key to be migrated; judging the storage state of the target hash bucket of the backup key to be migrated, if the storage state of the backup key to be migrated is idle, moving the backup key to be migrated to the target hash bucket to move the key-value pair to be optimized into the re-determined target hash bucket; or if the target hash bucket of the backup key to be migrated is full, moving the backup key to be migrated to other hash buckets with idle storage units based on the bitmap to move the key-value pair to be optimized into the corresponding target hash bucket.

[0084] For example, in the example, the backup key currently needed to be moved out can be selected from the backup keys. As mentioned above, the hash calculation is first performed on the backup key to be migrated to re-determine the target hash bucket. If the hash bucket is idle, the backup key can be normally moved to the idle storage unit of the hash bucket. When the hash bucket is full, unlike the above embodiment, the backup key to be migrated does not need to be migrated to the re-determined target hash bucket recursively, but can be directly moved to other hash buckets with idle storage units. Taking the above example as an example, as shown in FIG. 8, the backup of the backup key of Location 19 is also tried to be migrated, and the re-determined target hash bucket is Bucket 1. When Bucket 1 is full, Bucket 2 has an idle storage unit Location 10, so the backup key of Location 19 can be moved to Location 10 of Bucket 2, thereby freeing Location 19 and moving the key-value pair to be optimized into.

[0085] For the two methods mentioned above, the first method migrates the backup keys recursively, which has the advantage that multiple backup keys can be returned in one migration. The second method migrates to the hash bucket with idle storage units, which can quickly return the key-value pair to be optimized, reducing the movement of other keys and the calculation amount generated thereby. Therefore, the two methods can be selected according to the actual use occasion. It should be noted that the present application is not limited to the two methods, and those skilled in the art can flexibly adopt other migration strategies according to actual needs, which will not be limited herein.

[0086] Based on the same technical concept as above, the application further provides a hash table query method, which comprises:

[0087] locating a target hash bucket of a key-value pair to be queried in response to a query request for the key-value pair to be queried;

[0088] matching the key-value pair to be queried with key-value pairs stored in the target hash bucket to obtain a matching result for the key-value pair to be queried;

[0089] if the matching result is a matching success, returning a query success query result; if the matching result is a matching failure, determining a hash bucket with a backup key based on the position state of the key-value pairs in other hash buckets recorded by the bitmap, and matching the key-value pair to be queried with key-value pairs stored in the hash bucket with the backup key to return a query success or query failure query result.

[0090] For example, when a key value needs to be queried from a hash table, hash calculation is also needed for the query key to obtain the ideal position where the query key is supposed to be stored. Then, the query key is matched with all the stored keys in the target hash bucket one by one. If the matching is successful, it indicates that the query key is in the target hash bucket. If the matching fails, it indicates that the key-value pair is stored in other buckets in the form of a backup key. At this time, based on the storage state of other buckets recorded by the bitmap, a hash bucket with a backup key can be selected, and then the key-value pair is matched with the backup key in the hash bucket with the backup key. If the matching is successful, the position of the query key is returned. If the matching fails, it indicates that the hash table does not have the key-value pair.

[0091] In the embodiments of the application, when the query key and the stored key in the target hash bucket do not match successfully, the storage state of the backup key recorded by the bitmap can be combined to quickly query and obtain the query result of the key-value pair.

[0092] In summary, the application has the following advantages compared with the prior art:

[0093] 1) The hash bucket is combined with the bitmap to improve the correctness of data storage and ensure the table lookup performance;

[0094] 2) For hash conflicts, the storage state of the hash table recorded by the bitmap is combined to store in a backup position or migrate storage in a backup key, to improve the utilization rate of storage resources and reasonably use the idle storage space;

[0095] 3) The storage position of the backup key is optimized to maximize the storage of the key value in the target position and maximize the table lookup performance of the hash table;

[0096] 4) Utilize the key value itself attribute or the demand utilization of the key value, and store by using a proper storage strategy, further improve the access speed and reasonably optimize the storage resource utilization.

[0097] Based on the same technical concept, the application further provides a hash table processing device, the hash table comprising a plurality of hash buckets and a bitmap, the processing device comprising: a judging unit configured to determine a target hash bucket of a to-be-stored key-value pair in response to a storage request of the to-be-stored key-value pair, to judge whether there is a hash conflict according to a storage state of the target hash bucket of the to-be-stored key-value pair; a determining unit configured to determine a storage position of the to-be-stored key-value pair according to a position state of a key-value pair stored in the target hash bucket recorded by the bitmap and / or a storage state of the remaining hash buckets other than the target hash bucket in the case where there is the hash conflict; and a storage unit configured to store the to-be-stored key-value pair into the determined storage position and record the position state of the key-value pair by using the bitmap.

[0098] The specific embodiment details and more advantages of the hash table of the embodiment of the application can be referred to the above-mentioned embodiment of the hash table processing method, and will not be described in detail here.

[0099] Correspondingly, the application further provides a hash table query device, the hash table comprising a plurality of hash buckets and a bitmap, the query device comprising: a positioning unit configured to locate a target hash bucket of a to-be-queried key-value pair in response to a query request of the to-be-queried key-value pair; a matching unit configured to match the to-be-queried key-value pair with a key-value pair stored in the target hash bucket, to obtain a matching result of the to-be-queried key-value pair; and a query unit configured to return a query success query result if the matching result is a matching success, or to determine a hash bucket with a backup key based on a position state of a key-value pair in other hash buckets recorded by the bitmap, and match the to-be-queried key-value pair with a key-value pair stored in the hash bucket with the backup key, to return a query success or a query failure query result.

[0100] The specific embodiment details and more advantages of the hash table query device of the embodiment of the application can be referred to the above-mentioned embodiment of the hash table query method, and will not be described in detail here.

[0101] Correspondingly, the application further provides a non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are used to make the computer execute the hash table processing method or the hash table query method of the above-mentioned embodiments.

[0102] The computer program product of the first aspect can include one or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the operations of the first aspect. The computer program product of the first aspect can include a non-transitory computer-readable medium storing code for causing a computer to perform the operations of the first aspect.

[0103] The units described in the embodiments of the present application can be implemented by software, or by hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself. For example, the first obtaining unit can also be described as a unit for obtaining at least two Internet protocol addresses.

[0104] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0105] In the context of the present application, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0106] The above description merely illustrates the preferred embodiment of the application and a principle for applying the technology. It is understood by those skilled in the art that the disclosed range of the present application is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by the combinations of the technical features described above or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features with similar functions disclosed in the present application (but not limited to) can be used.

[0107] Further, although operations are depicted in a particular, chronological sequence, this should not be understood as requiring the operations to be performed in that order, or to be performed sequentially. In some circumstances, multitasking and parallel processing can be advantageous. Likewise, although specific implementation details are contained in the above discussion, these should not be construed as limiting the scope of the application. Some features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any suitable sub-combination.

[0108] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A processing method of a hash table, characterized by, The hash table comprises a plurality of hash buckets and a bitmap, and the processing method comprises: in response to a storage request of a key-value pair to be stored, determining a target hash bucket of the key-value pair to be stored, to determine whether there is a hash conflict according to a storage state of the target hash bucket of the key-value pair to be stored; in the case where the hash conflict exists, determining a storage position of the key-value pair to be stored according to a position state of a key-value pair stored in the target hash bucket recorded by the bitmap and / or a storage state of other hash buckets except the target hash bucket; storing the key-value pair to be stored into the determined storage position, and recording the position state of the key-value pair by the bitmap.

2. The treatment method according to claim 1, characterized in that, The determination of the storage position of the key-value pair to be stored according to the storage state of other hash buckets except the target hash bucket comprises: obtaining the storage state of the other hash buckets, and taking a hash bucket with an idle storage state as a candidate hash bucket for the key-value pair to be stored; comparing the load conditions of the candidate hash buckets, and taking a hash bucket with the lightest load as the storage position of the key-value pair to be stored.

3. The treatment method of claim 1, wherein The determination of the storage position of the key-value pair to be stored according to the position state of the key-value pair stored in the target hash bucket recorded by the bitmap comprises: determining whether there is a backup key in the target hash bucket according to the position state of the key-value pair stored in the target hash bucket recorded by the bitmap; in the case where there is a backup key in the target hash bucket, obtaining the storage state of other hash buckets except the target hash bucket; moving the backup key to one of the remaining hash buckets with an idle storage state, so that the target hash bucket serves as the storage position of the key-value pair to be stored.

4. The treatment method according to claim 3, characterized in that, The moving of the backup key to the remaining hash buckets with an idle storage state comprises: determining whether the remaining hash buckets with an idle storage state include the target hash bucket of the backup key; if the remaining hash buckets with an idle storage state include the target hash bucket of the backup key, moving the backup key to its target hash bucket, otherwise, moving the backup key to the remaining hash bucket with the lightest load.

5. The treatment method of claim 1, wherein The processing method further comprises: optimizing the storage position of the key-value pair in the hash bucket according to the position state of the key-value pair in the hash bucket recorded by the bitmap.

6. The treatment method according to claim 5, characterized in that, The optimization of the storage position of the key-value pair in the hash bucket according to the position state of the key-value pair in the hash bucket recorded by the bitmap comprises: traversing the position state of the key-value pair recorded in the bitmap, and determining a key-value pair with a position state not in its target position as a key-value pair to be optimized; recomputing the hash value of the key-value pair to be optimized by using a preset hash function, to redetermine the target hash bucket of the key-value pair to be optimized according to the hash value; maintaining or changing the storage position of the key-value pair to be optimized based on the storage state of the target hash bucket of the key-value pair to be optimized and / or the position state of the stored key-value pair.

7. The treatment method according to claim 6, characterized in that, The storage state of the target hash bucket based on the key-value pair to be optimized and / or the position state of the stored key-value pair keeps or changes the storage position of the key-value pair to be optimized, including: If the storage state of the target hash bucket of the key-value pair to be optimized is idle, the key-value pair to be optimized is migrated from its current storage position to the target hash bucket; If the storage state of the target hash bucket of the key-value pair to be optimized is full, according to the position state of the key-value pair stored in the target hash bucket, it is judged whether there is a backup key in the target hash bucket; If there is a backup key in the target hash bucket, the backup key is migrated to other hash buckets according to a preset migration strategy, so as to change the storage position of the key-value pair to be optimized to the target hash bucket, otherwise the storage position of the key-value pair to be optimized is kept unchanged.

8. The treatment method according to claim 7, characterized in that, If there is a backup key in the target hash bucket, the backup key is migrated to other hash buckets according to a preset migration strategy, so as to change the storage position of the key-value pair to be optimized to the target hash bucket, including: The target hash bucket of the backup key to be migrated is re-calculated, and the target hash bucket of the backup key is re-determined; If the storage state of the target hash bucket of the backup key to be migrated is idle, the backup key to be migrated is removed to the re-determined target hash bucket, and the key-value pair to be optimized is moved to its target hash bucket; or If the storage state of the target hash bucket of the backup key to be migrated is full, whether the target hash bucket of the backup key to be migrated has a backup key is judged based on the bitmap, and the above method is repeated to continue to judge whether the backup key in the target hash bucket of the backup key can be removed, until the migration of the backup key to be migrated is successful, and the key-value pair to be optimized is moved to its corresponding target hash bucket.

9. The treatment method of claim 7, wherein, If there is a backup key in the target hash bucket, the backup key is migrated to other hash buckets according to a preset migration strategy, so as to change the storage position of the key-value pair to be optimized to the target hash bucket, otherwise the storage position of the key-value pair to be optimized is kept unchanged, including: The backup key to be migrated is selected, and the target hash bucket of the backup key to be migrated is re-calculated to re-determine the target hash bucket of the backup key to be migrated; If the storage state of the target hash bucket of the backup key to be migrated is idle, the backup key to be migrated is moved to its target hash bucket to move the key-value pair to be optimized to the re-determined target hash bucket; or If the target hash bucket of the backup key to be migrated is full, the backup key to be migrated is moved to other hash buckets with idle storage units based on the bitmap to move the key-value pair to be optimized to its corresponding target hash bucket.

10. A method of searching a hash table, characterized by, The hash table includes a plurality of hash buckets and a bitmap, and the query method includes: In response to a query request for a key-value pair to be queried, the target hash bucket of the key-value pair to be queried is located; matching the key-value pair to be queried with the key-value pairs stored in the target hash bucket to obtain a matching result for the key-value pair to be queried; if the matching result is a matching success, returning a query success result; or if the matching result is a matching failure, determining a hash bucket with a backup key based on the position state of the key-value pairs in other hash buckets recorded by the bitmap, and matching the key-value pair to be queried with the key-value pairs stored in the hash bucket with the backup key to return a query success result or a query failure result.

11. A processing device of a hash table, characterized by, The hash table includes a plurality of hash buckets and a bitmap, and the processing device includes: a judging unit configured to determine a target hash bucket of a key-value pair to be stored in response to a storage request for the key-value pair to be stored, and determine whether there is a hash conflict according to a storage state of the target hash bucket of the key-value pair to be stored; a determining unit configured to determine a storage position of the key-value pair to be stored according to a position state of the key-value pairs stored in the target hash bucket recorded by the bitmap and / or a storage state of the remaining hash buckets other than the target hash bucket in a case where there is the hash conflict; a storing unit configured to store the key-value pair to be stored into the determined storage position, and record the position state of the key-value pair by using the bitmap.

12. A hash table search apparatus characterized by comprising: The hash table includes a plurality of hash buckets and a bitmap, and the query device includes: a positioning unit configured to locate a target hash bucket of a key-value pair to be queried in response to a query request for the key-value pair to be queried; a matching unit configured to match the key-value pair to be queried with the key-value pairs stored in the target hash bucket to obtain a matching result for the key-value pair to be queried; a querying unit configured to return a query success result if the matching result is a matching success, or determine a hash bucket with a backup key based on the position state of the key-value pairs in other hash buckets recorded by the bitmap, and match the key-value pair to be queried with the key-value pairs stored in the hash bucket with the backup key to return a query success result or a query failure result.

13. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to make the computer execute the processing method of the hash table according to any one of claims 1-9 or the query method of the hash table according to claim 10.

Images