A dynamic reconstruction method for key data structures

By using a global pointer to point to a node in danger, combined with the creation of a new hash table, dynamic reconstruction of the hash table is achieved. This solves the problem of temporary data loss and robustness during the reconstruction process, ensuring the continuity and integrity of data operations.

CN115098497BActive Publication Date: 2024-11-08NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210685649.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-16
Publication Date
2024-11-08
Estimated Expiration
2042-06-16

AI Technical Summary

Technical Problem

In existing technologies, the key parameters of hash tables cannot be dynamically adjusted, leading to robustness issues and temporary data loss, especially during the reconstruction process where data loss caused by concurrent operations cannot be effectively handled.

Method used

By pointing a global pointer to a node in danger and combining this with the method of creating a new hash table, dynamic reconstruction of the hash table is achieved, ensuring the accessibility and integrity of data during the reconstruction process.

Benefits of technology

It solves the problem of temporary data loss during the reconstruction process of hash tables, improves the robustness of hash tables, and ensures the effectiveness of data lookup, insertion, and deletion operations during reconstruction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115098497B_ABST
    Figure CN115098497B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for dynamic reconstruction of a key data structure. The method is applicable to a variety of data structures. Taking a hash table as an example, a new hash table is first created to store nodes in an old hash table; the nodes of each hash bucket in the old hash table are traversed, and for each node, a global pointer first points to the node, at which time the node enters a dangerous period, and then the node is deleted from the old hash table; according to the hash function of the new hash table, the position of the current node in the new hash table is determined, the current node is inserted into the new hash table, and the global pointer is synchronously set to NULL, and the current node ends the dangerous period state; the above operation is repeated until the number of nodes in the old hash table is 0, and the old hash table is deleted. The present invention reconstructs the hash table by dynamically changing the hash function to solve the robustness problem, and uses a global pointer to solve the problem of temporary data loss caused by concurrent insertion, deletion and query operations in the process of rebuilding the hash table.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of data processing, and in particular relates to a dynamic reconstruction method of a key data structure. Background Art

[0002] In existing computer systems, key parameters of data structures cannot be adjusted dynamically. For example, the hash function used in hash tables, the number of elements that each node in a B+ tree can accommodate, and other parameters. Once the program starts running, these key parameters are usually no longer adjustable. Existing adjustment methods are divided into two categories. (1) Lock-based dynamic adjustment method. When adjusting, first lock the data structure instance to be adjusted, and then transfer the data in the data structure instance to the new data structure instance. Correspondingly, the new data structure instance also needs to be locked. The use of locks will not only cause problems such as deadlock and livelock, but also reduce the performance of the data structure, such as increased latency or causing operations to be suspended. (2) Adjustment method of completely suspending the service. Specifically, when adjusting the key parameters of the data structure, first pause the operation being executed. Then, adjust the data structure. Finally, restart the previously suspended operation. This method will also cause unnecessary delays and reduce the performance of the data structure.

[0003] Take hash tables as an example. Most existing hash tables lack robustness. Even if the predefined hash function does not evenly distribute the incoming data to the hash table buckets, the hash function cannot be changed. This robustness problem affects the implementation of hash tables in a range of operating systems and programming languages, including the Linux kernel, PHP, and .NET. Researchers have proposed various techniques to address this robustness issue. The most studied solution is universal hashing, which theoretically provides good performance and solves the above robustness problem by randomly selecting a hash function from a set of carefully designed hash functions. Unfortunately, empirical evaluations have shown that universal hashing is not suitable for applications such as Perl, which is heavily used to process strings that are often selected from a restricted character set and therefore have unexpected distribution characteristics. In addition, new research shows that it is possible for an attacker to observe or guess the random selection of a predefined hash function, thereby rendering universal hashing useless.

[0004] The key to dynamically reconstructing a hash table is how to distribute (transfer) each node in the old hash table to the new hash table. In order to distribute each node, the reconstruction operation must update two hash tables. Insertion or deletion operations on a hash table can be atomic and non-blocking, but there is no non-blocking method to transfer a node from the old hash table to the new hash table, which will cause temporary data loss. For example, thread 1 first (1) deletes node A from the old hash table, and then (2) inserts node A into the new hash table. If there is a thread 2 that performs an operation to search for node A after thread 1 executes (1) and before executing (2), thread 2 will not be able to find node A in the old hash table and the new hash table. Only when thread 1 inserts node A into the new hash table can thread 2 find node A. This will cause a temporary loss of node A.

[0005] Therefore, there is an urgent need for a method of dynamically reconstructing key data structures to dynamically reconstruct the hash table to solve the robustness problem and the problem of temporary data loss. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a dynamic reconstruction method for key data structures, which solves the robustness problem by dynamically changing the hash function to rebuild the hash table, and uses a global pointer to solve the problem of temporary data loss caused by concurrent insertion, deletion and query operations during the reconstruction of the hash table.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] The embodiment of the present invention provides a method for dynamically reconstructing a key data structure, and the method comprises the following steps:

[0009] S1, create a new hash table to store the nodes in the old hash table;

[0010] S2, traverse the nodes of each hash bucket in the old hash table, for each node in each hash bucket, the global pointer points to the node, the node enters the danger period, and proceeds to step S3; the danger period refers to a period of time during the node distribution process when the specified node has been deleted from the old hash table but has not yet been stored in the new hash table;

[0011] S3, delete the node from the old hash table;

[0012] S4, according to the hash function of the new hash table, determine the position of the current node in the new hash table, insert the current node into the new hash table, and synchronously adjust the global pointer to NULL, so that the node ends the dangerous period state;

[0013] S5, repeat steps S2-S4 until the number of nodes in the old hash table is 0, and delete the old hash table.

[0014] Furthermore, when the reconstruction method is applied to a search operation, it includes the following specific steps:

[0015] S21, according to the key value of the node input externally, determine the key value of the node to be searched as key;

[0016] S22, determine the hash bucket where the node with the key value of key is located in the old hash table, traverse the hash bucket, and search for the node with the key value of key. If the node with the key value of key is found, return the pointer of the node and end the process; if the node with the key value of key is not found, go to step S23;

[0017] S23, determine whether there is a reconstruction operation currently being executed. If no reconstruction operation is being executed, return -ENOENT, indicating that there is no node with the key value of key in the hash table, and end the process; if a reconstruction operation is currently being executed, go to step S24;

[0018] S24, traverse the nodes in the danger period, search for the node with the key value key, if the node with the key value key is found, return the pointer of the node, and end the process; if the node with the key value key is not found, go to step S25;

[0019] S25, determine the hash bucket where the node with the key value of key is located in the new hash table, traverse the hash bucket, and search for the node with the key value of key. If the node with the key value of key is found, return the pointer of the node and end the process; if the node with the key value of key is not found, the search ends and -ENOENT is returned, indicating that there is no node with the key value of key in the hash table.

[0020] Furthermore, when the reconstruction method is applied to a deletion operation, it includes the following specific steps:

[0021] S26, determining the key value of the node to be deleted according to the key value of the node input externally;

[0022] S27, determine the hash bucket position of the node with the key value key in the old hash table, and try to delete the node with the key value key; if the deletion is successful, return SUCCESS and end the process; if the deletion is not successful, go to step S28;

[0023] S28, determine whether a reconstruction operation is being executed. If no reconstruction operation is being executed, the deletion ends and -ENOENT is returned, indicating that there is no node with the key value of key in the hash table and the deletion fails. If a reconstruction operation is being executed, proceed to step S29;

[0024] S29, traverse the nodes in the dangerous period, try to delete the node with the key value key, if the deletion is successful, return SUCCESS, and the process ends; if the deletion is not successful, go to step S30;

[0025] S30, determine the position of the hash bucket where the node with the key value of key is located in the new hash table, and try to delete the node with the key value of key. If the deletion is successful, SUCCESS is returned and the process ends; if the deletion is not successful, the deletion ends and -ENOENT is returned, indicating that there is no node with the key value of key in the hash table and the deletion fails.

[0026] Furthermore, when the reconstruction method is applied to an insertion operation, it includes the following steps:

[0027] S31, determining the key value of the node to be inserted according to the key value of the node input externally;

[0028] S32, determining whether a reconstruction operation is being performed, if no reconstruction operation is being performed, proceeding to step S33; if a reconstruction operation is being performed, proceeding to step S34;

[0029] S33: Determine the location of the hash bucket of the node with the key value key in the old hash table, and insert the node with the key value key; if the insertion is successful, return SUCCESS and the process ends; if the insertion fails, go to step S36;

[0030] S34, searching for a node with a key value of key in the old hash table and the nodes in danger; if a node with a key value of key is found, proceeding to step S36; if no node with a key value of key is found, proceeding to step S35;

[0031] S35, searching the new hash table for the location of the hash bucket where the node with the key value key is located, inserting the node with the key value key into the corresponding hash bucket, if the insertion is successful, returning SUCCESS and ending the process; if the insertion fails, proceeding to step S36;

[0032] S36, ends the insertion process and returns -EEXIST, indicating that there is a node with the key value key in the hash table and the insertion fails.

[0033] Beneficial effects:

[0034] First, the dynamic reconstruction method of key data structures proposed in the present invention uses a global pointer to point to the nodes in the danger period. When performing insertion, deletion and search operations, the nodes in the danger period can be accessed by accessing the global pointer, which solves the problem of temporary data loss.

[0035] Secondly, the dynamic reconstruction method of the key data structure proposed by the present invention realizes the dynamic reconstruction of the hash table by adding a hash table, thereby solving the robustness problem of the hash table.

[0036] Thirdly, the dynamic reconstruction method of the key data structure proposed by the present invention ensures that the search operation can find the node in the hash table regardless of whether a reconstruction operation is being executed, and will not cause temporary loss of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The figure is a flowchart of a reconstruction operation algorithm of a dynamic reconstruction method of a key data structure according to an embodiment of the present invention.

[0038] Figure 2 The figure is a flowchart of a search operation algorithm of a dynamic reconstruction method of a key data structure according to an embodiment of the present invention.

[0039] Figure 3 The figure is a flowchart of a deletion algorithm of a dynamic reconstruction method of a key data structure according to an embodiment of the present invention.

[0040] Figure 4 The figure is a flow chart of an insertion operation algorithm of a dynamic reconstruction method of a key data structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following examples will enable those skilled in the art to more fully understand the present invention, but are not intended to limit the present invention in any way.

[0042] Figure 1 1 is a schematic diagram of a reconstruction operation algorithm flow of a dynamic reconstruction method of a key data structure of an embodiment of the present invention. This embodiment proposes a dynamic reconstruction method of a key data structure, the method comprising the following steps:

[0043] S1, create a new hash table to store the nodes in the old hash table.

[0044] S2, traverse the nodes of each hash bucket in the old hash table. For each node in each hash bucket, the global pointer points to the node, and the node enters the danger period, and proceeds to step S3; the danger period refers to a period of time during the node distribution process when the specified node has been deleted from the old hash table but has not yet been stored in the new hash table.

[0045] S3, delete the node from the old hash table.

[0046] S4, according to the hash function of the new hash table, determine the position of the current node in the new hash table, insert the current node into the new hash table, and synchronously adjust the global pointer to NULL, so that the node ends the dangerous period state.

[0047] S5, repeat steps S2-S4 until the number of nodes in the old hash table is 0, and delete the old hash table.

[0048] 1. Reconstructing data structure

[0049] The specific steps for dynamically reconstructing the hash table are as follows:

[0050] Step 1: Create a new hash table to store the nodes in the old hash table.

[0051] Step 2: Traverse each bucket htbp in the old hash table, and for each htbp, traverse each node htnp in it, and the global pointer rebuild_cur points to htnp. At this time, the node htnp is in the hazard period.

[0052] Step 3: Delete the node htnp from the old hash table.

[0053] Step 4: According to the hash function of the new hash table, determine the position of the node htnp in the new hash table. Insert the node htnp into the new hash table.

[0054] Step 5: Set the global pointer rebuil_cur to NULL. htnp ends the hazard period state.

[0055] Step 6: Repeat steps 2-5 until the number of nodes in the old hash table is 0, and then delete the old hash table.

[0056] The danger period refers to a period of time during the node distribution process when the designated node is deleted from the old hash table and has not yet been stored in the new hash table.

[0057] 2. Perform a search operation

[0058] Figure 2 The following is a flow chart of the algorithm flow of the search operation of the dynamic reconstruction method of the key data structure of the embodiment of the present invention. The specific steps of performing the search operation are as follows:

[0059] Step 1: Based on the input, determine the key value of the node to be searched as key.

[0060] Step 2: Determine the bucket where the node with the key value key is located in the old hash table, traverse this bucket, and search for the node with the key value key. If a node with the key value key is found, return the pointer of the node; otherwise, go to step 3.

[0061] Step 3: Determine whether a reconstruction operation is currently being executed. If no reconstruction operation is being executed, return -ENOENT, that is, there is no node with the key value key in the hash table; otherwise, go to step 4.

[0062] Step 4: Traverse the nodes in the hazard period. If a node with the key value key is found, return the pointer of the node; otherwise, go to step 5.

[0063] Step 5: Determine the bucket where the node with the key value key is located in the new hash table, traverse this bucket, and search for the node with the key value key. If a node with the key value key is found, return the pointer of the node; otherwise, go to step 6.

[0064] Step 6: The search ends and -ENOENT is returned, which means there is no node with the key value key in the hash table.

[0065] For example, there is a node A in the old hash table. Taking searching for node A as an example, the specific search operation is as follows:

[0066] (1) When there is no rebuild operation, there is only one (old) hash table. Therefore, the search operation can definitely find node A in the old hash table.

[0067] (2) When there is a reconstruction operation, node A may exist in three locations, namely the old hash table, the hazard period, and the new hash table, as follows:

[0068] 1) When node A is in the old hash table, the lookup operation can find node A in the old hash table because the lookup operation first searches the old hash table.

[0069] 2) When node A is deleted from the old hash table and not inserted into the new hash table, node A is in the hazard period. The search operation first searches for node A in the old hash table. Since node A is in the hazard period, the search operation does not find node A in the old hash table. Next, the search operation traverses the nodes pointed to by rebuil_cur (i.e., the nodes in the hazard period). Therefore, the search operation will find node A in the nodes in the hazard period.

[0070] 3) When node A is in the new hash table, the search operation does not find node A in the old hash table or in the nodes in the hazard period. Then, the search operation searches for node A in the new hash table. Finally, the search operation finds node A.

[0071] According to the rebuild operation steps, before node A is deleted from the old hash table, rebuild_cur will point to node A. Then, node A is deleted from the old hash table and node A is in the hazard period. Subsequently, node A is inserted into the new hash table and rebuild_cur is set to NULL. Therefore, the search operation will definitely find node A in one of the three situations 1), 2), and 3).

[0072] 3. Execute the deletion operation

[0073] Figure 3 This is a schematic diagram of the deletion operation algorithm flow of the dynamic reconstruction method of the key data structure of an embodiment of the present invention. The specific steps for performing the deletion operation are as follows:

[0074] Step 1: Determine the key value of the node to be deleted.

[0075] Step 2: Determine the hash bucket position of the node with the key value key in the old hash table, and try to delete the node with the key value key. If the deletion is successful, return SUCCESS; otherwise, go to step 3.

[0076] Step 3: Check whether a rebuild operation is in progress. If no rebuild operation is in progress, return -ENOENT, which means that there is no node with the key value key in the hash table and the deletion fails; otherwise, go to step 4.

[0077] Step 4: Traverse the nodes in the hazard period. If the node with the key is successfully deleted, return SUCCESS; otherwise, go to step 5.

[0078] Step 5: Determine the hash bucket position of the node with the key value key in the new hash table, and try to delete the node with the key value key. If the deletion is successful, return SUCCESS; otherwise, go to step 6.

[0079] Step 6: Deletion is completed and -ENOENT is executed, that is, there is no node with the key value of key in the hash table, and the deletion fails.

[0080] 4. Perform insert operation

[0081] Figure 4 This is a schematic diagram of the insertion operation algorithm flow of the dynamic reconstruction method of the key data structure of an embodiment of the present invention. The specific steps of performing the insertion operation are as follows:

[0082] Step 1: Determine the key value of the node to be inserted.

[0083] Step 2: Determine whether a rebuild operation is being performed. If no rebuild operation is being performed, go to step 3; otherwise, go to step 4.

[0084] Step 3: Determine the location of the hash bucket of the node with the key value key in the old hash table and insert the node with the key value key. If the insertion is successful, return SUCCESS, otherwise, go to step 6.

[0085] Step 4: Search for a node with the key value key in the old hash table and the nodes in the hazard period. If a node with the key value key is found, go to step 6; otherwise, go to step 5.

[0086] Step 5: Determine the location of the hash bucket of the node with the key value key in the new hash table and insert the node with the key value key. If the insertion is successful, return SUCCESS, otherwise, go to step 6.

[0087] Step 6: Returns -EEXIST, which means that there is a node with the key value key in the hash table and the insertion fails.

[0088] The method of this embodiment is also applicable to other data structures, such as tree structures.

Claims

1. A method for dynamic reconstruction of a key data structure, characterized in that: The reconstruction method comprises the following steps: S1, create a new hash table to store the nodes in the old hash table; S2, traverse the nodes of each hash bucket in the old hash table, for each node in each hash bucket, the global pointer points to the node, the node enters the danger period, and proceeds to step S3; the danger period refers to a period of time during the node distribution process when the specified node has been deleted from the old hash table but has not yet been stored in the new hash table; S3, delete the node from the old hash table; S4, according to the hash function of the new hash table, determine the position of the current node in the new hash table, insert the current node into the new hash table, and synchronously adjust the global pointer to NULL, so that the node ends the dangerous period state; S5, repeat steps S2-S4 until the number of nodes in the old hash table is 0, and delete the old hash table.

2. The method for dynamically reconstructing a key data structure according to claim 1, characterized in that: When the reconstruction method is applied to a search operation, the following specific steps are included: S21, according to the key value of the node input externally, determine the key value of the node to be searched as key; S22, determine the hash bucket where the node with the key value of key is located in the old hash table, traverse the hash bucket, and search for the node with the key value of key. If the node with the key value of key is found, return the pointer of the node and end the process; if the node with the key value of key is not found, go to step S23; S23, determine whether there is a reconstruction operation currently being executed. If no reconstruction operation is being executed, return -ENOENT, indicating that there is no node with the key value of key in the hash table, and end the process; if a reconstruction operation is currently being executed, go to step S24; S24, traverse the nodes in the danger period, search for the node with the key value key, if the node with the key value key is found, return the pointer of the node, and end the process; if the node with the key value key is not found, go to step S25; S25, determine the hash bucket where the node with the key value of key is located in the new hash table, traverse the hash bucket, and search for the node with the key value of key. If the node with the key value of key is found, return the pointer of the node and end the process; if the node with the key value of key is not found, the search ends and -ENOENT is returned, indicating that there is no node with the key value of key in the hash table.

3. The method for dynamic reconstruction of a key data structure according to claim 1, characterized in that: When the reconstruction method is applied to a deletion operation, the specific steps include: S26, determining the key value of the node to be deleted according to the key value of the node input externally; S27, determine the hash bucket position of the node with the key value key in the old hash table, and try to delete the node with the key value key; if the deletion is successful, return SUCCESS and end the process; if the deletion is not successful, go to step S28; S28, determine whether a reconstruction operation is being executed. If no reconstruction operation is being executed, the deletion ends and -ENOENT is returned, indicating that there is no node with the key value of key in the hash table and the deletion fails. If a reconstruction operation is being executed, proceed to step S29; S29, traverse the nodes in the dangerous period, try to delete the node with the key value key, if the deletion is successful, return SUCCESS, and the process ends; if the deletion is not successful, go to step S30; S30, determine the position of the hash bucket where the node with the key value of key is located in the new hash table, and try to delete the node with the key value of key. If the deletion is successful, SUCCESS is returned and the process ends; if the deletion is not successful, the deletion ends and -ENOENT is returned, indicating that there is no node with the key value of key in the hash table and the deletion fails.

4. The method for dynamically reconstructing a key data structure according to claim 1, characterized in that: When the reconstruction method is applied to an insertion operation, the method comprises the following steps: S31, determining the key value of the node to be inserted according to the key value of the node input externally; S32, determining whether a reconstruction operation is being performed, if no reconstruction operation is being performed, proceeding to step S33; if a reconstruction operation is being performed, proceeding to step S34; S33: Determine the location of the hash bucket of the node with the key value key in the old hash table, and insert the node with the key value key; if the insertion is successful, return SUCCESS and the process ends; if the insertion fails, go to step S36; S34, searching for a node with a key value of key in the old hash table and the nodes in danger; if a node with a key value of key is found, proceeding to step S36; if no node with a key value of key is found, proceeding to step S35; S35, searching the new hash table for the location of the hash bucket where the node with the key value key is located, inserting the node with the key value key into the corresponding hash bucket, if the insertion is successful, returning SUCCESS and ending the process; if the insertion fails, proceeding to step S36; S36, ends the insertion process and returns -EEXIST, indicating that there is a node with the key value key in the hash table and the insertion fails.