Persistent memory dynamic hash index method, system, device and storage medium
By using preset data fingerprint insertion algorithm and splitting operations in persistent memory dynamic hash index, the problem of low efficiency of hash index writing operation in the existing technology is solved, the insertion performance and space utilization are improved, and the overall hashing is improved Index performance.
Patent Information
- Application Number
- CN202210001913.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-04
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-01-04
AI Technical Summary
The existing hash indexing scheme based on persistent memory accounts for only a small proportion of the write operations, but the overall index performance is not greatly improved, resulting in inefficiency.
Position the target data bucket in the dynamic hash structure through the preset data fingerprint insertion algorithm, insert the hash key into the target data bucket. If the target data bucket has no idle position, use the backup data bucket. After successful, the data fingerprint matching and refresh operations are performed; If the insertion fails, the local depth and the global depth are compared and the split operation is performed to optimize the hash index.
Improves the speed and efficiency of persistent memory dynamic hash indexes, improves insertion performance and space utilization, reduces unnecessary read and write overhead, and significantly improves the performance of persistent memory hash indexes.
Smart Images

Figure CN114385636B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of memory technology, and in particular to a persistent memory dynamic hash index method, system, device and storage medium. Background Art
[0002] In the past few decades, traditional memory technologies, including dynamic random access memory and static random access memory, have been used as main memory and cache in memory architectures. However, there are problems with high power consumption and limited scalability. Due to its advantages of high density, low power consumption and non-volatility, persistent memory (PM) is expected to become a storage medium to supplement or replace dynamic random access memory (DRAM) to build future storage systems.
[0003] The emergence of persistent memory has led to significant changes in memory architecture and its characteristics. Due to ignoring the properties of persistent memory, traditional data index structures are inefficient on it. For example, persistent memory has limitations in write durability. Compared with DRAM, PM has 3 times the read latency and similar write latency. In addition, the write operation of persistent memory not only consumes limited durability, but also has higher read and write latency than DRAM.
[0004] At present, hash indexing schemes based on persistent memory emerge in an endless stream. Existing hashing schemes can effectively adapt to PM. Since the write performance of persistent memory is far inferior to the read performance, the main focus of most research is to reduce PM writes in hash tables. However, in actual applications, the proportion of read operations is much higher than that of write operations. The general read operation of dynamic hashing is: when a key-value pair is inserted, the key prefix calculated by the hash function is first matched with the directory entry of the pointer array. When entering the data bucket, the traditional scalable hash directly uses a linear traversal method to find the first empty data slot to store the data. When searching, it also matches the corresponding directory entry based on the calculated key prefix, and then locates the data bucket through the pointer of the directory entry. Finally, the corresponding key-value pair is found by sequential traversal. The existing hash indexing scheme based on persistent memory seems to greatly alleviate the delay of writing data, but when the proportion of write operations is only a small part, the overall index performance is not greatly improved. Summary of the invention
[0005] The main purpose of the present invention is to provide a persistent memory dynamic hash index method, system, device and storage medium, aiming to solve the technical problem in the prior art that the hash index scheme based on persistent memory does not significantly improve the performance of the overall index when the write operation ratio accounts for only a small part.
[0006] In a first aspect, the present invention provides a persistent memory dynamic hash index method, the persistent memory dynamic hash index method comprising the following steps:
[0007] Locate a target data bucket in a dynamic hash structure according to a preset data fingerprint insertion algorithm, and insert a hash key into the target data bucket, wherein the dynamic hash structure includes a pointer array, a data bucket, and a segment;
[0008] When there is no free space in the target data bucket, the next logical data bucket is used as a spare data bucket, and the hash key is inserted into the spare data bucket;
[0009] When the insertion into the standby data bucket is successful, determine whether there is a matching data fingerprint in the standby data bucket. If there is a match, compare the key to be updated with the key value of the corresponding data slot that matches the data fingerprint, and perform a refresh operation according to the key comparison result;
[0010] When the insertion into the spare data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result.
[0011] Optionally, locating a target data bucket in a dynamic hash structure according to a preset data fingerprint insertion algorithm and inserting a hash key into the target data bucket includes:
[0012] Get the prefix and suffix of the pointer array in the dynamic hash structure;
[0013] Construct a preset data fingerprint insertion algorithm based on data fingerprint;
[0014] The index of the prefix and the suffix locates the target data bucket in the dynamic hash structure through a hash function according to the preset data fingerprint insertion algorithm, and inserts the hash key into the target data bucket.
[0015] Optionally, when the insertion into the standby data bucket is successful, determining whether there is a matching data fingerprint in the standby data bucket, and if there is a match, comparing the key to be updated with the key value of the corresponding data slot that matches the data fingerprint, and performing a refresh operation according to the key comparison result, including:
[0016] When the insertion into the spare data bucket is successful, it is determined whether there is a matching data fingerprint in the spare data bucket. If there is a match, the directory entry is located by the highest bit to find the target segment pointed to by the directory entry;
[0017] Determine the corresponding target data fingerprint according to the target segment, retrieve the target data fingerprint, compare the key to be updated in the target data fingerprint with the key value of the corresponding data slot matched by the data fingerprint, and generate a key comparison result;
[0018] A refresh operation is performed according to the key comparison result.
[0019] Optionally, performing a refresh operation according to the key comparison result includes:
[0020] When the key comparison result is that the key to be updated is the same as the key stored in each slot, update the value corresponding to the current key;
[0021] When the key comparison result is that the key to be updated is different from the key stored in each slot, and there is a free position in the current data bucket, the hash keys are inserted in sequence.
[0022] Optionally, when the insertion into the spare data bucket fails, comparing the local depth of the segment where the hash conflict occurs with the global depth, and performing a split operation according to the depth comparison result, includes:
[0023] When the insertion into the backup data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth to generate a depth comparison result;
[0024] When the depth comparison result is that the local depth is less than the global depth, additionally creating a new segment to migrate part of the data within the segment;
[0025] When the depth comparison result is that the local depth is equal to the global depth, directory expansion is performed and the current directory entry is switched.
[0026] Optionally, when the insertion into the spare data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and after the split operation is performed according to the depth comparison result, the persistent memory dynamic hash index method further includes:
[0027] When a delete instruction is received, the data in the current data bucket is deleted, the data in the logical next data bucket with the flag position being 1 is migrated to the deleted free position, and the flag position is set to 0.
[0028] Optionally, the directory entry of the persistent memory has one and only one pointer to a segment; the segment has several data buckets, the data bucket has several slots, and one slot stores one piece of data; the dynamic hash structure uses global depth and local depth to represent the quantitative relationship between the segment and the pointer pointing to the segment, wherein the global depth is used to determine the number of bits of the directory entry, and the local depth is used to indicate the length of the common hash key in the segment.
[0029] In a second aspect, to achieve the above-mentioned purpose, the present invention further proposes a persistent memory dynamic hash index system, the persistent memory dynamic hash index system comprising:
[0030] An insertion module, used to locate a target data bucket in a dynamic hash structure according to a preset data fingerprint insertion algorithm, and insert a hash key into the target data bucket, wherein the dynamic hash structure includes a pointer array, a data bucket, and a segment;
[0031] A spare module, used to use the next logical data bucket as a spare data bucket and insert the hash key into the spare data bucket when there is no free space in the target data bucket;
[0032] A refresh module is used to determine whether there is a matching data fingerprint in the standby data bucket when the insertion into the standby data bucket is successful. If there is a match, the key to be updated is compared with the key value of the corresponding data slot that matches the data fingerprint, and a refresh operation is performed according to the key comparison result;
[0033] The splitting module is used to compare the local depth of the segment where the hash conflict occurs with the global depth when the insertion into the spare data bucket fails, and perform a splitting operation according to the depth comparison result.
[0034] In the third aspect, to achieve the above-mentioned purpose, the present invention also proposes a persistent memory dynamic hash index device, which includes: a memory, a processor, and a persistent memory dynamic hash index program stored in the memory and executable on the processor, wherein the persistent memory dynamic hash index program is configured to implement the steps of the persistent memory dynamic hash index method as described above.
[0035] In a fourth aspect, in order to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a persistent memory dynamic hash index program is stored, and when the persistent memory dynamic hash index program is executed by a processor, the steps of the persistent memory dynamic hash index method described above are implemented.
[0036] The persistent memory dynamic hash indexing method proposed in the present invention locates the target data bucket in the dynamic hash structure through a preset data fingerprint insertion algorithm, and inserts the hash key into the target data bucket, wherein the dynamic hash structure includes a pointer array, a data bucket and a segment; when there is no free position in the target data bucket, the logical next data bucket is used as a spare data bucket, and the hash key is inserted into the spare data bucket; when the insertion into the spare data bucket is successful, it is determined whether there is a matching data fingerprint in the spare data bucket, and if there is a match, the key to be updated is compared with the key value of the corresponding data slot that matches the data fingerprint, and a refresh operation is performed according to the key comparison result; when the insertion into the spare data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result, which can improve the speed and efficiency of the persistent memory dynamic hash indexing, improve the insertion performance and space utilization, reduce unnecessary read and write overhead, and improve the performance of the persistent memory hash indexing. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention;
[0038] Figure 2 This is a flow chart of the first embodiment of the persistent memory dynamic hash indexing method of the present invention;
[0039] Figure 3 This is a flow chart of the second embodiment of the persistent memory dynamic hash indexing method of the present invention;
[0040] Figure 4 It is a flowchart of the third embodiment of the persistent memory dynamic hash index method of the present invention;
[0041] Figure 5 This is a flow chart of a fourth embodiment of a persistent memory dynamic hash indexing method according to the present invention;
[0042] Figure 6 This is a flowchart of a fifth embodiment of the persistent memory dynamic hash indexing method of the present invention;
[0043] Figure 7 A schematic diagram of the logical structure of the dynamic hash structure of the persistent memory dynamic hash indexing method of the present invention;
[0044] Figure 8 A schematic diagram of the internal structure of a data bucket of a dynamic hash structure of a persistent memory dynamic hash indexing method of the present invention;
[0045] Fig. 9 A schematic diagram of the indexing operation of the dynamic hash structure of the persistent memory dynamic hash indexing method of the present invention;
[0046] Fig.10 A schematic diagram of pointer array expansion of a dynamic hash structure of a persistent memory dynamic hash indexing method of the present invention;
[0047] Fig.11 This is a functional module diagram of the first embodiment of the persistent memory dynamic hash index system of the present invention.
[0048] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0050] The solution of the embodiment of the present invention is mainly: locating the target data bucket in the dynamic hash structure through a preset data fingerprint insertion algorithm, inserting the hash key into the target data bucket, and the dynamic hash structure includes a pointer array, a data bucket and a segment; when there is no free position in the target data bucket, the logical next data bucket is used as a spare data bucket, and the hash key is inserted into the spare data bucket; when the insertion into the spare data bucket is successful, it is determined whether there is a matching data fingerprint in the spare data bucket, if it is matched, the key to be updated and the key value of the corresponding data slot matched by the data fingerprint are compared, and a refresh operation is performed according to the key comparison result; when the insertion into the spare data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result, which can improve the speed and efficiency of the dynamic hash index of the persistent memory, improve the insertion performance and space utilization, reduce unnecessary read and write overhead, and improve the performance of the persistent memory hash index, and solve the technical problem that the hash index scheme based on persistent memory in the prior art does not improve the performance of the overall index much when the write operation ratio accounts for only a small part.
[0051] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.
[0052] like Figure 1As shown, the device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory, or it may be a stable memory (Non-Volatile Memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.
[0053] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0054] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a persistent memory dynamic hash index program.
[0055] The device of the present invention calls the persistent memory dynamic hash index program stored in the memory 1005 through the processor 1001, and performs the following operations:
[0056] Locate a target data bucket in a dynamic hash structure according to a preset data fingerprint insertion algorithm, and insert a hash key into the target data bucket, wherein the dynamic hash structure includes a pointer array, a data bucket, and a segment;
[0057] When there is no free space in the target data bucket, the next logical data bucket is used as a spare data bucket, and the hash key is inserted into the spare data bucket;
[0058] When the insertion into the standby data bucket is successful, determine whether there is a matching data fingerprint in the standby data bucket. If there is a match, compare the key to be updated with the key value of the corresponding data slot that matches the data fingerprint, and perform a refresh operation according to the key comparison result;
[0059] When the insertion into the spare data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result.
[0060] The device of the present invention calls the persistent memory dynamic hash index program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0061] Get the prefix and suffix of the pointer array in the dynamic hash structure;
[0062] A preset data fingerprint insertion algorithm is constructed based on the data fingerprint, and the index of the prefix and the suffix is located through a hash function in the dynamic hash structure according to the preset data fingerprint insertion algorithm, and the hash key is inserted into the target data bucket.
[0063] The device of the present invention calls the persistent memory dynamic hash index program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0064] When the insertion into the spare data bucket is successful, it is determined whether there is a matching data fingerprint in the spare data bucket. If there is a match, the directory entry is located by the highest bit to find the target segment pointed to by the directory entry;
[0065] Determine the corresponding target data fingerprint according to the target segment, retrieve the target data fingerprint, compare the key to be updated in the target data fingerprint with the key value of the corresponding data slot matched by the data fingerprint, and generate a key comparison result;
[0066] A refresh operation is performed according to the key comparison result.
[0067] The device of the present invention calls the persistent memory dynamic hash index program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0068] When the key comparison result is that the key to be updated is the same as the key stored in each slot, update the value corresponding to the current key;
[0069] When the key comparison result is that the key to be updated is different from the key stored in each slot, and there is a free position in the current data bucket, the hash keys are inserted in sequence.
[0070] The device of the present invention calls the persistent memory dynamic hash index program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0071] When the insertion into the backup data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth to generate a depth comparison result;
[0072] When the depth comparison result is that the local depth is less than the global depth, additionally creating a new segment to migrate part of the data within the segment;
[0073] When the depth comparison result is that the local depth is equal to the global depth, directory expansion is performed and the current directory entry is switched.
[0074] The device of the present invention calls the persistent memory dynamic hash index program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0075] When a delete instruction is received, the data in the current data bucket is deleted, the data in the logical next data bucket with the flag position being 1 is migrated to the deleted free position, and the flag position is set to 0.
[0076] The device of the present invention calls the persistent memory dynamic hash index program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0077] The directory entry of the persistent memory has one and only one pointer to a certain segment; the segment has several data buckets, the data bucket has several slots, and one slot stores one piece of data. The dynamic hash structure uses global depth and local depth to represent the quantitative relationship between the segment and the pointer pointing to the segment, wherein the global depth is used to determine the number of bits of the directory entry, and the local depth is used to indicate the length of the common hash key in the segment.
[0078] Through the above scheme, this embodiment locates the target data bucket in the dynamic hash structure through a preset data fingerprint insertion algorithm, and inserts the hash key into the target data bucket, and the dynamic hash structure includes a pointer array, a data bucket and a segment; when there is no free position in the target data bucket, the logical next data bucket is used as a spare data bucket, and the hash key is inserted into the spare data bucket; when the insertion into the spare data bucket is successful, it is determined whether there is a matching data fingerprint in the spare data bucket. If there is a match, the key to be updated and the key value of the corresponding data slot matching the data fingerprint are compared, and a refresh operation is performed according to the key comparison result; when the insertion into the spare data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result, which can improve the speed and efficiency of the persistent memory dynamic hash index, improve the insertion performance and space utilization, reduce unnecessary read and write overhead, and improve the performance of the persistent memory hash index.
[0079] Based on the above hardware structure, an embodiment of the persistent memory dynamic hash index method of the present invention is proposed.
[0080] Reference Figure 2 , Figure 2 This is a flow chart of the first embodiment of the persistent memory dynamic hash indexing method of the present invention.
[0081] In a first embodiment, the persistent memory dynamic hash index method comprises the following steps:
[0082] Step S10: locate the target data bucket in the dynamic hash structure according to a preset data fingerprint insertion algorithm, and insert the hash key into the target data bucket. The dynamic hash structure includes a pointer array, a data bucket and a segment.
[0083] It should be noted that the preset data fingerprint insertion algorithm is a pre-set algorithm based on data fingerprints. The dynamic hash structure includes a pointer array, a data bucket and a segment. The segment is an intermediate structure used to reduce the length of sequential search. According to the preset data fingerprint insertion algorithm, the target data bucket in the dynamic hash structure can be located, and the hash key can be inserted into the target data bucket, that is, the data bucket where the hash key is inserted is located by using the insertion algorithm based on data fingerprints, and the data bucket is selected for insertion first.
[0084] Step S20: When there is no free space in the target data bucket, use the next logical data bucket as a spare data bucket, and insert the hash key into the spare data bucket.
[0085] It can be understood that when there is no free space in the target data bucket, that is, when the target data bucket is full, the next logical data bucket can be selected as a backup data bucket to store data. Related operations include: insert operation, refresh operation, split operation and delete operation. The insert operation is to insert the hash key into the backup data bucket.
[0086] Step S30: When the insertion into the spare data bucket is successful, determine whether there is a matching data fingerprint in the spare data bucket. If there is a match, compare the key to be updated with the key value of the corresponding data slot that matches the data fingerprint, and perform a refresh operation based on the key comparison result.
[0087] It should be understood that when the insertion into the backup data bucket is successful, the data fingerprints are compared first. If they match, the key values are further compared to see if they are equal, that is, to determine whether there is a matching data fingerprint in the backup data bucket. If they match, the key to be updated can be compared with the key stored in each slot to generate a key comparison result for corresponding update, that is, a refresh operation is performed.
[0088] It can be understood that the refresh operation generally adopts in-situ refresh, that is, the data fingerprint of the key of the data to be refreshed is compared with the data fingerprint of the specified data bucket. If there is data with a matching data fingerprint, the key value is further compared. If the key values are consistent, it means that the data needs to be refreshed, and the old value is overwritten with the new value.
[0089] Step S40: When the insertion into the standby data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result.
[0090] It should be understood that when the insertion into the spare data bucket fails, a split operation may be performed, that is, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result.
[0091] It should be noted that the directory entry of the persistent memory has and only has one pointer to a certain segment; there are several data buckets in the segment, and there are several slots in the data bucket, one slot stores a piece of data, and the dynamic hash structure uses global depth and local depth to represent the quantitative relationship between the segment and the pointer pointing to the segment, wherein the global depth is used to determine the number of bits of the directory entry, and the local depth is used to indicate the length of the common hash key in the segment; the pointer array and the data bucket and an intermediate structure, called a segment, are used to reduce the length of the sequential search; a directory entry has and only has one pointer to a certain segment; there are multiple data buckets in the segment, and there are several slots in the bucket; one slot stores a piece of data; to solve the insertion and search problem in the three-layer structure, each data bucket has a serial number for matching the key suffix calculated by the hash function; in this way, when inserting a key-value pair, the segment to be inserted is first determined by the prefix, and then the suffix is matched to determine the last data bucket to be stored; like the traditional extensible hash, the index also uses global depth and local depth to represent the quantitative relationship between the segment and the pointer pointing to the segment; wherein, the global depth (Global Depth)G represents the number of bits used to determine the directory entry; therefore, it determines the maximum number of segments, that is, the pointer array has 2G directory entries; Local Depth (Local Depth) L indicates the length of the common hash key in the segment;.
[0092] Through the above scheme, this embodiment locates the target data bucket in the dynamic hash structure through a preset data fingerprint insertion algorithm, and inserts the hash key into the target data bucket, and the dynamic hash structure includes a pointer array, a data bucket and a segment; when there is no free position in the target data bucket, the logical next data bucket is used as a spare data bucket, and the hash key is inserted into the spare data bucket; when the insertion into the spare data bucket is successful, it is determined whether there is a matching data fingerprint in the spare data bucket. If there is a match, the key to be updated and the key value of the corresponding data slot matching the data fingerprint are compared, and a refresh operation is performed according to the key comparison result; when the insertion into the spare data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result, which can improve the speed and efficiency of the persistent memory dynamic hash index, improve the insertion performance and space utilization, reduce unnecessary read and write overhead, and improve the performance of the persistent memory hash index.
[0093] Further, Figure 3 FIG. 1 is a flow chart of a second embodiment of a persistent memory dynamic hash indexing method according to the present invention. Figure 3As shown, a second embodiment of the persistent memory dynamic hash index method of the present invention is proposed based on the first embodiment. In this embodiment, step S10 specifically includes the following steps:
[0094] Step S11, obtaining the prefix and suffix of the pointer array in the dynamic hash structure.
[0095] It should be noted that the prefix of the pointer array in the dynamic hash structure is the directory entry, each directory entry in the pointer array represents a prefix that can match the key calculated by the hash function, and the pointer points to a certain segment, and the suffix is the serial number corresponding to the data bucket, and each data bucket has a serial number for matching the key suffix calculated by the hash function.
[0096] Step S12: construct a preset data fingerprint insertion algorithm based on the data fingerprint, locate the target data bucket in the dynamic hash structure through a hash function according to the preset data fingerprint insertion algorithm, and insert the hash key into the target data bucket.
[0097] It can be understood that a corresponding insertion algorithm is constructed based on the data fingerprint, that is, a preset data fingerprint insertion algorithm. The index of the prefix and the suffix can use the insertion algorithm to locate the target data bucket for hash key insertion, and then insert the hash key into the target data bucket; the data fingerprint is a one-byte hash of the hash value, which acts as a filter by first scanning them during the search process to avoid detecting data that does not match the fingerprint. The data bucket index uses the data fingerprint to first determine whether the data to be found is stored in the bucket. If the data already exists, it is automatically updated. If not, the data is inserted into the data bucket. Therefore, setting an algorithm based on data fingerprint can pre-determine whether the data exists, which can reduce unnecessary reading and speed up operation performance.
[0098] This embodiment adopts the above scheme, by obtaining the prefix and suffix of the pointer array in the dynamic hash structure; constructing a preset data fingerprint insertion algorithm based on the data fingerprint, locating the target data bucket in the dynamic hash structure through a hash function according to the preset data fingerprint insertion algorithm for the index of the prefix and the suffix, and inserting the hash key into the target data bucket; the hash key can be quickly inserted into the target data bucket, which can improve the speed and efficiency of the persistent memory dynamic hash index.
[0099] Further, Figure 4 FIG. 1 is a flow chart of a third embodiment of a method for persistent memory dynamic hash indexing according to the present invention. Figure 4 As shown, a third embodiment of the persistent memory dynamic hash index method of the present invention is proposed based on the first embodiment. In this embodiment, step S30 specifically includes the following steps:
[0100] Step S31, when the insertion into the spare data bucket is successful, determine whether there is a matching data fingerprint in the spare data bucket, if it matches, locate the directory entry through the highest bit, and find the target segment pointed to by the directory entry.
[0101] It should be noted that when the insertion into the spare data bucket is successful, it is determined whether there is a matching data fingerprint in the spare data bucket. If it matches, the directory entry can be located through the highest bit to find the segment pointed to by the directory entry, that is, the target segment.
[0102] Step S32: determine the corresponding target data fingerprint according to the target segment, retrieve the target data fingerprint, compare the key to be updated in the target data fingerprint with the key value of the corresponding data slot matched by the data fingerprint, and generate a key comparison result.
[0103] It can be understood that the corresponding target data fingerprint can be determined based on the target segment, and the target data fingerprint can be retrieved. By comparing the key to be updated in the target data fingerprint with the key value of the corresponding data slot matching the data fingerprint, a key comparison result can be generated.
[0104] In specific implementations, many dynamic hash structures based on persistent memory often use new data to overwrite invalid data after the hash table is expanded in order to reduce read and write overhead. However, this will cause the same key-value pairs to appear in different data buckets at the same time, wasting space and likely to cause data reading errors. In-place refresh can remove duplicate key-value pairs, improve space utilization, and not introduce excessive read and write overhead.
[0105] Step S33: performing a refresh operation according to the key comparison result.
[0106] It should be understood that a refresh operation may be performed based on the key comparison result.
[0107] Furthermore, the step S33 specifically includes the following steps:
[0108] When the key comparison result is that the key to be updated is the same as the key stored in each slot, update the value corresponding to the current key;
[0109] When the key comparison result is that the key to be updated is different from the key stored in each slot, and there is a free position in the current data bucket, the hash keys are inserted in sequence.
[0110] It can be understood that if there is a key that is the same as the inserted key, the data fingerprint must be matched, which means that it is the data that needs to be refreshed, then the value corresponding to the current key is directly updated, and the old value is overwritten with the new value; if there is no key that is the same as the inserted key, if there are free positions in the data bucket, the hash keys are inserted sequentially, otherwise the insertion fails and the split operation is performed.
[0111] Through the above scheme, this embodiment determines whether there is a matching data fingerprint in the spare data bucket when the insertion into the spare data bucket is successful. If there is a match, the directory entry is located through the highest bit to find the target segment pointed to by the directory entry; the corresponding target data fingerprint is determined according to the target segment, the target data fingerprint is retrieved, and the key to be updated in the target data fingerprint is compared with the key value of the corresponding data slot matched by the data fingerprint to generate a key comparison result; a refresh operation is performed according to the key comparison result, which can improve the speed and efficiency of the dynamic hash index of the persistent memory, and improve the insertion performance and space utilization.
[0112] Further, Figure 5 FIG. 4 is a flow chart of a fourth embodiment of a method for persistent memory dynamic hash indexing according to the present invention. Figure 5 As shown, a fourth embodiment of the persistent memory dynamic hash index method of the present invention is proposed based on the first embodiment. In this embodiment, step S40 specifically includes the following steps:
[0113] Step S41: When the insertion into the standby data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth to generate a depth comparison result.
[0114] It should be noted that the global depth and local depth are used to represent the quantitative relationship between the segment and the pointer pointing to the segment. When the insertion of the backup data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth to obtain a depth comparison result corresponding to the comparison size.
[0115] In the specific implementation, the global depth G represents the number of bits used to determine the directory entry, that is, the pointer array has 2 G directory entries, the local depth L indicates the length of the common hash key in the segment, k represents the pointer to the data bucket, if the segment is pointed to by k directory entries, the local depth of the segment is L is used because after dynamic hash expansion, not necessarily one directory entry corresponds to one segment. Since the number of directory entries after expansion is greater than the original number of segments, a new segment is created only when the key-value pair needs to be moved to a newly created data bucket.
[0116] Step S42: When the depth comparison result is that the local depth is less than the global depth, additionally create a new segment to migrate part of the data within the segment.
[0117] It can be understood that when the depth comparison result is that the local depth is less than the global depth, that is, the local depth L of the segment where the hash conflict occurs is less than the global depth G, it is only necessary to create a new segment to migrate part of the data within the segment without expanding the directory (that is, the pointer array).
[0118] Step S43: When the depth comparison result is that the local depth is equal to the global depth, perform directory expansion and switch the current directory entry.
[0119] It should be understood that if the local depth L of the segment where the hash conflict occurs is equal to the global depth G, directory expansion and directory entry pointer switching are required.
[0120] In the specific implementation, the split operation includes two cases: when the local depth of the conflicting segment is less than the global depth, only a new segment is created, and a redundant directory entry pointer pointing to the original segment is changed to point to the new segment. The valid key-value pairs corresponding to the directory entry pointer are also transferred to the new segment, and the key-value pairs in the original segment are deleted; when the local depth of the conflicting segment is equal to the global depth, the pointer array is doubled, and the directory entry pointer also needs to point to the new segment again.
[0121] This embodiment adopts the above scheme, by comparing the local depth of the segment where the hash conflict occurs with the global depth when the insertion of the backup data bucket fails, to generate a depth comparison result; when the depth comparison result is that the local depth is less than the global depth, an additional new segment is created to migrate part of the data within the segment; when the depth comparison result is that the local depth is equal to the global depth, the directory is expanded and the current directory entry is switched, so that different split operations can be quickly determined according to different depth comparison results, thereby improving insertion performance and space utilization, reducing unnecessary read and write overhead, and improving the performance of persistent memory hash indexes.
[0122] Further, Figure 6 FIG. 5 is a flow chart of a fifth embodiment of a method for persistent memory dynamic hash indexing according to the present invention. Figure 6 As shown, a fifth embodiment of the persistent memory dynamic hash index method of the present invention is proposed based on the first embodiment. In this embodiment, after step S40, the persistent memory dynamic hash index method further includes the following steps:
[0123] Step S50: upon receiving a delete instruction, delete the data in the current data bucket, migrate the data in the logical next data bucket whose flag position is 1 to a deleted free position, and set the flag position to 0.
[0124] It should be noted that when a deletion instruction is received, the token flag technology can be used to perform the deletion operation, that is, to delete the data in the current data bucket, migrate the data with the flag position of 1 in the logical next data bucket to the deleted free position, and set the flag position to 0, thereby reducing the average linear detection length for searching the data in the bucket.
[0125] In the specific implementation, the dynamic hash structure based on persistent memory PM usually adopts lazy deletion to reduce the additional write overhead: whenever a new segment needs to be created and some key-value pairs of the original segment are migrated to the new segment, the key-value pairs of the migrated original data bucket are not deleted immediately. Instead, when new data is inserted into the original data bucket, the migrated key-value pairs are directly replaced. This can reduce the overhead of hash table expansion and thus improve performance. However, in fact, not deleting invalid data immediately will reduce space utilization and increase the frequency of hash expansion, which will increase system overhead overall.
[0126] Further, it is scalable to implement lock-free operation based on the dynamic hash index of the persistent memory of the present invention, that is:
[0127] A lock flag lock and version number v are attached to each data bucket, and the lock is acquired and released using the Compare And Set (CAS) mechanism. The CAS mechanism has three operands, namely the memory value value, the old expected value expect, and the value to be modified now. If and only if value is equal to expect, value is modified to now and true is returned. If it fails, false is returned. The CAS instruction is atomic, and the comparison and modification operations are not separated from each other. They can only be completed at the same time or not at the same time. Based on this, when HyDH applies for a lock, it only needs to change the lock flag from 0 to 1 through the CAS instruction, and when releasing the lock, it changes the lock flag from 1 to 0 through the CAS instruction. Therefore, only one thread will succeed in the CAS modification operation and return true, and other failed threads will keep retrying until true is returned. The version number v is the field information set in optimistic concurrency, which stipulates that as long as the data is modified, the version number must be increased by 1 at the same time. In this way, the query thread only needs to compare the version number information before and after the query to know whether other threads have made modifications during the query process. If the version numbers are consistent, it will return, otherwise it will retry.
[0128] It can be understood that the read-optimized dynamic hash index structure based on persistent memory of the present invention is different from the general dynamic hash index in that, during the secondary indexing process, the index structure uses an algorithm based on data fingerprints, so that for non-existent data, there is no need to traverse the entire data bucket like a general hash index, but the operation can be terminated immediately and an error can be returned; the index also provides lock-free query, using the CAS mechanism to reduce the additional overhead caused by the previous pessimistic lock; in addition, the in-place refresh and deletion operations of the index can facilitate operations such as search, so that at the same time, this operation can also reduce the split operation of the dynamic hash index structure compared to lazy deletion, thereby improving insertion performance.
[0129] See also Figure 7 , Figure 7 The following is a schematic diagram of the logical structure of the dynamic hash structure of the persistent memory dynamic hash index method of the present invention. Figure 7 As shown in the figure, the index adopts an intermediate structure called segment in structure compared with the traditional scalable hash. A segment consists of N data buckets. In order to solve the insertion and search problems in the three-layer structure, the hash prefix index and suffix index are introduced. The prefix index is implemented by the global depth G and the local depth L. The suffix index is used in the insertion algorithm to locate which data bucket the hash key is inserted into.
[0130] Accordingly, the data bucket structure example can refer to Figure 8 , Figure 8 This is a schematic diagram of the internal structure of a data bucket of a dynamic hash structure of the persistent memory dynamic hash index method of the present invention. The data bucket consists of metadata and key-value pairs, wherein the metadata consists of a version lock, a token, and a fingerprint.
[0131] See also Fig. 9 , Fig. 9 Schematic diagram of the index operation of the dynamic hash structure of the persistent memory dynamic hash index method of the present invention, as shown in Fig. 9 As shown, assuming that the given hash key is 01000010...00000001, the global depth G (GlobalDepth) of HyDH is 2 at the beginning, so the first two bits of the highest bit are used as the segment index, the least significant byte is used as the Bucket index, and L represents the local depth (Local Depth). k represents the number of pointers pointing to the data bucket. The local depth L is used because after the extensible hash expansion, not necessarily one directory entry corresponds to one segment. Since the number of directory entries after the expansion is greater than the original number of segments, a new segment will be created only when the key-value pair needs to be moved to the newly created data bucket.
[0132] For further information, see Fig.10 , Fig.10 Schematic diagram of pointer array expansion of dynamic hash structure of persistent memory dynamic hash indexing method of the present invention, as shown in Fig.10 As shown, the directory doubling operation: expansion according to MostSignificantBits, where white represents the directory entries before expansion and gray represents the newly added directory entries after expansion.
[0133] Insert operation: When the key 01000010...00000001 is inserted, first locate the 01 directory item through the Most Significant Bits, that is, the prefix index, find the segment pointed to by the 01 directory item pointer, and then find the insertable data bucket according to the hash function; first traverse the data fingerprints of the two insertable data buckets. If there is no matching data fingerprint and the data bucket is not full, the key is inserted into the first free slot in sequence and the data fingerprint matching the key value at the corresponding position is added; if the data bucket is full, a split operation is performed; in addition, data insertion involves data writing, so in order to ensure that the system crashes and data is inconsistent during insertion; the structure also introduces the token flag technology. If the bucket into which the data is inserted is the logical first bucket, the token is recorded as 0, and if it is inserted into the logical second bucket, the token is set to 1; if the system loses power unexpectedly before the token is persisted, some data has been successfully persisted at this time, because the token is not persisted to non-volatile storage (Non-Volatile In the NVM, the bit corresponding to the token is not modified, and the system considers the previously written data to be invalid, so it will repeat the previous insertion operation. Another possible situation is that the system crashes when writing the token, but there will still be no data consistency problem at this time. Because the size of the token is 1B, not more than 8B, and the size of the CPU atomic write is generally 8B, the write is atomic when the data volume does not exceed 8B, and there will be no partial write. There are only two situations: all 8B are successfully written or not written at all. Therefore, the token writing link will not cause crash consistency problems. Summarizing the above two situations, it can be found that the consistency of data is well guaranteed during the insertion process. At the same time, the lock-free operation will also work before and after the insertion operation: read the version number information v1 once before traversing the data bucket, which is used for version number matching later; after the operation is completed, read the version number information v2 again and match it with the previously read v1. If the version number is consistent, it means that no other thread has modified the data in the bucket during the reading process, and the query result is returned immediately; if the version number changes, it means that other threads may have modified the read data during the reading process, and a retry is required for data accuracy.
[0134] Refresh operation: For the refresh operation, the data fingerprints in the insertable data bucket will be compared during insertion. If a matching data fingerprint already exists, the value corresponding to the key in the slot is directly updated. That is, when a matching data fingerprint exists, the key value needs to be further matched. When the key value to be refreshed is consistent with the compared key value, the value is directly updated. In addition, version number flag and token flag technology are also used to ensure data consistency.
[0135] Deletion operation: When deleting a key-value, the operation of locating the data bucket is the same as the bucket insertion algorithm. After finding the data bucket, traverse the data fingerprint. If there is no matching data fingerprint, it means that the data does not exist and the function returns immediately; if there is a matching data fingerprint, just set the data fingerprint to null; when the operation is completed, add 1 to the version number of the bucket. Adding 1 to the version number indicates that the data in the bucket has been modified, and the modification can be perceived by the query thread through the version number; in addition, the version number flag and token flag technology are also used to ensure data consistency; when the data in the bucket is deleted, the data with token set to 1 in the logical next bucket can be migrated to the free position of the deleted data; in this way, the average linear detection length for searching the data in the bucket can be reduced.
[0136] Split operation: There are two types of split operations: 1. When the local depth L of the segment where the hash conflict occurs is less than the global depth G, it means that there are multiple directory entry pointers pointing to this segment at the same time; therefore, only a new segment needs to be created to migrate part of the data within the segment without expanding the directory (pointer array); 2. When the local depth L of the segment where the hash conflict occurs is equal to the global depth G, it means that there is only one pointer pointing to the segment, then it is necessary to expand the directory and switch the directory entry pointer; the global depth G and the local depth L of each segment must also change accordingly (the prefix index of the global depth is increased by one bit, and the local depth is modified according to the global depth and the number of pointers).
[0137] Through the above scheme, this embodiment deletes the data in the current data bucket when a deletion instruction is received, migrates the data with a flag position of 1 in the logical next data bucket to the deleted free position, and sets the flag position to 0, thereby reducing the average linear detection length of searching the bucket data, and further improving the performance of the persistent memory hash index.
[0138] Accordingly, the present invention further provides a persistent memory dynamic hash index system.
[0139] Reference Fig.11 , Fig.11 This is a functional module diagram of the first embodiment of the persistent memory dynamic hash index system of the present invention.
[0140] In a first embodiment of the persistent memory dynamic hash index system of the present invention, the persistent memory dynamic hash index system includes:
[0141] The insertion module 10 is used to locate the target data bucket in the dynamic hash structure according to a preset data fingerprint insertion algorithm, and insert the hash key into the target data bucket. The dynamic hash structure includes a pointer array, a data bucket and a segment.
[0142] The backup module 20 is used to use the next logical data bucket as a backup data bucket and insert the hash key into the backup data bucket when there is no free space in the target data bucket.
[0143] The refresh module 30 is used to determine whether there is a matching data fingerprint in the spare data bucket when the insertion into the spare data bucket is successful. If there is a match, the key to be updated is compared with the key value of the corresponding data slot matching the data fingerprint, and a refresh operation is performed according to the key comparison result.
[0144] The splitting module 40 is used to compare the local depth of the segment where the hash conflict occurs with the global depth when the insertion into the spare data bucket fails, and perform a splitting operation according to the depth comparison result.
[0145] The insertion module 10 is also used to obtain the prefix and suffix of the pointer array in the dynamic hash structure; construct a preset data fingerprint insertion algorithm based on the data fingerprint, locate the target data bucket in the dynamic hash structure through a hash function according to the preset data fingerprint insertion algorithm for the index of the prefix and the suffix, and insert the hash key into the target data bucket.
[0146] The refresh module 30 is also used to determine whether there is a matching data fingerprint in the spare data bucket when the insertion into the spare data bucket is successful. If there is a match, the directory entry is located through the highest bit to find the target segment pointed to by the directory entry; the corresponding target data fingerprint is determined according to the target segment, the target data fingerprint is retrieved, and the key to be updated in the target data fingerprint is compared with the key value of the corresponding data slot that matches the data fingerprint to generate a key comparison result; and a refresh operation is performed according to the key comparison result.
[0147] The refresh module 30 is also used to update the value corresponding to the current key when the key comparison result is that the key to be updated and the key stored in each slot are the same; when the key comparison result is that the key to be updated and the key stored in each slot are different, and there is free space in the current data bucket, the hash key is inserted in sequence.
[0148] The split module 40 is also used to compare the local depth of the segment where the hash conflict occurs with the global depth when the insertion of the spare data bucket fails, and generate a depth comparison result; when the depth comparison result is that the local depth is less than the global depth, create an additional new segment to migrate part of the data within the segment; when the depth comparison result is that the local depth is equal to the global depth, expand the directory and switch the current directory entry.
[0149] Furthermore, it also includes a deletion module, which is used to delete the data in the current data bucket when receiving a deletion instruction, migrate the data with a mark position of 1 in the logical next data bucket to the deleted free position, and set the mark position to 0.
[0150] Among them, the steps implemented by each functional module of the persistent memory dynamic hash index system can refer to the various embodiments of the persistent memory dynamic hash index method of the present invention, and will not be repeated here.
[0151] In addition, an embodiment of the present invention further provides a storage medium, on which a persistent memory dynamic hash index program is stored. When the persistent memory dynamic hash index program is executed by a processor, the following operations are implemented:
[0152] Locate a target data bucket in a dynamic hash structure according to a preset data fingerprint insertion algorithm, and insert a hash key into the target data bucket, wherein the dynamic hash structure includes a pointer array, a data bucket, and a segment;
[0153] When there is no free space in the target data bucket, the next logical data bucket is used as a spare data bucket, and the hash key is inserted into the spare data bucket;
[0154] When the insertion into the standby data bucket is successful, determine whether there is a matching data fingerprint in the standby data bucket. If there is a match, compare the key to be updated with the key value of the corresponding data slot that matches the data fingerprint, and perform a refresh operation according to the key comparison result;
[0155] When the insertion into the spare data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result.
[0156] Furthermore, when the persistent memory dynamic hash index program is executed by the processor, the following operations are also implemented:
[0157] Get the prefix and suffix of the pointer array in the dynamic hash structure;
[0158] A preset data fingerprint insertion algorithm is constructed based on the data fingerprint, and the index of the prefix and the suffix is located through a hash function in the dynamic hash structure according to the preset data fingerprint insertion algorithm, and the hash key is inserted into the target data bucket.
[0159] Furthermore, when the persistent memory dynamic hash index program is executed by the processor, the following operations are also implemented:
[0160] When the insertion into the spare data bucket is successful, it is determined whether there is a matching data fingerprint in the spare data bucket. If there is a match, the directory entry is located by the highest bit to find the target segment pointed to by the directory entry;
[0161] Determine the corresponding target data fingerprint according to the target segment, retrieve the target data fingerprint, compare the key to be updated in the target data fingerprint with the key value of the corresponding data slot matched by the data fingerprint, and generate a key comparison result;
[0162] A refresh operation is performed according to the key comparison result.
[0163] Furthermore, when the persistent memory dynamic hash index program is executed by the processor, the following operations are also implemented:
[0164] When the key comparison result is that the key to be updated is the same as the key stored in each slot, update the value corresponding to the current key;
[0165] When the key comparison result is that the key to be updated is different from the key stored in each slot, and there is a free position in the current data bucket, the hash keys are inserted in sequence.
[0166] Furthermore, when the persistent memory dynamic hash index program is executed by the processor, the following operations are also implemented:
[0167] When the insertion into the backup data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth to generate a depth comparison result;
[0168] When the depth comparison result is that the local depth is less than the global depth, additionally creating a new segment to migrate part of the data within the segment;
[0169] When the depth comparison result is that the local depth is equal to the global depth, directory expansion is performed and the current directory entry is switched.
[0170] Furthermore, when the persistent memory dynamic hash index program is executed by the processor, the following operations are also implemented:
[0171] When a delete instruction is received, the data in the current data bucket is deleted, the data in the logical next data bucket with the flag position being 1 is migrated to the deleted free position, and the flag position is set to 0.
[0172] Furthermore, when the persistent memory dynamic hash index program is executed by the processor, the following operations are also implemented:
[0173] The directory entry of the persistent memory has one and only one pointer to a certain segment; the segment has several data buckets, the data bucket has several slots, and one slot stores one piece of data. The dynamic hash structure uses global depth and local depth to represent the quantitative relationship between the segment and the pointer pointing to the segment, wherein the global depth is used to determine the number of bits of the directory entry, and the local depth is used to indicate the length of the common hash key in the segment.
[0174] Through the above scheme, this embodiment locates the target data bucket in the dynamic hash structure through a preset data fingerprint insertion algorithm, and inserts the hash key into the target data bucket, and the dynamic hash structure includes a pointer array, a data bucket and a segment; when there is no free position in the target data bucket, the logical next data bucket is used as a spare data bucket, and the hash key is inserted into the spare data bucket; when the insertion into the spare data bucket is successful, it is determined whether there is a matching data fingerprint in the spare data bucket. If there is a match, the key to be updated and the key value of the corresponding data slot matching the data fingerprint are compared, and a refresh operation is performed according to the key comparison result; when the insertion into the spare data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result, which can improve the speed and efficiency of the persistent memory dynamic hash index, improve the insertion performance and space utilization, reduce unnecessary read and write overhead, and improve the performance of the persistent memory hash index.
[0175] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0176] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0177] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A persistent memory dynamic hash indexing method, characterized in that: The persistent memory dynamic hash index method comprises: Locate a target data bucket in a dynamic hash structure according to a preset data fingerprint insertion algorithm, and insert a hash key into the target data bucket, wherein the dynamic hash structure includes a pointer array, a data bucket, and a segment; When there is no free space in the target data bucket, the next logical data bucket is used as a spare data bucket, and the hash key is inserted into the spare data bucket; When the insertion into the standby data bucket is successful, determine whether there is a matching data fingerprint in the standby data bucket. If there is a match, compare the key to be updated with the key value of the corresponding data slot that matches the data fingerprint, and perform a refresh operation according to the key comparison result; When the insertion into the spare data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result.
2. The persistent memory dynamic hash index method according to claim 1, characterized in that: The step of locating a target data bucket in a dynamic hash structure according to a preset data fingerprint insertion algorithm and inserting a hash key into the target data bucket includes: Get the prefix and suffix of the pointer array in the dynamic hash structure; A preset data fingerprint insertion algorithm is constructed based on the data fingerprint, and the index of the prefix and the suffix is located through a hash function in the dynamic hash structure according to the preset data fingerprint insertion algorithm, and the hash key is inserted into the target data bucket.
3. The persistent memory dynamic hash index method according to claim 1, characterized in that: When the insertion into the standby data bucket is successful, it is determined whether there is a matching data fingerprint in the standby data bucket. If there is a match, the key to be updated is compared with the key value of the corresponding data slot that matches the data fingerprint, and a refresh operation is performed according to the key comparison result, including: When the insertion into the spare data bucket is successful, it is determined whether there is a matching data fingerprint in the spare data bucket. If there is a match, the directory entry is located by the highest bit to find the target segment pointed to by the directory entry; Determine the corresponding target data fingerprint according to the target segment, retrieve the target data fingerprint, compare the key to be updated in the target data fingerprint with the key value of the corresponding data slot matched by the data fingerprint, and generate a key comparison result; A refresh operation is performed according to the key comparison result.
4. The persistent memory dynamic hash index method according to claim 3, characterized in that: The refreshing operation is performed according to the key comparison result, comprising: When the key comparison result is that the key to be updated is the same as the key stored in each slot, update the value corresponding to the current key; When the key comparison result is that the key to be updated is different from the key stored in each slot, and there is a free position in the current data bucket, the hash keys are inserted in sequence.
5. The persistent memory dynamic hash index method according to claim 1, characterized in that: When the insertion into the standby data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and a split operation is performed according to the depth comparison result, including: When the insertion into the backup data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth to generate a depth comparison result; When the depth comparison result is that the local depth is less than the global depth, additionally creating a new segment to migrate part of the data within the segment; When the depth comparison result is that the local depth is equal to the global depth, directory expansion is performed and the current directory entry is switched.
6. The persistent memory dynamic hash index method according to claim 1, characterized in that: When the insertion into the standby data bucket fails, the local depth of the segment where the hash conflict occurs is compared with the global depth, and after the split operation is performed according to the depth comparison result, the persistent memory dynamic hash index method further includes: When a delete instruction is received, the data in the current data bucket is deleted, the data in the logical next data bucket with the flag position being 1 is migrated to the deleted free position, and the flag position is set to 0.
7. The persistent memory dynamic hash index method according to claim 1, characterized in that: The directory entry of the persistent memory has one and only one pointer to a certain segment; the segment has several data buckets, the data bucket has several slots, and one slot stores one piece of data. The dynamic hash structure uses global depth and local depth to represent the quantitative relationship between the segment and the pointer pointing to the segment, wherein the global depth is used to determine the number of bits of the directory entry, and the local depth is used to indicate the length of the common hash key in the segment.
8. A persistent memory dynamic hash index system, characterized in that: The persistent memory dynamic hash index system includes: An insertion module, used to locate a target data bucket in a dynamic hash structure according to a preset data fingerprint insertion algorithm, and insert a hash key into the target data bucket, wherein the dynamic hash structure includes a pointer array, a data bucket, and a segment; A spare module, used to use the next logical data bucket as a spare data bucket and insert the hash key into the spare data bucket when there is no free space in the target data bucket; A refresh module is used to determine whether there is a matching data fingerprint in the standby data bucket when the insertion into the standby data bucket is successful. If there is a match, the key to be updated is compared with the key value of the corresponding data slot that matches the data fingerprint, and a refresh operation is performed according to the key comparison result; The splitting module is used to compare the local depth of the segment where the hash conflict occurs with the global depth when the insertion into the spare data bucket fails, and perform a splitting operation according to the depth comparison result.
9. A persistent memory dynamic hash index device, characterized in that: The persistent memory dynamic hash index device includes: a memory, a processor, and a persistent memory dynamic hash index program stored in the memory and executable on the processor, wherein the persistent memory dynamic hash index program is configured to implement the steps of the persistent memory dynamic hash index method as described in any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium stores a persistent memory dynamic hash index program, which, when executed by a processor, implements the steps of the persistent memory dynamic hash index method according to any one of claims 1 to 7.
Citation Information
Patent Citations
ACEH index structure and method for hotspot data based on memory
CN112395213A
Write optimization extensible Hash index structure based on nonvolatile memory and insertion, refreshing and deletion methods
CN113342706A
A method for controlling directory splits of theextendible hashing
KR1020010047384A
Multi-level nested open hashed data stores
US20060095458A1
