A read-write performance optimization method based on persistent memory B+ tree index

Abstract

The application discloses a read-write performance optimization method based on a persistent memory B+ tree index and belongs to the field of data storage, and comprises the following steps: establishing a B+ tree, non-leaf nodes of the B+ tree are stored in DRAM, and leaf nodes are stored in PM; introducing a concurrent hash table for storing hot data on the DRAM, transferring the access of the hot data from the PM to the DRAM, and relieving the read-write interference problem on the PM; based on the characteristic that there are hot spots in the work load under a real scene, a lightweight hot data identification and replacement strategy is also implemented, a weight is set for each storage element based on the access frequency, and the dynamic change of the hot data in the work load is dealt with; a low-overhead write operation is realized by combining an additional update mechanism and a self-verification insertion mechanism in the B+ tree, the data consistency guarantee overhead is greatly reduced, and a semi-sequenced leaf node is adopted, so that the query efficiency of the leaf node is improved, and meanwhile, the write efficiency is not affected. The application can provide high performance for read operation and write operation simultaneously.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of data storage, and more particularly, to a read-write performance optimization method based on a persistent memory B+ tree index. BACKGROUND

[0002] New persistent memory (PM) can provide excellent characteristics such as byte addressing, persistence, low latency and large capacity, making it a promising next-generation main memory medium. The emergence of the first commercial PM product, Intel Optane DC persistent memory modules (Optane DCPMM), further accelerates this trend. In recent years, researchers have done a lot of research on B+ tree index optimization based on persistent memory. However, these works cannot provide high-performance read and write operations at the same time. This problem is mainly caused by the following two reasons.

[0003] First, from the software point of view, the read optimization and write optimization mechanisms of the existing persistent memory-based B+ tree are mutually restrictive. In the traditional B+-tree index, all data items in the node are stored in order of data size to quickly find the target item for read operation. However, in persistent memory, the insertion operation of B+ tree not only needs to move half of the leaf node's data items on average, but also needs expensive persistent primitives (cache line flush and memory barrier instructions) to guarantee the atomicity of the sorting operation, which seriously reduces the write performance. Therefore, previous work uses an append-write method to insert new data to avoid the sorting operation caused by PM write. However, this mechanism makes the leaf node become unordered, thus reducing the efficiency of the query operation. In order to speed up the query operation of the unordered node, researchers introduce additional metadata to avoid scanning the entire node. However, this read-write performance improvement is at the expense of write performance, because each insertion operation needs to update the metadata. Although there are works that use data item moving mechanism to batch process metadata and data persistence, the metadata persistence overhead cannot be completely eliminated.

[0004] Secondly, from the hardware point of view, due to the read-write asymmetry of the PM and the hardware resource competition of the read-write threads, the PM has a serious read-write interference problem. Experiments show that this read-write interference can seriously reduce the system performance. For example, the read bandwidth of 6 Optane DCPMM can reach 37 GB / s, and the write bandwidth can reach 13 GB / s. Adding a read thread has no effect on the read bandwidth, but when a write thread is added, the read bandwidth is reduced to 32 GB / s. And with the increase of the write thread, the read-write interference becomes more and more serious, and the read bandwidth will be further reduced to 19 GB / s. Similarly, increasing the read thread will also reduce the write bandwidth to 7 GB / s. The read-write interference problem of the PM brings new challenges to the index structure based on the PM. However, the existing PM-based B+ tree work all ignore this problem, cannot fully utilize the performance characteristics of the PM, and cannot provide high-performance access under mixed read-write workloads. SUMMARY

[0005] In view of the defects of the prior art and the improvement needs, the present application provides a read-write performance optimization method based on a persistent memory B+ tree index, which aims to improve the read-write performance of the persistent memory.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, a read-write performance optimization method based on a persistent memory B+ tree index is provided, comprising:

[0007] establishing a B+ tree, the leaf nodes of which are stored in the persistent memory for persistently storing key-value pair elements; the non-leaf nodes of the B+ tree are stored in the DRAM;

[0008] establishing a hot data table in the DRAM; the hot data table is a concurrent hash table, which includes 2 g groups of hash buckets, each group of hash buckets includes 2 n hash buckets, and each hash bucket stores p elements and first metadata; the first metadata includes a first concurrency control lock; g and n are positive integers;

[0009] when performing an insertion operation or a range query operation, performing the corresponding operation in the B+ tree;

[0010] when performing a single-point query operation, querying the target element in the hot data table, if the queried target element is in the hot data table, returning the target element; if the single-point queried target element is not in the hot data table, querying the target element in the B+ tree, and inserting the target element into the hot data table after the target element is queried.

[0011] The read-write performance optimization method based on the persistent memory B+ tree index provided by the application optimizes the B+ tree structure, maintains a hot data table in the DRAM for storing read hot data, and when performing a single-point query operation, preferentially accesses the hot data table in the DRAM, and only when the single-point query in the hot data table is unsuccessful, the B+ tree is accessed, thereby transferring the read operation of the hot data from the persistent memory to the DRAM, improving the execution speed of the hot data read operation, effectively alleviating the read-write interference problem in the persistent memory, and effectively improving the overall read-write performance of the persistent memory.

[0012] Further, in the hot data table, each group of hash buckets further includes m hash buckets as overflow buckets; m is a positive integer.

[0013] And inserting an element into the hot data table comprises:

[0014] (S1) calculating the hash value of the element to locate the corresponding group and the corresponding hash bucket p of the element in the group c And determining whether there is a free slot in the hash bucket p c If yes, the element is inserted into the free slot in the hash bucket p c And the step (S4) is entered; otherwise, the step (S2) is entered.

[0015] (S2) determining whether there is a free slot in the next hash bucket p c of the hash bucket p n If yes, the element is inserted into the free slot in the hash bucket p n And the step (S4) is entered; otherwise, the step (S3) is entered.

[0016] (S3) determining whether there is a free slot in the m overflow buckets, if yes, the element is inserted into the free slot in the overflow bucket, and the step (S4) is entered; otherwise, an element in the hash bucket p c Is selected to replace the element to be inserted, and the step (S4) is entered.

[0017] (S4) insertion ends.

[0018] Before updating each hash bucket, a first concurrency control lock is acquired from the first metadata of the hash bucket, and after the update of the hash bucket ends, the first concurrency control lock of the hash bucket is released.

[0019] The present application realizes three mechanisms to solve hash conflicts in the hot data table: first, there are multiple slots in each hash bucket, which can store multiple hash elements; second, if there is no free slot in the target bucket, the hash element is allowed to be stored in the next hash bucket within the group, thereby utilizing the locality of data storage; third, if there is also no free slot in the next hash bucket, the hash element is allowed to be stored in the overflow bucket within the group. By combining the three mechanisms, the space utilization of the hash table can be effectively improved, and the hit rate of the hot data table can be improved, so that more read operations (i.e. single-point query operations) can be directly executed in the hot data table.

[0020] Further, the first metadata further includes: the weight of the element stored in the bucket, and the weight of the q elements stored in the overflow bucket; q is a positive integer;

[0021] Further, when performing the single-point query operation, if the target element of the query is in the hot data table, and the number of single-point query operations between the current single-point query operation and the last element weight update is greater than or equal to r, the weight of the target element is increased; r is a positive integer;

[0022] Further, in step (S3), the element replaced by the element to be inserted selected from the hash bucket p c is the element with the smallest weight in the hash bucket p c .

[0023] The present application maintains a weight for each element in the hash bucket in the hot data table, and the weight of the element is increased when the element hits in the hot data table, so the weight can record the read hotness of the element. Based on the weight, the present application can accurately identify the element with the smallest read hotness, i.e. the element with the smallest weight, when the element is replaced in the hot data table, so as to ensure that the elements stored in the hot data table are the elements with the highest read hotness, which is beneficial to improve the hit rate of the read operation in the hot data table.

[0024] Further, in the first metadata, the weight of each element occupies B bits; B is a positive integer;

[0025] Further, after increasing the weight of the element each time, the method further includes: if overflow occurs after the weight of the element is increased, the weights of all elements in the hash bucket where the element is located are reduced by the same proportion.

[0026] When the weight of the element overflows, the present application reduces the weight of all elements in the bucket, which can make the weight of the element adapt to the dynamic change of the load.

[0027] Further, the first metadata further includes: the element fingerprint corresponding to the weight of the stored element;

[0028] And, when performing a single-point query operation, querying a target element in the hot data table includes:

[0029] (T1) calculating a hash value of the target element to locate a target hash bucket corresponding to the target element in the group, and calculating a fingerprint of the target element;

[0030] (T2) determining whether there is a fingerprint f matching the fingerprint of the target element in the first metadata of the target hash bucket, if there is, proceeding to step (T3); otherwise, proceeding to step (T4);

[0031] (T3) obtaining the key of the element corresponding to the fingerprint f from the first metadata, and comparing it with the key of the target element, if they are equal, it is determined that the target element of the query is in the hot data table, and the target element of the query is returned, and the query ends; otherwise, proceeding to step (T4);

[0032] (T4) determining that the target element of the query is not in the hot data table.

[0033] The application further maintains element fingerprints in the metadata of the hot data table, and when performing a single-point query operation, first performs fingerprint matching, and then determines whether the target element of the query is in the hot data table according to the matched fingerprint. Since the size of a hash bucket is often set to multiple cache lines, and the number of bytes occupied by the element fingerprint is small, the fingerprints of all elements in a hash bucket can often be stored in a cache line. The application uses fingerprint matching to effectively reduce the number of read operations of the persistent memory, and further improves the execution speed of the single-point query operation. In addition, considering that there may be hash conflicts when storing elements, the application further compares the key values to confirm whether the matched element is the target element after successful fingerprint matching, which improves the operation execution speed while ensuring the accuracy of the operation.

[0034] Further, performing an insertion operation in the B+ tree includes:

[0035] locating a leaf node L1 in which the to-be-inserted element should be inserted in the B+ tree, and an idle slot st in which the to-be-inserted element will be inserted in the leaf node L1;

[0036] dividing the slots in the leaf node L1 into different segments according to the size of the cache line, reading the segment in which the idle slot st is located into a cache line, sorting the elements in the cache line, and then inserting the to-be-inserted element into the idle slot st, and then writing the cache line back to the persistent memory.

[0037] The size of a leaf node in a B+ tree is often set to multiple cache lines, and the application finds a free slot in the target leaf node in an append write manner after locating the target leaf node for element insertion, and only sorts the data in the cache line where the slot to be inserted is located, without sorting the entire leaf node, thereby avoiding the impact on write performance. Meanwhile, the entire leaf node is divided into segments according to the cache line size in the application, and the segments are unordered, but the elements in each segment are ordered, thereby realizing a semi-ordered leaf node, and the subsequent query operation can effectively improve the query efficiency when executed in the B+ tree.

[0038] Further, the information stored in the leaf node of the B+ tree further includes second metadata, and the second metadata includes a second concurrency control lock, a right sibling pointer and a lowkey; the information stored in the non-leaf node of the B+ tree further includes third metadata, and the third metadata includes a third concurrency control lock; and the lowkey is the lower bound of the key value range corresponding to the node in the B+ tree.

[0039] Further, the information stored in the leaf node of the B+ tree further includes second metadata, and the second metadata includes a second concurrency control lock, a right sibling pointer and a lowkey; the information stored in the non-leaf node of the B+ tree further includes third metadata, and the third metadata includes a third concurrency control lock; and the lowkey is the lower bound of the key value range corresponding to the node in the B+ tree.

[0040] (A1) locating a leaf node L to which an element <key, value> to be written should be written in the B+ tree; key and value are the key and value of the element to be written, respectively;

[0041] (A2) determining whether the number of bytes occupied by the key is greater than 8, if yes, go to step (A3); otherwise, write the value value, then atomically write the key key, and after writing is completed, go to step (A5);

[0042] (A3) splitting the key key into a persistent key and a remaining key, the remaining key being smaller than the lowkey of the leaf node L, and the persistent key occupying no more than 8 bytes;

[0043] (A4) writing the remaining key and the value value, then atomically writing the persistent key, and after writing is completed, going to step (A5);

[0044] (A5) the insertion operation is completed;

[0045] Before updating each leaf node, the second concurrency control lock is acquired from the second metadata of the leaf node, and after the update of the leaf node is completed, the second concurrency control lock of the leaf node is released.

[0046] Since the CPU can support 8-byte atomic operations, when the element in the cache line is written to the persistent memory, if the number of bytes occupied by the key of the element is not more than 8, the value of the element is written first, and then the key is atomically written, so that the key of the element is either completely written or not written at all; if the number of bytes occupied by the key of the element is more than 8, the key is split into a persistent key and a remaining key, and it is ensured that the number of bytes occupied by the persistent key is not more than 8 and the remaining key is less than the lowkey of the leaf node, and when writing, the value and the remaining key are written first, and then the persistent key is atomically written; based on this writing mechanism, after the system crashes, it is directly verified whether the key of the element is less than the lowkey of the leaf node where the element is located, if the key is less than the lowkey, it can be determined that the element is an invalid element, that is, the application realizes a self-verification mechanism, which avoids maintaining additional metadata to indicate the validity of the element and reduces the overhead of the persistent primitive on the basis of ensuring the consistency after the system crashes.

[0047] Further, the read-write performance optimization method based on the persistent memory B+ tree index provided by the application further comprises:

[0048] determining whether the key of the element is less than the lowkey of the corresponding leaf node, if yes, determining that the element is a valid element, otherwise, determining that the element is an invalid element.

[0049] Further, the read-write performance optimization method based on the persistent memory B+ tree index provided by the application further comprises:

[0050] reconstructing the B+ tree, comprising:

[0051] traversing the leaf nodes of the B+ tree in the persistent memory, checking the concurrent control lock in each leaf node and the validity of each element, and clearing the dirty lock and invalid elements;

[0052] dividing the leaf nodes into t regions, and using t threads to construct t sub-trees in parallel in the t regions; t is a positive integer;

[0053] using a thread to combine the constructed t sub-trees into one tree, thereby completing the recovery of the B+ tree.

[0054] When the system crashes, the application divides the leaf nodes of the B+ tree into t regions, respectively constructs sub-trees and combines them into complete sub-trees, and multiple sub-trees can be constructed in parallel, thereby effectively improving the efficiency of recovery after the system crashes.

[0055] Further, the first concurrency control lock, the second concurrency control lock and the third concurrency control lock are all optimistic locks.

[0056] Further, the second metadata and the third metadata further comprise: highkey; the highkey is an upper bound of a key value range corresponding to a node in the B+ tree.

[0057] The optimistic lock supports concurrent operation of multiple threads, and the read operation is a lock-free operation, and the concurrent control mechanism is realized by selecting the optimistic lock, so that the operation execution efficiency can be effectively improved while consistency is ensured.

[0058] According to another aspect of the present application, a computer readable storage medium is provided, comprising: a stored computer program, when executed by a processor, controls the device where the computer readable storage medium executes the read-write performance optimization method based on the persistent memory B+ tree index provided by the present application.

[0059] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0060] (1) In the present application, the leaf nodes of the B+ tree are stored on the persistent memory, and the non-leaf nodes are stored in the DRAM, which can directly persist the storage elements and accelerate the query of the non-leaf nodes by using the low latency of the DRAM; at the same time, a hash table for storing hot data is introduced in the DRAM, the hash table has optimized read performance, high space utilization and high scalability, by transferring the access of hot data to the DRAM, the read-write interference problem in the persistent memory can be significantly alleviated, and the overall read-write performance of the persistent memory can be effectively improved.

[0061] (2) The present application maintains a weight based on access frequency in the hot data table, and the hot data recognition can be realized by simple weight comparison, and the element with the smallest weight is selected as the replaced element when replacing, realizing a lightweight hot data recognition and replacement mechanism; combined with the periodic sampling mechanism, the influence on the foreground operation performance is reduced, so that the dynamic changes of hot data in the real scene workload can be adapted.

[0062] (3) In the present application, when inserting an element into the B+ tree, a semi-ordered leaf node is used to improve the query speed of the leaf node, and the write performance is not affected, and the overall read-write performance is high.

[0063] (4) The present application designs a self-verification mechanism to guarantee the crash consistency of data, and the B+ tree uses append write, and can indicate whether the element is valid by verifying the size of the key of the element, without maintaining additional metadata to indicate the validity of the element, thereby reducing the persistence overhead of the metadata, and reducing the overhead caused by guaranteeing data consistency. BRIEF DESCRIPTION OF DRAWINGS

[0064] Fig. 1 A read-write performance optimization method based on a persistent memory B+ tree index provided for an embodiment of the present application is shown in the schematic diagram;

[0065] Fig. 2 A B+ tree structure schematic diagram provided for an embodiment of the present application is shown in the schematic diagram;

[0066] Fig. 3 A hot data table structure schematic diagram provided for an embodiment of the present application is shown in the schematic diagram. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the present application clearer and more apparent, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0068] In the present application, the terms "first", "second", etc. (if any) in the present application and the drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0069] In order to effectively improve the overall read-write performance of the persistent memory, in an embodiment of the present application, a read-write performance optimization method based on a persistent memory B+ tree index is provided, as shown in the schematic diagram. Figs. 1-3 The present embodiment includes: establishing a B+ tree, the leaf nodes of which are stored in the persistent memory, for persistently storing key-value pair elements, and the number of elements that can be stored in each leaf node is b; the non-leaf nodes of the B+ tree are stored in the DRAM to accelerate the query of the non-leaf nodes by using the low latency of the DRAM, and the number of child node pointers stored in each non-leaf node is a.

[0070] In order to realize concurrent access to the nodes in the B+ tree, in the present embodiment, the metadata of each leaf node stores a corresponding concurrent control lock, and considering that the optimistic lock supports concurrent operation of multiple threads, and the read operation is a lock-free operation, as an optional implementation, the present embodiment selects the optimistic lock to realize the concurrent control mechanism, which can effectively improve the operation execution efficiency while ensuring consistency. When the lock is acquired, the value of the lock is atomically changed from an odd number to an even number using the compare_and_swap instruction, and after the write operation is executed, the lock is atomically changed to an odd number to release the lock; the read operation reads the value of the lock before and after reading the data, verifies whether it is changed, if so, retries the query operation, otherwise returns the result.

[0071] To support the optimistic concurrency control mechanism, as shown in Fig. 2 In the embodiment, the metadata of each non-leaf node includes an optimistic lock, a right sibling pointer, and a high key; and the metadata stored in each leaf node includes an optimistic lock, a right sibling pointer, a high key, and a low key; the high key and the low key are respectively the upper bound and the lower bound of the key value range corresponding to the node in the B+ tree.

[0072] In the B+ tree, each time a node is split, a node of the same size is constructed, the data in the old node is sorted, half of the data in the old node is migrated to the new node, the corresponding metadata is modified, the migrated data in the old node is deleted, and the parent nodes are sequentially updated upwards.

[0073] As a preferred embodiment, in the embodiment, the node size of the B+ tree is set to a multiple of 256 bytes, which aligns with the cache line of the Optane DCPMM, so as to improve the cache locality.

[0074] In order to reduce the influence of read-write interference in the persistent memory on read-write performance, the embodiment further includes: establishing a hot data table in the DRAM; the hot data table is a concurrent hash table, which includes 2 g groups of hash buckets, each group of hash buckets includes 2 n hash buckets, and each hash bucket stores 14 elements and 32 bytes of metadata; g and n are positive integers, and the number of elements stored in the hash bucket can be flexibly adjusted as needed;

[0075] The hot data table is used to store read hot data, and the insertion operation or the range query operation in the persistent memory is executed in the B+ tree; the single-point query operation is preferentially executed in the hot data table, and when the target element of the query is not in the hot data table, the B+ tree is executed, and when the target element of the single-point query is queried in the B+ tree, the target element is inserted into the hot data table.

[0076] Similarly, in order to realize concurrent access to the hash buckets in the hot data table, a concurrent control lock is stored in the metadata of each hash bucket, and in the embodiment, the concurrent control lock is specifically an optimistic lock of 4 bytes. The application stores read hot data in the hot data table in the DRAM, can transfer the access of the hot data to the DRAM, can significantly alleviate the read-write interference problem in the persistent memory, and effectively improves the overall read-write performance of the persistent memory.

[0077] In order to effectively solve the hash conflict, improve the utilization rate of the hash table, and improve the hit rate of subsequent single-point query operations in the hot data table, as shown in Fig. 3As shown, in this embodiment, each group of hash buckets further includes additional m hash buckets, which are overflow buckets; this embodiment proposes three hash collision mechanisms: first, each hash bucket has multiple slots, which can store multiple hash elements; second, if there is no free slot in the target bucket, the hash element is allowed to be stored in the next hash bucket within the group, thereby utilizing the locality of data storage; third, if there is also no free slot in the next hash bucket, the hash element is allowed to be stored in the overflow bucket within the group.

[0078] In order to effectively record the read heat degree of elements in the hot data table, in this embodiment, a weight based on access frequency is maintained for each element in the metadata of the hash bucket, and every interval r single-point query operation, the weight of the element hit in the hot data table will be increased; the metadata of the hash bucket also stores the weights of four elements stored in the overflow bucket;

[0079] Wherein, r is a positive integer, based on r, the application realizes a periodic sampling mechanism; the value of r can be set according to the load characteristics to avoid setting too small, which leads to frequent and invalid data replacement, or setting too large, which leads to that the weight of the element cannot be updated in time; if the value of r is 1, then every time the hot data table is hit, the weight of the element will be updated.

[0080] Based on the hash collision resolution mechanism of the hot data table in this embodiment and the recording method of the read heat degree of the element, in this embodiment, the element is inserted into the hot data table, including:

[0081] (S1) calculating the hash value of the element to locate the group corresponding to the element and the hash bucket p c in the group, and judging whether there is a free slot in the hash bucket p c , if yes, inserting the element into the free slot in the hash bucket p c , and turning to step (S4); otherwise, turning to step (S2);

[0082] Optionally, in this embodiment, the hot data table uses the first g bits of the hash value of the hash element to index the group address of the hash element, and uses the last n bits of the hash value to index the bucket position of the hash element in the group;

[0083] (S2) judging whether there is a free slot in the next hash bucket p c of the hash bucket p n , if yes, inserting the element into the free slot in the hash bucket p n , and turning to step (S4); otherwise, turning to step (S3);

[0084] (S3) judging whether there is a free slot in the m overflow buckets, if yes, inserting the element into the free slot in the overflow bucket, and turning to step (S4); otherwise, from the hash bucket pc The element with the minimum weight is selected to be replaced by the element to be inserted, and step (S4) is entered;

[0085] (S4) insertion is completed;

[0086] Wherein, before updating each hash bucket, the optimistic lock is obtained from the metadata thereof, and after the hash bucket is updated, the optimistic lock of the hash bucket is released.

[0087] In order to adapt to the dynamic change of hot data in the workload in the real scene, in the embodiment, the weight of each element in the hot data table occupies 4 bits, and after updating the element weight each time, it is further judged whether the element weight has overflowed, if yes, the weight of all elements in the bucket where the element is located is halved; it should be pointed out that the number of bits occupied by the element weight, and the proportion of the decrease of the element weight when the element weight overflows can be flexibly adjusted according to the actual application requirements.

[0088] Considering that the size of a hash bucket is often set to be multiple cache lines, in order to further improve the execution speed of the single-point query operation in the hot data table, as shown in Fig. 3 In the embodiment, the metadata stored in each hash bucket further includes: an element fingerprint corresponding to the element weight stored therein, each element fingerprint occupies 1 byte; and when performing the single-point query operation, the target element in the hot data table is queried, including:

[0089] (T1) calculating the hash value of the target element to locate the target hash bucket corresponding to the target element in the group, and calculating the fingerprint of the target element;

[0090] (T2) judging whether there is a fingerprint f matching the fingerprint of the target element in the first metadata of the target hash bucket, if yes, step (T3) is entered; otherwise, step (T4) is entered;

[0091] (T3) obtaining the key of the element corresponding to the fingerprint f from the first metadata, and comparing it with the key of the target element, if they are equal, it is determined that the target element of the query is in the hot data table, and the target element of the query is returned, and the query is completed; otherwise, step (T4) is entered;

[0092] (T4) determining that the target element of the query is not in the hot data table.

[0093] Since the number of bytes occupied by the element fingerprint is small, the fingerprints of all elements in a hash bucket can be concentrated in a cache line. The embodiment uses the fingerprint for matching, which can effectively reduce the number of read operations of the persistent memory and further improve the execution speed of the single-point query operation compared with directly matching through the key value. In addition, considering that there may be a hash conflict when the element is stored, the embodiment further compares the key values to confirm whether the matched element is the target element in the case of successful fingerprint matching, which can improve the operation execution speed while ensuring the accuracy of the operation.

[0094] In view of the problem that the read performance optimization and the write performance optimization of the existing B+ tree index structure in the persistent memory are mutually restricted, the embodiment implements a semi-ordered leaf node in the B+ tree. Specifically, the leaf node is divided into segments according to the cache line size, and when an element is inserted into the leaf node of the B+ tree, an idle slot is found in the target leaf node in an append write manner, and only the data in the cache line in which the slot to be inserted with the element is located is sorted, instead of sorting the entire leaf node. In the semi-ordered leaf node, the elements in the segments are ordered, and the segments are unordered. Since the size of a leaf node in the B+ tree is often set to multiple cache lines, the semi-ordered leaf node proposed in the embodiment can avoid affecting the write performance, and the order in the segment can effectively improve the query efficiency.

[0095] Based on the semi-ordered leaf node described above, the embodiment performs an insertion operation in the B+ tree, including:

[0096] Locating a leaf node L1 in which the to-be-inserted element should be inserted in the B+ tree, and an idle slot st in which the to-be-inserted element will be inserted in the leaf node L1. In the process of finding the idle slot st, if there is no idle slot in the located leaf node, the node splitting operation is performed, and then the leaf node is repositioned and the idle slot is found again;

[0097] The slots in the leaf node L1 are divided into different segments according to the cache line size, and the segment in which the idle slot st is located is read into a cache line. After the elements in the cache line are sorted, the to-be-inserted element is inserted into the idle slot st, and then the cache line is written back to the persistent memory.

[0098] In order to realize fast system recovery after system crash, the embodiment proposes a self-verification mechanism, that is, the order in which the keys and values in the elements are written to the persistent memory is controlled by using the characteristic that the system can support 8-byte atomic operations, so that when the element is not completely written, there is no key value in the persistent memory, or the key value is less than the lowkey of the leaf node in which the element is located. Specifically, the way in which the elements in the cache line are written to the persistent memory includes:

[0099] (A1) locating a leaf node L in which an element <key, value> to be written should be written in the B+ tree; key and value are the key and value of the element to be written, respectively;

[0100] (A2) determining whether the number of bytes occupied by the key key is greater than 8, if yes, go to step (A3); otherwise, write the value value, then atomically write the key key, and after the writing is completed, go to step (A5);

[0101] (A3) splitting the key key into a persistent key and a remaining key, the remaining key is less than the lowkey of the leaf node L, and the number of bytes occupied by the persistent key is not greater than 8;

[0102] (A4) writing the remaining key and the value value, then atomically writing the persistent key, and after the writing is completed, go to step (A5);

[0103] (A5) the insertion operation is completed;

[0104] Before updating each leaf node, a second concurrency control lock is acquired from the second metadata of the leaf node, and after the update of the leaf node is completed, the second concurrency control lock of the leaf node is released;

[0105] After the system crashes, the validity of the elements is verified by the following method:

[0106] determining whether the key of the element is less than the lowkey of the corresponding leaf node, if yes, determining that the element is a valid element; otherwise, determining that the element is an invalid element.

[0107] Based on the self-verification mechanism proposed, the embodiment further includes: after the system crashes, initializing the hot data table and rebuilding the B+ tree to complete system recovery;

[0108] rebuilding the B+ tree, including:

[0109] traversing the leaf nodes of the B+ tree in the persistent memory, checking the concurrency control lock in each leaf node and the validity of each element, and clearing the dirty lock and invalid elements;

[0110] dividing the leaf nodes into t regions, and using t threads to construct t sub-trees in parallel on the t regions; t is a positive integer;

[0111] using a thread to merge the t sub-trees constructed into one tree, thereby completing the recovery of the B+ tree.

[0112] In the embodiment, when rebuilding after the system crashes, the leaf nodes of the B+ tree are divided into t regions, and the sub-trees are constructed and then merged into a complete sub-tree, and multiple sub-trees can be constructed in parallel, thereby effectively improving the efficiency of recovery after the system crashes.

[0113] In another embodiment of the present application, a computer readable storage medium is also provided, comprising: a stored computer program, the computer program, when executed by a processor, controls a device where the computer readable storage medium is located to perform the read-write performance optimization method based on the persistent memory B+ tree index described above.

[0114] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A read-write performance optimization method based on a persistent memory B+ tree index, characterized in that, The method comprises the following steps: establishing a B+ tree, leaf nodes of which are stored in persistent memory, for persistently storing key-value pair elements; non-leaf nodes of the B+ tree are stored in DRAM; establishing a hot data table in the DRAM; The hot data table is a concurrent hash table, which includes 2 g Group hash buckets, each group of hash buckets includes 2 n There are p hash buckets, each storing p elements and first metadata; the first metadata includes a first concurrency control lock; g and n are both positive integers; when an insertion operation or a range query operation is performed, corresponding operations are performed in the B+ tree; when a single-point query operation is performed, a target element is queried in the hot data table, and if the queried target element is in the hot data table, the target element is returned; if the single-point queried target element is not in the hot data table, the target element is queried in the B+ tree, and after the target element is queried, the target element is inserted into the hot data table; in the hot data table, m hash buckets serving as overflow buckets are further included in each group of hash buckets; m is a positive integer; and inserting an element into the hot data table comprises: (S1) Calculate the element hash value to locate the group corresponding to the element and the hash bucket p corresponding in the group c , and determine whether there is a free slot in the hash bucket p c , if yes, insert the element into the free slot in the hash bucket p c , and go to step (S4); otherwise, go to step (S2); (S2) judging whether there is a free slot in the next hash bucket p c of the hash bucket p n , if yes, inserting the element into the free slot in the hash bucket p n and turning to step (S4); otherwise, turning to step (S3); (S3) judging whether there is a free slot in the m overflow buckets, if yes, inserting the element into the free slot in the overflow bucket and turning to step (S4); otherwise, selecting an element from the hash bucket p c to replace the element to be inserted and turning to step (S4); (S4) insertion ends; wherein before updating each hash bucket, a first concurrency control lock is acquired from the first metadata of the hash bucket, and after the hash bucket is updated, the first concurrency control lock of the hash bucket is released.

2. The read-write performance optimization method based on persistent memory B+ tree index according to claim 1, wherein, The first metadata further comprises weights of elements stored in the bucket and weights of q elements stored in the overflow bucket; q is a positive integer; and when a single-point query operation is performed, if the queried target element is in the hot data table and the number of single-point query operations between the current single-point query operation and the last element weight update is greater than or equal to r, the weight of the target element is increased; r is a positive integer; And, the element replaced by the element to be inserted in the step (S3) is the element with the smallest weight in the hash bucket p c selected from the hash bucket p c .

3. The read-write performance optimization method based on persistent memory B+ tree index according to claim 2, wherein, In the first metadata, the weight of each element occupies B bits; B is a positive integer; and after the weight of the element is increased each time, if overflow occurs after the weight of the element is increased, the weights of all elements in the hash bucket where the element is located are reduced by the same proportion.

4. The read-write performance optimization method based on persistent memory B+ tree index according to claim 2, wherein, The first metadata further comprises element fingerprints corresponding to the stored element weights; and when a single-point query operation is performed, querying a target element in the hot data table comprises: (T1) calculating the hash value of the target element to locate the target hash bucket corresponding to the target element in the group, and calculating the fingerprint of the target element; (T2) determining whether there is a fingerprint f matching the fingerprint of the target element in the first metadata of the target hash bucket, if there is, step (T3) is entered; otherwise, step (T4) is entered; (T3) acquiring the key of the element corresponding to the fingerprint f from the first metadata, and comparing it with the key of the target element, if they are equal, it is determined that the queried target element is in the hot data table, and the queried target element is returned, and the query ends; otherwise, step (T4) is entered; (T4) it is determined that the queried target element is not in the hot data table.

5. The read-write performance optimization method based on persistent memory B+ tree index according to any one of claims 1-4, characterized in that, Performing an insertion operation in the B+ tree comprises: locating a leaf node L1 in the B+ tree where the to-be-inserted element should be inserted, and an idle slot st in the leaf node L1 where the to-be-inserted element will be inserted; The slot st in the leaf node L1 is divided into different segments according to the cache line size, and the segment where the free slot st is located is read into a cache line, the elements in the cache line are sorted, and then the to-be-inserted element is inserted into the free slot st, and then the cache line is written back to the persistent memory.

6. The read-write performance optimization method based on persistent memory B+ tree index according to claim 5, wherein, The information stored in the leaf node of the B+ tree further includes second metadata, and the second metadata includes a second concurrency control lock, a right sibling pointer, and a lowkey; the information stored in the non-leaf node of the B+ tree further includes third metadata, and the third metadata includes a third concurrency control lock; the lowkey is the lower bound of the key value range corresponding to the node in the B+ tree; And the way of writing the elements in the cache line into the persistent memory comprises: (A1) locating a leaf node L where a to-be-written element <key, value> should be written in the B+ tree; key and value are the key and value of the to-be-written element, respectively; (A2) judging whether the number of bytes occupied by the key key is greater than 8, if yes, step (A3) is entered; otherwise, the value value is written, then the key key is atomically written, and after the writing is completed, step (A5) is entered; (A3) splitting the key key into a persistent key and a remaining key, the remaining key is smaller than the lowkey of the leaf node L, and the number of bytes occupied by the persistent key is not greater than 8; (A4) writing the remaining key and the value value, then atomically writing the persistent key, and after the writing is completed, step (A5) is entered; (A5) the insertion operation is completed; Wherein, before updating each leaf node, the second concurrency control lock is obtained from the second metadata of the leaf node, and after the leaf node is updated, the second concurrency control lock of the leaf node is released.

7. The read-write performance optimization method based on persistent memory B+ tree index according to claim 6, characterized in that, Further comprising: After the system crashes, the validity of the elements is verified by the following way: judging whether the key of the element is less than the lowkey of the corresponding leaf node, if yes, the element is determined to be a valid element; otherwise, the element is determined to be an invalid element.

8. The read-write performance optimization method based on persistent memory B+ tree index according to claim 7, characterized in that, Further comprising: After the system crashes, the hot data table is initialized and the B+ tree is rebuilt to complete system recovery; Rebuilding the B+ tree comprises: traversing the leaf nodes of the B+ tree in the persistent memory, checking the concurrency control lock in each leaf node and the validity of each element, and clearing the dirty lock and invalid elements; dividing the leaf nodes into t regions, and using t threads to construct t sub-trees in parallel; t is a positive integer; using a thread to merge the t sub-trees constructed into one tree, thereby completing the recovery of the B+ tree.

9. A computer-readable storage medium, characterized in that, Comprising: a stored computer program, when executed by a processor, controls the device where the computer readable storage medium executes the read-write performance optimization method based on the persistent memory B+ tree index of any one of claims 1-8.

Images