A data-driven hash index method based on persistent CPU cache

Abstract

The application belongs to the technical field of computers and specifically discloses a data-driven hash index method based on a persistent CPU cache, which comprises the following steps: a DRAM-PM hybrid architecture is adopted, a three-layer scalable hash structure of directory-segment-bucket is constructed in a persistent memory PM, and a physical separation layout of metadata and key-value pairs is configured in a segment, the metadata is stored in a segment header, and the key-value pairs are stored in a segment tail; a plurality of circular queues are set in DRAM, requests of index operations are collected and classified through the circular queues, a threshold is set for each circular queue, when the data volume in the circular queue reaches the threshold, batch processing of data in the corresponding circular queue is triggered, and the batch processing is used for batch persisting the data to the PM. Through data layout optimization of frequently accessed metadata and CPU cache optimization for scattered hash bucket access, the read-write amplification of the PM hash index can be effectively eliminated.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computers, and more particularly, to a data-driven hash index method based on persistent CPU cache. BACKGROUND

[0002] Hash-based index structures are widely used in in-memory databases and in-memory key-value stores. With the development of emerging storage technologies, hash indexes have evolved to better adapt to the characteristics of new hardware. Persistent Memory (PM) technology has become a new generation of storage media with the advantages of large capacity, byte-addressability, persistence, and low latency close to Dynamic Random Access Memory (DRAM). Existing PM technologies include PCM, ReRAM, 3D XPoint, and MRAM. To fully utilize the advantages of PM, many studies have optimized hash indexes on PM. Hash indexes on PM can retain the index structure in the event of system crashes, so when optimizing hash indexes, the crash consistency problem of PM must be addressed to ensure that the correct state of the index can be recovered after a system crash. Under the premise of ensuring crash consistency, improving the throughput of the index is the core goal of the design.

[0003] Initially, when the concept of PM was first proposed, researchers used Quartz and other DRAM-based PM simulators to explore the feasibility of deploying hash indexes on PM. Studies have found that DRAM-based PM simulators have the following characteristics: (1) the access delay of the simulator is between DRAM and HDD; (2) the write delay is higher than the read delay; (3) special instructions such as cache line flush instructions (CLFLUSH) and memory fence instructions (MFENCE) are required to persist data from the CPU cache to PM, thereby ensuring data consistency. During this period, the focus of index design was to minimize read and write operations on PM. A representative work, CCEH (Cache-Conscious Extendible Hashing), implemented an optimized extendible hashing scheme on PM simulators, which reduced the number of cache line accesses while using CLFLUSH and MFENCE instructions to ensure data consistency, thereby reducing the frequency of read and write accesses and improving the throughput of index operations. It is worth noting that based on the directory-bucket (hash bucket) two-layer structure of extendible hashing, CCEH proposed a directory-segment-bucket three-layer extendible hashing concept, which has been widely applied in subsequent research.

[0004] In recent years, a commercial PM supporting asynchronous DRAM refresh technology (ADR) named Optane Data Center Persistent Memory Module (DCPMM) has been introduced. The first generation of DCPMM guarantees the persistence of data in the PM, and has the following characteristics: (1) the minimum access unit of the PM (called XPLine) is 256 bytes, which determines that when writing less than 256 bytes of data in an XPLine, write amplification phenomenon will occur; (2) the PM write bandwidth is one third of the read bandwidth, and is five times lower than the DRAM write bandwidth; (3) CLFLUSH and MFENCE instructions are indeed needed in the PM to guarantee the consistency of CPU cache and PM data, however, these instructions have high write latency and will occupy the limited PM write bandwidth. In order to fully adapt to these characteristics, researchers further improved the hash index technology based on the previous development on the PM simulator. First, in order to adapt to the size of XPLine, the index of this period stores a certain amount of key-value pairs and their corresponding metadata in the same 256-byte block, so that accessing a single XPLine can read and write a pair of metadata and key-value pairs at the same time. For example, Dash optimizes the three-layer scalable hash structure based on CCEH, modifies the hash bucket size to XPLine size, and stores key-value pairs and their corresponding metadata in the same hash bucket. This setting becomes the general setting of hash index on PM. Second, in order to better utilize the limited bandwidth, some researches store unimportant or recoverable data in DRAM, thereby transferring part of the read and write operations on PM to DRAM. Finally, the use of CLFLUSH and MFENCE instructions is still necessary, and when designing the index structure, the research will reduce the number of times of using these instructions as a major point.

[0005] In recent years, the second generation of DCPMM that can support enhanced asynchronous DRAM refresh technology (eADR) has also been introduced. Under the eADR feature, when the machine experiences a power failure, the CPU's backup power can transfer data from the CPU cache to the persistent memory (PM), thereby making the CPU cache persistent. The eADR feature ensures the equivalence of data visibility and persistence, eliminating the possibility of data inconsistency between the CPU cache and the PM. Therefore, on a machine that supports the eADR feature, it is no longer necessary to explicitly use CLFLUSH and MFENCE instructions to persist data from the CPU cache to the PM.

[0006] Since the eADR feature was introduced, the index design based on this feature has been committed to taking advantage of persistent CPU cache to reduce the impact of PM write amplification on index performance. These designs mainly improve the performance of indexes deployed on PM from the following aspects: (1) The problem of mismatch between CPU cache line and XPLine granularity is considered in the design. When cache line eviction occurs, the data written to PM is in units of 64 bytes, which is smaller than 256 bytes of a PM row, causing write amplification; (2) Use lock-free technology, or integrate other technologies such as hardware transaction memory (HTM), cache allocation technology (Intel CAT), and persistent cache to enhance index performance. However, these studies still use the data layout designed under the ADR feature, and the optimization of persistent CPU cache is not sufficient. The present application believes that these hash index works have the following two defects.

[0007] The data layout is not optimized for frequently accessed metadata. As described above, storing key-value pairs and their corresponding metadata together in the same hash bucket of XPLine size is a common setting for hash indexes on PM. However, the access frequency of metadata is significantly higher than that of key-value pairs, resulting in a large amount of read and write amplification when accessing metadata. With the addition of the eADR feature, a more optimal data layout needs to be designed.

[0008] The CPU cache is not optimized for the scattered access of hash buckets. Ideally, for a hash bucket, as many insertion requests as possible are completed in the CPU cache before the data of the hash bucket is persisted from the CPU cache to the PM, so that multiple insertions can be completed in one PM write access. However, due to the random distribution of data inserted into the index, data will hit different hash buckets scatteredly, resulting in a large number of cache line evictions and a low cache hit rate, which causes the data of the same hash bucket to be evicted prematurely when it has not accumulated, resulting in a large amount of write amplification.

[0009] The above defects in the prior art cause read and write amplification of PM hash indexes, and effectively eliminating the read and write amplification of PM hash indexes is a technical problem to be solved in the field. SUMMARY

[0010] To overcome the defects of the prior art, the present application aims to optimize the data layout of frequently accessed metadata and optimize the CPU cache for scattered hash bucket access, in order to solve the problem of read and write amplification of PM hash indexes in the prior art.

[0011] To achieve the above-mentioned purpose, in a first aspect, the present application provides a data-driven hash index method based on persistent CPU cache, the method comprising:

[0012] The DRAM-PM hybrid architecture is adopted, a three-layer scalable hash structure of directory-segment-bucket is constructed in the persistent memory PM, and a physical separation layout of metadata and key-value pairs is configured in the segment, the physical separation layout including that the metadata is stored in the segment header and the key-value pairs are stored in the segment tail.

[0013] A plurality of circular queues are arranged in the dynamic random access memory (DRAM), requests of index operations are collected and classified through the circular queues, a threshold is set for each circular queue, when the data amount in the circular queue reaches the threshold, batch processing of data in the corresponding circular queue is triggered, and the batch processing is used for batch persisting the data to the PM.

[0014] It can be understood that, by designing the hash index part in the PM based on the CCEH directory-segment-bucket three-layer scalable hash structure, and designing a data layout of separated metadata and key-value pairs in the segment and the bucket, the read-write amplification caused by frequently accessed metadata can be effectively reduced.

[0015] By arranging a plurality of circular queues in the DRAM, the requests of index operations are collected and classified in the DRAM, and after the requests accumulate to a certain scale, the data in the requests are batch processed, which can effectively reduce the write amplification caused by scattered hash bucket access.

[0016] Therefore, by the above data layout optimization for frequently accessed metadata and CPU cache optimization for scattered hash bucket access, the read-write amplification of the PM hash index can be effectively eliminated.

[0017] In a possible implementation, the segment region includes a segment header, a segment middle part and a segment tail, and the local depth and the redundant bucket are arranged in the segment middle part.

[0018] It can be understood that, by arranging the redundant bucket composed of a small amount of key-value pairs and corresponding metadata in the middle part of the segment space, the splitting of the segment can be delayed. At the same time, by storing the local depth variable for managing the scalable hash level in the structure of the redundant bucket, the segment structure can be adapted to the size of the XPLine.

[0019] In a possible implementation, the above collecting and classifying the requests of index operations through the circular queues include:

[0020] The key-value pairs are classified into a circular queue according to the most significant bits (MSBs) of the key, and are stored in the empty slot at the tail of the circular queue;

[0021] The plurality of circular queues are allocated in a continuous memory block, and the slot position of the key-value pairs stored in the circular queue is located through an offset.

[0022] It can be understood that if a plurality of queues are managed using a pointer array, additional pointer dereferencing is introduced. The application allocates a continuous memory block for all queues in the DRAM, and the slot positions in the queue storing the key-value pairs are located through an offset, which can effectively eliminate the overhead caused by the pointer.

[0023] In a possible implementation, the above-mentioned batch persistence of data to the PM includes:

[0024] When the amount of data in the target circular queue reaches a threshold value, a directory structure related to the target circular queue in the PM is found, a directory snapshot is constructed in the DRAM, and the found directory structure is copied to the directory snapshot;

[0025] The data in the target circular queue is inserted into the directory snapshot;

[0026] The data in the directory structure related to the target circular queue in the PM is inserted into the directory snapshot;

[0027] The directory snapshot is persisted to the PM.

[0028] It can be understood that the directory snapshot concentrates the merging of the circular queue data and the index data in the directory snapshot through the pre-constructed index structure of the PM, reduces the access to the PM, and makes the batch writing become sequential access, thereby improving the efficiency and throughput of the persistence.

[0029] In a possible implementation, the above-mentioned batch persistence of data to the PM includes:

[0030] The plurality of circular queues are logically merged into a single queue at a cold start, and an initial threshold value is set according to the size of the merged queue;

[0031] As the directory is split to gradually split the queue, the control threshold value is dynamically increased according to the splitting progress until the merged queue is completely split, and then the threshold value is configured as a fixed value.

[0032] It can be understood that the above-mentioned batch persistence scheme based on the circular queue and the directory snapshot has the phenomenon that the PM bandwidth is not fully utilized. Specifically, first, in the cold start phase, sufficient data in the queue must be accumulated before entering the request processing phase of the index construction; in addition, if the threshold value is simply lowered, multiple queues will perform persistence at the same time, which will cause contention for the PM bandwidth. These two problems exacerbate the tail latency. In order to solve these problems, the application dynamically adjusts the merging and splitting of the queue in the index construction process, thereby improving the early PM bandwidth utilization and reducing the tail latency.

[0033] In a possible implementation, the above-mentioned control threshold value is dynamically increased according to the splitting progress until the merged queue is completely split, including:

[0034] Pre The second trigger request processing stage is triggered when the number of the first The second threshold is , , represents the number of the circular queue, represents the amount of data persisted each time,

[0035] From the beginning of the second trigger request processing stage, the threshold is configured as a fixed value.

[0036] It should be noted that, for the threshold If a fixed value is used, the accumulated data in the merged queue can be close to times of the threshold of a single original queue when a snapshot of the directory is triggered, which cannot effectively reduce the tail latency problem. In order to maintain a roughly constant amount of data during the execution of the persistence, the present application designs an algorithm for dynamically adjusting the threshold according to the above formula, ensuring that the amount of data persisted each time is about , effectively avoiding the waste of PM bandwidth and reducing the tail latency.

[0037] In a possible implementation, the method further includes:

[0038] The offset of the second directory is stored in the directory block through the first directory, the offset of the segment is stored in the segment block through the second directory, and the data structure in the PM is recovered by using the offset;

[0039] The key-value pair collected in the DRAM but not completed for persistence is recorded in the pre-write log WAL, and in the recovery process, the uncompleted data operation in the WAL is re-executed to recover the data structure in the DRAM.

[0040] It can be understood that remapping the physical address in the PM will cause the pointer to be invalid during recovery, and the present application manages the hash index structure by recording the offset, and the structure in the PM can be recovered immediately by using the offset. For the data structure of the DRAM, in the recovery process, by re-executing the uncompleted data operation in the WAL, data consistency can be ensured.

[0041] In a possible implementation, the method further includes:

[0042] The segment space allocator is configured as a recycling queue;

[0043] When a new segment is allocated, a recycling block is first attempted to be allocated from the recycling queue, and if the allocation is unsuccessful, the offset in the segment block is calculated, and an unused block is allocated from the segment block;

[0044] ​​Scanning the directory in the PM and recording active segments;

[0045] Reclaiming unused segments to a reclaim queue.

[0046] It can be understood that the segment space allocator only needs a small amount of memory, which can reside in the CPU cache, thereby minimizing the PM access overhead.

[0047] In a second aspect, the present application provides an electronic device / image signal generator / network device / transmitter / terminal / base station / industrial computer, comprising: at least one memory for storing programs; at least one processor for executing the programs stored in the memory, and when the programs stored in the memory are executed, the processor is used to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0048] In a third aspect, the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and when the computer program runs on the processor, the processor executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0049] It can be understood that the beneficial effects of the above-mentioned second aspect to third aspect can be referred to the related description in the first aspect, which will not be repeated here.

[0050] Overall, compared with the prior art, the above technical solutions conceived by the present application have the following beneficial effects:

[0051] (1) Through the data layout optimization for frequently accessed metadata and the CPU cache optimization for scattered hash bucket access, the read-write amplification of the PM hash index can be effectively eliminated.

[0052] (2) By setting a redundant bucket composed of a small number of key-value pairs and corresponding metadata in the middle of the segment space, the splitting of the segment can be delayed. At the same time, by storing the local depth variable used to manage the expandable hash level in the structure of the redundant bucket, the segment structure can adapt to the size of the XPLine.

[0053] (3) A continuous memory block is allocated for all queues in the DRAM, and the slot position of the key-value pair stored in the queue is located by offset, which can effectively eliminate the overhead caused by the pointer.

[0054] (4) The directory snapshot concentrates the merging of the circular queue data and the index data in the index structure of the pre-built PM, reduces the PM access, and makes the batch writing become sequential access, thereby improving the efficiency and throughput of the persistence.

[0055] (5) Dynamically adjust the merging and splitting of the queue during the index construction process, thereby improving the early PM bandwidth utilization and reducing the tail latency. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is a whole flow framework diagram provided by an embodiment of the present application;

[0057] Figure 2 is a segment structure diagram with metadata separation mode implemented in the PM provided by an embodiment of the present application;

[0058] Figure 3 is a diagram of a circular queue implemented in the DRAM provided by an embodiment of the present application;

[0059] Figure 4 is a flow diagram on the circular queue provided by an embodiment of the present application;

[0060] Figure 5 is a diagram of a method of directory snapshot and persistent data implemented in the DRAM provided by an embodiment of the present application;

[0061] Figure 6 is a flow diagram on the directory snapshot provided by an embodiment of the present application;

[0062] Figure 7 is a diagram of a method of optimizing the PM bandwidth usage rate at the cold start period provided by an embodiment of the present application;

[0063] Figure 8 is a flow diagram after introducing the optimization method when persisting the directory snapshot provided by an embodiment of the present application;

[0064] Figure 9 is a diagram of a persistent memory space allocator provided by an embodiment of the present application;

[0065] Figure 10 is a diagram of the throughput of the present application and other PM hash indexes under different thread numbers;

[0066] Figure 11 is a diagram of the insertion throughput of the present application and other PM hash indexes under the ADR platform and the eADR platform;

[0067] Figure 12 is a diagram of the load rate change of the present application and other PM hash indexes with data insertion;

[0068] Figure 13 is a diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0070] In the embodiments of the present application, the words such as "exemplary" or "for example" are used to mean serving as an example, instance, or illustration, any embodiment or aspect described as "exemplary" or "for example" in the embodiments of the present application is not necessarily to be construed as preferred or advantageous over other embodiments or aspects. Rather, use of the words such as "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0071] In the description of the embodiments of the present application, unless otherwise specified, "a plurality of" means two or more, for example, a plurality of processing units means two or more processing units, and the like; a plurality of elements means two or more elements, and the like.

[0072] The technical solution adopted in the present application comprises the following six steps.

[0073] Step 1, when designing a hash index, a DRAM-PM hybrid structure is adopted.

[0074] The DRAM-PM hybrid structure is a hybrid memory system. The hybrid memory system usually has multiple memory layers, or integrates two or more heterogeneous memory technologies. The DRAM-PM hybrid structure can be specifically a hybrid memory system constructed using two memory technologies of DRAM (for example, DDR4 DRAM) and PM (for example, byte-addressable persistent memory PM).

[0075] Step 2, in order to reduce the read-write amplification caused by frequently accessed metadata, the hash index part in the PM is designed based on the CCEH three-layer scalable hash structure of directory-segment-bucket, and a data layout separating metadata from key-value pairs (hereinafter referred to as "metadata separation") is designed in the segment and bucket.

[0076] Step 3, in order to reduce the write amplification caused by scattered hash bucket access, the requests of index operations are collected and classified in the DRAM, and the data in the requests are processed in batches after the requests accumulate to a certain scale.

[0077] Step 4, in order to improve the efficiency of persistence on the basis of step 3, when processing the requests in batches, the index structure is pre-constructed in the DRAM, and then the structure is written to the PM in batches.

[0078] Step 5, in order to improve the utilization rate of bandwidth in the early stage of index construction, the condition of triggering batch processing of requests is reduced through structure improvement.

[0079] Step 6, in order to ensure the crash consistency of PM, design a suitable PM space allocator.

[0080] The above step 1 is implemented by the following scheme:

[0081] The PM is managed by the App-Direct mode, and the use time of DRAM and PM is managed by the application.

[0082] The App-Direct mode is a kind of access mode of Optane data center persistent memory module (DCPMM), which allows the persistent memory to be directly accessed, and the data has persistence.

[0083] The above step 2 is implemented by the following scheme:

[0084] (1) The key-value pair and its corresponding metadata are no longer stored in the same hash bucket, and only the key-value pair is stored in each hash bucket, which is placed at the tail of the segment space, and the metadata in each bucket is stored separately in the head of the segment space;

[0085] (2) In the middle part of the segment space, a redundant bucket composed of a small number of key-value pairs and their corresponding metadata is set to delay the splitting of the segment. In order to adapt the size of XPLine, the local depth variable used to manage the expandable hash level is stored in the structure of the redundant bucket.

[0086] The above step 3 is implemented by the following scheme:

[0087] (1) Design several circular queues in DRAM to collect and classify the requests of index operations;

[0088] (2) Set a threshold for each circular queue, and when the data amount in a certain queue reaches the threshold, trigger batch processing of the data in the queue.

[0089] The above step 4 is implemented by the following scheme:

[0090] (1) When the data amount in a certain circular queue reaches the threshold, find the directory structure related to the queue in PM, construct a directory snapshot in DRAM, and copy the found directory structure to the directory snapshot;

[0091] (2) Insert the data in the circular queue into the directory snapshot;

[0092] (3) Insert the data in the directory structure related to the queue in PM into the directory snapshot;

[0093] (4) Persist the directory snapshot to PM.

[0094] Step 5 above is achieved through the following scheme:

[0095] (1) During the index initialization phase, multiple original queues are logically combined into a merge queue;

[0096] (2) For each original queue, set a dynamic threshold when the queue is in a merge state;

[0097] (3) The merge queue is gradually split as the corresponding directory in the index is split, and the threshold gradually increases during the splitting process;

[0098] (4) When the merged queue is split into the original queue, the threshold becomes a fixed value.

[0099] Step 6 above is achieved through the following scheme:

[0100] (1) Design directory blocks and segment blocks, and manage space by recording offsets;

[0101] (2) Design a segment space allocator to allocate and reclaim segments during persistence.

[0102] The embodiments of this application are described below with reference to the accompanying drawings.

[0103] Reference Figure 1 The diagram shown is the overall process framework of this application.

[0104] First, we introduce the overall structure of the index. In PM, a hash index based on the CCEH's three-layer scalable hash structure of directory-segment-bucket is designed, and the idea of ​​metadata separation is implemented in the data layout of the segment. In DRAM, several circular queues continuously collect and classify key-value pair insertion requests. After the number of requests accumulates to a certain amount, directory snapshots assist in completing the insertion into the index structure in PM.

[0105] The following describes the overall process of building an index. Figure 1 The red section in the diagram illustrates the main indexing process. For a requested key-value pair, the process can be divided into two phases. The first phase is the request collection phase, which involves collecting data based on the most significant bit of the key (…). ), the key-value pairs are collected and sorted into a circular queue in DRAM and stored into the empty slots of the queue. When the amount of data in the queue reaches a set threshold, all the requests in the queue enter the second phase. The second phase is the request processing phase, which first finds the directory in the PM corresponding to the queue, and then creates a structural snapshot for the directory in DRAM. Next, the data in the queue is inserted into the structural snapshot in the way of extensible hash insertion, and the data in the directory is inserted into the snapshot. After all the data is inserted, the directory snapshot is persisted to the hash index of the PM in the way of batch writing, and finally the snapshot is destroyed. Figure 1 The other parts in the present application are the optimization design of the index. First, in order to manage the segment space opened and recycled when persisting the queue, the present application designs a segment space allocator. Second, in order to maximize the utilization of PM bandwidth in the initial stage of index construction, the present application designs a merging and splitting mechanism for the queue. In the initial stage of index construction, multiple queues are logically merged into one whole, and as the amount of data increases, these queues are logically split.

[0106] Referring to Figure 2 , it is the segment structure implemented in the PM in the present application, and the data layout has a metadata separation mode.

[0107] In the ADR era, the CPU cache has not yet been included in the persistent domain. Therefore, in order to reduce PM writes while ensuring data consistency, the existing PM hash structure usually stores metadata and key-value pairs in the same hash bucket. However, this design is not applicable in the eADR era, as it fails to fully utilize the characteristics of the persistent cache. The present application observes that metadata occupies a small amount of space and has a very high access frequency, and is therefore very suitable for storage in the cache. Based on this observation, the present application proposes a metadata separation design that separates the storage of metadata and key-value pairs, and redesigns the data layout in the segment structure to make it more suitable for building indexes under the eADR platform.

[0108] Figure 2 The data layout of the segment structure in the present application is shown. In each segment, the metadata of the segment, the key-value pairs and the local depth are stored. The metadata is stored at the beginning of the segment, the local depth is in the middle, and the key-value pairs are stored at the end.

[0109] Specifically, most of the key-value pairs are stored in hash buckets, each segment contains 16 hash buckets, each bucket has 16 slots, and each slot occupies 16 bytes of space to store a key-value pair. When dividing the buckets in the segment, the present application uses the least significant bits (LSB) of the key to divide, so 4 bits are needed to distinguish 16 hash buckets. For example, assuming a segment stores a key-value pair with key 1110110, this key-value pair is stored in the 6th hash bucket of that segment. The storage order of metadata items corresponds one-to-one with the order of key-value pairs in the hash buckets. For example, the metadata item of the key-value pair in the first slot of the first bucket is stored in the first position of the metadata area. Each metadata item includes: (1) a 14-bit fingerprint (used to speed up the search); (2) a 1-bit valid bit (1 indicates that the item is valid, 0 indicates that it is invalid); (3) a 1-bit mark used to mark whether the corresponding position stores a value or a pointer (1 indicates that the value is stored directly in the bucket, 0 indicates that the bucket stores a pointer to the value). The fingerprint is used to speed up the search, and the valid bit is used to identify whether the data is valid at the position of the key-value pair corresponding to the metadata.

[0110] When the amount of data in any bucket reaches its capacity, the next insertion operation in that bucket will trigger a segment split. This involves rehashing the data in the entire segment and results in low space utilization. To reduce the frequency of segment splits, new key-value pairs to be inserted are first stored in a redundant bucket until the capacity of that area is insufficient before a segment split occurs. Therefore, a small portion of key-value pairs are stored in the redundant bucket. This application sets the redundant bucket in the segment to store a maximum of 28 key-value pairs and their corresponding metadata, with 16 bytes and 2 bytes reserved for each key-value pair and metadata, respectively. In addition, local depth is used to manage the hierarchy of scalable hashing, occupying 2 bytes. The position of this variable is arranged in the redundant bucket to adapt to the size of XPLine.

[0111] In the entire segment structure, the 16 hash buckets occupy a total of 16 XPLines, the metadata corresponding to these hash buckets occupies a total of 2 XPLines, and the redundant bucket occupies 2 XPLines. Therefore, this data layout makes better use of the access characteristics of PM while separating the metadata.

[0112] Reference Figure 3 The diagram shows a circular queue implemented in DRAM in this application, which collects and classifies insertion requests; see reference... Figure 4 The diagram shown is a flowchart of the circular queue in this application.

[0113] Previous work on PM-based hash indexing suffered from severe write amplification due to a mismatch between key-value pair sizes and XPLine sizes. This application proposes mitigating this problem by using a persistent CPU cache, collecting key-value pairs within a single XPLine and writing them back to the PM in a single persistence operation. However, experiments show that newly inserted key-value pairs have a very low cache hit rate, with only a small number of pairs being persisted along with others, leaving the write amplification problem unresolved. Therefore, this application uses DRAM to collect and categorize data before performing batch writes, thereby mitigating write amplification and reducing the PM write frequency.

[0114] The application creates multiple circular queues in the DRAM, serving the request collection phase of the index building process, where the start and end positions of the i-th queue are denoted as and In the request collection phase, the newly inserted data is stored at the end position of the queue, and a rear shift of 1 position is triggered. When the process enters the request processing phase, a rear shift of 1 position is triggered when each data is persisted to the PM. The capacity of each queue is a fixed value, denoted as .

[0115] In the request collection phase, according to the of the key, the key-value pair is classified into one of the queues and stored in the empty slot at the tail of the queue. Using a pointer array to manage multiple queues will introduce additional pointer dereferencing, so in order to eliminate the overhead caused by pointers, the application allocates a continuous memory block in the DRAM for all queues, and the slot in the queue that stores the key-value pair is located by offset. Figure 3 The process of locating the slot is illustrated by an example. In this example, it is assumed that each queue can accommodate at most data, and the blue blocks represent occupied slots. For the key-value pair with key 110110, it is assumed that the length of is 2, i.e., the first 2 bits are used for classification, then according to , the key-value pair is classified into the fourth queue . Since , the queue is not full and insertion can be performed, and since , the position of the key-value pair in the DRAM block is , i.e., the position of the green block in the figure. To summarize the process, the flowchart of the operation in the circular queue is shown in Figure 4 .

[0116] The total capacity of the circular queue structure is much larger than the number of orders of magnitude of the CPU cache capacity, and has a fixed size, which can accommodate a large amount of data when collecting data, but will not exceed the established capacity.

[0117] Referring to Figure 5 , it is a method for implementing a directory snapshot and persisting data in the DRAM according to the application; and referring to Figure 6 , it is a flowchart of the application on the directory snapshot.

[0118] In the circular queue, a threshold, called the persistence threshold T, is set for each queue. When the amount of data in the queue reaches T, the request processing phase of the index construction flow is entered. In this phase, the data in the PM index should be merged with the data in the DRAM circular queue. However, in-place merging can cause spatial contention of multi-threaded write operations, reducing concurrent performance. Therefore, the merging process is arranged in the catalog snapshot, and the write-once-copy technology is used in the catalog snapshot. In addition, a two-level catalog design is introduced to facilitate more efficient recovery.

[0119] As stated above, the hash index of the PM is a three-layer structure of catalog-segment-bucket. On this basis, in order to associate the circular queue in the DRAM with the catalog item in the PM index, so as to optimize the operation of the request processing phase, the catalog is divided into two levels. The first-level catalog is a static catalog, and each catalog item corresponds to a queue. The second-level catalog is a number of dynamic catalogs, wherein each second-level catalog corresponds to a first-level catalog item, and each second-level catalog is further divided into a number of second-level catalog items.

[0120] When the queue reaches the persistence threshold T, the method of the present application finds the catalog item corresponding to the queue in the first-level catalog in the PM by traversing the , and then further determines the storage location of the corresponding second-level catalog according to the pointer stored in the catalog item, and copies the second-level catalog to the catalog snapshot. In order to reduce the number of accesses to the XPLine, only the structure of the second-level catalog and the referenced segment are copied, and the data in the segment is not copied. After the structure snapshot is constructed, the method of the present application inserts the items in the queue into the snapshot in the order from to , and records which segments have generated data insertion in the process. After all the data in the queue is inserted, all the segments corresponding to the modified segments in the catalog snapshot are found in the PM index, and the data in these segments is merged into the segments in the catalog snapshot. In the process of inserting the queue data and the PM index data into the catalog snapshot, segment splitting and catalog doubling may occur, which may cause changes in the dynamic catalog. Finally, the modified segments and dynamic catalogs in the catalog snapshot are persisted to the PM, and the is updated to indicate the end of the persistence, The number of bits shifted is the number of key-value pairs persisted from the queue to the PM this time.

[0121] Figure 5 The example in the above illustrates the operation of the request processing phase. Assume , continue to use the example of the key-value pair with the key 110110, which has Therefore, the queues and PM indices shown in the diagram all begin with 11, and the first two digits of each data item are omitted in the diagram. The operation steps in the example are as follows: (1) Read and The corresponding dynamic directory in persistent memory, and create a directory snapshot in DRAM; (2) (3) For segments modified in the previous step's directory snapshot, read the corresponding segment's key-value pair from the PM index and merge it into the directory snapshot; (4) Persist the directory snapshot to PM and destroy the snapshot. Figure 5 In the diagram, the green area represents the read access to the PM, and the blue area represents the write access to the PM. To summarize this process, a flowchart of operations performed in the directory snapshot is shown below. Figure 6 As shown.

[0122] Directory snapshots reduce access to the PM by pre-building the PM's index structure and centralizing the merging of circular queue data and index data in the directory snapshot, and make batch writes sequential, thus improving throughput.

[0123] Reference Figure 7 The diagram illustrates the method used in this application to optimize PM bandwidth utilization during the cold start period; see reference... Figure 8 The diagram shown is the final flowchart after introducing optimization methods when persisting directory snapshots in this application.

[0124] The batch persistence scheme based on circular queues and directory snapshots mentioned above suffers from underutilization of PM bandwidth. Firstly, during the cold start phase, sufficient data must accumulate in the queues before the index building request processing phase begins. Secondly, simply lowering the threshold allows multiple queues to persist data simultaneously, leading to PM bandwidth contention. These two issues exacerbate tail latency. To address these problems, this application dynamically adjusts queue merging and splitting during the index building process, thereby improving early PM bandwidth utilization and reducing tail latency.

[0125] During the system initialization phase, the M queues are logically merged. Each independent queue is called the original queue, and the logically merged queue is called the merged queue. The corresponding first-level directories of the queues are also logically merged. At this point, assume that the queue partitioning is performed using... The number of digits is ( Indicates obtaining If the number of bits is M, then the number of bits used to identify the merged queue of M queues is... If a directory doubling occurs in the insertion of the catalog snapshot, the merge queue and its corresponding first-level directory are logically split. Specifically, when the data volume of a certain queue reaches the threshold T and thus enters the request processing phase, it is first checked whether the directory entry of the first-level directory corresponding to the queue is empty. If it is empty, it indicates that the current queue is an original queue in a merge queue. In this case, in order to find the head of the merge queue, each time is subtracted by 1, and it is checked whether the pointer in the first-level directory entry found with the bit number is valid, until the first valid pointer is found, and thus the second-level directory pointed to is found. Then, the catalog snapshot workflow in Figure 5 is continued, that is, a copy of the second-level directory structure in the PM is first copied in the catalog snapshot, then the data in the circular queue and the PM index are sequentially inserted into the snapshot, and finally the modification of the catalog snapshot is persisted to the PM and the snapshot is destroyed. It should be noted that for a snapshot of a merge queue, if a directory doubling occurs, it indicates that the split of the merge queue and its corresponding first-level directory occurs, and thus after the persistence, the first-level directory entries should be modified, that is, the first-level directory entries are directed to the correct second-level directory.

[0126] For the threshold T, if a fixed value is used, it will cause the data accumulated in the merge queue to be close to M times the threshold of a single original queue when a certain original queue in the merge queue triggers the catalog snapshot, which cannot effectively reduce the tail delay problem. In order to maintain a roughly constant data volume during the execution of the persistence, the present application designs an algorithm for dynamically adjusting the threshold. In the initialization of the queue, the of each original queue is set to the second slot of the queue, and the first slot is empty. For each original queue, the threshold of the i-th ( ) triggering of the request processing phase is . After the i-th persistence is completed, the total amount of data of the previous i times of persistence is . The present application sets k to , and thus after the k-th persistence is completed, the total amount of data that has been persisted is , and the last key-value pair that is persisted is the data at the last position in the queue. Under the current total amount of data that has been persisted, it is guaranteed that the first-level directory corresponding to the original queue is completely split, that is, the original queue is also no longer in the merge state. Next, when the k+1-th triggering of the request processing phase occurs, the first slot is no longer empty. Therefore, whether the original queue is a merge queue can be determined by whether the first slot is empty. Starting from the k+1-th triggering of the request processing phase, the persistence threshold T is set to a constant value slightly smaller than s.

[0127] Figure 7 An example of queue merging and splitting is shown. It is assumed that , , and the threshold T of all cyclic queues at the current time is 4. When inserting the key-value pair of key 110110, the operation steps on the catalog snapshot are as follows: (1) When inserting the key-value pair, it is found that the queue reaches the threshold, and the queue triggers the request processing phase; (2) The catalog corresponding to the merge queue is located, and the primary catalog item is found, and then the secondary catalog pointed to by is found; (3) The secondary catalog is copied to the catalog snapshot; (4) The segment splitting is triggered; (5) The catalog snapshot is persisted to the PM, the pointer of the primary catalog is changed, and the catalog snapshot is destroyed. In summary, after the optimization of adding the queue merge and splitting, the persistence of the catalog snapshot adds a transaction, that is, changing the pointer of the primary catalog. Specifically, it is necessary to judge whether a new secondary catalog is generated, and if a new secondary catalog is generated, the corresponding pointer in the primary catalog needs to be changed. After introducing the optimization method to the persistence of the catalog snapshot, the final flow chart is shown in Figure 8 .

[0128] By following this strategy, the present application ensures that the amount of data persisted each time is approximately s, effectively avoiding the waste of PM bandwidth.

[0129] Referring to Figure 9 , it is a persistent memory space allocator designed by the present application.

[0130] The method of the present application is equipped with an allocator in the PM, and the data structures in the PM and the DRAM are recovered by the following methods. (1) Recovering the data structure of the PM. Remapping the physical address in the PM will cause the pointer to be invalid during recovery, so the method of the present application manages the hash index structure by recording the offset. Specifically, the primary catalog stores the offset of the secondary catalog in the catalog block, and the secondary catalog stores the offset of the segment in the segment block. Therefore, the structure in the PM can be restored immediately by using the offset. (2) Recovering the data structure of the DRAM. The key-value pairs collected in the DRAM but not completed for persistence will be recorded in the write-ahead log (WAL). During the recovery process, the uncompleted data operations in the WAL are re-executed to ensure data consistency.

[0131] The method of the present application also introduces a segment space allocator for allocating and recycling segments during persistence, as shown in Figure 9The segment block contains used blocks, unused blocks, and recycled blocks. The segment space allocator is a recycle queue, which records the allocated and recycled blocks (recycled blocks) in the queue. When a new segment needs to be allocated, a recycled block is first attempted to be allocated from the recycle queue, and if the allocation is unsuccessful, an offset in the segment block is calculated, and an unused block is allocated from the segment block. When the system crashes, a new segment that is being persisted may be lost, resulting in memory leakage. To ensure memory safety, the directories in the PM are scanned and active segments are recorded. Any unused segment identified in this process is recycled into the allocator queue.

[0132] The PM allocator only requires a small amount of memory, which can reside in the CPU cache, thereby minimizing the PM access overhead.

[0133] In summary, due to the adoption of the above technical solutions, the application has the following beneficial effects.

[0134] (1) The high-performance scalable hash index method designed in the application achieves a higher throughput than other PM hash indexes. YCSB is a tool that can generate index operation requests, Figure 10 The throughput comparison of the application (Ours) and other PM hash indexes (Dash, Pea, Spash, Level, CCEH, and Clevel) under different thread numbers is shown under 6 workloads of YCSB. Among the 6 workloads used in the experiment, (a) to (c) are uniformly distributed workloads, and (d) to (f) are skewed workloads. (a) and (d) are write workloads, (b) and (e) are read-write mixed workloads, and (c) and (f) are read workloads. From the figure, Figure 10 It can be seen that the application outperforms other comparison indexes as a whole under various workloads, and performs better under write-intensive loads and large thread numbers.

[0135] (2) The high-performance scalable hash index method designed in the application can better utilize the eADR feature compared to other PM hash indexes. Figure 11 The insert throughput of the method of the application and other comparison indexes under the ADR platform and the eADR platform is compared. From the figure, Figure 11 It can be seen that the method of the application not only achieves the maximum throughput, but also has the highest performance improvement under the eADR feature.

[0136] (3) The hash index designed in the application performs similarly to other PM hash indexes in key performance indicators (such as load rate and recovery time). Table 1 shows the recovery time of the application, and from Table 1, it can be seen that the recovery time of the application can complete index recovery in milliseconds, which is slightly faster than existing hash indexes. Figure 12The load rate change of the present application with data insertion is shown from Figure 12 As can be seen from the above, the load rate of the present application presents a fluctuating state, and the overall data and the highest and lowest load rates are comparable to other hash indexes.

[0137] Table 1: Recovery time comparison table

[0138]

[0139] Based on the method in the above embodiment, an electronic device is provided in an embodiment of the present application, as shown in the figure, the electronic device can include: a processor (Processor) 810, a communications interface (Communications Interface) 820, a memory (Memory) 830 and a communications bus 840, wherein the processor 810, the communications interface 820, the memory 830 complete mutual communication through the communications bus 840. The processor 810 can call the logical instructions in the memory 830 to execute the method in the above embodiment. Figure 13

[0140] In addition, the logical instructions in the memory 830 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art or part of the technical solutions. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0141] Based on the method in the above embodiment, a computer readable storage medium is provided in an embodiment of the present application, and the computer readable storage medium stores a computer program. When the computer program runs on the processor, the processor executes the method in the above embodiment.

[0142] Based on the method in the above embodiment, a computer program product is provided in an embodiment of the present application, and when the computer program product runs on the processor, the processor executes the method in the above embodiment.

[0143] ​It can be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0144] The method steps in the embodiments of the present application can be implemented in the form of hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC.

[0145] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in or transmitted by a computer readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media sets. The available media can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0146] It can be understood that various numerical numbers involved in the embodiments of the present application are only distinguished for convenience of description, and are not used to limit the scope of the embodiments of the present application.

[0147] Those skilled in the art easily understand that the above only describes the preferred embodiments of the present application and is not used to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A data-driven hash index method based on persistent CPU cache, characterized in that, The application relates to a directory-segment-bucket three-layer scalable hash structure constructed in a persistent memory (PM) by adopting a DRAM-PM hybrid architecture, and a physical separation layout of metadata and key-value pairs configured in a segment, wherein the hash bucket is arranged at a tail part of the segment, the hash bucket is used for storing the key-value pairs, and the metadata corresponding to the key-value pairs in the hash bucket is stored at a head part of the segment. A plurality of circular queues are arranged in a dynamic random access memory (DRAM), the circular queues are used for collecting and classifying index operation requests, a threshold value is set for each circular queue, when the data amount in the circular queue reaches the threshold value, batch processing of the data in the corresponding circular queue is triggered, and the batch processing is used for batch persisting the data to the PM. The segment region comprises a segment head part, a segment middle part and a segment tail part, a redundancy bucket is further arranged in the segment, the redundancy bucket is arranged in the segment middle part, a local depth is stored in the redundancy bucket, a region before the local depth in the redundancy bucket is used for storing metadata corresponding to new key-value pairs, a region after the local depth in the redundancy bucket is used for storing the new key-value pairs, and segment splitting is performed when the region capacity of the redundancy bucket is insufficient, so that the frequency of segment splitting is reduced.

2. The data-driven hash index method based on persistent CPU cache according to claim 1, wherein, The collecting and classifying of the index operation requests through the circular queues comprises the following steps:

3. The data-driven hash index method based on persistent CPU cache according to claim 1, wherein, The key-value pairs are classified into a circular queue according to the most significant bits (MSBs) of the keys, and are stored in empty slots at the tail part of the circular queue. The plurality of circular queues are allocated in a continuous memory block, and the slot positions of the key-value pairs in the circular queues are positioned through an offset. The batch persisting of the data to the PM comprises the following steps:

4. The method of claim 1, wherein, When the data amount in the target circular queue reaches the threshold value, a directory structure related to the target circular queue in the PM is found, a directory snapshot is constructed in the DRAM, and the found directory structure is copied into the directory snapshot; The data in the target circular queue is inserted into the directory snapshot; The data in the directory structure related to the target circular queue in the PM is inserted into the directory snapshot; The directory snapshot is persisted to the PM. Further comprising:

5. The method of claim 1, wherein, When a cold start is performed, the plurality of circular queues are logically merged into a single queue, and an initial threshold value is set according to the size of the merged queue; With the gradual splitting of the directory, the control threshold value is dynamically increased with the splitting progress until the merged queue is completely split, and then the threshold value is set as a fixed value. The control threshold value is dynamically increased with the splitting progress until the merged queue is completely split, and the control threshold value is dynamically increased with the splitting progress until the merged queue is completely split.

6. The method of claim 5, wherein, Further comprising: Front The second trigger request processing stage is triggered when the first trigger request processing stage is completed The second threshold is , , The number of circular queues is represented by The amount of data persisted each time is represented by ; At the start of the first trigger request processing phase, the configuration threshold is set to a fixed value.

7. The data-driven hash index method based on persistent CPU cache according to any one of claims 1-6, characterized in that, The offset of the second-level directory is stored in a directory block through a first-level directory, the offset of the segment is stored in a segment block through the second-level directory, and the data structure in the PM is recovered by using the offset; Key-value pairs collected in the DRAM but not completed for persisting are recorded in a write-ahead log (WAL), and in the recovery process, the data operation in the WAL which is not completed is re-executed to recover the data structure in the DRAM. Further comprising:

8. The data-driven hash index method based on persistent CPU cache according to any one of claims 1-6, characterized in that, A segment space allocator is configured, and the segment space allocator serves as a recycling queue; When a new segment is allocated, a recycled block is first attempted to be allocated from the recycling queue, if the allocation is unsuccessful, an offset in a segment block is calculated, and an unused block is allocated from the segment block; The directories in the PM are scanned and active segments are recorded; The unused segments are recycled to the recycling queue. ​ 9. An electronic device, comprising: ​ at least one memory for storing a computer program; at least one processor for executing the program stored by the memory, the processor being configured to perform the method of any one of claims 1-8 when the program stored by the memory is executed.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. a computer program for making a processor execute the method of any one of claims 1-8 when the computer program is run on the processor.

Images