High performance key-value store method and system based on persistent cpu cache

Abstract

The application discloses a high-performance key-value storage method and system based on a persistent CPU cache, and relates to the technical field of key-value storage.The application establishes a memory table pool on the lowermost cache of the persistent CPU cache, which can flexibly adjust the number of memory tables according to write load, so as to reduce system resource competition; the update operation of the memory table index structure is delayed, and the asynchronous batch update is performed by a background thread, so as to reduce the software overhead on the system write critical path; the data write-back mechanism in the unit of the memory table is used to write the memory table from the CPU cache back to the persistent memory, so as to improve the write efficiency of the persistent memory device; the multiple memory tables written back to the persistent memory may have old or invalid data, which affects the search performance; the memory table index merging mechanism merges the index structures of the multiple memory tables, eliminates the invalid index nodes, and improves the system search efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of key value storage, and particularly to a high-performance key value storage method and system based on persistent CPU cache. BACKGROUND

[0002] In order to cope with large-scale, small data storage scenarios, key value storage is applied to a large number of commercial storage systems. The key value storage system takes key value pairs as storage objects, which include "key value" and "data value". The key value is a unique identifier of the key value pair, and the data value saves the corresponding data information. The key value storage system is usually based on a log-structured merge tree to establish, organize and save key value pairs in order on the disk. When the key value pair is written, it is first cached to the memory table on the memory, and the memory table index structure for quickly searching the memory table is inserted. In order to ensure the crash consistency of the data, the key value pair is also added to the write-ahead log located on the disk; then when the memory table is full, it is converted into an unmodifiable memory table; finally, the unmodifiable memory table is flushed to the disk, and the batch key value pairs are merged into the log-structured merge tree.

[0003] The emergence of new storage device persistent memory brings new opportunities for the design of key-value storage systems. With the rapid development of related electronic circuit technology, there have been persistent memory implementation technologies represented by phase change memory, resistive random access memory, and magnetoresistive memory, and the Intel developed Optane persistent memory has also been commercially available. Persistent memory has the characteristics of low latency, non-volatility, byte-addressable, and also has the advantages of high scalability, low energy consumption, and large storage density. According to the different implementation technologies, there are different access granularities and persistent domains in persistent memory devices. Differentiated hardware characteristics, such as Intel Optane persistent memory 100 series and 200 series, have a 256-byte access granularity (the smallest unit when reading and writing), but are equipped with different persistent domains (storage areas that guarantee data loss when power is off or the system crashes). The persistent domain of the 100 series only contains the memory controller and the persistent memory (ordinary persistent domain), while the 200 series contains the CPU cache on this basis (enhanced persistent domain). Existing research results can be divided into four categories: (1) replace the memory with persistent memory, use its non-volatility to eliminate the overhead of data consistency mechanism when writing to the memory table (Olzhas Kaiyrakhmet, Songyi Lee, Beomseok Nam, Sam H Noh, and Young-ri Choi. SLM-DB: Single-Level Key-Value Store with Persistent Memory. In Proc. of USENIX FAST, 2019); (2) add a persistent memory level between the memory and the disk to improve the merging efficiency of the log-structured merge tree (Ting Yao, Yiwen Zhang, Jiguang Wan, Qiu Cui, Liu Tang, Hong Jiang, Changsheng Xie, and Xubin He. MatrixKV: Reducing Write Stalls and Write Amplification in LSM-tree Based KV Stores with Matrix Container in NVM. In Proc. of USENIX ATC, 2020); (3) organize a hybrid level of memory and persistent memory to provide reliable large-capacity high-performance storage resources (Sudarsun Kannan, Nitish Bhat, Ada Gavrilovska, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. Redesigning LSMs for Non-Volatile Memory with NoveLSM. In Proc.(1) based on the high performance, low latency, byte-addressable, and other universal characteristics of persistent memory, and based on the access granularity of persistent memory devices, to optimize the design of key-value storage systems and give full play to the performance of persistent memory devices (Youmin Chen, Youyou Lu, Fan Yang, Qing Wang, Yang Wang, and Jiwu Shu. FlatStore: An Efficient Log-Structured Key-Value Storage Engine for Persistent Memory. In Proc. of ACM ALSPLOS, 2020; Wenhui Zhang, Xingsheng Zhao, Song Jiang, and Hong Jiang. ChameleonDB: A Key-Value Store for Optane Persistent Memory. In Proc. of ACM EuroSys, 2021); (2) based on the high performance, low latency, byte-addressable, and other universal characteristics of persistent memory, and based on the access granularity of persistent memory devices, to optimize the design of key-value storage systems and give full play to the performance of persistent memory devices (Youmin Chen, Youyou Lu, Fan Yang, Qing Wang, Yang Wang, and Jiwu Shu. FlatStore: An Efficient Log-Structured Key-Value Storage Engine for Persistent Memory. In Proc. of ACM ALSPLOS, 2020; Wenhui Zhang, Xingsheng Zhao, Song Jiang, and Hong Jiang. ChameleonDB: A Key-Value Store for Optane Persistent Memory. In Proc. of ACM EuroSys, 2021); (3) based on the high performance, low latency, byte-addressable, and other universal characteristics of persistent memory, and based on the access granularity of persistent memory devices, to optimize the design of key-value storage systems and give full play to the performance of persistent memory devices (Youmin Chen, Youyou Lu, Fan Yang, Qing Wang, Yang Wang, and Jiwu Shu. FlatStore: An Efficient Log-Structured Key-Value Storage Engine for Persistent Memory. In Proc. of ACM ALSPLOS, 2020; Wenhui Zhang, Xingsheng Zhao, Song Jiang, and Hong Jiang. ChameleonDB: A Key-Value Store for Optane Persistent Memory. In Proc. of ACM EuroSys, 2021). It is observed that the existing research results consider the high performance, low latency, byte-addressable, and other universal characteristics of persistent memory, and consider the access granularity of persistent memory devices, but all ignore the differences between different persistent domains, and directly apply the existing research results based on ordinary persistent domains to persistent memory devices equipped with enhanced persistent domains, which has some problems: (1) the existing research results based on ordinary persistent domains use special CPU instructions to bypass the CPU cache when writing to persistent memory, without utilizing the CPU cache; (2) if the special CPU instructions used by the existing research results to bypass the CPU cache are removed, the write-back timing of the data is controlled by the cache line replacement algorithm of the cache, which interferes with the original write order; (3) the existing research results have poor scalability and large software overhead, and cannot fully utilize the performance of the CPU cache. Therefore, based on persistent memory equipped with persistent CPU cache, how to design a key-value storage system to fully utilize its hardware characteristics is still a challenging and important problem in the field of key-value storage systems. SUMMARY

[0004] The purpose of the present invention is to address the above-mentioned shortcomings of the prior art and provide a high-performance key-value storage method based on persistent CPU cache. The persistent CPU cache is used as a persistent storage layer and added to the storage stack, on which a flexible and adjustable high-concurrency memory table pool is established to cope with dynamic workloads. A delayed batch update mechanism for memory table indexes is proposed to reduce system software overhead during writes. A memory table writeback mechanism is used to write the entire unmodifiable memory table in the CPU cache back to persistent memory, thereby improving persistent memory write efficiency and ensuring write order. A memory table index merge mechanism merges multiple memory table index structures, eliminating failed or invalid index nodes, and improving memory table search efficiency.

[0005] Another object of the present invention is to provide a high-performance key-value storage system based on persistent CPU cache, which utilizes high-performance, low-latency CPU cache to improve the performance of the key-value storage system.

[0006] A high-performance key-value storage method based on persistent CPU cache includes the following steps:

[0007] Step 1. Create, allocate, and flexibly adjust a CPU cache-based memory table pool.

[0008] 1.1 Allocate a space in the persistent memory as a memory table pool and map it to the lowest level of the CPU cache to prevent it from being replaced;

[0009] 1.2 Initialize the memory table pool and divide it into multiple memory tables of the same size;

[0010] 1.3 Allocate a memory table to the processor core that executes the write request. One memory table is exclusively used by one core.

[0011] 1.4 When the memory table is full, it is marked as an unmodifiable memory table, and another free memory table will be allocated to the processor core;

[0012] 1.5 When the number of memory tables cannot meet the concurrency of the workload, the capacity of each memory table will be reduced and the number of memory tables will be increased. When there are many idle memory tables, the capacity of the memory tables will be increased and the number of memory tables will be reduced to achieve flexible adjustment.

[0013] Step 2. Delayed update mechanism of memory table index structure;

[0014] 2.1 Initialize the background worker thread for background asynchronous batch execution of index structure updates;

[0015] 2.2 The background thread first checks whether the memory table index structure is created. If not, it newly creates the index structure in memory, and then performs synchronization between the memory table and the index structure, parses the key-value pairs of the memory table, adds them to the corresponding index structure, and synchronizes the data of the memory table and the index structure;

[0016] 2.3 Before executing a read request, the background thread is triggered to update the index structure to ensure correct execution of the read request;

[0017] 2.4 When the number of key-value pairs written in the memory table reaches a set threshold, the background thread is triggered to perform batch synchronization, and the counter is cleared to start writing count again;

[0018] Step 3. Memory table write-back mechanism based on copying;

[0019] 3.1 Initialize the background write-back thread for background memory table write-back work;

[0020] 3.2 The background write-back thread cyclically checks the state of the memory table to obtain the unmodifiable memory table in the memory table pool;

[0021] 3.3 Apply for a space on the persistent memory, with the same capacity as the memory table, and use memory copying bypassing the CPU cache to copy the memory table data to the space;

[0022] 3.4 The background write-back thread parses the key-value pair data in the unmodifiable memory table, synchronizes the memory table and the index structure, and modifies the address information of the index to point to the space applied for in step 3.3.

[0023] 3.5 Mark the memory table as idle to indicate that it can be allocated to the processor core again;

[0024] Step 4. Memory table index merging mechanism;

[0025] 4.1 Initialize the background thread for memory table index merging;

[0026] 4.2 Create a global index structure in memory to save valid memory table index nodes;

[0027] 4.3 Merge the memory table indexes one by one into the global index, and eliminate invalid index nodes during merging.

[0028] A high-performance key-value storage system based on persistent CPU cache, comprising: a memory pool table module based on persistent CPU cache, a memory table index structure delayed update module, a memory table write-back module based on copying, and a memory table index merging module;

[0029] The persistent CPU cache-based memory pool table module is used for processor core allocation and mapping memory table.

[0030] The memory table index structure delay update module is used for updating and synchronizing the memory table index structure.

[0031] The copy-based memory table write-back module is used for writing back the unmodifiable memory table in the CPU cache to the persistent memory and releasing the unmodifiable memory table resource.

[0032] The memory table index merging module is used for merging the memory table written back to the persistent memory to remove invalid or invalid nodes, so as to improve the system search efficiency.

[0033] Compared with the prior art, the present application has the following outstanding advantages:

[0034] 1. The present application organizes the memory table on the high-performance and low-delay persistent CPU cache, so as to improve the system write performance.

[0035] 2. The present application adopts the system design that the memory table corresponds to the processor core one by one, so as to completely eliminate the limitation caused by the memory table competition and improve the system concurrent performance.

[0036] 3. The present application can increase or reduce the number of memory tables according to the change of the work load, so as to cope with the dynamic change of the work load and reduce the influence of the change of the work load on the system performance.

[0037] 4. The memory table index structure delay update mechanism proposed by the present application can reduce the software overhead of the write operation.

[0038] 5. The memory table write-back mechanism based on the memory copy proposed by the present application increases the data granularity written back to the persistent memory from the cache line size to the memory table size, so as to improve the write bandwidth utilization efficiency of the persistent memory.

[0039] 6. The memory table index merging mechanism proposed by the present application can improve the system search performance. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1An example diagram of a memory table pool built based on a persistent CPU cache.

[0041] Figure 2 An example diagram of a memory table index structure delayed update process triggered by an update threshold.

[0042] Figure 3 An example diagram of a memory table write-back process based on a memory copy.

[0043] Figure 4 An example diagram of a memory table index structure merge process.

[0044] Figure 5 An example diagram of a prototype system structure of the present application.

[0045] Figure 6 An example diagram of experimental results of system write performance testing.

[0046] Figure 7 An example diagram of experimental results of system read performance testing.

[0047] Figure 8 An example diagram of detailed settings of each scenario of real load simulation performance testing.

[0048] Figure 9 An example diagram of experimental results of real load simulation performance testing.

[0049] Figure 10 An example diagram of experimental results of system performance testing in different concurrent scenarios.

[0050] Figure 11 An example diagram of experimental results of system performance testing in different background write-back thread numbers. DETAILED DESCRIPTION

[0051] The following embodiments will be further described with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

[0052] The core of the present application is to construct a high-concurrency key-value storage system based on a persistent CPU cache. The main idea of the system is to use a persistent CPU cache as a persistent storage layer, add a storage stack, and build a high-concurrency memory table pool that can be flexibly adjusted on the storage stack to cope with dynamic workloads. A memory table index delayed batch update mechanism is proposed to reduce the system software overhead during writing. Then, a memory table write-back mechanism is used to write the unmodifiable memory table in the CPU cache back to the persistent memory in blocks, improve the persistent memory writing efficiency, and ensure the writing order. A memory table index merge mechanism is used to merge multiple memory table index structures, eliminate invalid or ineffective index nodes, and improve the memory table search efficiency.

[0053] The embodiment of the present application specifically realizes the following steps:

[0054] 1) Creation, allocation and elastic adjustment of the memory table pool based on CPU cache:

[0055] (1.1) Initialize the persistent memory as a block device in the Linux system, deploy a file system supporting direct access characteristics for it, and mount it. Create a file under the directory mounted by the persistent memory device, the file size is the same as the preset memory table pool capacity, and call the memory mapping system call of the Linux system to map the file to the user process space of the system, and then write random data to the file by byte, triggering the physical page allocation of the file system;

[0056] (1.2) Select a certain processor core, use Intel CAT technology to limit it to only use the specified lowermost cache cache line, and limit the use of the remaining cores. The specified cache line capacity is the memory table pool size. On the selected processor core, turn off the interrupt and run the program to access the file mapped in step (1.1) several times by byte, at this time, it should be ensured that the remaining processor cores are in idle state. Then call the Intel CAT technology again to cancel the use permission of the processor core to the specified cache line, and realize the space locking of the CPU cache. At this time, the specified cache line saves the data of the mapped file in step (1.1), and the cache line content can be read and modified, but will not be replaced. Write data to the address of the mapped file, the data will be written to the lowermost cache of the CPU, and the mapped file will be used as the memory table pool.

[0057] (1.3) Initialize the global metadata structure of the memory table pool, and divide the mapped file into equal-sized memory tables, and initialize the memory tables to idle state.

[0058] (1.4) When the processor core performs a write operation, search the global metadata structure of the memory table pool, check whether to allocate a memory table, if not, search and allocate an idle memory table, and modify the global metadata structure to record the mapping relationship between the memory table and the core. If the allocation fails, the allocation operation is looped and the number of calls to the allocation operation is recorded.

[0059] (1.5) When the memory table corresponding to the processor core is full, modify the memory table to an unmodifiable memory table, and modify the global metadata structure to release the memory table.

[0060] (1.6) When the allocation operation call counter exceeds the preset threshold, trigger the elastic adjustment to reduce the memory table size to half of the original size, and set it to take effect at the next memory table allocation.

[0061] 2) Delayed update mechanism of memory table index structure:

[0062] (2.1) Create a background thread, initialize the data structure for inter-thread communication, and put the thread to sleep.

[0063] (2.2) When the system executes a write request, the key-value pair data is written to the memory table, and the memory table write count is increased. When the write count reaches the memory table index synchronization threshold, the memory table sequence number is written to the data structure initialized in step (2.1), and the background thread is awakened.

[0064] (2.3) When the system executes a read request, the background thread is awakened by the data structure initialized in step (2.1), and the index structure of all memory tables except the idle memory table is synchronized.

[0065] (2.4) After the background thread is awakened, it first checks the information recorded in the data structure to determine the memory table sequence number that needs to be updated, and then performs synchronization operations. The background thread determines the starting position of the synchronization operation in the memory table by comparing the metadata information of the memory table and the metadata information of the index structure (the state of the memory table after the last synchronization), and then parses the key-value pair and creates the corresponding index node, which is added to the index structure.

[0066] 3) Copy-based memory table write-back mechanism:

[0067] (3.1) Create a background write-back thread, which continuously polls and checks the status of all memory tables after starting. When it detects the presence of an unmodifiable memory table, it performs a write-back operation by setting the memory table to a write-back state. The system can create multiple background write-back threads to perform multiple memory table write-back operations in parallel.

[0068] (3.2) The background write-back thread checks the index structure of the unmodifiable memory table, and determines whether the index structure needs synchronization operations by comparing the index structure metadata and the memory table metadata. If it does, it performs synchronization operations.

[0069] (3.3) Create a file under the directory mounted by the persistent memory device, and call the memory mapping system call of the Linux system to map the file to the user process space of the system, which is used to save the data of the unmodifiable memory table.

[0070] (3.4) Call the memory copy function that bypasses the CPU cache to copy the data of the unmodifiable memory table to the file mapped in step (3.3), and update the address information of the index structure. Then release the unmodifiable memory table and modify its state to idle.

[0071] 4) Memory table index merging mechanism:

[0072] (4.1) Create a background worker thread, initialize the write-back memory table queue, and create a global index structure, which is the same type as the index structure used by the memory table.

[0073] (4.2) After each write-back is completed, the background write-back thread pushes the write-back memory table into the write-back memory table queue, and increases the write-back memory table counter. When the counter reaches the set index merging threshold, the background worker thread is awakened.

[0074] (4.3) The background worker thread merges the memory table index into the global index structure according to the order of the write-back memory table out of the queue. When the same index node appears, the original node is replaced to ensure the correctness of the merging.

[0075] The core of the embodiment of the present application is to implement a high-performance and high-concurrency key-value storage system based on a persistent memory device equipped with a persistent CPU cache. The specific implementation mainly consists of the following modules:

[0076] 1. Memory pool table module based on persistent CPU cache: This module establishes a memory table pool on the lowest cache of the persistent CPU cache, which contains multiple memory tables of the same size. Each memory table can be allocated to a processor core as needed, and the mapping relationship between the memory table and the processor core is recorded by a global metadata structure. The memory table is divided into five parts: (1) Key-value pair counter for recording the number of key-value pairs in the memory table, occupying 38 bits, (2) State flag bit for indicating the current state of the memory table, such as idle, unmodifiable, etc., occupying 2 bits, (3) Tail pointer for saving the offset of the next key-value pair write position in the memory table, occupying 24 bits, (4) Remaining space for recording the remaining capacity of the memory table, occupying 64 bits, (5) Key-value pair storage space for storing the key-value pair data stored in the memory table. The key-value pair counter, state flag bit, and tail pointer occupy a total of 64 bits, and an atomic operation is used to modify them when writing a key-value pair. Figure 1 A memory table pool diagram is given, in which processor core 1 is mapped to memory table 1, processor core N is mapped to memory table 5, and the remaining memory tables are not mapped.

[0077] 2. Memory table index structure delayed update module: This module is responsible for the update operation of the memory table index structure. When a key-value pair is written into a memory table, the index structure of the memory table also needs to be inserted into the corresponding node to ensure the correctness of the search. This module delays the update of the index structure and sets two update strategies: (1) When the difference between the key-value pair counter of the memory table and the counter of the index structure is greater than the set update threshold, the update of the index structure is triggered, (2) When a read request needs to be executed, the index structure of all memory tables is updated to ensure the correctness of the read operation. Figure 2The schematic diagram of the update strategy (1) is given, the key-value pair counter of the memory table is 5, the index structure counter is 3, the tail pointer offset is 0x100, the update threshold is 2, and the update is triggered at this time. The background work thread takes the index tail pointer as the starting position, parses the key-value pair in the memory table, and creates the corresponding index node, adds the index structure, and completes the update. After the update is completed, the index structure counter and the tail pointer are updated to 5 and 0x200 respectively.

[0078] 3. The copy-based memory table write-back module: this module is responsible for writing back the non-modifiable memory table in the CPU cache to the persistent memory, and releasing the non-modifiable memory table resource. The persistent memory space corresponding to the memory table is locked on the CPU cache, and writing back the persistent memory will cause the mapping to be invalid, so the memory copy method is used to copy the non-modifiable memory table data to other positions on the persistent memory. In order to avoid the cache pollution problem caused by memory copy, the module uses a cache-bypassing memory copy function to perform memory table write-back. Figure 3 The memory table write-back schematic diagram with the address of 0x200 is given, the background write-back thread calls the cache-bypassing memory copy function to copy the memory table data to the 0x300 position, and after the copying is completed, the memory table resource at the 0x200 position can be modified to be a free memory table.

[0079] 4. The memory table index merging module: this module is responsible for merging the memory table written back to the persistent memory. After the memory table is written back to the persistent memory by the CPU cache, multiple memory tables will be saved on the persistent memory, and there may be invalid or invalid data in the memory table. In order to improve the search efficiency, the module merges multiple memory table indexes into a global index structure, removes invalid or invalid nodes, and improves the search efficiency. Figure 4 The index merging example of two memory tables is given, wherein the non-modifiable memory table 1 has four nodes K1, K3, K4 and K6, and the non-modifiable memory table 2 has four nodes K2, K3, K4 and K6. When the background work thread merges their index structures into the global index, the K3 node of the memory table 1 and the K4 and K6 nodes of the memory table 2 are determined as invalid or invalid nodes, and the total number of index nodes is reduced from 8 to 5 after merging.

[0080] The system structure prototype realized by the application is as follows Figure 5As shown, it comprises a persistent CPU cache-based memory pool table module, a memory table index structure delayed update module, a copy-based memory table write-back module and a memory table index merging module. The persistent CPU cache-based memory pool table module allocates and maps the memory table for the processor core, the memory table index structure delayed update module is responsible for the update and synchronization of the memory table index structure, the copy-based memory table write-back module is responsible for the memory table data write-back and memory table resource release, and the memory table index merging module is responsible for the merging of multiple index structures, improving the system search efficiency.

[0081] The performance test of the present application is given below:

[0082] The prototype system of the present application is deployed on a single server, which is equipped with two Intel Xeon 5318Y processors, each processor is configured with 128GB memory, and 4 128GB Intel Optane 200 series persistent memory devices are configured on one of the processors, which adopts the interleaved application direct mode with 4KB granularity cross access, and the running operating system is Ubuntu 20.04. All tests are limited to run on the processor with persistent memory and its local memory, the key of the experimental setting key-value pair is 16 bytes, and all data is saved on the persistent memory during testing. The test system includes: (1) NoveLSM, setting up memory table in memory and persistent memory at the same time, and the index structure is stored together with the memory table; (2) NoveLSM-cache, based on NoveLSM, the memory table on the persistent memory is mapped to the CPU cache; (3) SLM-DB, organizing memory table only on persistent memory, and the index structure is also stored in the same medium; (4) SLM-DB-cache, based on SLM-DB, the memory table is also mapped to the CPU cache; (5) PCSM, representing the combination of module one and module three of the present application; (6) PCSM+LIU, representing the combination of module one, module two and module three of the present application; (7) CacheKV, representing the combination of all modules of the present application. Among them, systems (1), (2), (3), (4) are used for system performance comparison, (5), (6), (7) are used for analyzing the influence of system modules of the present application on system performance. The number of key-value pairs in each test operation is 10 million, the key size of the key-value pair is 16 bytes, the test is repeated 5 times, and the average value is taken as the test result, tests 1, 2 and 3 all adopt single user thread configuration. The experimental index to be concerned is the throughput of the system.

[0083] 1. System write performance test:

[0084] The write performance test uses the DB_Benche tool to test and evaluate the system throughput of the application and the rest of the comparative systems in the sequential write and random write scenarios under different key-value pair configurations, and the test results are shown in Figure 6 As the size of the key-value pair value increases, the throughput of the system improves, and the throughput of CacheKV in the sequential write scenario increases from 86.7 MB / s (value size of 16 bytes) to 488.5 MB / s (value size of 256 bytes). Compared with NoveLSM and SLM-DB, the average throughput of CacheKV is 5.1 times and 20.2 times higher, respectively; compared with NoveLSM-cache and SLM-DB-cache, the average throughput of CacheKV is 3.4 times and 7.8 times higher, respectively.

[0085] 2. System read performance test:

[0086] The read performance test uses the DB_Benche tool to test and evaluate the system throughput of the application and the rest of the comparative systems in the sequential read and random read scenarios under different key-value pair configurations, and the test results are shown in Figure 7 Compared with NoveLSM and NoveLSM-cache, the average throughput of CacheKV is 3.7% lower than them. Compared with SLM-DB and SLM-DB-cache, the average throughput of CacheKV is 1.4 times higher.

[0087] 3. Real load simulation performance test:

[0088] The real load simulation performance test uses the YCSB tool to simulate six different configurations of real load scenarios, and the detailed settings of each scenario are shown in Figure 8 The value size of the test key-value pair is 64 bytes, the number of test key-value pairs is five million, the performance index is the number of requests processed per second, and the test results are shown in Figure 9 Under all scenarios, the throughput of CacheKV is 42.9% higher than NoveLSM and NoveLSM-cache on average, and 8.9 times higher than SLM-DB and SLM-DB-cache on average.

[0089] 4. System performance test under different concurrent scenarios:

[0090] The test adjusts the concurrency level by configuring different user thread numbers, tests the random write throughput of CacheKV under different key-value pair value sizes and different user thread numbers, and the test results are shown in Figure 10As shown in the figure. In the configuration with a value less than 512 bytes, as the number of user threads increases, the system throughput increases; in the configuration with a value greater than or equal to 512 bytes, as the number of user threads increases, the system throughput first increases and then decreases. The test results prove that the system can adapt to different concurrent load scenarios.

[0091] 5. System performance test in different background write-back thread quantity scenarios:

[0092] The number of background write-back threads is tested, and the random write throughput of CacheKV under different user thread numbers and different background write-back thread quantities is tested, the value size of the key-value pair is 64 bytes, and the test results are as shown in the figure. Figure 11 As the number of background write-back threads increases, the write throughput of the system first increases and then remains stable. In the scenario with 2 user threads, as the number of background write-back threads increases from 1 to 4, the write throughput of the system increases by 97.4%; as the number of background write-back threads increases from 4 to 6, the write throughput of the system increases by 1.7%.

[0093] The application is based on persistent memory equipped with persistent CPU cache, and proposes a high-concurrency and high-performance key-value storage system design. The existing key-value storage system based on persistent memory does not consider persistent CPU cache or system concurrency performance, the existing key-value storage system cannot fully utilize the persistent CPU cache, and there is a problem of poor system concurrency performance. The application adds the persistent CPU cache to the persistent storage stack, organizes a high-concurrency memory pool table on it, and optimizes the system read-write process.

Claims

1. A high-performance key-value storage method based on persistent CPU cache, characterized by comprising the following steps: Step 1. Create, allocate, and flexibly adjust a CPU cache-based memory table pool. 1.1 Allocate a space in the persistent memory as a memory table pool and map it to the lowest level of the CPU cache to prevent it from being replaced; 1.2 Initialize the memory table pool and divide it into multiple memory tables of the same size; 1.3 Allocate a memory table to the processor core that executes the write request. One memory table is exclusively used by one core. 1.4 When the memory table is full, it is marked as an unmodifiable memory table, and another free memory table will be allocated to the processor core; 1.5 When the number of memory tables cannot meet the concurrency of the workload, the capacity of each memory table will be reduced and the number of memory tables will be increased. When there are many idle memory tables, the capacity of the memory tables will be increased and the number of memory tables will be reduced to achieve flexible adjustment. Step 2. Delayed update mechanism of memory table index structure; 2.1 Initialize the background worker thread for background asynchronous batch execution of index structure updates; 2.2 The background worker thread first checks whether the memory table index structure has been created. If not, it creates a new index structure for it in the memory. It then performs synchronization operations between the memory table and the index structure, parses the key-value pairs of the memory table, adds them to the corresponding index structure, and synchronizes the data between the memory table and the index structure. 2.3 Before executing a read request, a background thread will be triggered to update the index structure to ensure the correct execution of the read request; 2.4 When the number of key-value pairs written to the memory table reaches the set threshold, the background thread will be triggered to perform batch synchronization work, and the counter will be reset to zero and start writing again; Step 3. Copy-based memory table write-back mechanism; 3.1 Initialize the background write-back thread for background memory table write-back work; 3.2 The background write-back thread loop checks the memory table status to obtain the unmodifiable memory table in the memory table pool; 3.3 Allocate a space in persistent memory with the same capacity as the memtable. Use memory copy that bypasses the CPU cache to copy the memtable data to this space. 3.4 The background write-back thread parses the key-value pairs in the unmodifiable memory table, synchronizes the memory table with the index structure, and modifies the index address information to point to the space requested in step 3.3; 3.5 Mark the memory table as free, indicating that it can be allocated to the processor core again; Step 4. Memory table index merge mechanism; 4.1 Initialize the background worker thread for memory table index merging; 4.2 Create a global index structure in memory to store valid memory table index nodes; 4.3 Merge the memory table indexes into the global index one by one, and eliminate invalid index nodes during the merge.

2. A high-performance key-value storage system based on persistent CPU cache, characterized by The high-performance key-value storage method based on persistent CPU cache as claimed in claim 1 is adopted. The system includes a memory pool table module based on persistent CPU cache, a memory table index structure delayed update module, a copy-based memory table write-back module, and a memory table index merge module. Each module is connected in sequence. Specifically: The memory pool table module based on the persistent CPU cache is used to allocate and map memory tables to processor cores. The memory pool table module based on the persistent CPU cache establishes a memory table pool on the lowest cache layer of the persistent CPU cache, which includes multiple memory tables of the same size. Each memory table is allocated to a processor core on demand, and the mapping relationship between the memory table and the processor core is recorded by the global metadata structure. The memory table index structure delayed update module is used for updating and synchronizing the memory table index structure; When a key-value pair is written to the memory table, the corresponding node must also be inserted into the index structure of the memory table to ensure the correctness of the search; The copy-based memory table write-back module is used to write the unmodifiable memory table in the CPU cache back to the persistent memory to release the unmodifiable memory table resources; This module uses a cache-bypassing memory copy function to write back the memory table to avoid cache pollution caused by memory copy; The memory table index merging module is used to merge the memory table written back to the persistent memory and remove invalid or failed nodes to improve the system search efficiency.

Images