Key value storage method and system based on nonvolatile memory and solid state disk

Abstract

The invention discloses a key value storage method and system based on a nonvolatile memory and a solid state disk, and belongs to a key value storage system.The key value storage method comprises the steps that key value pairs are divided into large key value pairs and small key value pairs based on the size of storage space occupied by the key value pairs, the small key value pairs are stored in Smem and Simmm in an NVM and Skv in an SSD, and the large key value pairs are stored in Bmem and Bimmm in the NVM and Bkv in the SSD; small key value pairs are stored in order, and large key value pairs are not required to be stored in order; the key value pairs are firstly stored in Smem or Bmem in the NVM, and when the corresponding Smem or Bmem is full, the key value pairs are brushed down to the SSD; and maintaining a corresponding position recording structure in the NVM to record the storage position of each key value pair in the NVM or the SSD. According to the method, the characteristics of the NVM and the SSD can be fully utilized, and the key value storage performance is improved.

Description

Technical Field

[0001] The present invention belongs to the field of key-value storage, and more specifically, relates to a key-value storage method and system based on a non-volatile memory and a solid-state hard disk. Background Art

[0002] Key-value storage is a basic component of databases and various data-intensive applications. It is widely used in many fields such as web retrieval, e-commerce, cloud storage, social networks, etc., and has therefore received extensive attention and research from academia and industry. Existing key-value storage systems, such as leveldb and rocksdb, are mainly designed for disk and solid-state drive optimization. Specifically, since the sequential read and write performance of disks is better than random read and write performance, leveldb and rocksdb first write key-value pairs into memory, and then write the key-value pairs in memory to disk in batches in order to make full use of the characteristics of disks. At the same time, traditional storage solutions store key-value pairs in layers, and the key values ​​in the upper layer are newer than the key-value pairs in the lower layer. In the process of searching for key-value pairs, you need to search in the upper layer first. If you can't find it, you need to search in the lower layer until the last layer. In addition, with the use of key-value pairs, invalid key-value pairs will be generated in the key-value storage system. When the invalid key values ​​exceed a certain amount, a compaction operation needs to be performed. When performing the compaction operation, the key-value pairs in the disk need to be read into memory, the invalid key-value pairs need to be deleted and sorted, and then written to the disk again. This process will cause serious write amplification. Therefore, traditional key-value storage solutions have poor search performance.

[0003] DRAM technology is facing major challenges in terms of storage density and energy consumption. In order to overcome the shortcomings of DRAM, researchers have begun to study non-volatile memory technology (NVM), such as phase change memory and resistive random access memory. In general, NVM has the characteristics of high storage density, low static power consumption, no data loss during power failure, byte addressing, and low read and write latency. These characteristics make NVM likely to replace DRAM in the near future. With the continuous development and maturity of solid-state drives (SSDs), SSDs have been deployed on a large scale in computer systems. Similar to disks, SSD sequential write performance is better than random write performance. Unlike disks, SSD random read (multi-threaded) performance and sequential read performance are not much different. Therefore, hybrid storage devices composed of NVM and SSD can greatly improve the performance of storage systems.

[0004] However, existing key-value storage solutions are mainly designed for disk and solid-state drive optimization. When they are deployed on hybrid storage devices composed of NVM and SSD, they cannot fully utilize the characteristics of NVM and SSD, and may even lead to performance degradation. Summary of the invention

[0005] In view of the defects and improvement needs of the prior art, the present invention provides a key-value storage method and system based on non-volatile memory and solid-state drive, the purpose of which is to make full use of the characteristics of NVM and SSD to improve the key-value storage performance.

[0006] To achieve the above object, according to one aspect of the present invention, a key-value storage method based on a non-volatile memory and a solid state drive is provided, comprising:

[0007] Pre-allocate a memory block S_mem and a storage block B_mem in the non-volatile memory, respectively for storing small key-value pairs and large key-value pairs, and create a location record structure for recording the storage location of each key-value pair in the non-volatile memory or solid-state hard disk; the storage space occupied by the small key-value pair is less than a preset threshold, and the storage space occupied by the large key-value pair is not less than the preset threshold;

[0008] When an insert operation is received from the upper layer, the operation execution methods include:

[0009] (I1) For the key-value pair kv to be inserted i , determine its type, if it is a small key-value pair data, then transfer to (I2); otherwise, transfer to (I4);

[0010] (I2) Determine whether the current memory block S_mem is full. If so, convert the memory block S_mem into a memory block S_imm, flush it to the solid state drive, and reallocate the memory block S_mem in the non-volatile memory, and then proceed to (I3); if not, directly proceed to (I3);

[0011] (I3) The key-value pair kv i Insert the current memory block S_mem in order and transfer to (I6);

[0012] (I4) Determine whether the current memory block B_mem is full. If so, convert the memory block B_mem into the memory block B_imm, flush it to the solid state drive, and fully allocate the memory block B_mem in the non-volatile memory, and then proceed to (I5); if not, directly proceed to (I5);

[0013] (I5) The key-value pair kv i Insert the current memory block B_mem and transfer to (I6);

[0014] (I6) Record the key-value pair kv in the location record structure i The storage location of the insert operation is completed.

[0015] Furthermore, when a range query operation is received from an upper layer, the operation execution method includes:

[0016] For the key range to be queried, the position record structure is searched to obtain the storage location of each key-value pair data within the key range, and the corresponding key-value pairs are read in parallel according to the storage locations, and then merged and sorted.

[0017] Furthermore, the location record structure is a B+ tree, a skip list or a red-black tree.

[0018] Furthermore, when a delete operation is received from an upper layer, the operation execution method includes:

[0019] For the key-value pair kv to be deleted d , delete its storage location from the location record structure, and the key-value pair kv d Mark as invalid.

[0020] Further, the key-value pairs in each memory block S_imm are flushed to a data block S_kv in the solid state drive, and the key-value pairs in each memory block B_imm are flushed to a data block B_kv in the solid state drive;

[0021] Furthermore, when a compaction operation is received from the upper layer, the operation execution methods include:

[0022] For the data block S_kv to be operated, perform the following steps:

[0023] (C1) reading all key-value pairs in the data block S_kv to be operated into the volatile memory, and deleting the invalid key-value pairs therein;

[0024] (C2) after sorting the remaining key-value pairs according to the size of the key, the key-value pairs are divided into data blocks according to the size of the data block S_kv;

[0025] (C3) writing the divided data blocks in parallel into the data block S_kv in the solid state drive, and deleting the original data block S_kv;

[0026] For the data block B_kv to be operated, perform the following steps:

[0027] (C1') reading all the key-value pairs in the data block B_kv to be operated into the volatile memory, and deleting the invalid key-value pairs therein;

[0028] (C2') dividing the remaining key-value pairs into data blocks according to the size of the data block B_kv;

[0029] (C3′) The divided data blocks are written in parallel to the data blocks B_kv in the solid state drive, and the original data blocks B_kv are deleted.

[0030] Furthermore, when an update operation is received from an upper layer, the operation execution method includes:

[0031] The original key-value pair is deleted through the delete operation, and then a new key-value pair is inserted through the insert operation.

[0032] Furthermore, when a single-point query operation is received from an upper layer, the operation execution method includes:

[0033] For the key-value pair kv to be queried s , query its storage location from the location record structure;

[0034] Read the key-value pair kv according to the storage location s .

[0035] According to another aspect of the present invention, there is provided a key-value storage controller based on a nonvolatile memory and a solid-state hard disk, comprising: a computer-readable storage medium and a processor;

[0036] The computer readable storage medium is used to store the computer program;

[0037] The processor is used to read the computer program stored in the computer-readable storage medium and execute the key-value storage method based on the non-volatile memory and the solid-state hard disk provided by the present invention.

[0038] According to another aspect of the present invention, a key-value storage system is provided, comprising: a non-volatile memory, a solid-state drive, and the key-value storage controller based on the non-volatile memory and the solid-state drive provided by the present invention.

[0039] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0040] The present invention can make full use of the performance of NVM and SSD, and effectively improve the key-value storage performance. Specifically, the present invention divides key-value pairs into large key-value pairs and small key-value pairs according to the size of the occupied storage space. For small key-value pairs, orderly storage is performed, and for large key-value pairs, orderly storage is not required; when performing a range query operation, it is necessary to randomly read multiple key-value pairs. For large key-value pairs, due to the large storage space occupied and parallel reading, the corresponding reading is close to sequential reading. For small key-value pairs, due to the orderly storage, multiple key-value pairs to be read are stored in adjacent storage spaces, so that performance close to sequential reading can also be obtained. Therefore, the present invention can make full use of the characteristics of SSD's multi-threaded random read performance and sequential read performance that are not much different, and effectively improve the range query performance. In addition, the present invention does not require large key-value pairs to be stored in an orderly manner, so the number of compaction operations can be effectively reduced, thereby effectively reducing write amplification and improving write performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1An overall architecture diagram of the key-value storage method provided by an embodiment of the present invention;

[0042] Figure 2 A write operation flow chart provided for an embodiment of the present invention;

[0043] Figure 3 A flowchart of a read operation provided by an embodiment of the present invention;

[0044] Figure 4 A flowchart of a compaction operation provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0046] In the present invention, the terms "first", "second", etc. (if any) in the present invention and the accompanying drawings are used to distinguish similar objects but not necessarily to describe a specific order or sequence.

[0047] The overall performance of the key-value storage system is determined by the performance of key-value operations. In the key-value storage system, key-value operations include write operations, read operations, and compaction operations. Among them: read operations include insert operations, update operations, and delete operations. Insert operations insert a key-value pair, update operations update a key-value pair, and delete operations delete a key-value pair; read operations mainly include single-point query operations and range query operations. Single-point queries only need to find a key-value pair, and range queries need to find key-value pairs in a specified range; compaction operations mainly include garbage collection and sorting of key-value pairs.

[0048] In order to fully utilize the performance of NVM and SSD and effectively improve the key-value storage performance when the key-value storage system is deployed in a hybrid storage device composed of NVM and SSD, the present invention provides a key-value storage method and system based on non-volatile memory and solid-state drive, such as Figure 1As shown, in the present invention, the key-value pairs are first stored in the non-volatile memory and then flushed to the solid-state drive with larger capacity; small key-value pairs and large key-value pairs are stored separately, the small key-value pairs are stored in S_mem, S_imm in the NVM and S_kv in the SSD, and the large key-value pairs are stored in B_mem, B_imm in the NVM and B_kv in the SSD; the small key-value pairs are stored in each structure in order, while the large key-value pairs do not need to be stored in order; at the same time, the present invention maintains a corresponding data structure in the NVM for recording the storage location of the key-value pairs in the NVM or SSD.

[0049] In the present invention, large key-value pairs and small key-value pairs are distinguished based on the size of the storage space occupied by the key-value pairs. The key-value pairs that occupy a storage space size less than a preset threshold are small key-value pairs, and the key-value pairs that occupy a storage space size not less than the preset threshold are large key-value pairs. In practical applications, the threshold for distinguishing large key-value pairs from small key-value pairs should be set accordingly according to the read and write performance of the SSD in the hybrid storage device to ensure that when randomly reading large key-value pair data in the SSD, the performance is close to that of sequential reading.

[0050] The location record structure used to record the storage location of the key-value pair can select any data structure. In order to improve the query performance, especially the range query performance, the B+ tree, skip list, red-black tree, etc. that support range query are preferably used. Without loss of generality, in the following embodiments, the B+ tree is used as an example for explanation.

[0051] Based on the above concept, in the present invention, the execution process of the write operation is as follows: Figure 2 As shown, the insert operation first determines the size of the key-value pair. If the key-value pair is a small key-value pair, the key-value pair is inserted into S_mem, otherwise the key-value pair is inserted into B_mem, and finally the storage location of the key-value pair is inserted into the B+ tree. If S_mem (B_mem) is full, S_mem (B_mem) is converted to S_imm (B_imm), and then S_imm (B_imm) is flushed to the SSD. The delete operation The location of the key-value pair stored in the B+ tree is directly deleted. The update operation first performs the delete operation and then the insert operation.

[0052] The execution flow of the read operation is as follows Figure 3 As shown in the figure, a single-point query finds the location of the key-value pair in the B+ tree, and then reads the key-value pair at the specified location. A range query first finds the location of the key-value pair in a given range in the B+ tree, then reads the key-value pairs in parallel based on the found location, and finally merges and sorts the read results.

[0053] The execution process of compaction operation is as follows Figure 4As shown in the figure, after the compaction operation clears the invalid key-value pairs in S_kv, it is also necessary to ensure that the key-value pairs in S_kv are in order. After the compaction operation clears the invalid key-value pairs in B_kv, it is not necessary to ensure that the key-value pairs in B_kv are in order.

[0054] The following are examples.

[0055] Embodiment 1:

[0056] A key-value storage method based on non-volatile memory and solid-state drive, such as Figure 1 As shown, including:

[0057] Pre-allocate a memory block S_mem and a storage block B_mem in the non-volatile memory, respectively for storing small key-value pairs and large key-value pairs, and create a location record structure for recording the storage location of each key-value pair in the non-volatile memory or solid-state hard disk; the storage space occupied by the small key-value pair is less than a preset threshold, and the storage space occupied by the large key-value pair is not less than the preset threshold;

[0058] like Figure 2 As shown, when an insert operation is received from an upper layer, the operation execution method includes:

[0059] (I1) For the key-value pair kv to be inserted i , determine its type, if it is a small key-value pair data, then transfer to (I2); otherwise, transfer to (I4);

[0060] (I2) Determine whether the current memory block S_mem is full. If so, convert the memory block S_mem into a memory block S_imm, flush it to the solid state drive, and reallocate the memory block S_mem in the non-volatile memory, and then proceed to (I3); if not, directly proceed to (I3);

[0061] In this embodiment, after the memory block S_mem is converted into the memory block S_imm, the memory block S_imm no longer receives key-value pair data;

[0062] (I3) The key-value pair kv i Insert the current memory block S_mem in order and transfer to (I6);

[0063] (I4) Determine whether the current memory block B_mem is full. If so, convert the memory block B_mem into the memory block B_imm, flush it to the solid state drive, and fully allocate the memory block B_mem in the non-volatile memory, and then proceed to (I5); if not, directly proceed to (I5);

[0064] In this embodiment, after the memory block B_mem is converted into the memory block B_imm, the memory block B_imm no longer receives key-value pair data;

[0065] (I5) The key-value pair kv i Insert the current memory block B_mem and transfer to (I6);

[0066] (I6) Record the key-value pair kv in the location record structure i The storage location of the insert operation is completed.

[0067] It is easy to understand that after the memory block S_imm or B_imm is flushed to the SSD, the storage location of each key-value pair in the B+ tree will be updated accordingly.

[0068] like Figure 2 As shown, when a delete operation is received from an upper layer, the operation execution method includes:

[0069] For the key-value pair kv to be deleted d , delete its storage location from the B+ tree and set the key-value pair kv d Mark as invalid.

[0070] In actual applications, for each newly inserted key-value pair, the corresponding version number can be written at the same time. In subsequent operations, whether the key-value pair is valid can be determined based on the version number.

[0071] like Figure 2 As shown, when an update operation is received from the upper layer, the operation execution method includes:

[0072] The original key-value pair is deleted through the delete operation, and then a new key-value pair is inserted through the insert operation.

[0073] like Figure 3 As shown, when a single-point query operation is received from the upper layer, the operation execution method includes:

[0074] For the key-value pair kv to be queried s , query its storage location from the location record structure;

[0075] Read the key-value pair kv according to the storage location s .

[0076] like Figure 3 As shown, when a range query operation is received from an upper layer, the operation execution method includes:

[0077] For the key range to be queried, the position record structure is searched to obtain the storage location of each key-value pair data within the key range, and the corresponding key-value pairs are read in parallel according to the storage locations, and then merged and sorted.

[0078] like Figure 4As shown in the figure, when a compaction operation is received from the upper layer, the operation execution methods include:

[0079] For the data block S_kv to be operated, perform the following steps:

[0080] (C1) reading all key-value pairs in the data block S_kv to be operated into the volatile memory, and deleting the invalid key-value pairs therein;

[0081] (C2) after sorting the remaining key-value pairs according to the size of the key, the key-value pairs are divided into data blocks according to the size of the data block S_kv;

[0082] (C3) writing the divided data blocks in parallel into the data block S_kv in the solid state drive, and deleting the original data block S_kv;

[0083] For the data block B_kv to be operated, perform the following steps:

[0084] (C1') reading all the key-value pairs in the data block B_kv to be operated into the volatile memory, and deleting the invalid key-value pairs therein;

[0085] (C2') dividing the remaining key-value pairs into data blocks according to the size of the data block B_kv;

[0086] (C3′) The divided data blocks are written in parallel to the data blocks B_kv in the solid state drive, and the original data blocks B_kv are deleted.

[0087] In general, since this embodiment stores large key-value pairs and small key-value pairs separately, and small key-value pairs are stored in order, while large key-value pairs do not require ordered storage, the range query operation performance and write operation performance of the key-value query system are effectively improved, thereby improving the overall performance of the key-value storage system.

[0088] Embodiment 2:

[0089] A key-value storage controller based on non-volatile memory and solid-state hard disk includes: a computer-readable storage medium and a processor;

[0090] The computer readable storage medium is used to store the computer program;

[0091] The processor is used to read the computer program stored in the computer-readable storage medium and execute the key-value storage method based on the non-volatile memory and the solid-state hard disk provided in the above-mentioned embodiment 1.

[0092] Embodiment 3:

[0093] A key-value storage system includes: a non-volatile memory, a solid-state hard disk, and a key-value storage controller based on the non-volatile memory and the solid-state hard disk provided in the above-mentioned embodiment 2.

[0094] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A key-value storage method based on non-volatile memory and solid-state hard disk, characterized in that: include: A memory block S_mem and a storage block B_mem are pre-allocated in the non-volatile memory, respectively for storing small key-value pairs and large key-value pairs, and a location record structure is created to record the storage location of each key-value pair in the non-volatile memory or the solid-state hard disk; the storage space occupied by the small key-value pair is less than a preset threshold, and the storage space occupied by the large key-value pair is not less than the preset threshold; When an insert operation is received from the upper layer, the operation execution methods include: (I1) For the key-value pair kv to be inserted i , determine its type, if it is a small key-value pair data, then transfer to (I2); otherwise, transfer to (I4); (I2) determining whether the current memory block S_mem is full, if so, converting the memory block S_mem into a memory block S_imm, flushing it to the solid state drive, and reallocating the memory block S_mem in the non-volatile memory, and transferring to (I3); if not, directly transferring to (I3); (I3) The key-value pair kv i Insert the current memory block S_mem in order and transfer to (I6); (I4) determining whether the current memory block B_mem is full, if so, converting the memory block B_mem into a memory block B_imm, flushing it to the solid state drive, and fully allocating the memory block B_mem in the non-volatile memory, and then proceeding to (I5); if not, directly proceeding to (I5); (I5) The key-value pair kv i Insert the current memory block B_mem and transfer to (I6); (I6) Record the key-value pair kv in the location record structure i The storage location of the insert operation is completed.

2. The key-value storage method based on non-volatile memory and solid-state hard disk according to claim 1, characterized in that: When a range query operation is received from the upper layer, the operation execution methods include: For the key range to be queried, the position record structure is searched to obtain the storage location of each key-value pair data within the key range, and corresponding key-value pairs are read in parallel according to the storage locations, and then merged and sorted.

3. The key-value storage method based on non-volatile memory and solid-state hard disk according to claim 2, characterized in that: The location record structure is a B+ tree, a skip list or a red-black tree.

4. The key-value storage method based on non-volatile memory and solid state drive according to any one of claims 1 to 3, characterized in that: When a delete operation is received from the upper layer, the operation execution methods include: For the key-value pair kv to be deleted d , delete its storage location from the location record structure, and replace the key-value pair kv d Mark as invalid.

5. The key-value storage method based on non-volatile memory and solid-state hard disk according to claim 4, characterized in that: The key-value pairs in each memory block S_imm are flushed to a data block S_kv in the solid state drive, and the key-value pairs in each memory block B_imm are flushed to a data block B_kv in the solid state drive; Furthermore, when a compaction operation is received from the upper layer, the operation execution methods include: For the data block S_kv to be operated, perform the following steps: (C1) reading all key-value pairs in the data block S_kv to be operated into the volatile memory, and deleting the invalid key-value pairs therein; (C2) after sorting the remaining key-value pairs according to the size of the key, the key-value pairs are divided into data blocks according to the size of the data block S_kv; (C3) writing the divided data blocks in parallel into the data blocks S_kv in the solid state drive, and deleting the original data blocks S_kv; For the data block B_kv to be operated, perform the following steps: (C1') reading all the key-value pairs in the data block B_kv to be operated into the volatile memory, and deleting the invalid key-value pairs therein; (C2') dividing the remaining key-value pairs into data blocks according to the size of the data block B_kv; (C3′) writing the divided data blocks in parallel into the data blocks B_kv in the solid state drive, and deleting the original data blocks B_kv.

6. The key-value storage method based on non-volatile memory and solid state drive according to claim 4, characterized in that: When an update operation is received from the upper layer, the operation execution methods include: The original key-value pair is deleted through the delete operation, and then a new key-value pair is inserted through the insert operation.

7. The key-value storage method based on non-volatile memory and solid state drive according to any one of claims 1 to 3, characterized in that: When a single-point query operation is received from the upper layer, the operation execution methods include: For the key-value pair kv to be queried s , querying its storage location from the location record structure; Read the key-value pair kv according to the storage location s .

8. A key-value storage controller based on non-volatile memory and solid-state hard disk, characterized in that: include: A computer readable storage medium and a processor; The computer readable storage medium is used to store a computer program; The processor is used to read the computer program stored in the computer-readable storage medium and execute the key-value storage method based on non-volatile memory and solid-state hard disk according to any one of claims 1 to 7.

9. A key-value storage system, characterized in that: include: A non-volatile memory, a solid-state hard disk, and a key-value storage controller based on the non-volatile memory and the solid-state hard disk as described in claim 8.

Images