Data storage method and related device in a key-value pair system

Abstract

An embodiment of the invention discloses a storage method for data in a Key-Value system and a related device. By means of the method and the device, the storage operation efficiency of Value data can be improved. The method comprises steps as follows: whether the data volume of Value data in the Key-Value exceeds the threshold value of data is judged; if the data volume of Value data doesn't exceed the threshold value of data, the Value data is sliced, and M slice contents are obtained; slice information is generated for N slices according to the M slice contents and comprises the number of Value data slices, offset addresses of slice contents in the N slice contents, serial numbers ID of the slice contents in the N slice contents and (M-N) slice contents; and Key data and slice information are stored in an LSM-Tree (the log-structured merge-tree), the N slice contents are stored in a Key-Value database, and the Key data correspond to the slice contents.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a data storage method and a related device in a key-value pair system. Background technique [0002] LSM-Tree (The Log-Structured Merge-Tree, a log-structured merge tree) is an index mechanism introduced for frequently updating Value (such as frequent insertion or deletion) in a key-value pair (K-V, Key-Value) storage system , can be applied to scenarios where the frequency of querying Value is much lower than the frequency of updating Value, such as history tables and log files. [0003] The implementation method of LSM-Tree is: save the update of the Value data in the memory, and write these updates to the disk in batches when the Value stored in the memory reaches the specified threshold. Such as Figure 1-a with Figure 1-b As shown, specifically, the LSM-Tree passes through two layers of ordered datasets (C 0 tree and C 1 tree) or a multi-level ordered dataset (C 0 tr...

AI Technical Summary

Problems solved by technology

[0004] In the existing key-value pair storage system, the index data structure of the LSM-Tree described above is usually used, but in the existing LSM-Tree, a C i The tree (i is any value less than k) will bring the overhead of input and output (I / O, Input / Output) ports when updating the Value data. When the data volume of a Value data is large, the Value with a large amount of data Data needs to be moved between data sets at different levels during the rolling merge process, which will generate a large I / O port overhead, which will inevitably reduce the efficiency of storage and operation of the overall Value data

Images