Method for quickly reading key value storage data based on log structure merging tree

Abstract

The invention relates to a method for quickly reading key value storage data based on a log structure merging tree. A log structure merging tree is constructed through multiple data layers, multiple groups of key value data pairs are accessed to each data layer of the log structure merging tree, the data reading frequency in each data layer of the log structure merging tree is counted and analyzed, the log structure merging tree is divided into a data layer with high reading frequency and a data layer with low reading frequency, and a multi-layer cuckoo hash table application data reading layer is constructed for the data layer with high reading frequency; if the byte number of the keys in the key value data pairs in the data layer is greater than a byte threshold, MD5 encoding is carriedout on the keys and then the keys are stored in a cuckoo hash table; and when data index reading is carried out, firstly the valley hash index is inquired, and if the valley hash index is not hit, thestep of continuing to inquire the low-reading-frequency data layer is carried out. The method has the advantages that a data layer with high access frequency is selected through a heuristic strategy,an additional hash index is constructed for the data layer, and the reading performance is obviously improved by using a limited memory space.

Description

technical field [0001] The invention belongs to the field of computer science storage systems, and in particular relates to a fast reading method for key-value storage based on a log structure merge tree. Background technique [0002] Data-intensive enterprise-level applications often use persistent key-value storage systems for data reading and writing, such as web site crawlers, social networks, and image storage. Persistent key-value storage systems generally provide common operation interfaces, such as writing / updating key-value pairs, reading / point query and scanning (range query) of data to serve various applications. Among many key-value storage systems, key-value storage systems based on log-structured merge tree (LSM-tree) are particularly popular, mainly because they convert intensive random writes into sequential writes to make full use of disk I / O bandwidth. . The key-value storage system based on LSM-tree usually consists of a memory module and a disk module: ...

AI Technical Summary

Problems solved by technology

[0003] Aiming at the read amplification problem existing in the existing LSM-tree-based key-value storage system, the present invention proposes a method for speeding up data reading by using an additional memory index

Images