A key-value data indexing method for workload-adaptive single-layer lsmt

Abstract

The invention discloses a key-value data indexing method of a workload adaptive single-layer LSMT. This method optimizes the traditional Log-Structured-Merge Tree (LSMT), removes the multi-layer design and fixed memory table capacity design, and introduces the design of single-layer LSMT and dynamic capacity memory table. In this method, write operations are first written to the log file on the storage device in a sequential manner, and then the memory table is modified. When the size of the memory table reaches the capacity limit, it is converted into a read-only memory table, and the read-only memory table is merged into the single-layer LSMT structure on the storage device in a background thread. On this basis, the method can automatically optimize the storage structure according to the key-value read-write distribution in the workload. The indexing method can reduce the amplification of reading and writing to the storage device at the same time, and improve the system throughput and the service life of the storage device. At the same time, adaptive optimization is made for the workload to further improve system performance.

Description

technical field [0001] The invention belongs to the technical field of database systems, and in particular relates to a key-value data indexing method for workload adaptive single-layer LSMT. Background technique [0002] The key-value storage system based on Log-Structured-Merge Tree (LSMT) is widely used in data-intensive Internet applications due to its excellent processing-intensive write capability. However, the existing LSMT-based storage systems generally have problems of amplification and workload insensitivity. The amplification problem means that the amount of data that the user requests to read / write is much smaller than the amount of data that the system actually needs to read / write on the storage device, and the amplification factor can be used to quantify this problem. The problem of workload insensitivity refers to the fact that the existing LSMT system cannot optimize the storage structure more appropriately according to the read and write distribution in ...

AI Technical Summary

Problems solved by technology

The problem of workload insensitivity refers to the fact that the existing LSMT system cannot make a more appropriate optimization of the storage structure according to the distribution of reads and writes in the workload

[0003] In order to solve the problem of read and write amplification, researchers have proposed many methods, but these methods generally sacrifice read amplification in exchange for reduced write amplification (such as WiscKey, PebblesDB), and cannot guarantee efficient reading and writing.

As for the problem of unawareness of workload, few researchers have studied and solved it

Images