Read and merge method for insert operation and load of lsm tree storage system

Abstract

The invention provides an append operation method for an LSM tree memory system. The append operation method comprises following steps of: 1), constructing value to be appended and a key to be appended to be in a key-value structural body and utilizing a newly-constructed key-value structural body as latest data segments of the key to be appended for storage in a database; saving aggregate data amount of accumulated data segments before the key to be appended as for the newly-constructed key-value structural body and saving a storage position of a data segment before the key to be appended; and 2), updating table items of the key to be appended in the append table which records aggregate data amount of all data segments of all keys and storage positions of latest data segments. The invention further provides a reading and merging method for corresponding loads of append operation. The append operation method for the LSM tree memory system and the reading and merging method for loads of append operation have following beneficial effects: increase in system cost is avoided due to a read-write amplification problem of append operation; during the course of moving from a lower level to a higher level, loads of append operation are aggregated; and reading performance of loads of append operation can be improved.

Description

technical field [0001] The present invention relates to the field of information storage, in particular, the present invention relates to an insertion (ie Append) operation for an LSM tree (ie LSM Tree) storage system and a method for reading and merging corresponding loads. Background technique [0002] The LSM Tree storage system is a storage system optimized for the performance impact of random I / O on persistent storage. The full name of LSM Tree is Log-Structured Merge-Tree, figure 1 The structure and working principle of the existing LSM Tree storage system are shown, and its main design is: [0003] 1. The data is placed in layers, where the data at the lowest level is located in memory, and the data at other levels is located in persistent devices (HDD / SSD, etc.). [0004] 2. The data is updated in different places, and the data is written in log mode. [0005] 3. The background process of the system continuously merges the data from the lower level to the higher l...

AI Technical Summary

Problems solved by technology

However, in the LSM Tree storage system, when inserting data, it is necessary to read, concatenate, and write back the value in the K-V structure, and because the background process will continue to move the data from the lower layer to the upper layer, the insertion The number of read / write operations will actually be amplified, which may cause a significant increase in system overhead and affect storage performance

Images