Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Append operation method for LSM tree memory system and reading and merging method for loads of append operation

A technology of insertion operation and storage system, which is applied in the field of information storage, can solve the problems of affecting storage performance, increase of system overhead, amplification of read/write times of insertion operation, etc., to achieve the effect of improving read performance and avoiding increase of system overhead

Active Publication Date: 2015-12-09
中科海微(北京)科技有限公司
View PDF7 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in the LSMTree storage system, when inserting data, it is necessary to read, concatenate, and write back the value in the K-V structure, and because the background process will continue to move the data from the lower layer to the upper layer, the insertion operation The number of read / write times will actually be amplified, which may cause a significant increase in system overhead and affect storage performance

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Append operation method for LSM tree memory system and reading and merging method for loads of append operation
  • Append operation method for LSM tree memory system and reading and merging method for loads of append operation
  • Append operation method for LSM tree memory system and reading and merging method for loads of append operation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0044] According to an embodiment of the present invention, an insertion operation method in an LSMTree storage system is provided. This embodiment performs insert (i.e. Append) operation based on segmented storage technology, and solves the problem that the values ​​of different segments of the same key are related to each other through the transformation of key and the introduction of insert table, thereby realizing the normal function of insert operation while , avoiding the read / write amplification problem of the insert operation in the LSMTree storage system, thereby saving system overhead.

[0045] figure 2 A schematic diagram of the insertion operation principle of this embodiment is shown. like figure 2As shown, in order to improve the performance of inserting operation, set up inserting table in the present embodiment, in the key field of the K-V structure of Append class, increase new item, also set up SST evolution table simultaneously, insert table, Append firs...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an append operation method for an LSM tree memory system. The append operation method comprises following steps of: 1), constructing value to be appended and a key to be appended to be in a key-value structural body and utilizing a newly-constructed key-value structural body as latest data segments of the key to be appended for storage in a database; saving aggregate data amount of accumulated data segments before the key to be appended as for the newly-constructed key-value structural body and saving a storage position of a data segment before the key to be appended; and 2), updating table items of the key to be appended in the append table which records aggregate data amount of all data segments of all keys and storage positions of latest data segments. The invention further provides a reading and merging method for corresponding loads of append operation. The append operation method for the LSM tree memory system and the reading and merging method for loads of append operation have following beneficial effects: increase in system cost is avoided due to a read-write amplification problem of append operation; during the course of moving from a lower level to a higher level, loads of append operation are aggregated; and reading performance of loads of append operation can be improved.

Description

technical field [0001] The present invention relates to the field of information storage, in particular, the present invention relates to an insertion (ie Append) operation for an LSM tree (ie LSMTree) storage system and a method for reading and merging corresponding loads. Background technique [0002] The LSMTree storage system is a storage system optimized for the performance impact of random I / O on persistent storage. The full name of LSMTree is Log-StructuredMerge-Tree, figure 1 The structure and working principle of the existing LSMTree storage system are shown, and its main design is: [0003] 1. The data is placed in layers, where the data at the lowest level is located in memory, and the data at other levels is located in persistent devices (HDD / SSD, etc.). [0004] 2. The data is updated in different places, and the data is written in log mode. [0005] 3. The background process of the system continuously merges the data from the lower level to the higher level,...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
CPCG06F16/2237
Inventor 贾士博岳银亮
Owner 中科海微(北京)科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products