Key value storage system based on host management tile recording disk and data processing method

Abstract

The invention discloses a key value storage system based on a host management tile record disk and a data processing method, and belongs to the technical field of information storage. The system comprises an LSM tree key value storage module used for receiving a request of a user and a space management module; each layer of space of the LSM tree key value storage module is composed of partitions of a plurality of host management tile recording disks, and data of each layer is strictly separated; and the space management module is used for connecting the LSM tree key value storage module and the host management tile recording disk and is responsible for space allocation and recovery of the host management tile recording disk. According to the invention, random writing is converted into sequential writing; characteristics of a shingled record disk are met, each layer of data of the LSM tree key value storage system is separately stored in the disk, due to the fact that the data flow speed of each layer is different, expenditure of garbage collection is effectively reduced, disk space management combines zone allocation, space allocation and hierarchical storage, and the data of eachlayer is more centralized.

Description

technical field [0001] The invention belongs to the technical field of information storage, and more specifically relates to a key-value storage system and a data processing method based on a host-managed tile recording disk. Background technique [0002] With the continuous development of Internet technology, the amount of data generated by people in the process of using the Internet is increasing, the data structure is becoming more and more complex, and the total amount of stored information is increasing rapidly every year. In terms of storage devices, hard disk has always occupied a dominant position because of its large storage capacity, low unit storage cost and mature storage technology. However, due to the limitation of the superparamagnetic effect of the magnetic disk, its storage capacity is limited. In terms of storage systems, with the advent of the era of big data, traditional relational databases are increasingly unable to handle these data. In the face of ma...

AI Technical Summary

Problems solved by technology

[0004] However, in the existing technology, the random write performance of tile recording disks will be poor due to the overlap of their tracks. How to avoid random writing will be one of the hot issues of tile recording disks. Most of the existing tile recording disks use Divided into a series of consecutive partitions of the same size, each partition can only be written sequentially, and the data of the original partition needs to be migrated to the new partition when recycling, which will be a huge overhead, especially for the drive The management tile recording disk, because garbage collection is performed inside the disk, will cause a huge write amplification, while the host-aware tile recording disk has compatibility and has a more complicated structure, which will bring additional overhead. The host management tile recording disk can The garbage collection policy set by the user is the industry's most used watt-record disk. How to reduce the garbage collection overhead of the watt-record disk is a research hotspot.

[0005] Existing LSM tree key-value storage systems are all based on the file system. Directly storing the data of the LSM tree key-value storage system on the host management tile recording disk will result in fragmented data distribution, low space utilization, and garbage collection. The overhead becomes larger, and it is a key issue to correctly store the key-value storage system to the tile record disk

Images