Massive data storage method simultaneously applicable to disk and solid state disk reading and writing features

Abstract

The invention provides a massive data storage method simultaneously applicable to disk and solid state disk reading and writing features. Full sequencing of records in each block is changed into partial sequencing, a Bloom filter is added to the tail portion of each block, a Log-Structured Append-Tree is created, when the quantity of data stored in each block in the tree reaches a threshold and data in the block is directly added to corresponding child blocks, the data of the child blocks is composed of multiple collating sequences rather than full sequencing is achieved in the blocks in a merging sorting mode; each block in the tree stores one Bloom filter. According to the method, on the condition that no other properties are sacrificed, write amplification is greatly reduced, and the random writing efficiency is greatly improved. Besides, the service life of a solid state disk is better protected and prolonged. In read and write mixed scenes, the random read property is also enhanced, and the method has important market value.

Description

technical field [0001] The invention belongs to the field of mass data storage, and in particular relates to a storage tree. The method can simultaneously adapt to the reading and writing characteristics of disks and solid state hard disks. Background technique [0002] Commonly used index trees on existing hard disks include B-tree, LSM-tree, buffer-tree, and the like. Among them, B-tree is a traditional classic tree, but because it inevitably randomly writes to disk in random write scenarios, its performance is low when storing massive data, so its variant is often used when storing massive data, such as BigTable for A variant of B-tree is used in combination with LSM-tree. For the storage of massive data, LSM-tree or buffer-tree (also known as fractal-tree) is often used as the index tree. The common feature of the two is to postpone the writing of the records to be written, and wait until a certain amount is accumulated. Batch processing. This can better solve the pro...

AI Technical Summary

Problems solved by technology

[0005] The problem to be solved by the present invention is: the larger write amplification of the traditional tree makes the problem of low random write efficiency, and the larger write amplification in the solid-state hard disk also seriously affects the life of the solid-state hard disk

Images