Method and device for processing data as well as data storage system based on key value data base

Abstract

The invention provides a method and a device for processing data as well as a data storage system based on a key value data base. The method, the device and the system can improve the file access efficiency of the key value data base and can prolong the service life of a magnetic disc. The method comprises the steps as follows: slots corresponding to key names of data items to be stored at present are looked up in an index file according to corresponding relations between preset slots and the key names of the data items; metadata blocks of the data items to be stored at present are stored in the slots; in the corresponding relations, one slot corresponds to key names of a plurality of data items; the number of data items which correspond to one slot is preset, each slot comprises a recording area used for recording addresses of expansion slots connected to the slot; and under the condition that the slot is full, metadata blocks of data items to be stored at present are stored in expansion slots, which are not full, of the slot.

Description

technical field [0001] The invention relates to a data processing method and device based on a key-value database and a data storage system. Background technique [0002] A key-value database is a commonly used database in which data entries are stored in a key-value (Key-Value) manner. In the case of storing massive amounts of data, usually using figure 1 shown in the form of the storage system, figure 1 It is a schematic diagram according to the basic structure of the storage system in the prior art. Such as figure 1 As shown, the storage system 10 is divided into a plurality of storage units 11, 12...1N, and each storage unit includes index files and data files. The metadata blocks of multiple data entries are saved in the index file, and the key values ​​of multiple data entries are saved in the data file. As data entries are saved, the size of the data file also increases. In order to prevent a single data file from being too large, you can set the size of the dat...

AI Technical Summary

Problems solved by technology

In common databases at present, data entry generally all has very big quantity, causes the metadata block that often has multiple data entries to need to be saved on the same position of same index file, thereby above-mentioned skip times is huge, storage device The disk pointer moves frequently, resulting in low file access efficiency of the database and affecting disk life

Images