A data storage device, method and distributed data storage system

Abstract

A data storage device with high consistency, including: a read-write control unit, a redirection cache unit, and a storage unit; the read-write control unit includes a data writing management module, a metadata management module, a log management module, a metadata database, Metadata cache module, log storage module; redirection cache unit includes cache management module, SSD cache disk, index cache module, index storage module; data write management module receives data write request; cache management module writes in cache space The data in the request; if the data in the write request is written for the first time, mark the data as normal cache data; redirect the overwrite; combine the cache map and the metadata related to this write request to form a log data, The log management module writes the log data into the log storage module, and then updates the metadata to the metadata cache module, and writing the log to the log storage module is an atomic operation.

Description

technical field [0001] This application belongs to the technical field of software research and development, and in particular relates to a data storage device, method and distributed data storage system. The data storage device of the present invention can be used as an independent storage device, or as a device connected to the Internet, or as a distributed A storage node in a standard data storage system. Background technique [0002] The Key-Value Store has become a critical design building block for today's large-scale, high-performance data-sensitive applications. Such as e-commerce platform, data deduplication center, image storage and web object cache, etc. [0003] Ceph: Sage Weil of the University of California, Santa Cruz, a new generation of free software distributed file system designed for a doctoral dissertation. [0004] Data consistency: In a distributed environment, when there are multiple copies of data, if the network, server, or software fails, some co...

AI Technical Summary

Problems solved by technology

Disadvantages of FileStore: FileStore uses logs to ensure the transactionality of write requests. Although it can guarantee the transactionality of write requests, it has a double write amplification effect on IO.

However, variable length space management not only leads to larger metadata overhead, but also in continuous coverage scenarios, because redirecting writes will cause the space in an object to become more and more discrete, which will affect the performance of reading

Images