Data processing method and system for lightweight file system

Abstract

The invention discloses a data processing method and system of a lightweight file system. The data processing method comprises: setting the file system layout mode to the disk header storage label, super block, file data, metadata, data check value, Metadata check value information, the super block backup information is stored in the middle and the end of the disk; when the data in the file is written to the disk, the check value information of the data and metadata is calculated by an algorithm; the file system loading process is set to be calculated first Super block storage location, secondly read the super block information, and finally load the metadata information in the disk; set the writing, creating, deleting and reading methods of files in the file system according to the information in the metadata; the present invention can support the metadata of concurrent files Update, and increase the backup of super block, metadata of each file and data verification information of data to avoid the risk of data inconsistency caused by disk silence.

Description

technical field [0001] The invention relates to the field of storage cloud computing, in particular to a data processing method and system of a lightweight file system. Background technique [0002] The stand-alone storage engine is mainly used to solve how to quickly and efficiently organize data, index data, and store data on physical storage devices such as SSD, HDD, or NVMe, and provide an easy-to-use storage interface for the upper-layer storage system. In addition, it is necessary to consider how to provide superior performance in the storage engine, and how to ensure data recovery and data consistency in the event of a power failure or failure of a physical device. [0003] The Bluefs component in Ceph is a lightweight file system. It implements a lightweight log management module to record all metadata changes. However, it does not implement the Posix standard interface and does not support file overwrite operations. Only supports Append writing, there is no tree hi...

AI Technical Summary

Problems solved by technology

[0003] The Bluefs component in Ceph is a lightweight file system. It implements a lightweight log management module that records all metadata changes. However, it does not implement the Posix standard interface and does not support file overwrite operations. Append write, there is no tree hierarchy of the local file system, only a flat directory-to-file mapping relationship; in addition, it only provides a flat directory-to-file mapping relationship, and does not provide deep directory management; if the disk is silent, it will The resulting data inconsistency, if the disk is damaged, the data in the disk cannot be recovered

Images