Data writing and reading method

Abstract

The invention provides a data writing and reading method, and relates to the technical field of information storage. The data writing method is applied in a storage system composed of multiple OSDs, and a placement group to which target data belongs and a redundancy policy corresponding to the placement group are obtained; one or more OSDs needed by the placement group are selected through a CRUSHalgorithm; each OSD comprises multiple object storage modules; object storage modules needed by the placement group are selected from one or more OSDs; the target data is processed according to the redundancy policy to obtain processed results; the processed results are distributed to the corresponding object storage modules to conduct writing process. According to the method, by adjusting multiple object storage modules arranged in the OSDs, hard disks managed by each OSD can be flexibly and dynamically added or deleted, and moreover, the OSDs internally perform self-repair through the redundancy policy to avoid the problem of clustering repair caused by single hard disk failure.

Description

Technical field [0001] The present disclosure relates to the field of information storage technology, and in particular to a method for writing and reading data. Background technique [0002] The CEPH distributed file system is currently one of the most popular distributed storage software, which can run on common commercial hardware to build a storage cluster with self-repair functions. The core component of CEPH is OSD (Object Storage Device). OSD manages an independent hard disk and provides read and write access interfaces for Object-based Storage. The CEPH cluster is composed of many independent OSDs, and the number of OSDs can be dynamically added or deleted. The CEPH client distributes objects to different OSDs for storage through the CRUSH algorithm. [0003] CRUSH is a pseudo-random distribution algorithm. The algorithm first assigns Objects to a PG (Placement Group) through HASH, and then calculates the OSD stored by the PG. Therefore, Objects belonging to the same PG a...

AI Technical Summary

Problems solved by technology

[0013] 1) Too many OSDs consume too much resources such as CPU and memory;

[0014] 2) The performance of a single OSD is low. When a hard disk fails, it usually needs to restore data across servers, which requires a large amount of horizontal network bandwidth loss and affects the normal business IO of customers.

[0020] However, the existing RAID technology is inconsistent with CEPH's software-defined purpose (using general-purpose standard hardware without additional settings; flexible configuration, which can add or delete hard disks and nodes at will; define different storage forms through software), such as : The RAID group manages a fixed number of hard disks (usually within 10). After the RAID is configured, the number of managed hard disks cannot be changed flexibly

Images