Unlock instant, AI-driven research and patent intelligence for your innovation.

Data writing and reading method

A data writing and data reading technology, applied in the field of information storage, can solve problems such as large consumption of horizontal network bandwidth, large consumption of resources such as memory, and inability to flexibly change the number of hard disks for management, so as to avoid cluster repair problems.

Active Publication Date: 2020-11-10
NEW H3C TECH CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0013] 1) Too many OSDs consume too much resources such as CPU and memory;
[0014] 2) The performance of a single OSD is low. When a hard disk fails, it usually needs to restore data across servers, which requires a large amount of horizontal network bandwidth loss and affects the normal business IO of customers.
[0020] However, the existing RAID technology is inconsistent with CEPH's software-defined purpose (using general-purpose standard hardware without additional settings; flexible configuration, which can add or delete hard disks and nodes at will; define different storage forms through software), such as : The RAID group manages a fixed number of hard disks (usually within 10). After the RAID is configured, the number of managed hard disks cannot be changed flexibly

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data writing and reading method
  • Data writing and reading method
  • Data writing and reading method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0084] An embodiment of the present disclosure provides a data writing method, which is applied to a storage system composed of multiple OSDs, see figure 2 As shown, the method includes the following steps:

[0085] S101: Obtain a placement group to which target data belongs and a redundancy policy corresponding to the placement group.

[0086] S102: Select one or more required OSDs for the placement group through the CRUSH algorithm; each OSD includes multiple object storage modules.

[0087] S103: Select an object storage module required by the placement group from one or more OSDs.

[0088] S104: Process the target data according to the redundancy strategy to obtain a processing result.

[0089] S105: Distributing the processing results to corresponding object storage modules for writing operations.

[0090] Wherein, the target data is data to be written, and the redundancy strategy includes: at least one of EC redundancy, copy redundancy, and stripe redundancy. Before...

Embodiment 2

[0117] An embodiment of the present disclosure provides a data reading method, which is applied to a storage system composed of multiple OSDs, see Figure 7 As shown, the method includes the following steps:

[0118] S201: Obtain a placement group to which the target data belongs and a redundancy policy corresponding to the placement group.

[0119] S202: Select one or more required OSDs for the placement group through the CRUSH algorithm; each OSD includes multiple object storage modules.

[0120] S203: Select an object storage module required by the placement group from one or more OSDs.

[0121] S204: Read the required data from the required object storage module.

[0122] After reading the required data from the required object storage, it also includes: if the reading of the data fails, performing data recovery according to the redundancy strategy.

[0123] In the actual implementation, obtain the PG to which the target data belongs, and obtain the redundancy policy co...

Embodiment 3

[0126] An embodiment of the present disclosure provides a data writing device, which is applied to a storage system composed of multiple OSDs, see Figure 8 As shown, the device includes: an information acquisition module 31 , a first selection module 32 , a second selection module 33 , a data processing module 34 , and a data writing module 35 .

[0127] Among them, the information acquisition module 31 is used to obtain the placement group to which the target data belongs and the redundancy policy corresponding to the placement group; the first selection module 32 is used to select the required one or one for the placement group through the CRUSH algorithm. The above OSDs; each OSD includes a plurality of object storage modules; the second selection module 33 is used to select the object storage modules required by the placement group from one or more OSDs; the data processing module 34 is used to convert the target data Processing is performed according to the redundancy st...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a data writing and reading method, and relates to the technical field of information storage. The data writing method is applied in a storage system composed of multiple OSDs, and a placement group to which target data belongs and a redundancy policy corresponding to the placement group are obtained; one or more OSDs needed by the placement group are selected through a CRUSHalgorithm; each OSD comprises multiple object storage modules; object storage modules needed by the placement group are selected from one or more OSDs; the target data is processed according to the redundancy policy to obtain processed results; the processed results are distributed to the corresponding object storage modules to conduct writing process. According to the method, by adjusting multiple object storage modules arranged in the OSDs, hard disks managed by each OSD can be flexibly and dynamically added or deleted, and moreover, the OSDs internally perform self-repair through the redundancy policy to avoid the problem of clustering repair caused by single hard disk failure.

Description

technical field [0001] The present disclosure relates to the technical field of information storage, in particular to a data writing and reading method. Background technique [0002] The CEPH distributed file system is one of the most popular distributed storage software at present. It can run on common commercial hardware and build a storage cluster with self-healing function. The core component of CEPH is OSD (Object Storage Device, object storage component). OSD manages an independent hard disk and provides a read and write access interface for object storage (Object-basedStorage). A CEPH cluster consists of many independent OSDs, and the number of OSDs can be dynamically added or deleted. The CEPH client distributes Objects to different OSDs for storage through the CRUSH algorithm. [0003] CRUSH is a pseudo-random distribution algorithm. This algorithm first assigns Objects to a PG (Placement Group) through HASH, and then calculates the OSD stored in the PG. Therefore...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/182G06F16/13G06F16/18G06F11/14
CPCG06F11/1448G06F16/13G06F16/182G06F16/1847
Inventor 陈钊
Owner NEW H3C TECH CO LTD