Unlock instant, AI-driven research and patent intelligence for your innovation.

Main OSD adjusting method and device

An adjustment method and technology for adjustment devices, which are applied in the fields of instrumentation, computing, electrical digital data processing, etc., can solve the problem that the approximate balance cannot be guaranteed, the user data access requests cannot be guaranteed to be approximately evenly distributed, and the overall performance of the Ceph distributed storage system cannot be sufficient. Play and other issues to achieve the effect of overall performance

Active Publication Date: 2019-04-02
NEW H3C TECH CO LTD
View PDF5 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] In the Ceph distributed storage system, the user data to be stored is usually first cut into fixed-length objects, and each object belongs to a placement group (Placement Group, PG). The Ceph distributed storage system uses PG as the unit for storage. User data is managed, and the user data in each PG is finally stored on multiple object storage devices (Object Storage Device, OSD) according to the preset copy strategy. In order to distribute the stored user data evenly on each OSD, Ceph distributes The storage system uses the CRUSH (Controlled Replication Under Scalable Hashing) algorithm to obtain the OSD tuple corresponding to each PG according to the state information of each OSD. The OSD tuple includes multiple different OSDs arranged in sequence. Each PG and its multiple A copy is stored in multiple different OSDs in the corresponding OSD tuple, so as to achieve an approximately uniform distribution of multiple PGs on multiple OSDs as a whole. In each OSD tuple, the first OSD is the main OSD, the rest are slave OSDs, and the master OSD directly receives user data access requests. Therefore, the master OSD needs to process more intensive access requests than the rest of the slave OSDs. Although the CRUSH algorithm ensures that user data can be distributed approximately evenly different OSDs, but there is no guarantee that the number of PGs on each main OSD is approximately balanced, so it is impossible to ensure that user data access requests are approximately evenly distributed on the main OSDs, and ultimately the overall performance of the Ceph distributed storage system cannot be fully realized. to play

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Main OSD adjusting method and device
  • Main OSD adjusting method and device
  • Main OSD adjusting method and device

Examples

Experimental program
Comparison scheme
Effect test

no. 1 example

[0025] Please refer to image 3 , image 3 The flowchart of the main OSD adjustment method provided by the embodiment of the present invention is shown. The processing method includes the following steps:

[0026] Step S101: Calculate the OSD tuple corresponding to each PG, where the OSD tuple includes a master OSD determined from a plurality of OSDs and at least one slave OSD.

[0027] In the embodiment of the present invention, the data that the user needs to store is first cut into fixed-length objects, and each object is assigned to a PG through a hash function. PG is a Ceph distributed storage system that organizes, migrates, and The maintenance unit is a logical concept. A PG is responsible for organizing several objects, but one object can only be mapped to one PG. When the client 100 learns from the monitoring cluster that the cluster map is updated, it uses the CRUSH algorithm to calculate the OSD tuple corresponding to each PG according to the updated cluster map. One PG...

Embodiment approach

[0038] As an implementation manner, the method for determining the OSD tuple to be adjusted may be:

[0039] First, determine whether the total number of times corresponding to the main OSD in the current OSD tuple is within a preset deviation range, where the preset deviation range includes a lower limit value and an upper limit value.

[0040] In the embodiment of the present invention, the preset deviation range can be the average number of times each OSD in the Ceph distributed storage system is used as the main OSD fluctuates by a preset percentage, where each OSD in the Ceph distributed storage system is the average of the main OSD The number of times can be calculated by the formula: V=M / N, where V represents the average number of times each OSD in the Ceph distributed storage system serves as the main OSD, M represents the number of PGs in the Ceph distributed storage system, and N represents the Ceph distribution The number of OSDs in a distributed storage system, for exam...

no. 2 example

[0060] Please refer to Figure 5 , Figure 5 It shows a schematic block diagram of a main OSD adjustment apparatus 200 provided by an embodiment of the present invention. The main OSD adjustment device 200 is applied to the client 100 in the Ceph distributed storage system, and includes a calculation module 201; a statistics module 202; a traversal module 203; a first determination module 204; a second determination module 205; and an exchange module 206.

[0061] The calculation module 201 is configured to calculate the OSD tuple corresponding to each PG, where the OSD tuple includes a master OSD determined from a plurality of OSDs and at least one slave OSD.

[0062] In the embodiment of the present invention, the calculation module 201 is used to execute step S101.

[0063] The statistics module 202 is used to count the total number of times each OSD serves as the main OSD in multiple OSD tuples.

[0064] In the embodiment of the present invention, the statistics module 202 is us...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of distributed storage, and provides a main OSD adjustment method and device, and the method comprises the steps of calculating an OSD tuple correspondingto each PG; traversing the plurality of OSD tuples, and determining the OSD tuples meeting a preset adjustment condition as to-be-adjusted OSD tuples; determining a target OSD from the OSD from at least one of the OSD tuples to be adjusted; and exchanging the master and slave roles of the master OSD and the target OSD in the to-be-adjusted OSD tuple, so that the target OSD is adjusted to the adjusted master OSD of the corresponding to-be-adjusted OSD tuple. Compared with the prior art, and according to the present invention, by adjusting the main OSD in the OSD tuple of each PG, the number ofthe PGs on all the main OSDs is kept approximately balanced, and the user data access requests are approximately and uniformly distributed on the main OSDs.

Description

Technical field [0001] The present invention relates to the field of distributed storage technology, and in particular to a method and device for adjusting a main OSD. Background technique [0002] In the Ceph distributed storage system, the user data to be stored is usually first cut into fixed-length objects. Each object belongs to a Placement Group (PG). The Ceph distributed storage system uses PG as the unit to store User data is managed. The user data in each PG is finally stored on multiple Object Storage Devices (OSDs) with a preset copy strategy. In order to make the stored user data evenly distributed on each OSD, Ceph distribution The storage system uses the CRUSH (Controlled Replication UnderScalable Hashing) algorithm to obtain the OSD tuple corresponding to each PG according to the status information of each OSD. The OSD tuple includes a number of different OSDs arranged in order, each PG and its multiple Each copy is stored in multiple different OSDs in the corresp...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2458
Inventor 张天洁
Owner NEW H3C TECH CO LTD