Unlock instant, AI-driven research and patent intelligence for your innovation.

Data reconstruction method of distributed file system

A distributed file and data reconstruction technology, applied in the computer field, can solve the problems of inaccurate data reconstruction control scheme, affecting distributed cluster performance, and low data reconstruction performance.

Active Publication Date: 2020-03-24
LANGCHAO ELECTRONIC INFORMATION IND CO LTD
View PDF5 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The purpose of this application is to provide a data reconstruction method, device, server, and readable storage medium of a distributed file system, which are used to solve the problem that the traditional data reconstruction control scheme is not precise enough, resulting in low data reconstruction performance and even affecting Distributed Cluster Performance Issues

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data reconstruction method of distributed file system
  • Data reconstruction method of distributed file system
  • Data reconstruction method of distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0046] The following introduces Embodiment 1 of a data reconstruction method for a distributed file system provided by the present application, see figure 1 , embodiment one includes:

[0047]S101. Determine the number of hosts and the number of placement group copies in the storage pool where the target OSD is located;

[0048] S102. If the number of copies of the placement group is equal to the number of hosts, determine the water level of each OSD of the current host after the target OSD data is reconstructed;

[0049] S103. If the placement group copy is smaller than the number of hosts, determine the target host to which the target OSD data is expected to be reconstructed, and determine the water level of each OSD of the target host after the target OSD data is reconstructed;

[0050] S104. When the water level of each OSD does not exceed the reconstruction water level limit, perform data reconstruction on the target OSD.

[0051] This embodiment is applied to a distrib...

Embodiment 2

[0059] see figure 2 , embodiment two specifically includes:

[0060] S201. Perform state monitoring on the OSDs in the distributed file system, and if a faulty OSD is detected, determine that the OSD is a target OSD for data reconstruction to be performed;

[0061] S202. Determine the number of hosts and the number of placement group copies in the storage pool where the target OSD is located;

[0062] S203. If the number of copies of the placement group is equal to the number of hosts, determine the water level of each OSD of the current host after the target OSD data is reconstructed;

[0063] S204. If the placement group copy is less than the number of hosts, according to the CRUSH algorithm, determine the target host to which the target OSD data is expected to be reconstructed, and determine each of the target hosts after the target OSD data is reconstructed. OSD water level;

[0064] S205. Determine whether there is a water level exceeding 90% in the water level of eac...

Embodiment approach

[0077] As a specific embodiment, the device also includes:

[0078] Monitoring module: used to monitor the status of the OSD in the distributed file system. If a faulty OSD is detected, it is determined that the OSD is the target OSD for data reconstruction to be performed

[0079] The data reconstruction device of the distributed file system in this embodiment is used to realize the above-mentioned data reconstruction method of the distributed file system, so the specific implementation in the device can be seen in the above-mentioned data reconstruction method of the distributed file system In the embodiment part, for example, the data determination module 301, the first calculation module 302, the second calculation module 303, and the reconstruction module 304 are respectively used to implement steps S101, S102, and S103 in the data reconstruction method of the above-mentioned distributed file system, S104. Therefore, for the specific implementation manners thereof, refer...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data reconstruction method and device of a distributed file system, a server and a readable storage medium. According to the scheme, when the storage pool exceeds the water level, a decision strategy for judging whether data reconstruction is carried out or not can be more finely controlled; and in the actual process, the water level of the OSD bearing data reconstructionafter data reconstruction occurs is determined and calculated firstly, and then whether the water level of each OSD exceeds the reconstruction water level limit or not is judged, and data reconstruction is executed only when the water level of each OSD does not exceed the reconstruction water level limit. Compared with a traditional scheme that whether data reconstruction is executed or not is decided on the basis of the water level of the storage pool, the scheme decides whether data reconstruction is executed or not according to the water level of each OSD bearing data reconstruction, so that the control granularity of the scheme is finer, and the data reconstruction can utilize the disk space to the maximum extent, and the data reconstruction performance is improved.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a data reconstruction method, device, server and readable storage medium of a distributed file system. Background technique [0002] In a distributed file system, the main function of OSD is to store data, copy data, balance data, restore data, etc. Generally, one hard disk corresponds to one OSD, and the OSD manages the hard disk storage. When the hard disk attached to the OSD fails, the data of the OSD needs to be recovered. The process of recovering the data in the failed OSD from other OSDs is called data reconstruction. [0003] When data reconstruction occurs, the client will perform read and write operations on the cluster and data reconstruction at the same time. In order to reduce the impact of data reconstruction on the client's read and write performance, the data reconstruction speed of the cluster can be adjusted. At present, existing solutions achieve ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06F16/182
CPCG06F3/0607G06F3/0629G06F3/065G06F3/067G06F3/0679G06F16/182
Inventor 刘鑫孟祥瑞
Owner LANGCHAO ELECTRONIC INFORMATION IND CO LTD