Unlock instant, AI-driven research and patent intelligence for your innovation.

Distributed file system load balance implementation method

A distributed file and implementation method technology, applied in the field of distributed file system load balance implementation, can solve the problems of small bandwidth data volume, uneven storage device IO request, storage device timeout processing, etc., to solve the problem of excessive processing delay Effect

Inactive Publication Date: 2017-05-31
ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] The distributed file system consists of a large number of storage devices. Currently, consistent hashing is generally used to control the distribution of data. However, the distribution of IO requests cannot be well controlled based on original user information such as names. The IO distribution is uneven in a short period of time, resulting in The IO requests of the storage device are uneven, resulting in a high processing delay of the storage device
[0003] Especially in small file application scenarios, the IOPS requirements for storage devices are high, but the amount of bandwidth data is small, and the IO load is unbalanced, which may cause timeout processing of individual storage devices in severe cases

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed file system load balance implementation method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0023] The distributed file system load balancing implementation method is as follows: according to the storage device IO load statistics of the distributed file system, the metadata of the idle storage device is distributed, and the client performs consistent hashing according to the distributed metadata to realize the data IO according to the IO Pressure distribution.

[0024] In the distributed file system metadata distribution process, select metadata based on the IO statistics of storage devices and IO load control, and adjust the IO distribution of all storage devices in the system by controlling the distribution of metadata;

[0025] The steps are as follows:

[0026] 1) Data will be distributed to different storage devices based on metadata;

[0027] 2) The metadata distribution module distributes metadata according to the storage device;

[0028] 3) The pressure of sending IO on the storage device;

[0029] 4) The metadata distribution module selects storage devices with light ...

Embodiment 2

[0032] The specific operation steps of the distributed file system load balancing implementation method are as follows:

[0033] 1) Client write request, send request to metadata distribution module, apply for creation of metadata;

[0034] 2) The metadata distribution module selects the metadata with the lightest IO pressure of the storage device according to the IO pressure of the device;

[0035] 3) The client receives the metadata response, and completes the distributed writing of the data through the consistent hash algorithm according to the metadata information, and writes the data to the storage device with the least IO pressure.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a distributed file system load balance implementation method. The implementation method comprises the following steps that metadata of a leisure storage device is distributed according to storage device IO load statistics of a distributed file system, a client performs consistent hash according to the distributed metadata, and the purpose that the data IO is distributed according to IO pressure is achieved. Compared with the prior art, the distribute file system load balance implementation method disclosed by the invention has the advantages that IO pressure distribution is effectively adjusted through IO statistics of the storage device; a metadata distribution list is managed according to the storage device, so taht the problem of excessive processing delay of the storage device is effectively solved.

Description

Technical field [0001] The invention relates to the technical field of distributed file system IO load balancing, in particular to a method for realizing distributed file system load balancing. Background technique [0002] The distributed file system consists of a large number of storage devices. Currently, consistent hashing is generally used to control the distribution of data. However, the distribution of IO requests cannot be well controlled based on the original user information such as names. The IO distribution is uneven in a short period of time, resulting in The IO requests of the storage device are uneven, causing the processing delay of the storage device to be too high. [0003] Especially in the small file application scenario, the IOPS requirements of the storage device are high, but the amount of bandwidth data is small, and the IO load is unbalanced, which may cause overtime processing of individual storage devices in severe cases. [0004] Related published patent ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/50
CPCG06F9/5083
Inventor 李雪生
Owner ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com