QoS (Quality of Service) control method based on Ceph distributed storage Osd end data recovery

A distributed storage and control method technology, applied in the field of distributed storage, can solve the problems of inability to accurately control data recovery transmission bandwidth, affect business IO processing speed, etc., and achieve the effect of reducing the impact of business IO and concurrency

Active Publication Date: 2019-11-05
北京易捷思达科技发展有限公司
View PDF4 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

At present, data recovery mainly uses the method of controlling how many objects an Osd pg can recover per second. However, since the amount of data in each object is uncertain,

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • QoS (Quality of Service) control method based on Ceph distributed storage Osd end data recovery
  • QoS (Quality of Service) control method based on Ceph distributed storage Osd end data recovery

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] The present invention is described in further detail now in conjunction with accompanying drawing.

[0021] A QoS control method based on Ceph distributed storage Osd end data Recovery, which is used to realize the flow control method when limiting data balance, and can be effectively applied when the system changes, including trigger conditions such as the addition and removal of Osd services. During the data balancing process, the speed of data recovery is precisely controlled to reduce the impact on business IO.

[0022] Token Bucket (TokenBucket) is a container for storing tokens. The container has a background process that releases tokens into the container at a certain rate continuously. When the total number of tokens is greater than the maximum token that can be accommodated in the token bucket, Discard excess tokens, keep the maximum amount, and increase the number of tokens when the total number of tokens is less than the maximum token limit of the token bucke...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a QoS (Quality of Service) control method based on Ceph distributed storage Osd end data recovery. A QoS mechanism is mainly added to a distributed storage Osd server; controlling is carried out according to a byte stream method; a token bucket is arranged at each Osd end, wherein the maximum number of the tokens is the maximum current limiting size of the Osd server, whether the number of the tokens is enough or not is judged when the data is balanced each time, if so, a request IO is started to be issued, and if not, a background process is waited for to issue the tokens to a token bucket, and when the tokens are sufficient, a waiting IO request is started to be issued. By controlling the token issuing rate of the background process, the purpose of accurately controlling the data flow can be achieved, and meanwhile, the QoS has the effect of coping with the burst IO, so that the data balancing process is closer to the scene of controlling the data flow by a user scene.

Description

technical field [0001] The invention belongs to the field of distributed storage, and in particular relates to a QoS control method based on Ceph distributed storage Osd end data Recovery. Background technique [0002] The traditional storage system uses a centralized server storage method to improve the performance, capacity, network, etc. of a single mainframe to improve the current growing data demand, but because the traditional storage system can only be limited to a single large computer, and The disadvantage of data storage on a single server is that on the one hand, the performance of the storage is limited by the server and becomes a bottleneck; on the other hand, the reliability and security of the data are severely tested. The birth of distributed storage has greatly improved and broken through the bottleneck of traditional storage systems. It integrates many server resources even across multiple data centers, and uses hash and CRUSH algorithms to disperse and sto...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L12/819H04L29/08H04L47/21
CPCH04L47/215H04L67/1097
Inventor 王洋温大帅于治国杨东升古轶特陈刘忠
Owner 北京易捷思达科技发展有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap