Storage data migration method and device

A technology for storing data and data, applied in the input/output process of data processing, electrical digital data processing, instruments, etc., can solve the problems that affect the efficiency of data writing operations, there is no special priority queuing method, and the writing operation cannot be performed in time. To achieve the effect of reducing the suspension lock time and improving the product experience

Pending Publication Date: 2020-09-11
GUANGDONG EFLYCLOUD COMPUTING CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

When the ceph cluster needs to make major adjustments, the number of pgs that need to be migrated is large, so write operations are often suspended, and these pgs cannot be written in time because they are locked, which seriously affects the efficiency of data write operations. This is current problems
[0005] In the prior art, when the data is migrated, there is no special priority queuing method, but the data block pg is simply migrated

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Storage data migration method and device
  • Storage data migration method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0023] Such as figure 1 As shown, the present invention adds a "lock timer module" to all small data blocks pg that need to be migrated. This module records whether the current data block is locked, and if locked, how long it has been currently locked. When migrating, those small data blocks pg that are locked are migrated first, so that these small data blocks pg can be unlocked as soon as possible and can be written.

[0024] Therefore, the present invention provides a method for storage data migration, such as figure 1 shown, including the following steps:

[0025] S11. The data client writes data to the ceph distributed storage system, and the ceph distributed storage system divides the data into several small data blocks, and dispersely stores the small data blocks on each storage device;

[0026] S12. When it is necessary to migrate small data blocks, that is, to migrate all small data blocks on a certain storage device to other storage devices, it is necessary to lock...

Embodiment 2

[0033] This embodiment provides a storage data migration device on the basis of Embodiment 1, such as figure 2 As shown, including data client, ceph distributed storage system, locking module, locking timer module and unlocking module, described ceph distributed storage system includes several storage devices;

[0034] The data client is used to read and write data to the ceph distributed storage system;

[0035] The ceph distributed storage system is used to divide data into several small data blocks, and store the small data blocks scattered on each storage device;

[0036] The locking module is used for: when the small data block needs to be migrated, the locking module locks the small data block so that it can only be read but not written;

[0037] Described locking timer module is used for: For each locked small data block, calculates the locking time of small data block, and sorts small data block according to locking time from big to small, then lock timer priority lo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a storage data migration method and device, and the method comprises the steps that a data client writes in data, and a ceph distributed storage system divides the data into aplurality of small data blocks and stores the small data blocks in all storage equipment; when the small data blocks need to be migrated, the small data blocks are locked, so that the small data blocks can only be read and cannot be written; the locking timer module is used for calculating the locking time of each small data block, sorting the small data blocks from large to small according to thelocking time, and then preferentially migrating the small data blocks with longer locking duration by the locking timer; and the small data block which has been successfully migrated to the new storage device is unlocked, so that the small data block can be read and written. According to the method and device, the locking timer is added and used for counting the locking duration of each small data block needing to be migrated, the small data blocks with the long locking duration are migrated preferentially, the migration efficiency of the small data blocks is improved, and the influence on writing operation of a user is reduced.

Description

technical field [0001] The invention relates to the technical field of data storage migration, in particular to a storage data migration method and device thereof. Background technique [0002] At present, ceph, like other distributed storage, generally divides a piece of data to be stored into several small data blocks of the same size, and then stores these small data blocks in different storage units according to a certain algorithm. . Name these small data blocks after pg in ceph. [0003] However, when a storage unit fails or some new storage units are added, the existing pg needs to be redistributed to conform to the distribution algorithm of ceph (when a storage unit fails, the pg inside will be disappear, need to be rebalanced; when a new storage unit is added, it is empty and can store pg, so this situation also needs to be rebalanced). [0004] When ceph rebalances the pg, in order to prevent the data update from being out of sync, it will write-protect the pg t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F3/06
CPCG06F3/0647G06F3/067G06F3/0604Y02D10/00
Inventor 麦剑史伟闵宇
Owner GUANGDONG EFLYCLOUD COMPUTING CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap