Method for improving data transfer efficiency under automatic data hierarchical storage frame

Abstract

The invention provides a method for improving the data transfer efficiency under an automatic data hierarchical storage frame. The method involves a data block IO monitoring module, a data block transfer push module and a data block transfer extract module; the data block IO monitoring module is used for monitoring and counting the IO flow, from a user, of each hierarchy of a whole hierarchical storage pool and monitoring the maximum IO flow, set by the user, of each hierarchy of the hierarchical storage pool; the data block transfer push module is responsible for scanning data blocks of hierarchical equipment where the data block transfer push module is located and scanning counting information from the data block IO monitoring module; the data block transfer extract module is used for extracting IO from a transfer queue of the hierarchical equipment according to an IO quantity extracting threshold value from the data block IO monitoring module. Compared with the prior art, the method enables data block information which is accessed recently to be counted in a memory, the part of the counting information is scanned rapidly, the transfer real-time performance and the transfer efficiency of data are improved, the influence on normal IO is reduced, and the use efficiency of storage equipment is improved.

Description

technical field [0001] The invention relates to the technical field of computer communication, in particular to a method for improving data migration efficiency under an automatic data hierarchical storage framework. Background technique [0002] In the tiered storage system based on block-level data, infrequently accessed data is automatically migrated to devices with lower storage costs, releasing higher-cost storage space for more frequently accessed or higher-priority data , thereby greatly reducing the space occupied by non-important data on the first-level local disk, speeding up the storage performance of the entire system, reducing the cost of ownership of the entire storage system, and obtaining better cost performance. Hierarchical storage based on data blocks is a fine-grained data hierarchical management solution, which can refine the granularity of data management to the extended block level defined according to application requirements, so the management of dat...

AI Technical Summary

Problems solved by technology

However, in the existing hierarchical storage migration scheme based on data blocks, there is a problem that the migration efficiency of data blocks is relatively low, and data migration has a certain negative impact on the normal business of users. The main reasons are as follows:

[0003] (1) When the data migration cycle is reached at a certain moment, intensive IO access occurs. At this time, in order to ensure data consistency, the entire hierarchical storage pool will be locked during IO operations. At this time, trigger The migration operation will cause the operation to be unable to complete in time due to the locking and mutual exclusion of the data blocks. It is necessary to wait for the next or more migration cycles before attempting to perform the upgrade operation of hot data, resulting in the inability to upgrade hot data quickly and affecting the hierarchical management of data The actual application effect; when the data migration obtains the tiered storage pool, a large amount of data migration will be performed. At this time, the data migration will also occupy a certain IO bandwidth of the storage pool, which has a negative performance impact on the normal business of users;

[0004] (2) When the data migration cycle is reached at a certain moment, all data blocks in the entire storage pool are often traversed and scanned. When the amount of data in the hierarchical storage pool reaches the ZB level, the data in the entire storage pool in this scenario The migration efficiency is very low

Images