Unlock instant, AI-driven research and patent intelligence for your innovation.

Data scheduling method and system based on hierarchical storage and electronic equipment

A tiered storage and data scheduling technology, which is applied in electrical digital data processing, data processing input/output process, instruments, etc., can solve the waste of investment, cannot timely and accurately reflect the peak value of tiered storage workload, Large investment costs and other issues, to achieve the effect of reducing deployment costs

Pending Publication Date: 2021-10-19
WUXI CHINAC DATA TECH SERVICE
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The applicant pointed out that the prior art of characterizing the storage overhead and status of tiered storage based on the workload mentioned above cannot reflect the actual workload formed by the tiered storage data to be scheduled due to the "long tail effect" in a timely and accurate manner. As a result, the storage capacity of the storage device using tiered storage cannot be effectively utilized and utilized, and as a whole, the IOPS performance and throughput of the storage device using tiered storage and the computer system containing the storage device will decrease.
In addition, the technical means of collecting and learning workload (workload) in a fixed cycle will lead to the inability to meet the data read and write capabilities required for specific access requests in the face of sudden high load scenarios, resulting in the deployment of tiered storage. Investment causes waste, too large (or stronger performance) or too small (or weaker performance) tiered storage will lead to excessive investment costs and investment waste

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data scheduling method and system based on hierarchical storage and electronic equipment
  • Data scheduling method and system based on hierarchical storage and electronic equipment
  • Data scheduling method and system based on hierarchical storage and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0074] Tiered Storage, also known as "Hierarchical Storage Management". Hierarchical storage is to store data in different levels of storage media, and perform automatic or manual data migration, replication and other operations between different media. Hierarchical storage is also a specific application and realization of information lifecycle management.

[0075] A data scheduling method based on hierarchical storage shown in this embodiment aims to reveal the data scheduling method based on hierarchical storage or Figure 10 (or Figure 13 or Figure 14 ) shown in the tiered storage device 20 (20A) with different read / write performance of the memory groups to perform data scheduling operations on cold data and hot data.

[0076] ginseng image 3 As shown, in this embodiment, the data scheduling method based on hierarchical storage (hereinafter referred to as "data scheduling method") includes the following steps S1 to S3.

[0077] The data dispatch method runs on Fig...

Embodiment 2

[0104] ginseng Figure 16 and Figure 17 As shown, this embodiment shows a modified embodiment of a data scheduling method based on hierarchical storage in the present invention.

[0105] Compared with the first embodiment, the main difference of this embodiment is that step S2 specifically includes the following steps: step S21, pre-judging the data to be scheduled for hierarchical storage processing. Step S22, issue an optimization policy that configures two QoS thresholds (ie, the first QoS threshold and the second QoS threshold) to the hierarchical storage, and divide the workload cycle into three cycle segments with different workloads through the two QoS thresholds , and output the workload mode that is dynamically set for the tiered storage within the preset period and matches the period segment. This embodiment is intended to disclose based on hierarchical storage or Figure 10 (or Figure 13 or Figure 14 ) shows in detail the scheduling process of cold data, war...

Embodiment 3

[0110] ginseng Figure 13 As shown, based on the hierarchical storage-based data scheduling method disclosed in Embodiment 1, this embodiment also discloses a hierarchical storage-based data scheduling system (hereinafter referred to as "data scheduling system").

[0111] In this embodiment, a data scheduling system based on hierarchical storage includes: a hierarchical storage controller 21 , a storage cluster 30 mounted to the hierarchical storage controller, and a power spectral density function detection module 10 . The tiered storage controller 21 deploys the workload computing module 25 . The power spectral density function detection module 10 collects hierarchically stored historical workload data, and determines the workload period based on the power spectral density function corresponding to the historical workload data. The workload calculation module 25 dynamically sets a workload pattern that matches the workload period for the tiered storage within a preset perio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a data scheduling method and system based on hierarchical storage and electronic equipment, and the data scheduling method comprises the steps: collecting historical workload data of hierarchical storage, and determining a workload period based on a power spectrum density function corresponding to the historical workload data; dynamically setting a working load mode matched with the working load period for the hierarchical storage in a preset period according to the working load period; and setting a migration priority of the to-be-scheduled data in a preset period according to the working load mode, so as to schedule the to-be-scheduled data to at least one storage medium in the memory group which forms hierarchical storage and has different read / write performances according to the migration priority. According to the method and the device, the effect that the to-be-scheduled data is better guided to be reasonably migrated to at least one storage medium in the memory group with different read / write performance of the hierarchical storage according to the performance of the hierarchical storage is realized, and the storage performance of the hierarchical storage is fully exerted.

Description

technical field [0001] The present invention relates to the technical field of data storage, in particular to a hierarchical storage-based data scheduling method, system and electronic equipment. Background technique [0002] Tiered storage (Tiered Storage) is a technology that allows data to be migrated and managed between different storage levels. Tiered storage can save more costs while providing the required performance. Among all the data, the data with high access frequency is stored in the high-performance storage layer, while most of the other data are stored in the storage layer with low performance but large capacity and low price. Users do not need to know where the data is stored, and the system will Data is retrieved automatically. From the perspective of the physical structure of storage media, the latest development of hierarchical storage is the combination of mechanical hard disk and flash memory. Hierarchical storage in the storage system is to store the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
CPCG06F3/0604G06F3/0647G06F3/0644Y02D10/00
Inventor 陈铎
Owner WUXI CHINAC DATA TECH SERVICE