Unlock instant, AI-driven research and patent intelligence for your innovation.

Cloud storage data unit scheduling method and device

A technology of data unit and scheduling method, applied in the field of information processing, to achieve the effect of reducing the amount of calculation and strong robustness

Pending Publication Date: 2020-01-24
ALIBABA GRP HLDG LTD
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The existing technology does not involve multiple-input and multiple-output scenarios, but is based on single-input and single-output scenarios, and the method of single-input and single-output scenarios cannot be reused in multiple-input and multiple-output scenarios

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cloud storage data unit scheduling method and device
  • Cloud storage data unit scheduling method and device
  • Cloud storage data unit scheduling method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0070] In a cloud computing environment, a VM will have at least one system disk, which stores the operating system, and may have multiple data disks, which store corresponding business data. Each system disk or data disk will be divided into several segments, each Segments are scheduled to a physical machine, and each segment is connected to the underlying distributed file system. When the user's disk IO comes out of the VM, it will find the corresponding segment according to the IO offset (offset), and send the IO to the corresponding segment. The physical machine where it is located, and then sent to the underlying distributed file system.

[0071] like figure 1 As shown, the embodiment of the present invention provides a cloud storage data unit scheduling method in order to realize the self-adaptive scheduling of the new MIMO cloud storage segment, including:

[0072] S101. Collect load parameters of each physical machine in the cloud storage system;

[0073] S102. Deter...

Embodiment 2

[0102] like image 3 As shown, this embodiment provides a cloud storage data unit scheduling device, including:

[0103] The physical machine data collection module 100 is configured to collect the load parameters of each physical machine in the cloud storage system;

[0104] The physical machine selection module 200 is configured to determine a candidate physical machine according to the feedback information of each physical machine and the collected load parameters of each physical machine;

[0105] The disk scheduling module 300 is configured to evaluate the load parameters of the candidate physical machines, determine the target physical machine, and schedule the current data unit set of the disk to the target physical machine.

[0106] In this embodiment, the described device also includes:

[0107] The disk scheduling and grouping module 400 is configured to group the data units of the disk and determine multiple sets of data units.

[0108] The disk scheduling and gr...

Embodiment 3

[0125] This embodiment illustrates the process of grouping by the disk scheduling grouping module 400:

[0126] The disk scheduling and grouping module 400 is a preprocessing module for cloud storage system scheduling. In a distributed cloud storage system, the size of a single disk has no upper limit in theory and depends on the size of the cluster. When the size of the disk reaches a certain capacity , the number of divided segments is greater than the number of physical machines. At the same time, the unavailability of a single segment will lead to the unavailability of the entire disk. Therefore, if there is no grouping, the slight jitter of the entire cluster and the abnormality of a single machine will bring a lot to the disk IO of the VM. Therefore, the magnetic scheduling and grouping module 400 will group all the segments of the disk. Since the number of N segment permutation and grouping groups is very large, the amount of calculation for finding the optimal solution ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a cloud storage data unit scheduling method and device. The cloud storage data unit scheduling method comprises the steps of collecting load parameters of each physical machinein a cloud storage system; determining candidate physical machines according to the feedback information of each physical machine and the acquired load parameters of each physical machine; evaluatingthe load parameters of the candidate physical machines; and determining a target physical machine, and scheduling the current data unit set of the disk to the target physical machine. The cloud storage data unit scheduling method determines a grouping scheme to realize dimensionality reduction and reduce the calculated amount of scheduling, and meanwhile, does not influence the use of a user and the scheme of the SLA under the condition of a node fault. The evaluation scheme provided by the invention has a self-adaptive effect, and can adjust the weight and give the most suitable target physical machine according to the scale change of the cluster and the promotion of the operation in the cluster. And disturbance and noise in the cluster can be effectively suppressed, so that the whole system has very high robustness.

Description

technical field [0001] The present invention relates to the technical field of information processing, in particular to a cloud storage data unit scheduling method and device. Background technique [0002] In a cloud computing environment, the computing resources of a data center are divided into a large number of VMs (Virtual Machines) through virtualization technology. Each VM has at least one block device, that is, a disk. Each disk has an independent It is provided by the storage cluster pool. Different clusters have different capabilities, that is, different IOPS (Input / OutputOperations Per Second, the number of read and write operations per second), throughput, delay, size, cost, etc. In the new cloud storage environment, users can choose disks with different capabilities according to the different applications deployed in their VMs. For example, some are development and test business, small-load database or system disks, some are I / O-intensive applications, medium and...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F3/06
CPCG06F3/0607G06F3/0614G06F3/0664G06F3/067Y02D10/00
Inventor 张坤余辰侃
Owner ALIBABA GRP HLDG LTD