Unlock instant, AI-driven research and patent intelligence for your innovation.

Optimization method, estimation method and processing method and device for data migration

An optimization method and an optimization device technology, applied in the computer field, can solve problems such as the lack of evaluation of data dependencies in new businesses, and achieve the effects of reducing risks and improving efficiency

Active Publication Date: 2017-09-29
ALIBABA GRP HLDG LTD
View PDF14 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] An embodiment of the present invention provides an optimization method, evaluation method, and processing method and device for data migration, so as to solve the defects caused by the generation of new services and the lack of evaluation of data dependencies in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Optimization method, estimation method and processing method and device for data migration
  • Optimization method, estimation method and processing method and device for data migration
  • Optimization method, estimation method and processing method and device for data migration

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0059] like image 3 As shown, it is one of the schematic flowcharts of the optimization method for data migration in Embodiment 1 of the present invention, and the optimization method includes the following steps 101 to 103.

[0060] Step 101: According to the principle of preferentially copying one or more data units to be migrated with a large amount of first dependent data as data units to be copied to the target cluster, and then switching computing clusters, multiple data migration plans are generated. Wherein, the first dependent data volume is all dependent data volumes of the data unit to be migrated. The first dependent data volume may include the dependent data volume within the same project unit, or the dependent data volume of other project units other than the project unit where the data unit is located. In fact, there will be cross-cluster dependent data volumes Depends on the amount of data, etc. In addition, the switching computing cluster mentioned here ref...

Embodiment 2

[0094] This embodiment relates to an evaluation method for data migration, which is used to simulate and evaluate a data migration scheme to determine its feasibility before actually performing data migration operations. like Figure 8 As shown, it is a schematic flow chart of the evaluation method of data migration in Embodiment 2 of the present invention. The evaluation method includes:

[0095] Step 201: Obtain the second dependent data volume of one or more data units to be copied from the source cluster to the target cluster before switching the computing cluster, where the second dependent data volume is the same as that in the above-mentioned embodiment The meaning is consistent, that is, the amount of dependent data between the data unit to be copied and other data units outside the target cluster. In this step, the data units to be copied can be all the data units to be migrated that need to be migrated, or some data units to be migrated that need to be migrated. Mi...

Embodiment 3

[0100] This embodiment relates to a processing method for data migration, such as Figure 9 As shown, it is a schematic flow chart of the data migration processing method in Embodiment 3 of the present invention, including:

[0101] Step 301: Prioritize copying one or more data units to be migrated with a large amount of first dependent data as data units to be copied to the target cluster, wherein the first dependent data amount is all the data units to be migrated Depends on data volume. .

[0102] Step 302: switch computing clusters. The switching computing cluster here can be specifically a

[0103] All computing tasks in one or more project units to be migrated are switched to the target cluster. After the computing cluster is switched, the new data generated by the computing task will be stored in the target cluster by default.

[0104] Step 303: Migrate the remaining one or more data units to be migrated to the target cluster.

[0105] Before the above step 301, i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention provides an optimization method, an estimation method and a processing method and device for data migration. The optimization method includes the steps that based on the principle of preferentially copying one or more to-be-migrated data units with a large dependency data quantity to a target cluster and then switching computing clusters, multiple data migration schemes are generated; inter-cluster bandwidth state data generated after computing cluster switching is calculated; the data migration schemes are optimally selected according to the bandwidth state data. According to the optimization method of data migration, the principle of preferentially copying the hot data units with the large dependency data quantity and then switching the computing clusters is adopted, prediction and evaluation are conducted on the schemes based on the bandwidth state data, and then optimal selection is conducted; thus, an optimal data migration scheme can be obtained, the data migration efficiency can be improved, and the risk of data migration failure can be lowered.

Description

technical field [0001] Embodiments of the present invention relate to an optimization method, evaluation method, and processing method and device for data migration, which belong to the field of computer technology. Background technique [0002] The data migration method in the existing technology is to first copy all the data units of the project unit to be migrated from the source cluster to the target cluster. During this period, all computing tasks related to the migrated data are still running on the source cluster until the complete copy is completed. Then switch the computing task from the source cluster to the target cluster. For large-scale data migration (such as project units with a relatively large amount of data), the whole process will take a long time. Moreover, the existing data is not evaluated based on data dependencies before migration, that is, the impact of data dependencies on clusterware bandwidth after migration is not considered. [0003] In the pr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/214H04L67/1095H04L67/61G06F16/119G06F16/00G06F3/0605G06F3/0647G06F3/067
Inventor 黄俨何乐史英杰张杰张辰
Owner ALIBABA GRP HLDG LTD