Unlock instant, AI-driven research and patent intelligence for your innovation.

Data replication method, device and system and electronic equipment

A data replication and data technology, applied in the computer field, can solve the problems of difficulty in guaranteeing recovery point objectives, increasing the bearing pressure and impact of replication resources, etc.

Active Publication Date: 2020-05-08
ALIBABA GRP HLDG LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Every time a file is uploaded, a copy task will be generated. If the same file is uploaded multiple times in a short period of time, the number of copy tasks will increase sharply, which will increase the load on the copy resources. Copy tasks must be executed serially, otherwise data consistency will occur problems; the replication tasks of different customers are not isolated and will affect each other, making it difficult to guarantee the recovery point objective (RPO) commitment given to users; when there is a problem with the specified replication bandwidth between clusters, the user's data replication delay will appear uncontrollable prolongation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data replication method, device and system and electronic equipment
  • Data replication method, device and system and electronic equipment
  • Data replication method, device and system and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0047] Based on the idea of ​​the above data replication scheme, such as figure 2 As shown, it is the flow of the data replication method shown in the embodiment of the present invention Figure 1 , the execution body of this method can be deployed in figure 1 The scheduler 110 in. Such as figure 2 As shown, the data replication method includes the following steps:

[0048] S210. Acquire m first copy tasks for the user to copy data from the first storage space to the second storage space.

[0049] In terms of scheduler deployment, this solution sets up a scheduler for each user to copy data from a specified original storage space (first storage space) to the target storage space (second storage space), and these schedulers Exclusively schedule the replication tasks on the corresponding synchronization edge. For example, user A has a storage space at the source end and a storage space at the remote end, and the data synchronization relationship between the two storage sp...

Embodiment 2

[0059] This embodiment and figure 2 The difference between the shown methods is that this embodiment adopts two preferred implementation solutions of merging m first copy tasks to obtain n second copy tasks.

[0060] First, if image 3 Shown is the flow of the data replication method of the embodiment of the present invention Figure II . As shown in the figure, after step S210, the following steps are performed:

[0061] S221. For multiple first copy tasks corresponding to multiple versions of the same file, use the first copy task corresponding to the latest version of the file as the merged second copy task.

[0062] In actual application scenarios, users may perform multiple uploads on the same file within a short period of time to update the file. When the user uploads the update file shortly after, he immediately updates and uploads the same file. At this time, the file currently stored by the system is the new version file corresponding to the later upload operatio...

Embodiment 3

[0070] Such as Figure 5 Shown is the flow of the data replication method of the embodiment of the present invention Figure four . This embodiment and figure 2 The difference of the shown methods is that this embodiment adopts a preferred implementation solution of acquiring m first copy tasks for the user to copy data from the first storage space to the second storage space. Such as Figure 5 As shown, at the very beginning of this embodiment, the following steps are performed:

[0071] S510. Obtain a plurality of log records formed when the user uploads a file to the first storage space, where the log record records a file version corresponding to the log record.

[0072] Every time a user uploads a file to the first storage space, a log record (binlog) will be left in OSS, and the binlog will record the file version corresponding to the current log record.

[0073] S520. Compare the first file version recorded in the log record with the second file version of the sam...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention provides a data replication method, device and system and electronic equipment, and the method comprises the steps: obtaining m first replication tasks of a user for replicating data from a first storage space to a second storage space; merging the m first copy tasks to obtain n second copy tasks, wherein n is smaller than m; and executing the second copy task. According to the scheme of the embodiment of the invention, under the condition that the cross-regional bandwidth is limited, the file data can be quickly copied by combining the copying tasks.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a data replication method, device, system and electronic equipment. Background technique [0002] With the continuous development of the Internet, the data generated by many enterprises has exceeded the storage limit of a single cluster, or a single cluster cannot meet the needs of data processing. At this time, these data need to be split according to business units and stored in multiple clusters. There are often dependencies between business units, so it is often necessary to read data across clusters. As the number of businesses increases, the number of businesses that read data across clusters in different places will also increase, resulting in more serious waste of bandwidth resources. In order to save bandwidth resources, the industry generally adopts the method of keeping a copy of the business data stored on each cluster in other clusters, so that when the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06H04L29/08
CPCG06F3/0614G06F3/065G06F3/067H04L67/10
Inventor 仓利基
Owner ALIBABA GRP HLDG LTD