Data migration method, device, control equipment and readable storage medium

Abstract

This application provides a data migration method, apparatus, control device, and readable storage medium. The method includes: creating a migration link between a source database and a destination database of data to be migrated; splitting the data to be migrated and the migration link, and stopping the splitting if the running status information of the migration link becomes abnormal. In this application, the method of splitting the migration link significantly increases the overall migration rate of the entire data migration; and by obtaining the running status information of the migration link, a controllable increase in the migration rate is achieved, effectively improving the migration speed.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data migration method, apparatus, control device and readable storage medium. Background Art

[0002] During the project delivery process, we often encounter scenarios where the customer's online database needs to be migrated to the cloud database. When the existing data is large (>1TB), the migration speed is often slow. Summary of the invention

[0003] The problem that this application solves is that the migration speed is slow when migrating large amounts of existing data.

[0004] To solve the above problems, the first aspect of the present application provides a data migration method, including:

[0005] Create a migration link between the source database and the destination database for the data to be migrated;

[0006] The data to be migrated and the migration link are split, and the splitting is stopped when the running state information of the migration link is abnormal.

[0007] A second aspect of the present application provides a data migration device, which includes:

[0008] A link creation module, which is used to create a migration link between a source database and a destination database for data to be migrated;

[0009] The link splitting module is used to split the to-be-migrated data and the migration link, and stop the splitting when the running status information of the migration link is abnormal.

[0010] A third aspect of the present application provides a control device, comprising: a memory and a processor;

[0011] The memory is used to store programs;

[0012] The processor, coupled to the memory, is configured to execute the program to:

[0013] Create a migration link between the source database and the destination database for the data to be migrated;

[0014] The data to be migrated and the migration link are split, and the splitting is stopped when the running state information of the migration link is abnormal.

[0015] A fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the above-mentioned data migration method.

[0016] In the present application, the total migration rate of the entire data migration is greatly increased by splitting the migration link; and by monitoring / obtaining the operating status information of the migration link, a controllable migration rate increase is performed, thereby effectively improving the migration speed.

[0017] This application can automatically adjust the migration rate. During the adjustment process, statistics will be collected on the errors reported by the source and destination ends to ensure that the stability of the source and destination ends is not affected during the rate adjustment process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of a data migration method according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of a data migration method S200 according to an embodiment of the present application;

[0020] Figure 3 This is the architecture diagram of the data migration solution according to this application;

[0021] Figure 4 is a structural block diagram of a data migration device according to an embodiment of the present application;

[0022] Figure 5 4 is a structural block diagram of a control device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below in conjunction with the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0024] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0025] During the project delivery process, we often encounter scenarios where the customer's online database needs to be migrated to the cloud database. When the existing data is large (>1TB), the migration speed is often slow.

[0026] The speed of data migration is limited by the reading rate of the source end, the migration rate of the intermediate migration link, the writing rate of the destination end, etc. This application mainly solves the migration rate problem of the intermediate migration link, assuming that the bandwidth of the intermediate migration link is not limited.

[0027] Synchronization tools like DTS can increase the speed by adjusting link specifications and parameters, which has a certain speed-up effect. However, because the rate adjustment is always performed on one synchronization link, the migration rate cannot be linearly expanded due to the CPU / memory performance of a single machine.

[0028] In response to the above problems, the present application provides a new data migration solution, which can solve the problem of slow migration speed when migrating large amounts of existing data by automatically splitting the migration link.

[0029] For ease of understanding, the following terms that may be used are explained here:

[0030] Isomorphic databases: refers to two databases with similar compatibility, such as ORACLE and PolarDB.

[0031] ORACLE: A popular relational database.

[0032] PolarDB: A relational database developed by Alibaba Cloud with independent intellectual property rights, highly compatible with ORACLE syntax.

[0033] Data migration: Migrate data from the source to the destination.

[0034] Adaptive rate: The migration tool automatically adjusts the migration rate.

[0035] DTS: A data migration tool.

[0036] The present application embodiment provides a data migration method, which can be performed by a data migration device, and the data migration device can be integrated in electronic devices such as pads, computers, servers, computers, server clusters, data centers, etc. Figure 1 As shown, it is a flow chart of a data migration method according to an embodiment of the present application; wherein the data migration method includes:

[0037] S100, creating a migration link between a source database and a destination database for data to be migrated;

[0038] S200: Split the data to be migrated and the migration link, and stop splitting when the running status information of the migration link is abnormal.

[0039] It can be understood that for a cloud computing system, there are many links to choose from from the source database to the destination database. The migration link splitting in this application, taking the splitting of one link into two links as an example, is essentially selecting two new links in the system and deleting the old link.

[0040] In the present application, the operation status information of the migration link can be obtained by the system monitoring the migration link, or by other methods such as dedicated detection tools, etc., and the present application does not limit the specific acquisition method.

[0041] In the present application, for the migration link before or after the split, without considering other factors, the migration rate is not much different, so splitting a link into two or more links can increase the total migration rate of the entire data migration by a coefficient. In the present application, the total migration rate is proportional to the number of links.

[0042] In this application, the specific method of splitting the migration link into multiple new migration links can be completed by synchronization tools, such as DTS, ETL and other tools involving synchronization or data migration between the source and target ends. In this application, the specific splitting of the migration link is not limited or explained.

[0043] In the present application, the total migration rate of the entire data migration is greatly increased by splitting the migration link; and by monitoring the operating status information of the migration link, a controllable migration rate increase is performed, thereby effectively improving the migration speed.

[0044] In one implementation, the migration link and the corresponding data to be migrated are split synchronously.

[0045] The migration links and the data to be migrated are split synchronously, a plurality of the split migration links and a plurality of the data to be migrated are associated one by one, and the data to be migrated is migrated through the associated migration links.

[0046] In one implementation, the data to be migrated is split according to at least one dimension of a database dimension, a table dimension in a database, a row dimension in a table, and a field dimension in a row.

[0047] It can be understood that the data to be migrated may be several databases. In this case, the data to be migrated should be split according to the database dimension.

[0048] It can be understood that the data to be migrated is split according to the database dimension, that is, the data to be migrated is split with the database as the smallest unit; similarly, the data to be migrated is split according to the table dimension in the database, the row dimension in the table, and the field dimension in the row, that is, the data to be migrated is split with the table in the database, the row in the table, and the field in the row as the smallest unit.

[0049] It can be understood that splitting according to two dimensions of the database dimension, the table dimension in the database, the row dimension in the table, and the field dimension in the row means that the data to be migrated is first split according to the large dimension and then split according to the small dimension; for example, splitting according to the database dimension and the table dimension in the database means that the data to be migrated is split with the database as the smallest unit until each database is split into a separate migration link, and then each database is split with the table in the database as the smallest unit until each table is split into a separate migration link or the splitting stops.

[0050] Similarly, the specific process of splitting according to multiple dimensions of the database dimension, the table dimension in the database, the row dimension in the table, and the field dimension in the row can be understood. This application will not be repeated.

[0051] In one embodiment, Figure 2 As shown, the splitting of the to-be-migrated data and the migration link includes:

[0052] S201, splitting the data to be migrated and the migration link into a preset number of data to be migrated and migration links according to a preset strategy, and configuring each of the split data to be migrated to a corresponding split migration link for migration;

[0053] S202: Continue to split each of the split data to be migrated and the configured migration link according to the preset strategy until the total running status information of the migration link becomes abnormal; the total running status information of the migration link is the sum of the running status information of each migration link.

[0054] It can be understood that each of the split data to be migrated is configured on a split migration link corresponding to each of the split data, wherein each of the split data to be migrated corresponds to a split migration link.

[0055] In this application, the specific details of the preset strategy are not limited.

[0056] It can be understood that if one copy of the data to be migrated is split into N copies of the data to be migrated, each of the N copies of the data to be migrated can be split twice; each copy of the data to be migrated obtained by the second split can be split three times; the splitting is performed progressively in this way until the overall operation status information of the migration link is abnormal. The overall operation status information of the migration link is abnormal, which means that it is caused by the current splitting action. Stopping the splitting means stopping the execution of the current splitting action and returning to the previous step.

[0057] For example, when four copies of data to be migrated are split into eight copies of data to be migrated, the overall running status information of the migration link is abnormal. At this time, the splitting into eight copies of data to be migrated is stopped and returned, and only the splitting result of four copies of data to be migrated is retained.

[0058] In one implementation, when the data to be migrated is split according to the table dimension in the database, the data volume of each table in the database is obtained, and large tables whose data volume is greater than the configured threshold are screened out based on a preset configuration threshold;

[0059] The data to be migrated and the migration links are split according to the number of the large tables until each large table is configured on a corresponding migration link.

[0060] It can be understood that in this step, the situation where the data to be migrated is split according to the table dimension in the database is limited. There are many tables in the database. Large tables are selected according to the amount of data, and only the large tables are effectively split. The remaining small tables in the database can be randomly or on demand assigned to the migration links configured for the large tables, which greatly reduces the number of migration links that need to be split.

[0061] In one implementation, the operation status information of the migration link includes a migration rate, a source end read error amount, and a destination end write error amount; when at least one of the migration rate, the source end read error amount, and the destination end write error amount of the migration link is abnormal, it is confirmed that the operation status information of the migration link is abnormal.

[0062] It can be understood that as long as any one of the migration rate, the source end reading error amount, and the destination end writing error amount of the migration link is abnormal, it is confirmed that the operation status information of the migration link is abnormal.

[0063] In this application, the source end reading and destination end writing rates have bottlenecks. When the migration rate of the migration link is increased without limit, if the source end reading and destination end writing rates are approaching their limits, source end reading errors and destination end writing errors are likely to occur.

[0064] The present application can bring the rate of the entire data migration close to the limit by monitoring the operating status information of the migration link.

[0065] In one implementation, when the total migration rate of the migration link after the split is less than the total migration rate of the migration link before the split, it is determined that the migration rate of the migration link is abnormal; or,

[0066] When the difference between the source end reading error amount of the migration link after the split and the source end reading error amount of the migration link before the split is greater than a first preset threshold, it is determined that the source end reading error amount of the migration link is abnormal; or,

[0067] When the difference between the write error amount of the destination end of the migration link after the split and the write error amount of the destination end of the migration link before the split is greater than a second preset threshold, it is confirmed that the write error amount of the destination end of the migration link is abnormal.

[0068] In one implementation, an upper limit condition of at least one state indicator of the migration link is obtained, and after at least one of the state indicators exceeds its upper limit condition, it is confirmed that the operation state information of the migration link is abnormal.

[0069] In the present application, the state indicator of the migration link can be obtained by system monitoring, or by other methods such as dedicated detection tools, etc., and the present application does not limit the specific acquisition method.

[0070] It is understandable that some limitations of the source end, intermediate link, and destination end can be obtained through historical data, and capacity expansion based on these limitations can more easily reach the limit. Exceeding the upper limit limit means exceeding the limit, thereby confirming that the operating status information of the migration link is abnormal.

[0071] This application sets a rate cap to improve the migration rate in a controllable manner, effectively reducing the migration time.

[0072] The present application can also use this method to test the migration link bottleneck without setting an upper limit on the migration rate, optimize in advance, and reduce the migration time during the formal migration.

[0073] In the present application, for the data migration scenario, the user does not need to worry about the data migration rate at all. The data migration method of the present application can automatically adjust the migration rate, and the user only needs to prepare the resource pool used for the migration link.

[0074] This solution can not only automatically adjust the migration rate, but also collect statistics on the errors reported by the source and destination during the adjustment process to ensure that the stability of the source and destination is not affected during the rate adjustment process.

[0075] This application is not only applicable to homogeneous data migration, but also to heterogeneous data migration; heterogeneous data needs to be converted on the intermediate link during transmission; for example, two fields into one field requires aggregation action.

[0076] Specifically, Figure 3 As shown, taking the table dimension as an example, the specific process of data migration of this application is described in detail in combination with the content in the figure.

[0077] It should be noted that Figure 3 A worker is a link; the number of workers is determined by the configuration of the virtual machine (VM), and there is no restriction on the configuration of the workers in this application.

[0078] The data migration solution for this application includes:

[0079] Create the initial migration link: The initial task is created according to the synchronization requirements of the entire database / instance. When creating a synchronization task through the synchronization tool, all data tables in the database to be synchronized are included. After the migration link starts data migration, the synchronization rate is monitored, such as 20MB / s, for subsequent migration rate comparison. Among them, the real-time transmission rate of the current link can be directly obtained by the synchronization tool.

[0080] Get the source database table metadata:

[0081] After the migration tool connects to the source database, it obtains the total number of rows (rows) and the average row length (avg_length) of each table from the statistical information. The source information of the database stores the amount of data in each table, as well as the number of rows, average row length and other data. The size of each table is estimated based on the total number of rows and average row length. If the size of the entire table exceeds 100GB, the table is marked as a large table. In this way, all large tables exceeding 100GB in the entire library / instance have been marked. For the convenience of subsequent description, it is assumed that there are 4 large tables in the source database, namely bigtable_a (100GB), bigtable_b (200GB), bigtable_c (500GB), and bigtable_d (1TB).

[0082] Split the link according to the table dimension:

[0083] The above steps have counted a list of large tables with a size of more than 100GB. First, sort them in reverse order according to the table size, and split the tasks according to the overall data volume from the table dimension. Split one link into two links, and the data volume of each link is about 1 / 2 of the overall data volume. There are two links in total, analogous to two buckets A and B. First put bigtable_d (1TB) into bucket A, and then put bigtable_a (100GB), bigtable_b (200GB), and bigtable_c (500GB) into bucket B in turn. Small tables less than 100GB can be randomly put into A and B. Split the tables in bucket A into a separate migration link, and split the tables in bucket B into a separate migration link. After the split is completed, delete the old link. After the link split is completed, count the migration rate of each link and summarize it into the overall rate rate'.

[0084] If the rate rate' after link splitting is less than rate, the newly split link is directly merged into the previous link, and the whole process ends here. If the rate rate' after link splitting is greater than rate, it proves that the link splitting has a speed-up effect, but at the same time, it is necessary to determine whether there are a large number of errors when the source end reads data and the destination end inserts data, or whether the number of errors is one order of magnitude more than before the link splitting. If there are no abnormalities, you can go directly to the next step to continue speeding up.

[0085] If each large table is in a separate link, the table dimension split is complete, and the link split is performed according to the row dimension. If more than two large tables are in the same link, follow this step to split the link in the table dimension again for this link until the migration rate does not increase.

[0086] Split the link by row dimension:

[0087] After the above steps, each large table is in a separate migration link. If you want to increase the rate, you can split the task of a table into two parts according to the row dimension. Similar to the logic of the above steps, if a large table has a total of 100 million rows, the median of the maximum and minimum values ​​of the primary key is used as the dividing line. The rows less than the median of the primary key are split into separate links, and the rows greater than the median of the primary key are split into another single-path link. After the link splitting is completed, the migration rate of each link is counted and summarized into the overall rate rate'.

[0088] If the rate rate' after link splitting is less than rate', the newly split link is directly merged into the previous link. If the rate rate' after link splitting is greater than rate', it proves that the link splitting has a speed-up effect, but at the same time, it is necessary to determine whether there are a large number of errors when the source end reads data and the destination end inserts data, or whether the number of errors is one order of magnitude higher than before link splitting. If there are no abnormalities, you can directly follow this step to split the link in the row dimension again.

[0089] An embodiment of the present application provides a data migration device for executing the data migration method described in the above content of the present application. The data migration device is described in detail below.

[0090] like Figure 4 As shown, the data migration device includes:

[0091] A link creation module 101 is used to create a migration link between a source database and a destination database of data to be migrated;

[0092] The link splitting module 102 is used to split the to-be-migrated data and the migration link, and stop the splitting when the running status information of the migration link is abnormal.

[0093] In one implementation, the migration link and the corresponding data to be migrated are split synchronously.

[0094] In one implementation, the data to be migrated is split according to at least one dimension of a database dimension, a table dimension in a database, a row dimension in a table, and a field dimension in a row.

[0095] In one implementation, the link splitting module 102 is further configured to:

[0096] The data to be migrated and the migration link are split into a preset number of data to be migrated and migration links according to a preset strategy, and each split data to be migrated is configured to a corresponding split migration link for migration; each split data to be migrated and the configured migration link are continuously split according to the preset strategy until the total running status information of the migration link is abnormal; the total running status information of the migration link is the sum of the running status information of each migration link.

[0097] In one implementation, the link splitting module 102 is further configured to:

[0098] When the data to be migrated is split according to the table dimension in the database, the data volume of each table in the database is obtained, and large tables whose data volume is greater than the configuration threshold are screened out based on a preset configuration threshold; the data to be migrated and the migration links are split according to the number of the large tables until each large table is configured on a migration link corresponding to the large table.

[0099] In one implementation, the operation status information of the migration link includes a migration rate, a source end read error amount, and a destination end write error amount; the link splitting module 102 is further used to:

[0100] When at least one of the migration rate, the source end reading error amount, and the destination end writing error amount of the migration link is abnormal, it is confirmed that the operation status information of the migration link is abnormal.

[0101] In one implementation, the link splitting module 102 is further configured to:

[0102] When the total migration rate of the migration link after the split is less than the total migration rate of the migration link before the split, it is confirmed that the migration rate of the migration link is abnormal; or, when the difference between the source-end read error amount of the migration link after the split and the source-end read error amount of the migration link before the split is greater than a first preset threshold, it is confirmed that the source-end read error amount of the migration link is abnormal; or, when the difference between the destination-end write error amount of the migration link after the split and the destination-end write error amount of the migration link before the split is greater than a second preset threshold, it is confirmed that the destination-end write error amount of the migration link is abnormal.

[0103] In one implementation, the link splitting module 102 is further configured to:

[0104] An upper limit condition of at least one state indicator of the migration link is obtained, and after at least one of the state indicators exceeds the upper limit condition, it is confirmed that the operation state information of the migration link is abnormal.

[0105] The data migration device provided in the above-mentioned embodiment of the present application and the data migration method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0106] The above describes the internal functions and structure of the data migration device. Figure 5 As shown, in practice, the data migration apparatus can be implemented as a control device, including: a memory 301 and a processor 303 .

[0107] The memory 301 may be configured to store programs.

[0108] In addition, the memory 301 may also be configured to store various other data to support operations on the control device. Examples of such data include instructions for any application or method operating on the control device, contact data, phone book data, messages, pictures, videos, etc.

[0109] The memory 301 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0110] The processor 303 is coupled to the memory 301 and is configured to execute a program in the memory 301 to:

[0111] Create a migration link between the source database and the destination database for the data to be migrated;

[0112] The data to be migrated and the migration link are split, and the splitting is stopped when the running state information of the migration link is abnormal.

[0113] In one implementation, the migration link and the corresponding data to be migrated are split synchronously.

[0114] In one implementation, the data to be migrated is split according to at least one dimension of a database dimension, a table dimension in a database, a row dimension in a table, and a field dimension in a row.

[0115] In one implementation, the processor 303 is specifically configured to:

[0116] The data to be migrated and the migration link are split into a preset number of data to be migrated and migration links according to a preset strategy, and each split data to be migrated is configured to a corresponding split migration link for migration; each split data to be migrated and the configured migration link are continuously split according to the preset strategy until the total running status information of the migration link is abnormal; the total running status information of the migration link is the sum of the running status information of each migration link.

[0117] In one implementation, the processor 303 is specifically configured to:

[0118] When the data to be migrated is split according to the table dimension in the database, the data volume of each table in the database is obtained, and large tables whose data volume is greater than the configuration threshold are screened out based on a preset configuration threshold; the data to be migrated and the migration links are split according to the number of the large tables until each large table is configured on a migration link corresponding to the large table.

[0119] In one implementation, the operation status information of the migration link includes a migration rate, a source end read error amount, and a destination end write error amount; the processor 303 is specifically configured to:

[0120] When at least one of the migration rate, the source end reading error amount, and the destination end writing error amount of the migration link is abnormal, it is confirmed that the operation status information of the migration link is abnormal.

[0121] In one implementation, the processor 303 is specifically configured to:

[0122] When the total migration rate of the migration link after the split is less than the total migration rate of the migration link before the split, it is confirmed that the migration rate of the migration link is abnormal; or, when the difference between the source-end read error amount of the migration link after the split and the source-end read error amount of the migration link before the split is greater than a first preset threshold, it is confirmed that the source-end read error amount of the migration link is abnormal; or, when the difference between the destination-end write error amount of the migration link after the split and the destination-end write error amount of the migration link before the split is greater than a second preset threshold, it is confirmed that the destination-end write error amount of the migration link is abnormal.

[0123] In one implementation, the processor 303 is specifically configured to:

[0124] An upper limit condition of at least one state indicator of the migration link is obtained, and after at least one of the state indicators exceeds the upper limit condition, it is confirmed that the operation state information of the migration link is abnormal.

[0125] In this application, Figure 5 Only some components are shown schematically, which does not mean that the control device only includes Figure 5 Components shown.

[0126] The control device provided in this embodiment is based on the same inventive concept as the data migration method provided in the embodiment of the present application, and has the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0127] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0128] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0131] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0132] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0133] The present application also provides a computer-readable storage medium corresponding to the data migration method provided in the aforementioned embodiment, on which a computer program (ie, a program product) is stored. When the computer program is executed by a processor, the data migration method provided in any of the aforementioned embodiments is executed.

[0134] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0135] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the data migration method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0136] It should be noted that in the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0137] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0138] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A data migration method, characterized in that: include: Create a migration link between the source database and the destination database for the data to be migrated; The data to be migrated and the migration link are split, and the splitting is stopped when the running status information of the migration link is abnormal; the running status information of the migration link includes a migration rate, a source end reading error amount, and a destination end writing error amount; when at least one of the running status information of the migration link is abnormal, it is confirmed that the running status information of the migration link is abnormal; The splitting of the migration link refers to splitting one link into multiple links, selecting the split new links in the cloud computing system, and deleting the old links.

2. The method according to claim 1, characterized in that The migration link and the corresponding data to be migrated are split synchronously.

3. The method according to claim 1, characterized in that The data to be migrated is split according to at least one dimension of a database dimension, a table dimension in a database, a row dimension in a table, and a field dimension in a row.

4. The method according to claim 1, characterized in that: The splitting of the data to be migrated and the migration link includes: Splitting the data to be migrated and the migration link into a preset number of data to be migrated and migration links according to a preset strategy, and configuring each of the split data to be migrated to a corresponding split migration link for migration; The split data to be migrated and the configured migration link are continuously split according to the preset strategy until the total running status information of the migration link becomes abnormal; the total running status information of the migration link is the sum of the running status information of each migration link.

5. The method according to claim 3, characterized in that: When the data to be migrated is split according to the table dimension in the database, the data volume of each table in the database is obtained, and large tables whose data volume is greater than the configured threshold are screened out based on a preset configuration threshold; The data to be migrated and the migration links are split according to the number of the large tables until each large table is configured on a corresponding migration link.

6. The method according to claim 1, characterized in that If the total migration rate of the migration link after the split is less than the total migration rate of the migration link before the split, it is determined that the migration rate of the migration link is abnormal; or, When the difference between the source end reading error amount of the migration link after the split and the source end reading error amount of the migration link before the split is greater than a first preset threshold, it is confirmed that the source end reading error amount of the migration link is abnormal; or, When the difference between the write error amount of the destination end of the migration link after the split and the write error amount of the destination end of the migration link before the split is greater than a second preset threshold, it is confirmed that the write error amount of the destination end of the migration link is abnormal.

7. The method according to claim 1, characterized in that An upper limit condition of at least one state indicator of the migration link is obtained, and after at least one of the state indicators exceeds the upper limit condition, it is confirmed that the operation state information of the migration link is abnormal.

8. A data migration device, characterized in that: include: A link creation module, which is used to create a migration link between a source database and a destination database for data to be migrated; a link splitting module, which is used to split the data to be migrated and the migration link, and stop the splitting when the running status information of the migration link is abnormal; the running status information of the migration link includes the migration rate, the amount of errors reported when reading from the source end, and the amount of errors reported when writing from the destination end; when at least one of the running status information of the migration link is abnormal, confirm that the running status information of the migration link is abnormal; The splitting of the migration link refers to splitting one link into multiple links, selecting the split new links in the cloud computing system, and deleting the old links.

9. A control device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program to: Create a migration link between the source database and the destination database for the data to be migrated; The data to be migrated and the migration link are split, and the splitting is stopped when the running status information of the migration link is abnormal; the running status information of the migration link includes a migration rate, a source end reading error amount, and a destination end writing error amount; when at least one of the running status information of the migration link is abnormal, it is confirmed that the running status information of the migration link is abnormal; The splitting of the migration link refers to splitting one link into multiple links, selecting the split new links in the cloud computing system, and deleting the old links.

Images