Data migration method and device thereof, and computer storage medium

Abstract

The embodiment of the disclosure provides a data migration method and device, a computer storage medium, an electronic device and a computer program product, and belongs to the technical field of computers. The data migration method comprises the following steps: in the case that the selected data migration type is incremental data migration when performing data migration between a source data end and a target data end, a preconfigured incremental migration index table is acquired, the incremental migration index table at least comprises a first source table primary key column, and each row in the first source table primary key column can be used for recording a source table primary key value corresponding to a piece of to-be-migrated data in a source data table; and according to each source table primary key value in the incremental migration index table, incremental data migration is performed between the source data end and the target data end.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular to a data migration method and device, computer storage medium, electronic device and computer program product. BACKGROUND

[0002] With the development of business, the amount of data is increasing, and the database single table of the source data end has several hundred GB, and even the single table TB level data is common, and the super large data volume also tests the performance of the business server. Therefore, how to migrate the data of the source data end to the target data end for storage to ensure the data consistency between the source data end and the target data end becomes a technical problem to be solved. SUMMARY

[0003] The present disclosure provides a data migration method and device, computer storage medium, electronic device and computer program product to overcome or alleviate the above technical problems in the prior art.

[0004] The technical solution adopted by the present disclosure is:

[0005] A data migration method comprises:

[0006] In the case that the selected data migration type for data migration between the source data end and the target data end is incremental data migration, a preconfigured incremental migration index table is obtained, the incremental migration index table at least comprises a first source table primary key column, and each row in the first source table primary key column can be used to record the source table primary key value corresponding to a piece of data to be migrated in the source data table;

[0007] According to each source table primary key value in the incremental migration index table, incremental data migration is performed between the source data end and the target data end.

[0008] Optionally, before the incremental data migration between the source data end and the target data end, in the case that the selected data migration type for data migration between the source data end and the target data end is full data migration,

[0009] a preconfigured full migration index table is obtained, the full migration index table at least comprises a second source table primary key column, and each row in the second source table primary key column can be used to record the source table primary key value corresponding to a piece of data to be migrated in the source data table;

[0010] According to each source table primary key value in the full migration index table, full data migration is performed between the source data end and the target data end.

[0011] Optionally, the incremental migration index table and the full migration index table each further comprises: an integer auto-increment column, configured to configure the amount of data that can be migrated in each batch during batch migration.

[0012] Optionally, the incremental migration index table further comprises: a data operation type, the data operation type comprising at least one of insertion, deletion, and update.

[0013] Optionally, the full data migration between the source data end and the target data end according to each source table primary key value in the full migration index table comprises: batch-wise full data migration between the source data end and the target data end according to the amount of data that can be migrated in each batch and the corresponding source table primary key value configured in the full migration index table, and according to the amount of data that can be migrated in each batch.

[0014] Optionally, the data migration between the source data end and the target data end according to each source table primary key value in the full migration index table further comprises: monitoring the data that has completed full migration at the source data end to obtain corresponding incremental data during the batch-wise full data migration between the source data end and the target data end.

[0015] According to the amount of data that can be migrated in each batch and the corresponding source table primary key value configured in the incremental migration index table, the batch-wise incremental migration of the incremental data between the source data end and the target data end is performed according to the data operation type and the amount of data that can be migrated in each batch.

[0016] Optionally, the monitoring of the full migration data at the source data end to obtain corresponding incremental data comprises: monitoring the data that has completed full migration at the source data end according to a set trigger to obtain corresponding incremental data.

[0017] Optionally, the batch-wise full data migration between the source data end and the target data end according to the amount of data that can be migrated in each batch and the corresponding source table primary key value configured in the full migration index table comprises:

[0018] determining the amount of data of the last batch of migration successful data to determine the starting data for the next batch of data migration, and performing batch-wise full data migration between the source data end and the target data end according to the amount of data that can be migrated in each batch and the source table primary key value corresponding to the starting data configured in the full migration index table, and according to the amount of data that can be migrated in each batch.

[0019] Optionally, the performing the batch-by-batch incremental migration of the incremental data between the source data end and the target data end according to the data operation type and the amount of migratable data in each batch comprises: for the incremental migration data, the data operation type is the newly added data of insertion and update, and the performing the batch-by-batch data replacement of the incremental data between the source data end and the target data end is through a data replacement instruction and a source table primary key value corresponding to a maximum integer auto-increment value corresponding to each newly added data.

[0020] Optionally, the performing the batch-by-batch incremental migration of the incremental data between the source data end and the target data end according to the data operation type and the amount of migratable data in each batch comprises: for the incremental migration data, the data operation type is the newly added data of insertion and update, and the performing the batch-by-batch data replacement of the incremental data between the source data end and the target data end is through a data replacement instruction and a source table primary key value corresponding to a maximum integer auto-increment value corresponding to each newly added data.

[0021] A data migration apparatus comprises:

[0022] The first processing unit is configured to, in a case where the selected data migration type for performing data migration between a source data end and a target data end is incremental data migration, acquire a preconfigured incremental migration index table, the incremental migration index table comprising at least a first source table primary key column, each row in the first source table primary key column being used to record a source table primary key value corresponding to a piece of to-be-migrated data in a source data table.

[0023] The second processing unit is configured to perform incremental data migration between the source data end and the target data end according to each source table primary key value in the incremental migration index table.

[0024] A computer storage medium, the computer storage medium storing a computer executable program, the computer executable program being executed to implement the method according to any one of the embodiments of the present disclosure.

[0025] An electronic device, the electronic device comprising a memory and a processor, the memory being configured to store a computer executable program, and the processor being configured to execute the computer executable program to implement the method according to any one of the embodiments of the present disclosure.

[0026] A computer program product, comprising a computer program, the computer program being executed by a processor to implement the method according to any one of the embodiments of the present disclosure.

[0027] In embodiments of this disclosure, when the selected data migration type for data migration between the source data endpoint and the target data endpoint is incremental data migration, a pre-configured incremental migration index table is obtained. This index table includes at least a first source table primary key column. Each row in the first source table primary key column can be used to record the source table primary key value corresponding to a piece of data to be migrated in the source data table. Based on each source table primary key value in the incremental migration index table, incremental data migration is performed between the source data endpoint and the target data endpoint. This ensures that incremental data migration is based on the incremental migration index table rather than the source data table, avoiding table locking and thus achieving incremental data migration while guaranteeing data consistency between the source data endpoint and the target data endpoint. Attached Figure Description

[0028] Figure 1A The diagram illustrates an application scenario of Embodiment 1 of this disclosure.

[0029] Figure 1B The diagram illustrates another application scenario of the present disclosure.

[0030] Figure 1C The diagram illustrates another application scenario of the embodiments of this disclosure.

[0031] Figure 2 The illustration shows a flowchart of a data migration method according to an embodiment of the present disclosure.

[0032] Figure 3 The illustration shows a flowchart of a data migration method according to an embodiment of the present disclosure.

[0033] Figure 4 The illustration shows a flowchart of a data migration method according to an embodiment of the present disclosure.

[0034] Figure 5 The schematic diagram illustrates the structure of a data migration apparatus according to an embodiment of the present disclosure.

[0035] Figure 6 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0036] To make the technical problems, technical solutions and advantages to be solved by this disclosure clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0037] Figure 1A A schematic diagram illustrating an application scenario of Embodiment 1 of this disclosure is provided. For example... Figure 1AAs shown, the application scenario includes a source data end and a target data end. In this embodiment, the source data end is specifically a source data table, and the target data end includes multiple target data tables, which can be located on the same target server or different target servers.

[0038] In this embodiment, data migration is performed one-to-one between the target data table and the source data table.

[0039] Figure 1B A schematic diagram of another application scenario of the embodiment of the present disclosure is schematically shown. Figure 1B As shown, the application scenario includes a source data end and a target data end. In this embodiment, the source data end is specifically a source data table, and the target data end includes multiple target data tables, which can be located on the same target server or different target servers.

[0040] Unlike the above Figure 1A embodiments, in this embodiment, data migration is performed one-to-many between the target data table and the source data table.

[0041] Figure 1C A schematic diagram of still another application scenario of the embodiment of the present disclosure is schematically shown. Figure 1C As shown, data synchronization is required between two servers, one of which is a source server (which can also be referred to as a source database instance), and the other of which is a target server (which can also be referred to as a target database instance). The source server serves as a source data end, and the target server serves as a target data end. The source server is provided with multiple source data tables, and the target server is provided with multiple target data tables. Data migration is performed between the multiple source data tables and the multiple target data tables to migrate all data in the source data tables to the corresponding target data tables.

[0042] In this embodiment, data migration is performed multiple-to-multiple between the target data table and the source data table.

[0043] In addition, similar to the above Figure 1C When data migration is performed multiple-to-multiple between the target data table and the source data table, in another embodiment, the multiple target data tables can be located on different servers, and the source data table can also be located on different servers, or in other words, distributed migration of data.

[0044] The aforementioned servers are, for example, cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0045] In one approach, data migration is performed directly based on the source table. This process can lead to table locking, preventing further migration and compromising data consistency between the target and source tables, thus failing to meet the needs of typical business scenarios. Furthermore, table locking prevents the migration of incremental data, further compromising data consistency between the target and source tables.

[0046] In the following embodiments of this disclosure, when the selected data migration type for data migration between the source data terminal and the target data terminal is incremental data migration, a pre-configured incremental migration index table is obtained. The incremental migration index table includes at least a first source table primary key column. Each row in the first source table primary key column can be used to record the source table primary key value corresponding to a piece of data to be migrated in the source data table. Incremental data migration is performed between the source data terminal and the target data terminal based on each source table primary key value in the incremental migration index table. This avoids table locking by using the incremental migration index table instead of the source data table, thereby realizing incremental data migration and ensuring data consistency between the source data terminal and the target data terminal.

[0047] The following provides an exemplary description of the data migration process in the above application scenarios. However, it should be noted that the following embodiments are merely illustrative and not unique. For those skilled in the art, other substitutions or modifications can be made to the following steps without departing from the spirit of this disclosure.

[0048] Figure 2 The illustration shows a schematic flowchart of a data migration method according to an embodiment of this disclosure. Figure 2 As shown, the data migration method provided in this embodiment is executed after a full data migration is performed between the source data endpoint and the target data endpoint. Specifically, the data migration method in this embodiment includes:

[0049] S201. When the selected data migration type for data migration between the source data end and the target data end is incremental data migration, obtain the pre-configured incremental migration index table.

[0050] In this embodiment, the incremental migration index table (also referred to as an incremental temporary table) includes at least a first source table primary key column, each row in the first source table primary key column can be used to record a source table primary key value corresponding to a piece of to-be-migrated data in a source data table, and the source table primary key value is also referred to as an ID of the to-be-migrated data.

[0051] Optionally, the incremental migration index table can further include an integer auto-increment column, which is used to configure an amount of data that can be migrated in each batch when performing batch migration. In this way, when step S202 is performed, batch incremental data migration can be implemented, resource consumption can be reduced, smooth data migration can be ensured, and the migration process is imperceptible to users.

[0052] Optionally, the incremental migration index table can further include a data operation type, and the data operation type includes at least one of insertion (also referred to as insert), deletion (also referred to as delete), and update (also referred to as update). The data operation type is referred to as DM (Data Manipulation). Through the insertion, a piece of data can be added in the source data table. Through the deletion, old data in the source data table can be deleted. Through the update, an existing piece of data in the source data table can be modified. By including the data operation type in the incremental migration index table, when step S202 is performed subsequently, incremental data migration can be performed according to the data operation type, so that the accuracy and efficiency of data migration are ensured.

[0053] Specifically, the incremental migration index table can be configured with three fields, namely a primary key field, a data operation type field, and an integer auto-increment field, which are respectively filled with a primary key value, an amount of data that can be migrated in each batch when performing batch migration, and a data operation type, so that the incremental migration index table can record a source table primary key value corresponding to a piece of to-be-migrated data in a source data table, configure the amount of data that can be migrated in each batch, and record the data operation type. For example, a value of the data operation type field is 1, which represents insert; is 2, which represents update; and is 3, which represents delete.

[0054] S202, according to each source table primary key value in the incremental migration index table, performing incremental data migration between the source data end and the target data end.

[0055] Optionally, in this embodiment, according to each source table primary key value in the incremental migration index table, performing incremental data migration between the source data end and the target data end can include performing batch incremental migration of the incremental data between the source data end and the target data end according to the data operation type and the amount of data that can be migrated in each batch.

[0056] As mentioned above, considering that the increment is possible due to the different data operation types, for this, when performing step S202, the increment migration is implemented in combination with the different data operation types. In addition, in order to reduce the consumption of resources by migration, the increment migration is performed in batches.

[0057] For this purpose, optionally, the batch-by-batch increment migration of the increment data between the source data end and the target data end according to the data operation type and the amount of data that can be migrated per batch includes: for the newly added data in the increment migration data, the data operation type is insertion and update, the batch-by-batch data replacement of the increment data between the source data end and the target data end is performed by a data replacement instruction and the source table primary key value corresponding to the maximum integer auto-increment value corresponding to each newly added data. Specifically, the format of the data replacement instruction is replace to.

[0058] In this embodiment, batch migration can be specifically performed based on the integer auto-increment column in the increment migration index table. Specifically, as mentioned above, the integer auto-increment column is used to configure the amount of data that can be migrated per batch for batch migration, for this, after each batch of increment migration is completed, the auto-increment value corresponding to the newly added data is formed in an incremental manner in the integer auto-increment column, for example, if the amount of data that can be migrated per batch is 1000, after each batch of increment data migration is successfully completed, the auto-increment value will increase by 1000, that is, it indicates that 1000 newly added data increment migration is completed, for this, in the process of increment migration, the newly added data of the data operation type of insertion and update can directly implement the batch-by-batch data replacement of the increment data by the source table primary key value corresponding to the maximum integer auto-increment value corresponding to each newly added data. At the same time, due to the existence of the maximum integer auto-increment value, even if a failure occurs or the migration process is paused during a batch of data migration, data migration can be restarted based on the maximum integer auto-increment value as the starting point of migration, without the need to start from the first piece of data to be migrated and perform data migration again, thereby improving the efficiency of data migration.

[0059] Optionally, the batch-by-batch increment migration of the increment data between the source data end and the target data end according to the data operation type and the amount of data that can be migrated per batch includes: for the newly added data in the increment migration data, the data operation type is deletion, and the batch-by-batch data deletion of the increment data between the source data end and the target data end is performed according to the amount of data that can be migrated per batch. In this step of batch-by-batch deletion, the amount of data deleted per batch can also be implemented by referring to the above-mentioned integer auto-increment column.

[0060] In addition, it should be noted that the batch data deletion and batch data replacement processes described above are not limited in time sequence and can be executed sequentially or in parallel. Further, the batch data deletion and batch data replacement processes can be assigned to the same task or different tasks.

[0061] Specifically, when the amount of data to be migrated is large, in order to improve the migration efficiency, multiple tasks can be created and the steps in the above Figure 2 are executed in parallel to achieve parallel migration.

[0062] Figure 3 The flowchart of the data migration method according to an embodiment of the present disclosure is schematically shown. As shown in Figure 3 , in the data migration method provided in the present embodiment, before the incremental data migration between the source data end and the target data end, a full data migration process is performed on the data. Specifically, in the present embodiment, when the selected data migration type for the data migration between the source data end and the target data end is full data migration, the data migration method comprises:

[0063] S301, obtaining a preconfigured full migration index table, the full migration index table comprising at least a second source table primary key column, each row in the second source table primary key column being used to record a source table primary key value corresponding to a piece of data to be migrated in the source data table.

[0064] Optionally, the full migration index table (also referred to as a full temporary table) can further comprise an integer auto-increment column for configuring the amount of data that can be migrated in each batch during batch migration. Accordingly, the incremental migration in batches can be realized when step S302 is executed, the resource consumption is reduced, and the smooth migration of data is ensured, so that the migration process is imperceptible to users.

[0065] Specifically, the incremental migration index table can be configured with two fields, namely a primary key field and an integer auto-increment field, which are respectively filled with a primary key value and the amount of data that can be migrated in each batch during batch migration, so that the full migration index table can record the source table primary key value corresponding to a piece of data to be migrated in the source data table and configure the amount of data that can be migrated in each batch.

[0066] S302, performing full data migration between the source data end and the target data end according to each source table primary key value in the full migration index table.

[0067] S303, monitoring the data that has completed full migration in the source data end to obtain corresponding incremental data.

[0068] Optionally, the full-quantity migrated data is monitored at the source data end to obtain corresponding incremental data, comprising: monitoring the full-quantity migrated data at the source data end according to a set trigger to obtain corresponding incremental data, so as to perform the above-mentioned Figure 2 incremental migration processing.

[0069] The specific set trigger can include an insertion trigger, an update trigger, and a deletion trigger, so as to monitor the full-quantity migrated data at the source data end, and record the corresponding data operation type and the corresponding primary key value into the data operation type field and the primary key field of the incremental migration index respectively when data insertion, data update, and data deletion in the source data table are monitored. The trigger can be specifically set on the source data table.

[0070] Optionally, in the case that the selected data migration type is full-quantity data migration, the full-quantity data migration between the source data end and the target data end according to each source table primary key value in the full-quantity migration index table comprises: performing batch full-quantity data migration between the source data end and the target data end according to each batch migratable data quantity and the corresponding source table primary key value configured in the full-quantity migration index table.

[0071] Optionally, the batch full-quantity data migration between the source data end and the target data end according to each batch migratable data quantity and the corresponding source table primary key value configured in the full-quantity migration index table comprises:

[0072] determining the data quantity of the last batch of migrated data to determine the starting data for the next batch of data migration, and performing batch full-quantity data migration between the source data end and the target data end according to each batch migratable data quantity and the source table primary key value corresponding to the starting data configured in the full-quantity migration index table.

[0073] Specifically, the full-amount migration index table includes an integer auto-increment column, and the batch migration is performed based on the integer auto-increment column. Specifically, as described above, the integer auto-increment column is configured to configure the amount of data that can be migrated in each batch for batch migration. To this end, after each batch of full-amount migration is completed, an auto-increment value corresponding to the successfully migrated data is formed in the integer auto-increment column in an incremental manner. For example, if the amount of data that can be migrated in each batch is 1000, the auto-increment value is increased by 1000 after each batch of incremental data migration is successfully completed, indicating that full-amount migration of 1000 data is completed. The batch data replacement of the full-amount data can be directly implemented by using the maximum integer auto-increment value. At the same time, due to the existence of the maximum integer auto-increment value, even if a failure occurs or the migration process is paused during the batch data migration, the data migration can be restarted based on the maximum integer auto-increment value as the starting point of the migration, without the need to start the data migration from the first piece of data to be migrated, thereby improving the efficiency of the data migration.

[0074] Figure 4 An example of a flowchart of a data migration method is schematically shown. As shown in the figure, the method includes the following steps. Figure 4

[0075] S401, creating and configuring a full-amount migration index table and an incremental migration index table.

[0076] In step S401, two fields are added to the incremental migration index table, namely, a primary key field and an integer auto-increment field. Three fields are added to the incremental migration index table, namely, a primary key field, a data operation type field, and an integer auto-increment field.

[0077] S402A, obtaining full-amount data from the source data end based on the full-amount migration index table.

[0078] S403A, loading the to-be-migrated data involved in the full-amount migration to the target data end.

[0079] S404A, updating the maximum auto-increment value in the integer auto-increment field of the full-amount migration index table for the next batch of full-amount migration.

[0080] S402B, monitoring and obtaining incremental data in the data for which the full-amount migration is completed through a set trigger, and obtaining incremental data from the source data end based on the incremental migration index table.

[0081] S403B, migrating the incremental data to the target data end.

[0082] As described above, the incremental data is migrated to the target data end by referring to the processing of the batch data deletion and the batch data replacement.

[0083] ​S404B, updating the maximum increment value in the integer increment field of the increment migration index table for the next batch of increment migration.

[0084] After the full migration and the increment migration are completed, the following cleanup steps are performed.

[0085] S405, deleting the trigger.

[0086] S406, deleting the increment migration index table.

[0087] S407, deleting the full migration index table.

[0088] In this embodiment, the execution order of steps S405-S407 is not particularly limited.

[0089] By performing the above cleanup steps, storage space is saved, resource consumption is reduced, and re-creation is performed when data migration needs to be performed again.

[0090] Figure 5 The structure of a data migration device according to an embodiment of the present disclosure is schematically shown. As shown in the figure, it includes: Figure 5

[0091] The first processing unit 501 is configured to, in the case where the selected data migration type for data migration between the source data end and the target data end is increment data migration, obtain a preconfigured increment migration index table, the increment migration index table including at least a first source table primary key column, each row in the first source table primary key column being used to record a source table primary key value corresponding to a piece of data to be migrated in the source data table.

[0092] The second processing unit 502 is configured to perform increment data migration between the source data end and the target data end according to each source table primary key value in the increment migration index table.

[0093] Optionally, the first processing unit 501 is further configured to, before performing increment data migration between the source data end and the target data end, in the case where the selected data migration type for data migration between the source data end and the target data end is full data migration, obtain a preconfigured full migration index table, the full migration index table including at least a second source table primary key column, each row in the second source table primary key column being used to record a source table primary key value corresponding to a piece of data to be migrated in the source data table.

[0094] The second processing unit 502 is further configured to perform full data migration between the source data end and the target data end according to each source table primary key value in the full migration index table.

[0095] ​Optionally, the incremental migration index table and the full migration index table each further comprises: an integer auto-increment column, used to configure the amount of data that can be migrated in each batch during batch migration.

[0096] Optionally, the incremental migration index table further comprises: a data operation type, the data operation type comprising at least one of insertion, deletion, and update.

[0097] Optionally, in the case where the selected data migration type is full data migration, the second processing unit 502 is specifically configured to perform batched full migration of data between the source data end and the target data end according to the amount of data that can be migrated in each batch and the corresponding source table primary key values configured in the full migration index table, and according to the amount of data that can be migrated in each batch.

[0098] Optionally, the second processing unit 502 is specifically configured to monitor the data that has completed full migration at the source data end during the batched full migration of data between the source data end and the target data end, to obtain corresponding incremental data; and perform batched incremental migration of the incremental data between the source data end and the target data end according to the data operation type and the amount of data that can be migrated in each batch, and according to the amount of data that can be migrated in each batch and the corresponding source table primary key values configured in the incremental migration index table.

[0099] Optionally, the second processing unit 502 is specifically configured to monitor the data that has completed full migration at the source data end according to a set trigger, to obtain corresponding incremental data.

[0100] Optionally, the second processing unit 502 is specifically configured to determine the amount of data of the data of the last batch of migration that is successful, to determine the starting data for the next batch of data migration, and to perform batched full migration of data between the source data end and the target data end according to the amount of data that can be migrated in each batch and the corresponding source table primary key values configured in the full migration index table, and according to the amount of data that can be migrated in each batch.

[0101] Optionally, the second processing unit 502 is specifically configured to perform batched data replacement of the incremental data between the source data end and the target data end by means of a data replacement instruction and the source table primary key value corresponding to the maximum integer auto-increment value corresponding to each new data, for the new data in the incremental migration data, the data operation type of which is insertion or update.

[0102] Optionally, the second processing unit 502 is specifically configured to perform batched data deletion of the incremental data between the source data end and the target data end according to the amount of data that can be migrated in each batch, for the new data in the incremental migration data, the data operation type of which is deletion.

[0103] Figure 6 A schematic diagram of an electronic device is shown schematically. As shown, the electronic device includes a memory 601 and a processor 602, the memory has stored thereon a computer executable program, and the processor is configured to execute the computer executable program to implement the method according to any one of the embodiments of the present disclosure. Figure 6

[0104] The embodiments of the present disclosure further provide a computer storage medium, which has stored thereon a computer executable program, and the computer executable program is executed to implement the method according to any one of the embodiments of the present disclosure.

[0105] The embodiments of the present disclosure further provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the method according to any one of the embodiments of the present disclosure.

[0106] The above-described embodiments are merely specific implementations of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, but not to limit the same. The protection scope of the present disclosure is not limited thereto. Even though the present disclosure has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any modification or easy-to-think change, or equivalent replacement of some technical features of the technical solutions recorded in the foregoing embodiments can be made within the technical scope disclosed by the present disclosure by any person skilled in the art. Such modification, change or replacement does not cause the corresponding technical solution to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.​

Claims

1. A data migration method, characterized by, The application comprises: In the case that the selected data migration type between the source data end and the target data end is incremental data migration, a pre-configured incremental migration index table is obtained, the incremental migration index table at least comprising a first source table primary key column, each row in the first source table primary key column being used to record a source table primary key value corresponding to a piece of data to be migrated in the source data table; According to each source table primary key value in the incremental migration index table, incremental data migration is performed between the source data end and the target data end, the incremental migration index table being configured with three fields, namely a primary key field, a data operation type field and an integer auto-increment field, which are filled with a primary key value, a data amount of each batch to be migrated in batch migration and a data operation type, so that the incremental migration index table can record a source table primary key value corresponding to a piece of data to be migrated in the source data table, configure a data amount of each batch to be migrated and record a data operation type, the full-amount migration index table comprising an integer auto-increment column for configuring a data amount of each batch to be migrated in batch migration, after each batch of full-amount migration is completed, an auto-increment value corresponding to successfully migrated data is formed in the integer auto-increment column in an incremental manner, and the maximum integer auto-increment value is directly used to realize batch replacement of full-amount data, and the existence of the maximum integer auto-increment value ensures that, even if a failure occurs or the migration process is paused during a batch of data migration, data migration can be restarted based on the maximum integer auto-increment value as a starting point, without performing data migration from the first piece of data to be migrated.

2. The method of claim 1, wherein, Further comprising: Before performing incremental data migration between the source data end and the target data end, In the case that the selected data migration type between the source data end and the target data end is full-amount data migration, a pre-configured full-amount migration index table is obtained, the full-amount migration index table at least comprising a second source table primary key column, each row in the second source table primary key column being used to record a source table primary key value corresponding to a piece of data to be migrated in the source data table; According to each source table primary key value in the full-amount migration index table, full-amount data migration is performed between the source data end and the target data end.

3. The method of claim 2, wherein, The incremental migration index table and the full-amount migration index table each further comprise an integer auto-increment column for configuring a data amount of each batch to be migrated in batch migration.

4. The method of claim 3, wherein, The incremental migration index table further comprises a data operation type, the data operation type comprising at least one of insertion, deletion and update.

5. The method of claim 3, wherein, The full-amount data migration according to each source table primary key value in the full-amount migration index table between the source data end and the target data end comprises: According to the data amount of each batch to be migrated in the full-amount migration index table and the corresponding source table primary key value, batch full-amount data migration is performed between the source data end and the target data end according to the data amount of each batch to be migrated.

6. The method of claim 5, wherein, The full-quantity data migration between the source data end and the target data end according to each source table primary key value in the full-quantity migration index table further comprises a process of batch-wise full-quantity data migration between the source data end and the target data end, The source data end is monitored to obtain corresponding incremental data after full-quantity migration is completed; According to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and the target data end according to the data operation type and the batch-wise incremental data migration between the source data end and 7. The method of claim 6, wherein, ​ ​ 8. The method of claim 6, wherein, ​ ​ 9. The method of claim 6, wherein, ​ ​ 10. The method of claim 6, wherein, ​ ​ 11. A data migration apparatus, characterized by comprising: ​ ​ The second processing unit is configured to perform incremental data migration between the source data end and the target data end according to each source table primary key value in the incremental migration index table. The incremental migration index table includes three fields, i.e., a primary key field, a data operation type field, and an integer auto-increment field, which are filled with a primary key value, a data amount of each batch of migratable data configured during batch migration, and a data operation type, respectively. The incremental migration index table can record a source table primary key value corresponding to a piece of data to be migrated in a source data table, configure a data amount of each batch of migratable data, and record a data operation type. The full migration index table includes an integer auto-increment column for configuring a data amount of each batch of migratable data during batch migration. After each batch of full migration is completed, an auto-increment value corresponding to successfully migrated data is formed in the integer auto-increment column in an incremental manner. Full data is replaced by batch data directly by using a maximum integer auto-increment value. The presence of the maximum integer auto-increment value ensures that, even if a batch of data migration fails or the migration process is paused, data migration can be restarted based on the maximum integer auto-increment value as a starting point for migration, without the need to start data migration from the first piece of data to be migrated.

12. A computer storage medium, characterized in that The computer storage medium stores a computer executable program, and the computer executable program is run to implement the method according to any one of claims 1-10.

13. An electronic device, comprising: The electronic device includes a memory and a processor. The memory stores a computer executable program. The processor is configured to run the computer executable program to implement the method according to any one of claims 1-10.

14. A computer program product, characterised in that, The computer program is executed by a processor to implement the method according to any one of claims 1-10.

Images