Data migration method and device, equipment, storage medium and computer program product

Abstract

The invention discloses a data migration method and device, equipment, a storage medium and a computer program product, and relates to the technical field of data migration, and the method comprises the steps: carrying out the slicing of transaction data in an original database based on the features of a to-be-migrated demand side, and obtaining the sliced data; dividing the slice data according to the data size of storage resources occupied by the slice data in the original database to obtain gray scale batch data and formal batch data; and after the correctness of the migration process is verified through the gray scale batch data, migrating the formal batch data from the original database to the target database in batches. The data migration time is shortened.

Description

Technical Field

[0001] The present application relates to the technical field of data migration, and in particular to a data migration method, a data migration apparatus, a data migration device, a storage medium and a computer program product. Background Art

[0002] With the continuous development of information technology, in the actual operation of enterprises, the original database data often needs to be migrated due to the need for upgrading, relocation or expansion of the original database. Generally speaking, the data involved in the migration is huge, and the traditional data migration solution directly migrates large-scale data, resulting in a long migration time.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present application is to provide a data migration method, a data migration device, a data migration equipment, a storage medium and a computer program product, aiming to solve the technical problem of long data migration time.

[0005] To achieve the above objectives, the present application proposes a data migration method, which includes:

[0006] Slice the transaction data in the original database based on the characteristics of the demand side to be migrated to obtain sliced ​​data;

[0007] According to the amount of storage resources occupied by the slice data in the original database, the slice data is divided to obtain grayscale batch data and formal batch data;

[0008] After verifying the correctness of the migration process through the grayscale batch data, the formal batch data is migrated from the original database to the target database in batches.

[0009] In one embodiment, while migrating the formal batch data from the original database to the target database in batches, the method further includes:

[0010] Obtain the transaction read request and transaction write request of the demand side to be migrated in the traffic gateway;

[0011] Shielding the transaction write request of the party to be migrated to the original database;

[0012] Based on the migration status of the transaction data of the demand party to be migrated, the transaction read request of the demand party to be migrated is processed through an online service cluster connected to the database, wherein the online service cluster receives the read and write requests to the database forwarded by the traffic gateway during the migration process.

[0013] In one embodiment, the step of processing the transaction read request of the demand party to be migrated through the online service cluster connected to the database based on the migration status of the transaction data of the demand party to be migrated includes:

[0014] If the transaction data of the demand party to be migrated is in the migrating state or the non-migrated state, the transaction read request of the demand party to be migrated is processed through the original online service cluster connected to the original database;

[0015] If the transaction data of the demand party to be migrated is in a migrated state, the transaction read request of the demand party to be migrated is processed through the target online service cluster connected to the target database.

[0016] In one embodiment, the method further comprises:

[0017] Determine whether transaction data will change within a preset period of time;

[0018] If the transaction data will change within the preset period, the slice data will be regarded as active transaction data;

[0019] If the transaction data does not change within the preset period, the slice data will be used as historical transaction data;

[0020] During the migration process, active transaction data is migrated first.

[0021] In one embodiment, while migrating the formal batch data from the original database to the target database in batches, the method further includes:

[0022] In the original database, build a backup table of the migrated transaction data;

[0023] If the target database is down, the migrated transaction data is restored to the original database according to the backup table, and the transaction read requests and write requests of the party to be migrated are processed through the original database.

[0024] In one embodiment, the step of slicing the transaction data in the original database based on the characteristics of the demand party to be migrated to obtain the sliced ​​data includes:

[0025] Obtaining a database object of an original database, wherein the database object includes a table and an index of the table;

[0026] Establish a database object consistent with the original database to obtain the target database;

[0027] A target online service cluster connected to the target database is established, wherein the target online service cluster receives read and write requests to the target database forwarded by a traffic gateway during the migration process.

[0028] In addition, to achieve the above purpose, the present application also proposes a data migration device, which includes: a slicing module, which is used to slice the transaction data in the original database based on the characteristics of the demand party to be migrated to obtain sliced ​​data;

[0029] A partitioning module is used to partition the slice data according to the amount of storage resources occupied by the slice data in the original database to obtain grayscale batch data and formal batch data;

[0030] The migration module is used to migrate the formal batch data from the original database to the target database in batches after verifying the correctness of the migration process through the grayscale batch data.

[0031] In addition, to achieve the above objectives, the present application also proposes a data migration device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data migration method described above.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data migration method described above are implemented.

[0033] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the data migration method described above are implemented.

[0034] Compared with the prior art, when performing data migration, large-scale data is directly migrated, resulting in a long migration time; the present application first slices the transaction data in the original database based on the characteristics of the demand side to be migrated, and then divides the sliced ​​data according to the data volume to obtain grayscale batch data and formal batch data. After verifying the correctness of the migration process, the formal batch data is migrated from the original database to the target database in batches, and large-scale data is sliced, classified, and migrated in batches. Through the new migration strategy, the migration time is shortened. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the data migration method of the present application;

[0038] Figure 2 A schematic diagram of a migration architecture provided for the first embodiment of the data migration method of the present application;

[0039] Figure 3 A schematic diagram of request forwarding provided for the first embodiment of the data migration method of the present application;

[0040] Figure 4 A brief flowchart of the second embodiment of the data migration method of the present application is provided;

[0041] Figure 5 This is a schematic diagram of the module structure of the data migration device of this application;

[0042] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the data migration method of this application.

[0043] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0045] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0046] In this embodiment, for ease of description, the device that executes the data migration program is used as the execution subject for explanation below, wherein the device that executes the data migration can be a computing service device with data processing, network communication and program running functions, or an electronic device, processor, etc. that can realize the above functions.

[0047] Based on this, the present application embodiment provides a data migration method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the data migration method of the present application.

[0048] In this embodiment, the data migration method includes steps S10 to S30:

[0049] Exemplarily, the data migration method of the present application is applied to a banking system, that is, the holder and updater of the database is a bank, and correspondingly, the party requiring migration is a customer of the bank.

[0050] Step S10, slicing the transaction data in the original database based on the characteristics of the demand side to be migrated to obtain sliced ​​data;

[0051] It should be noted that the demand party to be migrated (i.e., the customer) is the party to which the transaction data in the original database belongs. For example, the transaction data stored in the database is the customer's transaction data; the characteristics of the demand party to be migrated include the customer's unique identifier (such as a bank card number), transaction type, transaction date, etc.; transaction data includes transaction subject information, transaction time information, transaction amount information, transaction products, etc.; sliced ​​data is a part of the transaction data, similar to cutting out sliced ​​data from all the transaction data in the original database according to certain rules (characteristics of the demand party to be migrated). Exemplarily, the characteristics of the demand party to be migrated can also be expressed as a customer dimension. When slicing, the transaction data in the original database is sliced ​​according to the customer dimension to obtain sliced ​​data.

[0052] Since the scale of transaction data stored in the original database is very large, if the large-scale transaction data is directly migrated, the migration time will be long; therefore, through step S10, the transaction data stored in the original database is sliced ​​based on the characteristics of the demand party to be migrated, and the large-scale transaction data is sliced ​​according to certain rules (i.e., the characteristics of the demand party to be migrated) to obtain multiple sliced ​​data. When performing subsequent data migration, smaller data volumes can be migrated, which can shorten the migration time.

[0053] In one implementation, before step S10, the data migration method includes steps A10 to A30:

[0054] Step A10, obtaining a database object of the original database, wherein the database object includes a table and an index of the table;

[0055] Step A20, establishing a database object consistent with the original database to obtain a target database;

[0056] In this embodiment, the type of database is not limited.

[0057] It should be noted that the purpose of obtaining the tables contained in the original database and the indexes and other database objects created for these tables is to establish a table structure (including the table name, column name, column data type, column constraints, etc.) and a corresponding index structure (including the index name, the column corresponding to the index, the index type, etc.) that is exactly the same as the original database to obtain the target database.

[0058] Through the above operations of establishing database objects, a new database will eventually be formed, which is the target database. It is exactly the same as the original database in terms of the structure of database objects, but it can be in different operating environments, such as different servers, different database management system versions, etc., and there is no data in the target database initially, but it has the same structure as the original database. Later, data can be filled in according to specific needs, such as migrating data from the original database.

[0059] Step A30: Building a target online service cluster connected to the target database, wherein the target online service cluster receives read and write requests to the target database forwarded by the traffic gateway during the migration process.

[0060] It should be noted that the target online service cluster is a collection of online servers that cooperate with each other, and its purpose is to provide customers with online services that interact with the target database. Online services can be some software programs or processes that run on the server and are responsible for processing various database-related operation requests, such as receiving customer transaction read requests, executing data write operations, etc. By forming such a cluster, it is possible to achieve the advantages of efficient processing of database access, load balancing, and high availability, so that the online service can communicate effectively with the target database, thereby being able to receive and process various external operation requests related to the target database.

[0061] The traffic gateway is on the critical path of network traffic and is mainly responsible for receiving external read and write requests to the target database. Read and write requests are requests to read data (such as querying a record in the database, obtaining certain statistical data, etc.) and requests to write data (such as inserting a new record into the database, updating an existing record, etc.).

[0062] During the migration process, the target online service cluster will receive external read and write requests to the target database through the traffic gateway. That is, when a customer or other application wants to read or write to the target database, the request will first be received by the traffic gateway, and then the traffic gateway will forward these requests to the target online service cluster according to certain rules, and the target online service cluster will further process these requests, thereby realizing normal read and write operations on the target database.

[0063] For example, after the target database and the target online service cluster are configured, Figure 2As shown, the traffic gateway sends the customer's request to different online service clusters for processing on different databases according to whether the customer has been migrated. Specifically, the original online service cluster processes the requests sent by non-migrated customers for the original database, and the target online service cluster processes the requests sent by migrated customers for the target database.

[0064] In this implementation, the traffic gateway sends the customer's request to different online service clusters for processing different databases according to the result of whether the customer has migrated. During the data migration period, customers who have not yet migrated can still access the original database normally through the original online service cluster and continue to carry out daily business operations, such as querying orders and making payments, without being affected by the data migration of some customers. Migrated customers can interact with the target database through the target online service cluster and successfully complete various business activities in the new database environment, which can ensure business continuity and help to carry out various tasks efficiently and safely during data migration.

[0065] Step S20, dividing the slice data according to the amount of storage resources occupied by the slice data in the original database to obtain grayscale batch data and formal batch data;

[0066] It should be noted that the amount of storage resources occupied by slice data in the original database refers to the size of the storage space actually occupied by slice data in the original database. For example, a group of slice data occupies 50MB of storage space in the original database. Divide the slice data that occupies a large amount of storage resources, for example, divide 50GB of slice data into 1 grayscale batch data of 1GB and 49 formal batch data. During the data migration process, first conduct preliminary testing on the grayscale batch data, and then migrate the formal batch data, so as to effectively ensure the smooth progress of data migration.

[0067] Step S30, after verifying the correctness of the migration process through gray-scale batch data, the formal batch data is migrated from the original database to the target database in batches.

[0068] It should be noted that the grayscale batch data is a relatively small part of the selected data, which is used to verify the correctness of the data migration process within a limited scope, wherein the data migration process includes: the steps of migrating from the original database to the target database, and correctness means that the same grayscale batch data as in the original database can be queried in the target database. It can be imagined that after the verification is correct, the grayscale batch data already exists in the target database. The formal batch data is the sliced ​​data remaining except the grayscale batch data after the division of the grayscale batch data and the formal batch data. After the grayscale test of the formal batch data is completed and the correctness has been verified, the formal batch data will be migrated from the original database to the target database to ensure the safe migration of the formal batch data.

[0069] like Figure 3 As shown, after all the data in the original database is migrated to the target database, the traffic gateway directly forwards all customer requests to the target online service cluster to request the target database.

[0070] In one embodiment, while migrating the formal batch data from the original database to the target database in batches, the data migration method includes steps B10 to B20:

[0071] Step B10, constructing a backup table of the migrated transaction data in the original database;

[0072] It should be noted that in the original database, the backup table of the migrated transaction data is constructed in order to retain a copy of the original migrated data in the original database, which can be used as a disaster recovery backup of the target database in a short period of time after the migration.

[0073] After step B10, the migrated transaction data in the original database is cleaned up, and the table data in the original database is cleaned up. On the one hand, the data volume of the original table object can be reduced and the table query performance can be improved; on the other hand, the transaction data that has not been migrated can be intuitively displayed. After all migrations are completed, the table object data will be empty.

[0074] Step B20: If the target database is down, the migrated transaction data is restored to the original database according to the backup table, and the transaction read requests and write requests of the demand party to be migrated are processed through the original database.

[0075] It should be noted that if the target database is down and cannot provide services at all, the migrated data can be restored from the backup table to the original database, and the original database can be used to provide services to the party to be migrated.

[0076] In this embodiment, when an unexpected situation such as a target database crash occurs, the migrated transaction data can be quickly restored to the original database according to the backup table, so that the business can be immediately switched to the original database to continue running, and the party requiring migration can still issue transaction read requests and write requests normally, and they are processed in time through the original database, avoiding business interruption due to target database failure, effectively preventing data loss caused by target database crash, ensuring data integrity, and allowing all migrated transaction data to be retained, thereby improving overall reliability and stability.

[0077] This embodiment provides a data migration method, which slices the transaction data stored in the original database based on the characteristics of the demand party to be migrated, and slices the large-scale transaction data according to the characteristics of the demand party to be migrated to obtain multiple sliced ​​data; then determine the actual storage space occupied by the sliced ​​data in the original database, and then divide the sliced ​​data into two parts based on the size of the data as a division standard: one part is grayscale batch data for small-scale testing, and the other part is formal batch data that will be processed according to the normal migration process after the grayscale test is completed; large-scale data is sliced, classified, and migrated in batches, and the migration time is shortened through a new migration strategy.

[0078] During data migration, it is necessary to consider the writing of incremental data in data migration to ensure data consistency. The current method to ensure data consistency during data migration is to close the original database. During the migration process, the entire online service cluster cannot provide external services.

[0079] Based on this, in the second embodiment of the present application, by shielding the transaction write requests of the online service cluster, it is ensured that no incremental data is written during the migration process, and the consistency of the data before and after the database migration is guaranteed; by controlling the transaction write requests of the migration demander through the online service cluster, only the transaction write requests of the small number of customers being migrated are affected, and the transaction write requests of other customers in the entire online service cluster are not affected, which greatly improves the availability of the online service cluster during the data migration process.

[0080] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can refer to the above introduction, and will not be repeated in the following. Figure 4 , while migrating the formal batch data from the original database to the target database in batches, the data migration method further includes steps D10 to D30:

[0081] Step D10, obtaining the transaction read request and transaction write request of the demand party to be migrated in the traffic gateway;

[0082] It should be noted that a transaction read request refers to a request issued by the party to be migrated who wants to view and analyze transaction data from a database storing transaction data; a transaction write request refers to a request issued by the party to be migrated who wants to modify, add or delete transaction data from a database storing transaction data; a read request for transaction data information issued by the party to be migrated who wants to manage and control network traffic and / or a write request to modify transaction data are received by the traffic gateway responsible for managing and controlling network traffic.

[0083] In this embodiment, the manner in which the traffic gateway determines the database operation request from the data traffic is not limited.

[0084] Step D20, shielding the transaction write request of the party to be migrated to the original database;

[0085] It should be noted that in order to avoid writing incremental data during the migration process and to ensure the consistency of data before and after migration, it is necessary to shield the requests from the party to be migrated to modify, add or delete transaction data on the original database, so as to avoid changes in the data in the original database due to the write requests from the party to be migrated, and ensure that the migrated data is accurate, complete and consistent with the data values ​​before migration.

[0086] Exemplarily, only the transaction write requests to the original database from the clients who are migrating are shielded, and the transaction write requests to the original database from the clients who are not migrating are not affected.

[0087] Step D30, based on the migration status of the transaction data of the demand party to be migrated, the transaction read request of the demand party to be migrated is processed through the online service cluster connected to the database, wherein the online service cluster receives the read and write requests to the database forwarded by the traffic gateway during the migration process.

[0088] It should be noted that during the data migration process, transaction data will be in different stages or states. For example, the migration may have just begun and the data has not yet undergone any substantial movement operations; or some data has been migrated and the remaining data is being migrated; or all data has been migrated. Different processing methods can be used to respond to transaction read requests in different states. For example, if the data has been migrated, the processing of the read request can obtain data from the target database; if it is still in the early stages of migration, the data can be obtained from the original database.

[0089] In one implementation, in step D30, the data migration method further includes:

[0090] If the transaction data of the demand party to be migrated is in the migrating state or the non-migrated state, the transaction read request of the demand party to be migrated is processed through the original online service cluster connected to the original database;

[0091] If the transaction data of the demand party to be migrated is in a migrated state, the transaction read request of the demand party to be migrated is processed through the target online service cluster connected to the target database.

[0092] It should be noted that the "Migrating" state means that the transaction data of the party to be migrated is currently in the process of being transferred from the original database to the target database. At this stage, some data may have been successfully migrated to the target database, while some data remains in the original database waiting for further migration. The "Not Migrated" state means that the transaction data of the party to be migrated has not yet started the migration operation, and all of it is completely stored in the original database. The "Migrated" state means that the transaction data of the party to be migrated has been successfully migrated from the original database to the target database, and can be used and accessed normally in the target database.

[0093] When the transaction data is in the migrating state or the unmigrated state, since the transaction data has not been completely migrated from the original database to the target database, the read request is processed by the original online service cluster connected to the original database. When the transaction data is in the migrated state, the transaction data is completely migrated from the original database to the target database, and the read request is processed by the target online service cluster connected to the target database.

[0094] In this embodiment, according to the different migration states of the transaction data of the party to be migrated, the corresponding original online service cluster connected to the original database or the target online service cluster connected to the target database is used to process the transaction read request issued by the party to be migrated, so as to ensure that the required transaction data can be accurately and efficiently obtained from the corresponding database to meet the query requirements of the party to be migrated. The entire data migration process does not affect the read request of the party to be migrated of the online service cluster, thereby improving the availability of the online service cluster during the data migration process.

[0095] In this embodiment, by shielding write requests, it is ensured that the migrated data is based on the original state of the original database without interference, thereby ensuring the accuracy and integrity of the target database data after migration. Processing transaction read requests based on the migration status of the transaction data of the party to be migrated can meet the query needs of the party to be migrated for transaction data at different stages of data migration and maintain the normal operation of the business.

[0096] At present, when performing data migration, all transaction data are migrated as a whole. Among them, some transaction data and non-transaction data are very large in volume. It is easy for a large amount of data to occur during the data migration process, which increases the interruption time of customer transactions.

[0097] Based on this, the third embodiment of the present application proposes a method for classifying transaction data, which divides transaction data into active transaction data and historical transaction data, wherein the active transaction data has a smaller data volume and a shorter migration time, thereby avoiding the occurrence of transaction data with a larger amount of migration data during the data migration process and shortening the impact time on customer transactions; the historical transaction data has a larger data volume and a longer migration time, but the historical transaction data does not affect the customer's transaction requests.

[0098] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction, and will not be described in detail later. The data migration method further includes steps T10 to T40:

[0099] Step T10, determining whether the transaction data will change within a preset period of time;

[0100] It should be noted that the preset time period can be within the next day, within a week, or within the period from the beginning of the month to the end of the month, and the specific setting is not limited; for example, a data monitoring system can be built to check the status of transaction data in real time or regularly, and by comparing the data values ​​before and after, it can be determined whether any data has changed within the preset time period.

[0101] Step T20, if the transaction data will change within the preset period, the slice data is used as active transaction data;

[0102] It should be noted that within the preset time period, the transaction data is expected to change, be updated, etc., in which case the sliced ​​transaction data will be regarded as active transaction data. For example, in the business of financial institutions, if the preset time period is the beginning of each month, due to the concentrated development of businesses such as salary payment and financial product expiration processing, the transaction amount, transaction frequency and other transaction data of the demand side to be migrated will change, in which case the transaction amount and transaction frequency will be regarded as active transaction data.

[0103] Step T30: if the transaction data has not changed within the preset period, the slice data is used as historical transaction data;

[0104] It should be noted that within the preset time period, the transaction data will not change or be updated. For example, if the preset time period is within a week, and the customer purchased a ten-year financial product a year ago, this part of the transaction data of the demand side to be migrated will be treated as historical transaction data and will not be easily changed.

[0105] Step T40: during the migration process, active transaction data is migrated first.

[0106] It should be noted that during the migration process, since the amount of active transaction data is small and closely related to the current business activities, migrating this data first can enable customers who rely on active transaction data to obtain the latest and most critical transaction data as soon as possible during the migration process, thereby ensuring business continuity and reducing the impact of business interruptions or delays that may be caused by data migration.

[0107] In this embodiment, by determining changes in transaction data, classifying and slicing data, and prioritizing the migration of active transaction data, it is possible to avoid migrating large amounts of transaction data during the data migration process, thereby shortening the impact time on customer transactions; this helps to ensure the smooth progress of business and reduce the impact of business interruptions or delays that may be caused by data migration.

[0108] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data migration method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0109] This application also provides a data migration device, please refer to Figure 5 , the data migration device comprises:

[0110] The slicing module 10 is used to slice the transaction data in the original database based on the characteristics of the demand party to be migrated to obtain sliced ​​data;

[0111] A division module 20 is used to divide the slice data according to the amount of storage resources occupied by the slice data in the original database to obtain grayscale batch data and formal batch data;

[0112] The migration module 30 is used to migrate the formal batch data from the original database to the target database in batches after verifying the correctness of the migration process through the grayscale batch data.

[0113] Optionally, the slicing module 10 is further used to obtain a database object of the original database, wherein the database object includes a table and an index of the table;

[0114] Establish a database object consistent with the original database to obtain the target database;

[0115] A target online service cluster connected to the target database is established, wherein the target online service cluster receives read and write requests to the target database forwarded by a traffic gateway during the migration process.

[0116] Optionally, the migration module 30 is further used to obtain a transaction read request and a transaction write request of the demand party to be migrated in the traffic gateway;

[0117] Shielding the transaction write request of the party to be migrated to the original database;

[0118] Based on the migration status of the transaction data of the demand party to be migrated, the transaction read request of the demand party to be migrated is processed through an online service cluster connected to the database, wherein the online service cluster receives the read and write requests to the database forwarded by the traffic gateway during the migration process.

[0119] Optionally, the migration module 30 is further configured to process a transaction read request of the demand party to be migrated through an original online service cluster connected to an original database if the transaction data of the demand party to be migrated is in a migrating state or a non-migrated state;

[0120] If the transaction data of the demand party to be migrated is in a migrated state, the transaction read request of the demand party to be migrated is processed through the target online service cluster connected to the target database.

[0121] Optionally, the migration module 30 is further used to clean up the migrated transaction data in the original database;

[0122] In the original database, build a backup table of the migrated transaction data;

[0123] If the target database is down, the migrated transaction data is restored to the original database according to the backup table, and the transaction read requests and write requests of the party to be migrated are processed through the original database.

[0124] The data migration device further comprises: determining whether the transaction data will change within a preset period of time;

[0125] If the transaction data will change within the preset period, the slice data will be regarded as active transaction data;

[0126] If the transaction data does not change within the preset period, the slice data will be used as historical transaction data;

[0127] During the migration process, active transaction data is migrated first.

[0128] The data migration device provided by the present application adopts the data migration method in the above embodiment, which can solve the technical problem of long data migration time. Compared with the prior art, the beneficial effects of the data migration device provided by the present application are the same as the beneficial effects of the data migration method provided by the above embodiment, and other technical features in the data migration device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0129] The present application provides a data migration device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data migration method in the above-mentioned first embodiment.

[0130] Reference below Figure 6 , which shows a schematic diagram of the structure of a data migration device suitable for implementing an embodiment of the present application. The data migration device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The data migration device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0131] like Figure 6 As shown, the data migration device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the data migration device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data migration device to communicate with other devices wirelessly or by wire to exchange data. Although the data migration device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0132] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0133] The data migration device provided by the present application adopts the data migration method in the above embodiment, which can solve the technical problem of long data migration time. Compared with the prior art, the beneficial effects of the data migration device provided by the present application are the same as the beneficial effects of the data migration method provided by the above embodiment, and other technical features in the data migration device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0134] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0135] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0136] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the data migration method in the above-mentioned embodiment.

[0137] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0138] The computer-readable storage medium may be included in the data migration device; or may exist independently without being assembled into the data migration device.

[0139] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the data migration device, the data migration device: slices the transaction data in the original database based on the characteristics of the demand party to be migrated to obtain sliced ​​data; divides the sliced ​​data according to the amount of data of the storage resources occupied by the sliced ​​data in the original database to obtain grayscale batch data and formal batch data; after verifying the correctness of the migration process through the grayscale batch data, migrate the formal batch data from the original database to the target database in batches.

[0140] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0143] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data migration method, and can solve the technical problem of long data migration time. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the data migration method provided in the above-mentioned embodiment, and will not be repeated here.

[0144] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned data migration method when executed by a processor.

[0145] The computer program product provided by this application can solve the technical problem of long data migration time. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the data migration method provided by the above embodiment, which will not be repeated here.

[0146] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A data migration method, characterized in that: The data migration method comprises: Slice the transaction data in the original database based on the characteristics of the demand side to be migrated to obtain sliced ​​data; According to the amount of storage resources occupied by the slice data in the original database, the slice data is divided to obtain grayscale batch data and formal batch data; After verifying the correctness of the migration process through the grayscale batch data, the formal batch data is migrated from the original database to the target database in batches.

2. The data migration method according to claim 1, characterized in that: While migrating the formal batch data from the original database to the target database in batches, the method further includes: Obtain the transaction read request and transaction write request of the demand side to be migrated in the traffic gateway; Shielding the transaction write request of the party to be migrated to the original database; Based on the migration status of the transaction data of the demand party to be migrated, the transaction read request of the demand party to be migrated is processed through an online service cluster connected to the database, wherein the online service cluster receives the read and write requests to the database forwarded by the traffic gateway during the migration process.

3. The data migration method according to claim 2, characterized in that: The step of processing the transaction read request of the demand party to be migrated through the online service cluster connected to the database based on the migration status of the transaction data of the demand party to be migrated includes: If the transaction data of the demand party to be migrated is in the migrating state or the non-migrated state, the transaction read request of the demand party to be migrated is processed through the original online service cluster connected to the original database; If the transaction data of the demand party to be migrated is in a migrated state, the transaction read request of the demand party to be migrated is processed through the target online service cluster connected to the target database.

4. The data migration method according to claim 1, characterized in that: The method further comprises: Determine whether transaction data will change within a preset period of time; If the transaction data will change within the preset period, the slice data will be regarded as active transaction data; If the transaction data does not change within the preset period, the slice data will be used as historical transaction data; During the migration process, active transaction data is migrated first.

5. The data migration method according to claim 1, characterized in that: While migrating the formal batch data from the original database to the target database in batches, the method further includes: In the original database, build a backup table of the migrated transaction data; If the target database is down, the migrated transaction data is restored to the original database according to the backup table, and the transaction read requests and write requests of the party to be migrated are processed through the original database.

6. The data migration method according to claim 1, characterized in that: The step of slicing the transaction data in the original database based on the characteristics of the demand party to be migrated to obtain the sliced ​​data includes: Obtaining a database object of an original database, wherein the database object includes a table and an index of the table; Establish a database object consistent with the original database to obtain the target database; A target online service cluster connected to the target database is established, wherein the target online service cluster receives read and write requests to the target database forwarded by a traffic gateway during the migration process.

7. A data migration device, characterized in that: The data migration device comprises: A slicing module is used to slice the transaction data in the original database based on the characteristics of the demand side to be migrated to obtain sliced ​​data; A partitioning module is used to partition the slice data according to the amount of storage resources occupied by the slice data in the original database to obtain grayscale batch data and formal batch data; The migration module is used to migrate the formal batch data from the original database to the target database in batches after verifying the correctness of the migration process through the grayscale batch data.

8. A data migration device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data migration method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data migration method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the data migration method according to any one of claims 1 to 6 are implemented.

Images