Method and apparatus for implementing database transformation migration

Abstract

The present disclosure provides a method, device, equipment and storage medium for implementing database transformation migration, which can be applied to the fields of computer technology and financial technology. The method comprises the following steps: obtaining usage specifications corresponding to a first database, a second database and a data replication tool respectively; obtaining, according to the usage specification corresponding to the first database, to-be-migrated data corresponding to a migration task from the first database; determining a migration strategy according to the migration task; and executing the migration task according to the migration strategy by using the data replication tool based on the respective usage specifications, so as to realize migration of the to-be-migrated data to the second database.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer and the technical field of financial technology, and in particular, relates to a method, device, equipment, medium and program product for realizing database transformation migration. BACKGROUND

[0002] There are various implementation methods for transforming an Oracle database to a Gauss database. A commonly used tool is a DRS (Data Replication Service) data replication tool. The tool is simple to use, but often encounters the following problems: migration involving large tables results in poor timeliness; full migration has a large impact on the host; after transformation to the Gauss database, some fields are insufficient in length; the processing capacity for LOB (Large Object) objects in the Oracle database is insufficient; and the incremental processing of tables without primary keys is limited.

[0003] Therefore, how to realize smooth transformation migration between an Oracle database and a Gauss database is a technical problem to be solved in the related art. SUMMARY

[0004] In view of the above problems, the present disclosure provides a method, device, equipment, medium and program product for realizing database transformation migration.

[0005] According to a first aspect of the present disclosure, a method for realizing database transformation migration is provided, comprising:

[0006] obtaining usage specifications respectively corresponding to a first database, a second database and a data replication tool;

[0007] obtaining, according to the usage specification corresponding to the first database, to-be-migrated data corresponding to a migration task from the first database;

[0008] determining a migration strategy according to the migration task; and

[0009] based on the respective usage specifications, performing the migration task according to the migration strategy by using the data replication tool, so as to realize migration of the to-be-migrated data to the second database.

[0010] According to an embodiment of the present disclosure, the to-be-migrated data corresponding to the migration task is obtained from the first database according to the usage specification corresponding to the first database, comprising:

[0011] starting an archive log of the first database according to the usage specification corresponding to the first database; and

[0012] The to-be-migrated data is obtained from the content of the archive log record.

[0013] According to an embodiment of the present disclosure, the determining the migration strategy according to the migration task and the to-be-migrated data comprises:

[0014] In a case where the migration task comprises a full migration task, a first strategy corresponding to the host and the standby machine is determined; and

[0015] In a case where the migration task comprises the full migration task and an incremental migration task, a second strategy corresponding to the host and the standby machine is determined.

[0016] According to an embodiment of the present disclosure, the performing the migration task according to the migration strategy by using the data replication tool based on the respective corresponding usage specifications, so as to realize the migration of the to-be-migrated data to the second database comprises:

[0017] In a case where the migration strategy represents the second strategy, the to-be-migrated data is split into task data executed by each of the standby machines, wherein the to-be-migrated data represents data required to be migrated by the full migration task;

[0018] For each of the standby machines, the migration task corresponding to the standby machine is executed by using the data replication tool, so as to realize the migration of the task data to the second database.

[0019] According to an embodiment of the present disclosure, the method for realizing the database transformation migration further comprises:

[0020] For the incremental migration task, the target data in the supplementary log is migrated to the second database by using the host, with a target time point at which the full migration task starts as a reference, wherein the target data represents data changed after the to-be-migrated data at the target time point.

[0021] According to an embodiment of the present disclosure, the method for realizing the database transformation migration further comprises:

[0022] Before the migration of the target data, a primary key constraint is created for a table in which a target operation statement exists in the target data.

[0023] According to an embodiment of the present disclosure, the method for realizing the database transformation migration further comprises:

[0024] Before the data migration, a first constraint condition in a data migration process and a second constraint condition corresponding to the second database are set, so as to perform the data migration according to the first constraint condition and the second constraint condition.

[0025] According to an embodiment of the present disclosure, the splitting of the to-be-migrated data into task data executed by each of the standby machines includes:

[0026] For the full-volume migration task, in the case that the to-be-migrated data includes a primary key table and a non-primary key table, the primary key table and the non-primary key table are separately subjected to task creation to obtain task data corresponding to the primary key table and the non-primary key table respectively, so as to execute the respective task data by using the standby machines.

[0027] In the case that the task data corresponding to the non-primary key table includes a plurality of non-primary key tables, the task data corresponding to the non-primary key table is split into a plurality of task sub-data, so as to execute the plurality of task sub-data by using the standby machines.

[0028] A second aspect of the present disclosure provides an apparatus for implementing database transformation migration, comprising a first obtaining module, a second obtaining module, a determining module and an executing module. The first obtaining module is configured to obtain usage specifications corresponding to a first database, a second database and a data replication tool respectively. The second obtaining module is configured to obtain to-be-migrated data corresponding to a migration task from the first database according to the usage specification corresponding to the first database. The determining module is configured to determine a migration strategy according to the migration task. The executing module is configured to execute the migration task according to the migration strategy by using the data replication tool based on the respective usage specifications, so as to implement migration of the to-be-migrated data to the second database.

[0029] A third aspect of the present disclosure provides an electronic device, comprising one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the above method.

[0030] A fourth aspect of the present disclosure further provides a computer-readable storage medium having stored executable instructions, which, when executed by a processor, cause the processor to perform the above method.

[0031] A fifth aspect of the present disclosure further provides a computer program product comprising a computer program, which, when executed by a processor, implements the above method.

[0032] The method, device, equipment, medium and program product for implementing database transformation migration provided according to the present disclosure, based on the formulated usage specification corresponding to the first database, obtain the to-be-migrated data corresponding to the migration task from the first database, and determine the migration strategy according to the migration task, so that the migration task can be performed according to the migration strategy by using the data replication tool based on the formulated usage specification corresponding to the first database, the second database and the data replication tool respectively, so as to realize the migration of the to-be-migrated data to the second database, and the smooth transformation migration between the Oracle database and the Gauss database can be realized, and abnormal error caused by improper use in the migration process can be avoided. BRIEF DESCRIPTION OF DRAWINGS

[0033] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure taken in conjunction with the accompanying drawings, in which:

[0034] Figure 1 An application scenario diagram of the method for implementing database transformation migration according to an embodiment of the present disclosure is schematically shown;

[0035] Figure 2 A flowchart of the method for implementing database transformation migration according to an embodiment of the present disclosure is schematically shown;

[0036] Figure 3 A flowchart of performing a migration task according to an embodiment of the present disclosure is schematically shown;

[0037] Figure 4 A flowchart of formulating a migration best practice according to an embodiment of the present disclosure is schematically shown;

[0038] Figure 5 A structural block diagram of the device for implementing database transformation migration according to an embodiment of the present disclosure is schematically shown; and

[0039] Figure 6 A block diagram of an electronic device suitable for implementing the method for implementing database transformation migration according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0040] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it is to be understood that these descriptions are only exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that one or more embodiments can be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques have been omitted to avoid unnecessarily obscuring the concept of the present disclosure.

[0041] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the terms "comprises", "comprising", "includes", "including" and the like are, unless otherwise stated, taken to mean the statement that one includes, i.e. at least one, but not excluding others.

[0042] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein are defined as having meanings that are consistent with the context of the specification in which they are used, and should not be interpreted in an idealized or overly formal way.

[0043] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should generally be interpreted to include at least one of A, B, or C, unless otherwise defined (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B together, a system having A and C together, a system having B and C together, and / or a system having A, B, and C together, etc.).

[0044] In the technical solutions of the present application, the user information (including but not limited to user personal information, user image information, user equipment information, such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved are information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards of relevant countries and regions, necessary security measures are taken, public order and good customs are not violated, and corresponding operation entrances are provided for users to choose authorization or refusal.

[0045] It is found in the process of implementing the present disclosure that the data replication service is a cloud service that is one-time, stable, efficient, and used for database online migration and real-time synchronization of databases. DRS surrounds cloud databases, reduces the complexity of data flow between databases, and effectively helps to reduce the cost of data transmission. Its main features are: online migration can completely move the opposite database under the condition that the business does not stop; real-time synchronization can synchronize the required data on the basis of millisecond delay to realize the sharing between businesses.

[0046] The main implementation principle of the smooth migration of the Oracle database to the Gauss database by using the DRS includes: log pulling, that is, reading the original database log by using an Oracle Logminer (a log analysis tool of the Oracle database) and storing the log to a local disk; reading-> analyzing-> integrating, that is, reading the log data from the local disk, analyzing the effective change records in the log, and integrating the records into a transaction according to the order of occurrence of the original database; and THL landing file, that is, converting the integrated data into a storage format of the DRS and writing the data to the disk. However, the DRS tool is based on the Oracle Logminer to read the original database log, and is limited by the log capture capability of the Logminer, so that the tool itself has certain limitations. Once the user operates carelessly, the migration error or even the data exception may occur.

[0047] Therefore, how to realize the smooth transformation migration between the Oracle database and the Gauss database is a technical problem to be solved in the related art.

[0048] To this end, an embodiment of the present disclosure provides a method for realizing database transformation migration, including: obtaining usage specifications respectively corresponding to a first database, a second database and a data replication tool; obtaining, according to the usage specification corresponding to the first database, to-be-migrated data corresponding to a migration task from the first database; determining a migration strategy according to the migration task; and performing the migration task according to the migration strategy by using the data replication tool based on the respective usage specifications, so as to realize migration of the to-be-migrated data to the second database.

[0049] Figure 1 An application scenario diagram of the method for realizing database transformation migration according to an embodiment of the present disclosure is schematically shown.

[0050] As shown in Figure 1 According to the application scenario 100 of the embodiment, the first terminal device 101, the second terminal device 102, the third terminal device 103, the network 104 and the server 105 can be included. The network 104 is used as a medium to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0051] A user can use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0052] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc.

[0053] The server 105 can be a server providing various services, such as a background management server providing support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only as an example). The background management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal devices.

[0054] For example, the usage specifications corresponding to the first database, the second database, and the data replication tool can be obtained through the server 105, and the to-be-migrated data corresponding to the migration task can be obtained from the first database according to the usage specification corresponding to the first database, and the migration strategy can be determined according to the migration task, so that finally the migration task is executed according to the migration strategy based on the respective usage specifications using the data replication tool, to realize migration of the to-be-migrated data to the second database.

[0055] It should be noted that the method for implementing database transformation migration provided by the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the apparatus for implementing database transformation migration provided by the embodiments of the present disclosure can generally be arranged in the server 105. The method for implementing database transformation migration provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Accordingly, the apparatus for implementing database transformation migration provided by the embodiments of the present disclosure can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105.

[0056] It should be understood that, Figure 1The number of terminal devices, networks and servers in the figure is only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

[0057] The following will be based on Figure 1 The described scenario, by Figures 2 to 4 The method for implementing database transformation migration of the disclosed embodiment is described in detail.

[0058] Figure 2 The flowchart of the method for implementing database transformation migration according to the embodiments of the present disclosure is schematically shown.

[0059] As Figure 2 The method 200 includes operation S210 to operation S240.

[0060] In operation S210, the usage specifications corresponding to the first database, the second database and the data replication tool respectively are acquired.

[0061] According to the embodiments of the present disclosure, the first database can represent an Oracle database, the second database can represent a Gauss database, and the data replication tool can represent a DRS.

[0062] According to the embodiments of the present disclosure, the usage specifications corresponding to the first database, the second database and the data replication tool respectively can be formulated according to the summary of the problems occurred in the past transformation migration process of the Oracle database, the Gauss database and the data replication tool. The usage specifications can represent the specifications that need to be followed in the migration process using the first database, the second database and the data replication tool.

[0063] According to the embodiments of the present disclosure, the usage specification corresponding to the first database can include setting the log mode to the archive mode; opening at least the pk / uk level of the supplemental log (pk, primary key, primary key value, primary key constraint; uk, unique key, unique value, unique constraint); the archive log should have a certain retention time (such as 3-5 days) to facilitate sufficient reaction time to repair data when problems occur; do not set the no logging parameter to write data, otherwise the DRS cannot be captured; before performing the reset log operation, the DRS data replication service must be notified, and the DRS needs to specify the SCN (System Change Number) restart, otherwise the DRS will not be able to synchronize data.

[0064] The SCN can represent a time identifier of an Oracle database. In the case that the Oracle database is reset log, the logs in the database are renumbered. In the case that the original logs corresponding to the migration task are directly copied and migrated by using the DRS, the correct logs cannot be identified due to the change of the log numbers. Therefore, the database needs to be restarted, and a new SCN needs to be specified. The SCN can represent the time point after the reset log.

[0065] According to an embodiment of the present disclosure, the usage specification corresponding to the second database can include that the Gauss database can only perform read operations and cannot write data, otherwise, data conflicts and inconsistency may occur on both sides; and the trigger needs to be disabled, otherwise, data conflicts may occur on both sides.

[0066] In the process of migration, the second database can be accessed for verification; and the trigger is disabled to prevent the second database from performing additional operations in the process of migration.

[0067] According to an embodiment of the present disclosure, the usage specification corresponding to the data replication tool can include that the DRS currently does not support synchronization of Oracle UDT (UDP-based Data Transfer Protocol; User Datagram Protocol), such as GIS (Geography Information System), BFILE (a data type) and nested tables; only table structures, indexes and constraints are supported in the process of full migration, and synchronization of functions, stored procedures, triggers and other database objects is not supported; the DRS currently does not support incremental synchronization of DDL (Data Definition Language) of heterogeneous databases; and in the process of full synchronization, a business low peak period is preferably selected to reduce the impact on the original database.

[0068] The synchronization of the UDT, BFILE and nested tables by the data replication tool will result in an error; the DDL can include CREATE, ALTER, DROP and the like, and can be used to define or change the structure of a table, a data type, a link between tables and a constraint and the like.

[0069] In operation S220, according to the usage specification corresponding to the first database, the to-be-migrated data corresponding to the migration task is obtained from the first database.

[0070] According to an embodiment of the present disclosure, before migration, the first database can be set to an archive mode in a log mode according to the usage specification corresponding to the first database, so that the to-be-migrated data corresponding to the migration task can be obtained from the archive log in the subsequent migration process.

[0071] The migration task can include a full migration task and can also include an incremental migration task.

[0072] In operation S230, a migration strategy is determined according to the migration task.

[0073] According to an embodiment of the present disclosure, the migration strategy executed by the host and the standby machine can be determined according to the migration task. The to-be-migrated data can represent the data required to be migrated by the full migration task.

[0074] In operation S240, the migration task is executed according to the migration strategy by using the data replication tool based on the respective usage specifications, so as to realize migration of the to-be-migrated data to the second database.

[0075] According to an embodiment of the present disclosure, in the migration process, the first database, the second database, and the data replication tool all need to be based on the corresponding usage specifications, and the migration task is executed according to the migration strategy by using the data replication tool, so as to migrate the to-be-migrated data to the second database.

[0076] According to an embodiment of the present disclosure, based on the usage specification corresponding to the first database, the to-be-migrated data corresponding to the migration task is obtained from the first database, and the migration strategy is determined according to the migration task and the to-be-migrated data, so that the migration task can be executed according to the migration strategy by using the data replication tool based on the usage specifications corresponding to the first database, the second database, and the data replication tool, respectively, so as to realize migration of the to-be-migrated data to the second database. By executing the migration task according to the respective usage specifications, smooth transformation migration between the Oracle database and the Gauss database can be realized, and abnormal error caused by improper use in the migration process can be avoided.

[0077] According to an embodiment of the present disclosure, the to-be-migrated data corresponding to the migration task is obtained from the first database according to the usage specification corresponding to the first database, including: starting the archive log of the first database according to the usage specification corresponding to the first database; and obtaining the to-be-migrated data from the content recorded in the archive log.

[0078] According to an embodiment of the present disclosure, before the migration operation, the log mode of the first database needs to be set to an archive mode, that is, the archive log is opened, so as to save the content of the redo log in the first database to the archive log.

[0079] According to an embodiment of the present disclosure, in the migration process, the to-be-migrated data can be obtained by acquiring the content recorded in the archive log.

[0080] According to an embodiment of the present disclosure, the content of the redo log in the first database can be saved by using the archive log, so that the to-be-migrated data can be directly obtained according to the content recorded in the archive log, for subsequent data migration.

[0081] According to an embodiment of the present disclosure, according to the migration task, the migration strategy is determined, including: in the case that the migration task includes a full migration task, determining a first strategy corresponding to the host and the standby machine; in the case that the migration task includes a full migration task and an incremental migration task, determining a second strategy corresponding to the host and the standby machine.

[0082] According to an embodiment of the present disclosure, in the case that the migration task only includes a full migration task, if there is no LOB data in the to-be-migrated data, the first strategy can represent that the full migration task is executed by using the host, or can represent that the full migration task is executed by using the standby machine; if there is LOB data in the to-be-migrated data, the first strategy can represent that the full migration task is executed by using the host, or can represent that the full migration task is executed by using the standby machine.

[0083] According to an embodiment of the present disclosure, in the case that the migration task includes a full migration task and an incremental migration task, if there is no LOB object in the data corresponding to the incremental task, the second strategy can represent that the incremental migration task is executed by using the host, and the full migration task is executed by using the standby machine, or the second strategy can represent that the full migration task and the incremental migration task are executed by using the standby machine; if there is an LOB object in the data corresponding to the incremental task, the second strategy can represent that the incremental migration task is executed by using the host, and the full migration task is executed by using the standby machine.

[0084] In the case that there is no LOB object in the data corresponding to the incremental task, the incremental migration task can also be executed by using the standby machine, but once the task is interrupted in the process of executing the incremental migration task by using the standby machine, the consistency of the data cannot be guaranteed, and it is recommended to execute the incremental migration task by using the host.

[0085] In the case that there is no LOB object in the data corresponding to the incremental migration task, the incremental migration task can also be executed by using the standby machine, but once the task is interrupted in the process of executing the incremental migration task by using the standby machine, the consistency of the data cannot be guaranteed, and it is recommended to execute the incremental migration task by using the host.

[0086] According to an embodiment of the present disclosure, the host can be used to execute a migration task sensitive to time delay, and the standby machine can be used to execute a migration task not sensitive to time delay. In the case that the performance requirement of the host is high or there is a time limit for batch processing, the standby machine can be introduced to execute the migration task, so as to avoid damage to the host caused by the migration task.

[0087] According to an embodiment of the present disclosure, in the migration process, if a large table is involved, a full table scan needs to be performed on the large table, which has a great impact on the performance of the host. Therefore, in the case where a large table is involved in the full migration task, the standby machine can be used to perform the migration task; in the case where a large table is involved in the incremental migration task and there is no LOB object in the data corresponding to the incremental migration task, the standby machine can be used to perform the migration task, so as to avoid the impact of the large table on the performance of the host. However, in the case where there is an LOB object in the data corresponding to the incremental migration task, the host can only be used to perform the migration task.

[0088] According to an embodiment of the present disclosure, since the standby machine does not support migration of the LOB object, in the case where the migration task includes the full migration task and the incremental migration task, the host can be used to perform the incremental migration task, and the standby machine can be used to perform the full migration task, so as to share the performance pressure of the host and improve the migration efficiency.

[0089] Figure 3 A flowchart of performing a migration task according to an embodiment of the present disclosure is schematically shown.

[0090] As shown in Figure 3 The method 300 includes operation S310 and operation S320.

[0091] In operation S310, in the case where the migration strategy represents the second strategy, the to-be-migrated data is split into task data performed by each standby machine.

[0092] According to an embodiment of the present disclosure, the to-be-migrated data can represent data required to be migrated by the full migration task.

[0093] According to an embodiment of the present disclosure, in the case where the migration strategy represents the second strategy, the full migration task can be performed by the standby machine first, and then the incremental migration task can be performed by the host.

[0094] According to an embodiment of the present disclosure, in the case where the full migration task is performed by the standby machine, the to-be-migrated data can be split into task data performed by each standby machine, so as to release the performance pressure of the host, that is, the full migration task can be split into multiple migration concurrent tasks, wherein the migration concurrent task can represent migration of the task data corresponding to the standby machine.

[0095] For example, in the case where there are six standby machines, the to-be-migrated data can be split into six pieces of task data, so as to perform the corresponding task data by each standby machine.

[0096] In operation S320, for each standby machine, a migration task corresponding to the standby machine is performed by using a data replication tool, so as to realize migration of the task data to the second database.

[0097] According to an embodiment of the present disclosure, each standby machine can perform a corresponding migration task by using a data replication tool to migrate corresponding task data into the second database, so that the to-be-migrated data can be migrated into the second database.

[0098] According to an embodiment of the present disclosure, in the case where the migration strategy represents the first strategy and the first strategy represents performing a full migration task by using the standby machine, the same operations as the operation S310 and the operation S320 can be performed by using the standby machine.

[0099] According to an embodiment of the present disclosure, introducing the standby machine to create multiple migration concurrent tasks can not only solve the performance impact on the host during the migration process, but also improve the migration efficiency.

[0100] According to an embodiment of the present disclosure, the method for implementing database transformation migration further includes: for the incremental migration task, taking the target time at which the full migration task starts as a reference, using the host to migrate the target data in the supplement log to the second database.

[0101] According to an embodiment of the present disclosure, the target data can represent data that is changed after the target time of the to-be-migrated data, and the target data can represent data corresponding to the incremental migration task. After the full migration task is completed by using the standby machine, the incremental migration task can be performed by using the host.

[0102] According to an embodiment of the present disclosure, before the migration operation is performed, the supplement log in the first database is opened based on the usage specification corresponding to the first database. The supplement log can be used to record data that is changed after the target time of the to-be-migrated data.

[0103] According to an embodiment of the present disclosure, for the primary key table, the incremental synchronization can be performed on the premise of ensuring data consistency, but for the table without a primary key, the incremental synchronization cannot be achieved, so the supplement log at the pk / uk level is opened to ensure the incremental synchronization.

[0104] According to an embodiment of the present disclosure, the host can perform the incremental migration task by using the data replication tool to migrate the target data into the second database. During the process of performing the incremental migration task by using the host, the standby machine needs to be disconnected from the synchronization to ensure the consistency of the host and the standby machine.

[0105] According to an embodiment of the present disclosure, the host is used to migrate the target data in the supplement log into the second database to ensure the consistency of the data.

[0106] According to an embodiment of the present disclosure, the method for implementing database transformation migration further includes: before the target data is migrated, creating a primary key constraint for a table in which a target operation statement exists in the target data.

[0107] According to an embodiment of the present disclosure, the table that needs to be synchronized is a table that needs to have a primary key or a unique index, otherwise the DRS can have a duplicate data problem, thereby causing inconsistent data on both sides. Among them, the primary key or the unique index can represent the unique identifier of the table to distinguish different tables.

[0108] According to an embodiment of the present disclosure, before migrating the target data, that is, before performing the incremental migration task, a primary key constraint needs to be created for the table in the target data that has a target operation statement, so as to maintain data consistency.

[0109] Among them, the target operation statement can represent a DML (Data Manipulation Language), and the DML can include select, update, insert, delete and the like, which can be used to operate data in the database.

[0110] According to an embodiment of the present disclosure, the DML statement supports the incremental migration task, and the target data all have the DML statement, that is, only the data with the DML statement can perform the incremental migration task.

[0111] According to an embodiment of the present disclosure, before performing the incremental migration task, a primary key constraint is created for the table in the target data to achieve data consistency.

[0112] According to an embodiment of the present disclosure, the method for realizing database transformation migration further includes: before performing data migration, setting a first constraint condition in the data migration process and a second constraint condition corresponding to the second database, so as to perform data migration according to the first constraint condition and the second constraint condition.

[0113] According to an embodiment of the present disclosure, before performing data migration, that is, before performing the migration task, a first constraint condition in the data migration process and a second constraint condition corresponding to the second database can be set, so as to perform data migration according to the first constraint condition and the second constraint condition.

[0114] According to an embodiment of the present disclosure, the first constraint condition can include that the username, password, permission or port number connected with the first database and the second database is not allowed to be modified or deleted in the migration synchronization process; the DRS synchronization partition table cannot be used to avoid converting the partition table into a non-partition table.

[0115] According to an embodiment of the present disclosure, the second constraint condition can include that a non-primary key index should not be created when creating a table to improve migration efficiency; the object names such as database, schema, table and column in the migration table list must be in lowercase; the table to be synchronized in the migration needs to disable the foreign key and the trigger; and it is suggested that the second database needs to be consistent with the table structure and the primary key of the first database.

[0116] If a non-primary key index is created, the index also needs to be maintained during the migration process, thereby increasing the migration time; the migration table list can include data that needs to be migrated, and the object names in the migration table list are converted to lowercase to make the data comply with the Gauss database.

[0117] In the case where the table structure and the primary key of the first database and the second database are inconsistent, the following cases can cause the migration task to fail: the target column is required to be non-empty and has no default value, and the target column is set to a fixed default value. The target can represent the second database.

[0118] According to the first constraint condition and the second constraint condition, data migration is performed to achieve smooth transformation migration between the Oracle database and the Gauss database, and abnormal error caused by improper use during the migration process is avoided.

[0119] According to an embodiment of the present disclosure, the to-be-migrated data is split into task data executed by each standby machine, including: for a full migration task, in the case where the to-be-migrated data includes a primary key table and a non-primary key table, the primary key table and the non-primary key table are separately created for a task to obtain task data corresponding to the primary key table and the non-primary key table respectively, so as to execute the respective task data by using the standby machine; in the case where the task data corresponding to the non-primary key table includes multiple non-primary key tables, the task data corresponding to the non-primary key table is split into multiple task sub-data, so as to execute the multiple task sub-data by using the standby machine.

[0120] According to an embodiment of the present disclosure, the difference between the primary key table and the non-primary key table is that the non-primary key table does not exist non-empty and unique constraints. In the case where a row in the non-primary key table is updated, multiple rows can be updated, and in the case where a LOB object exists in the non-primary key table, the row can be misplaced, so that the task is interrupted and the consistency of the data cannot be guaranteed. As can be seen, the non-primary key table cannot exist in the incremental migration task, that is, there is no primary key table, but the non-primary key table can exist in the full migration task.

[0121] According to an embodiment of the present disclosure, for a full migration task, in the case where the to-be-migrated data includes both a primary key table and a non-primary key table, the primary key table and the non-primary key table are separately created for a task, that is, the primary key table is allocated to a task and the non-primary key table is allocated to another task, to obtain task data corresponding to the primary key table and the non-primary key table respectively, so that the task data corresponding to the primary key table can be executed by using one standby machine and the task data corresponding to the non-primary key table can be executed by using another standby machine.

[0122] According to an embodiment of the present disclosure, in the full volume migration task, in the case that the data volume of the primary key table and the non-primary key table in the data to be migrated is not large and the downtime window is multiple, the primary key table and the non-primary key table can be merged into one task.

[0123] According to an embodiment of the present disclosure, in the case that there are more non-primary key tables in the task data corresponding to the non-primary key tables, the task data corresponding to the non-primary key tables can be split into multiple task sub-data, so as to execute the task sub-data by using different standby machines, thereby improving the migration efficiency.

[0124] According to an embodiment of the present disclosure, for the migration of the non-primary key table in the full volume migration task, non-downtime migration is not recommended, that is, downtime migration is recommended. The table which is both a non-primary key table and a partition table is not supported for synchronous migration.

[0125] According to an embodiment of the present disclosure, the primary key table and the non-primary key table can be separately created for the task, and in the case that there are more non-primary key tables, the non-primary key tables can be split into multiple task sub-data, so as to improve the migration efficiency.

[0126] Figure 4 A flowchart for formulating migration best practices according to an embodiment of the present disclosure is schematically shown.

[0127] As shown in Figure 4 , the method 400 includes operation S410 to operation S450.

[0128] In operation S410, investigation is performed.

[0129] According to an embodiment of the present disclosure, the migration scenario is obtained, and the relevance analysis is performed on the migration scenario, and the information search is performed on the data volume, the business pressure, and the specification configuration, so as to form a migration network.

[0130] In operation S420, a scheme is formulated.

[0131] According to an embodiment of the present disclosure, the migration scheme / tool is discussed through a project start meeting, the migration network is evaluated, and thus the migration scheme is formed based on the user authority, the parameter, and the migration mode.

[0132] In operation S430, technology verification is performed.

[0133] According to an embodiment of the present disclosure, based on the migration scheme, the technology verification is performed on the data migration speed and the business influence, and then the risk assessment and the emergency plan are performed on the migration scheme, so as to form a preliminary scheme review, and a test report of the migration scheme is generated according to the preliminary scheme review.

[0134] In operation S440, pre-migration is performed.

[0135] According to an embodiment of the present disclosure, based on the migration scheme, an environment is built to perform pre-migration, and after migration, the migrated data in the database is reviewed. In the case of review error, the migration scheme is modified until the review is correct. In the case of correct review, the final scheme can be determined.

[0136] In operation S450, migration is implemented.

[0137] According to an embodiment of the present disclosure, based on the final scheme, migration preparation, environment monitoring and tool deployment are performed to implement migration. After migration, performance verification and function verification are performed, and review and data comparison are performed to ensure the accuracy of migration according to the final scheme.

[0138] According to an embodiment of the present disclosure, based on the formulated best migration practice, the above-mentioned operations S210-S240 can be performed, so that in the process of transformation migration, the performance pressure of the host is shared through the introduction of the standby machine, the overall migration timeliness is improved, the smooth migration between the Oracle database and the Gauss database is realized, and based on the formulated use specification, abnormal error caused by improper use in the migration process is avoided.

[0139] Based on the above-mentioned method for realizing database transformation migration, the present disclosure further provides an apparatus for realizing database transformation migration. The following will be described in detail in combination with Figure 5 The apparatus is described in detail.

[0140] Figure 5 The structure block diagram of the apparatus for realizing database transformation migration according to an embodiment of the present disclosure is schematically shown.

[0141] As shown in Figure 5 The apparatus 500 for realizing database transformation migration of this embodiment includes a first acquisition module 510, a second acquisition module 520, a determination module 530 and an execution module 540.

[0142] The first acquisition module 510 is configured to acquire the use specification corresponding to the first database, the second database and the data replication tool respectively. In an embodiment, the first acquisition module 510 can be configured to perform the operation S210 described above, which will not be repeated here.

[0143] The second acquisition module 520 is configured to acquire the to-be-migrated data corresponding to the migration task from the first database according to the use specification corresponding to the first database. In an embodiment, the second acquisition module 520 can be configured to perform the operation S220 described above, which will not be repeated here.

[0144] The determination module 530 is configured to determine the migration strategy according to the migration task. In an embodiment, the determination module 530 can be configured to perform the operation S230 described above, which will not be repeated here.

[0145] The execution module 540 is configured to perform the migration task according to the migration policy by using the data replication tool based on the respective corresponding usage specification, so as to realize migration of the to-be-migrated data to the second database. In an embodiment, the execution module 540 can be configured to perform the operation S240 described above, and details are not described herein again.

[0146] According to an embodiment of the present disclosure, the second acquisition module 520 comprises an opening unit and an acquisition unit.

[0147] The opening unit is configured to open the archive log of the first database according to the usage specification corresponding to the first database.

[0148] The acquisition unit is configured to acquire the to-be-migrated data from the content recorded in the archive log.

[0149] According to an embodiment of the present disclosure, the determination module 530 comprises a first determination unit and a second determination unit.

[0150] The first determination unit is configured to determine the first strategy corresponding to the master and the standby in the case where the migration task comprises the full migration task.

[0151] The second determination unit is configured to determine the second strategy corresponding to the master and the standby in the case where the migration task comprises the full migration task and the incremental migration task.

[0152] According to an embodiment of the present disclosure, the execution module 540 comprises a splitting unit and an execution unit.

[0153] The splitting unit is configured to split the to-be-migrated data into task data executed by each standby in the case where the migration policy represents the second strategy, wherein the to-be-migrated data represents data required to be migrated by the full migration task.

[0154] The execution unit is configured to perform the migration task corresponding to each standby by using the data replication tool, so as to realize migration of the task data to the second database.

[0155] According to an embodiment of the present disclosure, the execution module 540 further comprises a migration unit.

[0156] The migration unit is configured to migrate target data in the supplement log to the second database by using the master in the case of the incremental migration task, wherein the target time is the starting time of the full migration task, and the target data represents data changed after the to-be-migrated data at the target time.

[0157] According to an embodiment of the present disclosure, the execution module 540 further comprises a creation unit.

[0158] The creating unit is configured to create a primary key constraint for a table having a target operation statement in the target data before the target data is migrated.

[0159] According to an embodiment of the present disclosure, the apparatus 500 for implementing database transformation migration further includes a setting module.

[0160] The setting module is configured to set a first constraint condition in a data migration process and a second constraint condition corresponding to a second database before the data migration is performed, so as to perform the data migration according to the first constraint condition and the second constraint condition.

[0161] According to an embodiment of the present disclosure, the splitting unit includes an obtaining sub-unit and a splitting sub-unit.

[0162] The obtaining sub-unit is configured to, for a full-amount migration task, separate a primary key table and a non-primary key table in the to-be-migrated data, create a task for each of the primary key table and the non-primary key table, and obtain task data corresponding to each of the primary key table and the non-primary key table, so as to execute the task data corresponding to each of the primary key table and the non-primary key table by using a backup machine.

[0163] The splitting sub-unit is configured to, in a case where the task data corresponding to the non-primary key table includes a plurality of non-primary key tables, split the task data corresponding to the non-primary key table into a plurality of task sub-data, so as to execute the plurality of task sub-data by using the backup machine.

[0164] According to an embodiment of the present disclosure, any one or more of the first obtaining module 510, the second obtaining module 520, the determining module 530, and the executing module 540 can be combined in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of other modules, and implemented in one module. According to an embodiment of the present disclosure, at least one of the first obtaining module 510, the second obtaining module 520, the determining module 530, and the executing module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware by integrating or packaging circuits, or in any one of software, hardware, and firmware or in any appropriate combination of any of them. Alternatively, at least one of the first obtaining module 510, the second obtaining module 520, the determining module 530, and the executing module 540 can be at least partially implemented as a computer program module that can perform corresponding functions when the computer program module is run.

[0165] Figure 6A block diagram of an electronic device suitable for implementing a method for implementing database transformation migration according to embodiments of the present disclosure is shown schematically.

[0166] As shown in Figure 6 The electronic device 600 according to embodiments of the present disclosure includes a processor 601 that can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 602 or loaded into a random access memory (RAM) 603 from a storage section 608. The processor 601 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chipset and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), and the like. The processor 601 can also include an on-board memory for cache use. The processor 601 can include a single processing unit or multiple processing units for executing different actions of the method processes according to embodiments of the present disclosure.

[0167] In the RAM 603, various programs and data required for the operation of the electronic device 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method processes according to embodiments of the present disclosure by executing the programs in the ROM 602 and / or the RAM 603. Note that the programs can also be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 can also perform various operations of the method processes according to embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0168] According to embodiments of the present disclosure, the electronic device 600 can also include an input / output (I / O) interface 605 that is also connected to the bus 604. The electronic device 600 can also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as necessary. A removable medium 611 such as a magnetic disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 610 as necessary, so that a computer program read out therefrom is installed in the storage section 608 as necessary.

[0169] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present disclosure.

[0170] According to an embodiment of the present disclosure, the computer readable storage medium can be a non-volatile computer readable storage medium, which can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer readable storage medium can include one or more memories, such as the ROM 602 and / or the RAM 603 described above, and / or one or more memory units other than the ROM 602 and the RAM 603.

[0171] The embodiments of the present disclosure also include a computer program product, which includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the item recommendation method provided by the embodiments of the present disclosure.

[0172] The above functions defined in the system / apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 601. According to an embodiment of the present disclosure, the system, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0173] In one embodiment, the computer program can rely on a tangible storage medium, such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of a signal via a network medium, and be downloaded and installed via the communication part 609 and / or installed from the detachable medium 611. The program codes contained in the computer program can be transmitted via any appropriate network medium, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0174] In such embodiments, the computer program can be downloaded and installed from the network via the communication section 609, and / or installed from the removable media 611. When the computer program is executed by the processor 601, the above-described functions defined in the system of the embodiments of the present disclosure are executed. According to the embodiments of the present disclosure, the system, device, apparatus, module, unit, and the like described above can be implemented by the computer program modules.

[0175] According to the embodiments of the present disclosure, the program code for executing the computer program provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages, and specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming language, and / or assembly / machine language. The programming language includes, but is not limited to, such as Java, C++, python, "C" language, or similar programming language. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected through the Internet by using an Internet service provider).

[0176] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that shown in the figures. For example, two blocks noted in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the flowcharts or block diagrams, and combinations of blocks in the flowcharts or block diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0177] Those skilled in the art can understand that the features described in various embodiments of the present disclosure and / or claims can be combined or / and integrated, even if such combinations or integrations are not explicitly described in the present disclosure. In particular, the features described in various embodiments of the present disclosure and / or claims can be combined and / or integrated in various combinations, without departing from the spirit and teachings of the present disclosure. All these combinations and / or integrations fall within the scope of the present disclosure.

[0178] The above described embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be advantageously used in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present disclosure, and all such substitutions and modifications shall fall within the scope of the present disclosure.

Claims

1. A method for implementing database transformation migration, comprising: obtaining usage specifications corresponding to a first database, a second database and a data replication tool respectively; obtaining, according to the usage specification corresponding to the first database, to-be-migrated data corresponding to a migration task from the first database; determining a migration strategy according to the migration task, wherein, in the case that the migration task comprises a full migration task and an incremental migration task, the migration strategy is determined as performing the incremental migration task by a host and performing the full migration task by a standby machine; in the case that a large table is involved in the incremental migration task and there is no LOB object in the data corresponding to the incremental migration task, the incremental migration task is performed by the standby machine; and performing the migration task according to the migration strategy by using the data replication tool based on the respective usage specifications, so as to migrate the to-be-migrated data to the second database; the method further comprises: setting a first constraint condition in a data migration process and a second constraint condition corresponding to the second database before data migration, so as to perform data migration according to the first constraint condition and the second constraint condition.

2. The method of claim 1, wherein, the step of obtaining, according to the usage specification corresponding to the first database, to-be-migrated data corresponding to a migration task from the first database, comprises: starting an archive log of the first database according to the usage specification corresponding to the first database; and obtaining the to-be-migrated data from the recorded content of the archive log.

3. The method of claim 1, wherein, the step of determining a migration strategy according to the migration task, comprises: in the case that the migration task comprises a full migration task, determining a first strategy corresponding to a host and a standby machine; and in the case that the migration task comprises the full migration task and an incremental migration task, determining a second strategy corresponding to the host and the standby machine.

4. The method of claim 3, wherein, the step of performing the migration task according to the migration strategy by using the data replication tool based on the respective usage specifications, so as to migrate the to-be-migrated data to the second database, comprises: in the case that the migration strategy represents the second strategy, splitting the to-be-migrated data into task data performed by each of the standby machines, wherein the to-be-migrated data represents data required to be migrated by the full migration task; for each of the standby machines, performing a migration task corresponding to the standby machine by using the data replication tool, so as to migrate the task data to the second database. 5.The method of claim 4, further comprising: for the incremental migration task, migrating, by using the host, target data in a supplement log to the second database based on a target time point at which the full migration task starts, wherein the target data represents data changed after the to-be-migrated data at the target time point. 6.The method of claim 5, further comprising: before migrating the target data, creating a primary key constraint for a table in which a target operation statement exists in the target data.

7. The method of claim 4, wherein, the step of splitting the to-be-migrated data into task data performed by each of the standby machines, comprises: For the full quantity migration task, in the case that the to-be-migrated data includes a primary key table and a non-primary key table, the primary key table and the non-primary key table are separately subjected to task creation, to obtain task data corresponding to the primary key table and the non-primary key table respectively, so as to execute the respective corresponding task data by using the standby machine; and In the case that the task data corresponding to the non-primary key table includes a plurality of non-primary key tables, the task data corresponding to the non-primary key table is split into a plurality of task sub-data, so as to execute the plurality of task sub-data by using the standby machine. 8.An apparatus for implementing database transformation migration, comprising: a first obtaining module configured to obtain respective usage specifications of a first database, a second database and a data replication tool; a second obtaining module configured to obtain, according to the usage specification corresponding to the first database, to-be-migrated data corresponding to a migration task from the first database; a determining module configured to determine a migration strategy according to the migration task, wherein, in the case that the migration task includes a full quantity migration task and an incremental migration task, the migration strategy of executing the incremental migration task by using a host and executing the full quantity migration task by using a standby machine is determined; in the case that a large table is involved in the incremental migration task and there is no LOB object in the data corresponding to the incremental migration task, the incremental migration task is executed by using the standby machine; and an executing module configured to execute the migration task according to the migration strategy by using the data replication tool based on the respective usage specifications, so as to implement migration of the to-be-migrated data to the second database. The apparatus further comprises: a setting module configured to set a first constraint condition in a data migration process and a second constraint condition corresponding to the second database before the data migration, so as to perform the data migration according to the first constraint condition and the second constraint condition. 9.An electronic device, comprising: one or more processors; a storage device configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method according to any one of claims 1-7. 10.A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1-7. 11.A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images