Data migration method, device, equipment and medium

Abstract

The present application provides a data migration method, device, equipment and medium, which relates to the field of database technology and is used to solve the problem that DATAX-WEB does not support table structure migration. The method is applied in the data synchronization tool DATAX-WEB, and the method includes: obtaining relevant information of the data to be migrated; the relevant information includes indication information of whether to perform table structure migration; if the indication information indicates to perform table structure migration, the plug-in of the source database is called through the unified interface of DATAX-WEB to obtain the table structure information of the source database; the plug-in of the source database is defined according to the operation method of the source database; the plug-in of the target database is called through the unified interface, and the target table is created in the target database according to the table structure information of the source database; the plug-in of the target database is defined according to the operation method of the target database; the data to be migrated is migrated to the target table. The method calls the plug-in of the database through the unified interface of DATAX-WEB, reads and writes the table structure information, so that DATAX-WEB can realize table structure migration.

Description

Technical Field

[0001] The present application relates to the field of database technology and provides a data migration method, device, equipment and medium. Background Art

[0002] DataX is an open source and widely used offline data synchronization tool / platform that implements efficient data synchronization between various heterogeneous data sources, such as: MySQL, Oracle, OceanBase, SqlServer, Postgre, HDFS, Hive, ADS, HBase, TableStore (OTS), MaxCompute (ODPS), Hologres, DRDS, databend, etc.

[0003] DATAX-WEB is a B / S architecture, which is built on top of DATAX. It provides developers with easier data source management, migration task management, online editing of migration configuration files, task timeout control, automatic retry upon failure, online configuration of Java virtual machine (JVM) parameters, etc. However, it does not support migration of table structures and some domestic databases. Summary of the invention

[0004] The embodiments of the present application provide a data migration method, apparatus, device and medium for solving the problem that DATAX-WEB does not support table structure migration.

[0005] In a first aspect, a data migration method is provided, in which a data synchronization tool DATAX-WEB is applied, and the data migration method comprises:

[0006] Obtaining relevant information about the data to be migrated; the relevant information includes indication information on whether to migrate the table structure;

[0007] If the instruction information indicates to migrate the table structure, the plug-in of the source database is called through the unified interface of the data synchronization tool DATAX-WEB to obtain the table structure information of the source database; the plug-in of the source database is defined according to the operation method of the source database;

[0008] Calling a plug-in of the target database through the unified interface, and creating a destination table in the target database according to the table structure information of the source database; the plug-in of the target database is defined according to the operation method of the target database;

[0009] Migrate the data to be migrated from the source database to the target table.

[0010] Optionally, before obtaining relevant information of the data to be migrated, the data migration method further includes:

[0011] According to the syntax of each database in multiple databases, write the SQL statement corresponding to each database;

[0012] Package the SQL statements corresponding to each database to obtain the plug-in for each database.

[0013] Optionally, the multiple databases include multiple first-category databases supported by the data synchronization tool DATAX-WEB; and writing SQL statements corresponding to each database according to the syntax of each database in the multiple databases includes:

[0014] According to the syntax of each of the first-category databases, a first SQL statement corresponding to each of the first-category databases is compiled; the first SQL statement includes an SQL statement for operating table structure information.

[0015] Optionally, the multiple databases include multiple second-category databases that are not supported by the data synchronization tool DATAX-WEB; and writing SQL statements corresponding to each database according to the syntax of each database in the multiple databases includes:

[0016] According to the syntax of each second-category database, write a second SQL statement corresponding to each second-category database; the second SQL statement includes an SQL statement for operating table structure information, an SQL statement for operating table data, and an SQL statement for returning an error; the error is used to indicate the reason for the failure of migration of the data to be migrated.

[0017] Optionally, the relevant information includes a type of a source database; and the step of migrating the data to be migrated from the source database to the target table includes:

[0018] If it is determined, according to the type of the source database, that the source database belongs to the second type of database, calling a plug-in of the source database through the unified interface to read the data to be migrated from the source database;

[0019] The data to be migrated is written into the destination table.

[0020] Optionally, the relevant information includes a type of a target database; and writing the to-be-migrated data into the target table includes:

[0021] According to the type of the target database, if it is determined that the target database belongs to the second type of database, a plug-in of the target database is called through the unified interface to write the data to be migrated into the destination table.

[0022] Optionally, after migrating the data to be migrated to the target table, the data migration method further includes:

[0023] If the migration fails, an error from the source database and / or the target database is returned.

[0024] In a second aspect, a data migration device is provided, wherein the data migration device is arranged in the data synchronization tool DATAX-WEB, and the data migration device comprises:

[0025] A related information acquisition module, used to acquire related information of the data to be migrated; the related information includes indication information of whether to migrate the table structure;

[0026] A table structure information acquisition module, for invoking a plug-in of a source database through a unified interface of the DATAX-WEB to acquire the table structure information of the source database if the indication information indicates to migrate the table structure; the plug-in of the source database is defined according to the operation method of the source database;

[0027] A table structure migration module, used to call a plug-in of a target database through the unified interface, and create a destination table in the target database according to the table structure information of the source database; the plug-in of the target database is defined according to the operation method of the target database;

[0028] The table data migration module is used to migrate the data to be migrated to the target table.

[0029] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the data migration method described in the first aspect.

[0030] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement the data migration method described in the first aspect.

[0031] Compared with the prior art, the beneficial effects of the embodiments of the present application are as follows:

[0032] The embodiment of the present application provides a data migration method, which is applied in the data synchronization tool DATAX-WEB, and the method includes: obtaining relevant information of the data to be migrated; the relevant information includes indication information of whether to migrate the table structure; if the indication information indicates to migrate the table structure, the plug-in of the source database is called through the unified interface of DATAX-WEB to obtain the table structure information of the source database; the plug-in of the source database is defined according to the operation method of the source database; the plug-in of the target database is called through the unified interface, and the target table is created in the target database according to the table structure information of the source database; the plug-in of the target database is defined according to the operation method of the target database; the data to be migrated is migrated to the target table. The method calls the plug-ins of the source database and the target database through the unified interface of DATAX-WEB, and can read and write the table structure information, so that DATAX-WEB can realize the table structure migration. This plug-in design provides a plug-in driver configuration for DATAX-WEB. Even if the subsequent database version changes, it only needs to modify the jar logic corresponding to the plug-in of each database, without changing the DATAX-WEB code, and the invasiveness to the existing DATAX-WEB code is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0034] Attached Figure 1 A schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application;

[0035] Attached Figure 2 A schematic diagram of a flow chart of a data migration method provided in an embodiment of the present application;

[0036] Attached Figure 3 Another schematic diagram of a data migration method according to an embodiment of the present invention;

[0037] Attached Figure 4 A schematic diagram of the structure of a data migration device provided in an embodiment of the present application.

[0038] Markings in the figure: 101 - processor, 102 - communication bus, 103 - network interface, 104 - user interface, 105 - memory. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0040] In order to solve the problem that DATAX-WEB does not support table structure migration, the present application provides a data migration method, which can be executed by a computer device in which a data synchronization tool DATAX-WEB is installed. Before introducing the specific technical solution of the present application, the hardware operating environment involved in the embodiment of the present application is first introduced.

[0041] See attached Figure 1 , attached Figure 1 A schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application.

[0042] As attached Figure 1 As shown, the computer device may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 104 may also include a standard wired interface and a wireless interface. The network interface 103 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 105 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 105 may optionally also be a storage device independent of the aforementioned processor 101.

[0043] Those skilled in the art will appreciate that Figure 1 The structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0044] As attached Figure 1 As shown, the memory 105 as a storage medium may include an operating system, a data configuration module, a network communication module, a user interface module and an electronic program.

[0045] In the attached Figure 1 In the computer device shown, the network interface 103 is mainly used for data communication with the network server; the user interface 104 is mainly used for data interaction with the user; the processor 101 and the memory 105 in the computer device of the present invention can be set in the computer device, and the computer device calls the data migration device stored in the memory 105 through the processor 101, and executes the data migration method provided in the embodiment of the present application.

[0046] Based on the computer device of the aforementioned embodiment, Figure 2 A data migration method provided in an embodiment of the present application is introduced, and the steps are as follows:

[0047] S201: Obtain relevant information of the data to be migrated.

[0048] The data to be migrated refers to the data that needs to be migrated from the source database to the target database. The data to be migrated can be part of the data in the source database or all of the data in the source database. The relevant information is used to indicate information related to the data to be migrated, including the type and connection information of the source database, the type and connection information of the target database, the table information of the data to be migrated, and the indication information of whether to migrate the table structure. The source database refers to the database where the data to be migrated originally resides, and the target database refers to the database to which the data to be migrated is finally copied, such as MySQL, Oracle, SQL Server, etc. The connection information includes the host name, IP address, port number, database name, user name and password of the database. The connection information of the source database is used to establish a connection with the source database, and the connection information of the target database is used to establish a connection with the target database. The table information refers to the specific table name or table name pattern that needs to be migrated (such as using wildcards to specify multiple tables), as well as the structure and field information of the table.

[0049] In the specific implementation process, DATAX-WEB can provide a visual interface, and users can perform configuration operations on the visual interface by clicking the mouse, dragging, filling in forms, etc. The computer device can respond to the configuration operations performed by the user on the visual interface to obtain relevant information of the data to be migrated. For example, the user selects the type of source database, enters the connection information of the source database, and then selects the table information of the data to be migrated on the visual interface of DATAX-WEB in the computer device, and then selects the type of target database, enters the connection information of the target database, and selects whether to migrate the table structure, etc. Alternatively, after other devices obtain the relevant information of the data to be migrated, they can send it to the computer device, so that the computer device obtains the relevant information of the data to be migrated.

[0050] In a possible embodiment, before obtaining relevant information of the data to be migrated, an SQL statement corresponding to each database may be written according to the syntax of each database in the multiple databases; the SQL statement corresponding to each database may be packaged to obtain a plug-in for each database.

[0051] In the specific implementation process, multiple databases include multiple first-class databases supported by the data synchronization tool DATAX-WEB and multiple second-class databases not supported by the data synchronization tool DATAX-WEB. The first-class databases include: MySQL, Oracle, OceanBase, SqlServer, Postgre, HDFS, Hive, ADS, HBase, TableStore (OTS), MaxCompute (ODPS), Hologres, DRDS, databend and other databases, and the second-class databases include domestic databases such as DAMO database and Xugu database. For different types of databases, the SQL statements written are different, and the corresponding plug-in functions are different, which are introduced below.

[0052] In a possible embodiment, the step of writing an SQL statement corresponding to each database according to the syntax of each database in the multiple databases includes:

[0053] According to the syntax of each first-category database, a first SQL statement corresponding to each first-category database is compiled; the first SQL statement includes an SQL statement for operating table structure information.

[0054] In an embodiment of the present application, according to the syntax of each first-category database, SQL statements for operating table structure information, such as SQL statements for querying, reading, and writing table structure information, are written, and then the first SQL statements corresponding to each first-category database are packaged to obtain a plug-in corresponding to each first-category database. The plug-in can be used to implement the migration function of the first-category database for table structure information.

[0055] In a possible embodiment, the step of writing an SQL statement corresponding to each database according to the syntax of each database in the multiple databases includes:

[0056] According to the syntax of each second-category database, a second SQL statement corresponding to each second-category database is compiled; the second SQL statement includes an SQL statement for operating table structure information, an SQL statement for operating table data, and an SQL statement for returning an error.

[0057] In the specific implementation process, according to the syntax of each second-category database, SQL statements for operating table structure information, SQL statements for operating table data, and SQL statements for returning errors are written. Among them, SQL statements for operating table structure information include SQL statements for querying, reading, and writing table structure information, SQL statements for operating table data include SQL statements for querying, reading, and writing table data, errors are used to indicate the reason for the failure of migration of the data to be migrated, and SQL statements for returning errors include SQL statements for returning the reason for the failure of migration of the data to be migrated.

[0058] In the embodiment of the present application, the plug-in packaged by the SQL statement for operating the table structure information can realize the migration function of the second type of database to the table structure information, the plug-in packaged by the SQL statement for operating the table data can realize the migration function of the second type of database to the table data, and the plug-in packaged by the SQL statement for returning errors can realize the error enumeration function of the second type of database as the source database or the target database. By building a plug-in for the second type of database, the problem of the limitations of DATAX-WEB in database support is solved, and DATAX-WEB supports domestic databases such as DAMO and XUGU.

[0059] Taking Xugu database as an example, the following will introduce how to build a plug-in for Xugu database.

[0060] (1) Add query for metadata of Xugu database.

[0061] In the source code, create a class XuguDatabaseMeta that inherits the BaseDatabaseMeta class and implements the DatabaseInterface interface. This interface provides methods for obtaining table information, column information, primary key information, maximum id, mode information, etc. XuguDatabaseMeta writes SQL statements for querying these meta information according to the syntax of the Xugu database. Among them, XuguDatabaseMeta is a class in the Xugu database driver, which is used to obtain and describe database metadata information. Metadata is data about objects such as databases, tables, columns, indexes, views, etc. It contains the structure, properties and other related information of these objects. By using the XuguDatabaseMeta class, you can get the metadata information of the Xugu database.

[0062] (2) Add judgment on the reader and writer of Xugu database.

[0063] In the package com.wugui.datax.admin.tool.datax.reader, the XuguReader class is added to inherit the BaseReaderPlugin class, implement the DataxReaderInterface interface, and rewrite the getName and sample methods to achieve the function of obtaining the plug-in name and sample configuration. In this way, when using Datax, you can use the XuguReader plug-in in the way specified in the configuration file to read the data in the Xugu database.

[0064] In the package com.wugui.datax.admin.tool.datax.writer, the XuguWriter class is added to inherit the BaseWriterPlugin class, implement the DataxWriterInterface interface, and rewrite the getName and sample methods to achieve the function of obtaining the plug-in name and sample configuration. In this way, when using Datax, you can use the XuguWriter plug-in in the way specified in the configuration file to write data into the Xugu database.

[0065] In the DataxJsonHelper class, the initReader and initWriter methods are used to initialize the DataX plug-in. In these two methods, an if statement is used to determine whether the incoming plug-in instance is a newly added XuguReader or XuguWriter. If so, the corresponding initialization operation is performed. Otherwise, the initialization operation of other plug-ins is performed, or an exception is thrown.

[0066] (3) Add query tools for Xugu database.

[0067] In the com.wugui.datax.admin.tool.query class, add the XuguQueryTool class to inherit BaseQueryTool, implement the QueryToolInterface interface, provide a constructor with a JobDatasource type parameter, and call the parent class's constructor. Add the getXuguQueryToolInstance method in the QueryToolFactory class, which adds a XuguQueryTool return, and getByDbType adds the judgment logic for the Xugu database.

[0068] (4) Add error return to Xugu database.

[0069] In the com.wugui.datax.admin.util class, a new method xuguConnectionErrorAna is added. In the DBUtilErrorCode class, an error enumeration for using Xugu database as the source database or target database is added. In the asConnException method, the judgment logic for Xugu database is added.

[0070] S202: If the instruction information indicates to migrate the table structure, the plug-in of the source database is called through the unified interface of DATAX-WEB to obtain the table structure information of the source database.

[0071] In the specific implementation process, after determining the relevant information of the data to be migrated, the computer device can first establish a connection with the source database according to the connection information of the source database, and establish a connection with the target database according to the connection information of the target database. After establishing the connection with the source database and the target database respectively, it is determined whether to migrate the table structure according to the indication information.

[0072] Furthermore, if the instruction information indicates to migrate the table structure, the plug-in of the source database is called through the unified interface of DATAX-WEB. The unified interface refers to the JDBC (Java Database Connectivity) interface. JDBC is an application program interface (API) provided by Java for connecting and operating with databases. By using the JDBC interface, Java applications can interact with various databases to perform SQL queries, update data, and perform transactions. The plug-in of the source database is defined according to the operation method of the source database. By calling the plug-in of the source database, the table structure information of the source database can be obtained.

[0073] Table structure information refers to the definition of the table in the database, including table name, column name, data type, constraints and other information. The following introduces some common table structure information.

[0074] 1. Table name: The name of the table, used to uniquely identify the table in the database.

[0075] 2. Column name: The column name in the table, used to identify each field in the table.

[0076] 3. Data type: Each column has a specific data type, which is used to define the type of data that can be stored in the column, such as integers, strings, dates, etc.

[0077] 4. Length / precision: Some data types require the specified length (character type) or precision (numeric type) to limit the range of stored data.

[0078] 5. Constraints: Rules used to limit the integrity and validity of data in a table. Common constraints include primary key constraints, unique constraints, foreign key constraints, and check constraints.

[0079] 6. Default value: Specify the default value of a column. If no value for the column is provided when inserting a new record, the default value is used.

[0080] 7. Auto-increment columns: Some databases support auto-increment columns, which automatically generate a unique value when a new record is inserted.

[0081] 8. Index: A data structure used to speed up table queries, which can be sorted and searched based on the values ​​of one or more columns.

[0082] It should be noted that if the data to be migrated is part of the data in the source database, when obtaining the table structure information of the source database, only the table structure information of the source table where the data to be migrated is located can be obtained to reduce the amount of data processing.

[0083] In a possible embodiment, after obtaining relevant information of the data to be migrated, if the instruction information indicates not to migrate the table structure, S204 is directly executed, that is, the data to be migrated is migrated from the source database to the target database.

[0084] S203: Call the plug-in of the target database through the unified interface, and create a destination table in the target database according to the table structure information of the source database.

[0085] In the specific implementation process, after obtaining the table structure information of the source database, the plug-in of the target database can be called. The plug-in of the target database is defined according to the operation method of the target database. The table structure information of the source database is written into the target database through the plug-in of the target database, and the target table is created in the target database. The target table is an empty table with the same table structure information as the source table where the data to be migrated is located, including the table name, field name, data type and constraints, etc. For each field, it is necessary to ensure that its data type in the target table matches the data type in the source table. If the data type does not match, type conversion is required. For constraints such as primary keys, unique constraints, indexes, etc., it is necessary to re-establish the constraints in the target table to ensure data consistency and integrity.

[0086] S204: Migrate the data to be migrated from the source database to the target table.

[0087] In the specific implementation process, after creating the target table, you need to perform data migration operations to read the data to be migrated from the source table of the source database and write the data to be migrated into the target table. Different methods are used to read the data to be migrated from the source database for different types of source databases, which are introduced below.

[0088] In a possible embodiment, according to the type of the source database, if it is determined that the source database belongs to the second type of database, a plug-in of the source database is called through a unified interface to read the data to be migrated from the source database and write the data to be migrated into a destination table.

[0089] In the embodiment of the present application, for the second type of database not supported by DATAX-WEB, the corresponding plug-in is called through the unified interface of DATAX-WEB to read the data to be migrated from the source database, and the table data reading function of the database not supported by DATAX-WEB is added, thereby realizing DATAX-WEB's support for domestic databases such as DAMO and XUGU.

[0090] In another possible embodiment, according to the type of the source database, if it is determined that the source database belongs to the first type of database, the data to be migrated is directly read from the source database, and the data to be migrated is written into the destination table.

[0091] In an embodiment of the present application, for the first type of database supported by DATAX-WEB, the data synchronization function provided by DATAX-WEB can be used to directly and quickly read the data to be migrated from the source database. Compared with the plug-in reading method, the efficiency of reading the data to be migrated can be improved, thereby improving the efficiency of data migration.

[0092] Furthermore, for different types of target databases, the methods of writing the data to be migrated into the destination table are different, which are introduced below respectively.

[0093] In a possible embodiment, according to the type of the target database, if it is determined that the target database belongs to the second type of database, a plug-in of the target database is called through a unified interface to write the data to be migrated into the destination table.

[0094] In the embodiment of the present application, for the second type of database not supported by DATAX-WEB, the corresponding plug-in is called through a unified interface to write the data to be migrated into the destination table in the target database, and a table data writing function for databases not supported by DATAX-WEB is added, thereby realizing DATAX-WEB's support for domestic databases such as DAMO and XUGU.

[0095] In a possible embodiment, according to the type of the target database, if it is determined that the target database belongs to the first type of database, the data to be migrated is directly written into the destination table.

[0096] In an embodiment of the present application, for the first type of database supported by DATAX-WEB, the data synchronization function of DATAX-WEB can be used to directly and quickly write the data to be migrated into the destination table in the target database. Compared with the plug-in writing method, the efficiency of writing the data to be migrated can be improved, thereby improving the efficiency of data migration.

[0097] In a possible embodiment, after the data to be migrated is migrated from the source database to the target table, if the migration fails, an error from the source database and / or the target database is returned.

[0098] For example, if Xugu database is used as the source database, the error from Xugu database may be: "Failed to read Xugu source data, please check whether the source table information is correct", or for example, if Xugu database is used as the target database, the error from Xugu database may be: "Failed to write Xugu target data, please check whether the target table information is correct".

[0099] Please refer to the attached Figure 3 , is another flow chart of the data migration method provided in the embodiment of the present application, and Figure 3 Taking the Xugu database as an example, the following Figure 3 Make an introduction.

[0100] S301. Select Xugu database as the data source.

[0101] First, the user performs configuration operations in the visual interface provided by DATAX-WEB. When selecting the data source, select Xugu Database as the source database or target database. After completing the configuration operations for the data to be migrated, click the "Submit" button.

[0102] S302, start executing DATAX-WEB task.

[0103] After the user completes the configuration operation for the data to be migrated, click the "Execute" button in the visual interface provided by DATAX-WEB to start the execution of the DATAX-WEB task.

[0104] S303: Determine whether to perform table structure migration.

[0105] According to the instruction information configured by the user, it is determined whether to migrate the table structure. If the instruction information indicates to migrate the table structure, S304 is executed, that is, the plug-in is called to migrate the table structure. If the instruction information indicates not to migrate the table structure, S305 is directly executed, that is, the table data is migrated.

[0106] S304. Call the plug-in to migrate the table structure.

[0107] The plug-in of the source database is called through the unified interface of DATAX-WEB to obtain the table structure information of the data to be migrated, and the plug-in of the target database is called through the unified interface of DATAX-WEB to write the table structure information of the data to be migrated into the target database. For the specific migration process, please refer to the contents discussed in S202 and S203 above, which will not be repeated here. After migrating the table structure, S305 is executed again, that is, migrating the table data.

[0108] S305. Migrate table data.

[0109] The data to be migrated is migrated from the source database to the target database. For the specific migration process, please refer to the content discussed in S204 above, which will not be repeated here.

[0110] In summary, the embodiment of the present application provides a data migration method, which calls the external plug-in of the database, i.e., the jar package, through the unified interface of DATAX-WEB to read and write table structure information, thereby realizing the table structure migration function of DATAX-WEB. By calling the external plug-in of the database, i.e., the jar package, through the unified interface of DATAX-WEB to read and write table data, more database type support is added, so that the migration of domestic databases can also make full use of the high performance of DATAX itself. This plug-in design makes the existing DATAX-WEB code less invasive, that is, it enhances the function of DXTAX-WEB, and also ensures the stability of the software and improves performance. Later, as the database version changes, it only needs to modify the logic of the external jar package, which is completely decoupled from the DATAX-WEB data migration, ensuring the maintainability and scalability of DATAX-WEB.

[0111] Based on the same inventive concept, as shown in the attached Figure 4As shown, the embodiment of the present application further provides a data migration device, which is arranged in the data synchronization tool DATAX-WEB, and the data migration device includes:

[0112] A related information acquisition module is used to acquire related information of the data to be migrated; the related information includes indication information on whether to migrate the table structure;

[0113] The table structure information acquisition module is used to call the plug-in of the source database through the unified interface of DATAX-WEB to obtain the table structure information of the source database if the indication information indicates to migrate the table structure; the plug-in of the source database is defined according to the operation method of the source database;

[0114] The table structure migration module is used to call the plug-in of the target database through a unified interface and create a target table in the target database according to the table structure information of the source database; the plug-in of the target database is defined according to the operation method of the target database;

[0115] The table data migration module is used to migrate the data to be migrated to the target table.

[0116] It should be noted that each module in the data migration device in this embodiment corresponds one-to-one to each step in the data migration method in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned data migration method, which will not be repeated here.

[0117] In addition, in one embodiment, the present application also provides a computer device, which includes a processor, a memory, and a computer program stored in the memory, and the computer program implements the aforementioned data migration method when executed by the processor.

[0118] In addition, in one embodiment, the present application also provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the aforementioned data migration method is implemented.

[0119] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0120] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0121] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0122] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0123] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0124] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0126] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A data migration method, characterized in that: In the application data synchronization tool DATAX-WEB, the data migration method includes: Obtaining relevant information about the data to be migrated; the relevant information includes indication information on whether to migrate the table structure; If the instruction information indicates to migrate the table structure, the plug-in of the source database is called through the unified interface of the data synchronization tool DATAX-WEB to obtain the table structure information of the source database; the plug-in of the source database is defined according to the operation method of the source database; Calling a plug-in of the target database through the unified interface, and creating a destination table in the target database according to the table structure information of the source database; the plug-in of the target database is defined according to the operation method of the target database; Migrating the data to be migrated from the source database to the target table; Before obtaining relevant information of the data to be migrated, the data migration method further includes: According to the syntax of each database in multiple databases, write the SQL statement corresponding to each database; Package the SQL statements corresponding to each database and obtain the plug-in corresponding to each database; The multiple databases include multiple first-category databases supported by the data synchronization tool DATAX-WEB and multiple second-category databases not supported by the data synchronization tool DATAX-WEB; the step of writing an SQL statement corresponding to each database according to the syntax of each database in the multiple databases includes: According to the syntax of each of the first-category databases, write a first SQL statement corresponding to each of the first-category databases; the first SQL statement includes an SQL statement for operating table structure information; According to the syntax of each second-category database, write a second SQL statement corresponding to each second-category database; the second SQL statement includes an SQL statement for operating table structure information, an SQL statement for operating table data, and an SQL statement for returning an error; the error is used to indicate the reason for the failure of migration of the data to be migrated.

2. The data migration method according to claim 1, characterized in that: The relevant information includes the type of the source database; and the step of migrating the data to be migrated from the source database to the target table includes: If it is determined, according to the type of the source database, that the source database belongs to the second type of database, calling a plug-in of the source database through the unified interface to read the data to be migrated from the source database; The data to be migrated is written into the destination table.

3. The data migration method according to claim 2, characterized in that: The relevant information includes the type of the target database; and the step of writing the data to be migrated into the target table includes: According to the type of the target database, if it is determined that the target database belongs to the second type of database, a plug-in of the target database is called through the unified interface to write the data to be migrated into the destination table.

4. The data migration method according to claim 2, characterized in that: After migrating the data to be migrated to the target table, the data migration method further includes: If the migration fails, an error from the source database and / or the target database is returned.

5. A data migration device, characterized in that: The data migration device is arranged in the data synchronization tool DATAX-WEB, and the data migration device comprises: A related information acquisition module, used to acquire related information of the data to be migrated; the related information includes indication information of whether to migrate the table structure; A table structure information acquisition module, for invoking a plug-in of a source database through a unified interface of the DATAX-WEB to acquire the table structure information of the source database if the indication information indicates to migrate the table structure; the plug-in of the source database is defined according to the operation method of the source database; A table structure migration module, used to call a plug-in of a target database through the unified interface, and create a destination table in the target database according to the table structure information of the source database; the plug-in of the target database is defined according to the operation method of the target database; A table data migration module, used to migrate the data to be migrated to the target table; Before obtaining relevant information of the data to be migrated, the data migration method further includes: writing an SQL statement corresponding to each database according to the syntax of each database in the multiple databases; packaging the SQL statement corresponding to each database to obtain a plug-in corresponding to each database; the multiple databases include multiple first-category databases supported by the data synchronization tool DATAX-WEB and multiple second-category databases not supported by the data synchronization tool DATAX-WEB; The step of writing an SQL statement corresponding to each database according to the syntax of each database in the multiple databases includes: writing a first SQL statement corresponding to each first-category database according to the syntax of each first-category database; the first SQL statement includes an SQL statement for operating table structure information; writing a second SQL statement corresponding to each second-category database according to the syntax of each second-category database; the second SQL statement includes an SQL statement for operating table structure information, an SQL statement for operating table data, and an SQL statement for returning an error; the error is used to indicate the reason for the failure of migration of the data to be migrated.

6. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the data migration method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the processor executes the computer program to implement the data migration method according to any one of claims 1 to 4.

Images