Metadata migration method and device between databases, storage medium and equipment

Abstract

The invention discloses a method and a device for migrating metadata between databases, a storage medium and computer equipment, and the method comprises the following steps: in response to a metadata migration instruction, determining a first database type of a source database and a second database type of a target database; based on the first database type and the second database type, determining a first conversion strategy corresponding to a front mapper and a second conversion strategy corresponding to a rear mapper; obtaining to-be-migrated metadata information from the source database according to a target operation control parameter, executing the first conversion strategy on the to-be-migrated metadata information through the front mapper to obtain a first conversion result, and executing the second conversion strategy on the first conversion result through the rear mapper to obtain a second conversion result; obtaining a second conversion result; and storing the second conversion result in the target database.

Description

Technical Field

[0001] The present application relates to the technical field of database migration, and in particular to a method and apparatus for metadata migration between databases, a storage medium, and a computer device. Background Art

[0002] Migrating a database to a domestic or open source database is an important step in achieving database autonomy and control, and database metadata migration is the most important first step in the database migration process. This is because metadata contains important information such as the database structure, table structure, index, view, stored procedure, trigger, foreign key relationship, etc., which is the basis for database operation and management. Therefore, how to efficiently and flexibly complete metadata migration is the basis for all subsequent work.

[0003] Currently, although there are many commercial or open source database metadata migration tools, these tools generally only support migration between specific types of databases and lack scalability and flexibility. Summary of the invention

[0004] In view of this, the present application provides a method and device for metadata migration between databases, a storage medium, and a computer device. By using the target data type as an intermediate bridge, the mapping is performed in two steps during metadata migration, which uncouples the source database from the target database, effectively solves the problem of metadata migration between different database types, improves the scalability of the migration business, and is conducive to improving the efficiency and accuracy of metadata migration. In addition, the entire migration process realizes the automatic conversion and storage of the metadata information to be migrated from the source database to the target database.

[0005] According to one aspect of the present application, a method for migrating metadata between databases is provided, comprising:

[0006] In response to the metadata migration instruction, determining a first database type of the source database and a second database type of the target database;

[0007] Based on the first database type and the second database type, determining a first conversion strategy corresponding to a front mapper and a second conversion strategy corresponding to a rear mapper;

[0008] According to the target operation control parameter, the metadata information to be migrated is obtained from the source database, and the first conversion strategy is executed on the metadata information to be migrated by the front mapper to obtain a first conversion result, and the second conversion strategy is executed on the first conversion result by the rear mapper to obtain a second conversion result;

[0009] The second conversion result is stored in the target database.

[0010] According to another aspect of the present application, a metadata migration device between databases is provided, comprising:

[0011] A database type determination module, configured to determine a first database type of a source database and a second database type of a target database in response to a metadata migration instruction;

[0012] A conversion strategy determination module, used to determine a first conversion strategy corresponding to a front mapper and a second conversion strategy corresponding to a rear mapper based on the first database type and the second database type;

[0013] a conversion module, configured to obtain metadata information to be migrated from the source database according to target operation control parameters, and execute the first conversion strategy on the metadata information to be migrated through the front mapper to obtain a first conversion result, and execute the second conversion strategy on the first conversion result through the rear mapper to obtain a second conversion result;

[0014] A storage module is used to store the second conversion result in the target database.

[0015] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned metadata migration method between databases is implemented.

[0016] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned metadata migration method between databases when executing the program.

[0017] By means of the above technical scheme, the present application provides a method and device for metadata migration between databases, a storage medium, and a computer device. After receiving a metadata migration instruction, the first database type of the source database and the second database type of the target database are first identified. Based on the first database type of the source database and the second database type of the target database, a corresponding conversion strategy can be selected. Here, the conversion strategy includes a first conversion strategy corresponding to the front mapper and a second conversion strategy corresponding to the back mapper. Among them, the first conversion strategy is generally used to convert the metadata of the source database from the first database type to the target data type, and the second conversion strategy is generally used to convert from the target data type to the second database type corresponding to the target database. According to the target operation control parameter, the metadata information to be migrated is extracted from the source database. Then, the front mapper is used to perform a first conversion on the extracted metadata information to be migrated according to the first conversion strategy, and the metadata information to be migrated is converted from the first database type to the target data type to obtain a first conversion result. Subsequently, the back mapper is used to execute the second conversion strategy on the first conversion result, and the first conversion result is converted from the target data type to the second database type to obtain the final second conversion result. The metadata information to be migrated that has undergone two conversions, that is, the second conversion result, is stored in the target database. The embodiment of the present application uses the target data type as an intermediate bridge, and performs mapping in two steps during metadata migration, thereby uncoupling the source database and the target database, effectively solving the problem of metadata migration between different database types, improving the scalability of the migration business, and facilitating the efficiency and accuracy of metadata migration. In addition, the entire migration process realizes the automatic conversion and storage of the metadata information to be migrated from the source database to the target database.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A schematic diagram of a process flow of a metadata migration method between databases provided in an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of a process flow of another metadata migration method between databases provided in an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram showing a process of determining a first conversion strategy and a second conversion strategy provided by an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram of the structure of a metadata migration device between databases provided in an embodiment of the present application is shown;

[0024] Figure 5 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0026] In this embodiment, a method for migrating metadata between databases is provided. Figure 1 As shown, the method includes:

[0027] Step 101 : In response to a metadata migration instruction, determine a first database type of a source database and a second database type of a target database.

[0028] A metadata migration method between databases provided in an embodiment of the present application can be applied to metadata migration between databases of the same type, and can also be applied to metadata migration between heterogeneous databases. The method can be implemented in a computing environment at one end where the migration operation is performed, such as a server or a developer's local machine. First, when a user wants to perform metadata migration between databases, a metadata migration instruction can be triggered. After receiving the metadata migration instruction, the server or local machine first identifies the first database type of the source database, that is, determines the database type of the source database. Here, the source database can be a relational database, such as MySQL, Oracle, or a non-relational database, such as MongoDB, Cassandra, etc. At the same time, the second database type of the target database, that is, the database type of the database that receives the migrated metadata, is determined. The target database can also be a relational database or a non-relational database.

[0029] Step 102: Determine a first conversion strategy corresponding to a front mapper and a second conversion strategy corresponding to a rear mapper based on the first database type and the second database type.

[0030] In this embodiment, based on the first database type of the source database and the second database type of the target database, a corresponding conversion strategy can be selected. Here, the conversion strategy includes a first conversion strategy corresponding to the front mapper and a second conversion strategy corresponding to the back mapper. Among them, the first conversion strategy is usually used to convert the metadata of the source database from the first database type to the target data type, and the second conversion strategy is usually used to convert from the target data type to the second database type corresponding to the target database. Among them, the target data type can be determined according to demand, as an intermediate bridge from the first database type corresponding to the source database to the second database type corresponding to the target database. The biggest advantage of this is that the coupling is released. If a new source database is added, only a conversion strategy from the source database to the front mapper of the target data type needs to be added; similarly, if only a new target database is added, only a conversion strategy from the target data type to the back mapper of the target database needs to be added. This method can cope with metadata migration between multiple different source databases and target databases.

[0031] Step 103, according to the target operation control parameters, obtain the metadata information to be migrated from the source database, and execute the first conversion strategy on the metadata information to be migrated through the front mapper to obtain a first conversion result, and execute the second conversion strategy on the first conversion result through the post mapper to obtain a second conversion result.

[0032] In this embodiment, the metadata information to be migrated is extracted from the source database according to the target operation control parameters. Here, the target operation control parameters are parameters used to indicate the migration content and migration requirements corresponding to different metadata migration tasks. Specifically, the target operation control parameters may include the selection range of the metadata table, whether to migrate partitions, whether to migrate foreign keys and other control parameters. Then, using the front mapper, the first conversion is performed on the extracted metadata information to be migrated according to the first conversion strategy, and the metadata information to be migrated is converted from the first database type to the target data type to obtain a first conversion result. Subsequently, through the post mapper, the second conversion strategy is executed on the first conversion result, and the first conversion result is converted from the target data type to the second database type to obtain the final second conversion result. The above steps are performed to ensure that the metadata information to be migrated is successfully converted from the first database type to the second database type.

[0033] Step 104: store the second conversion result in the target database.

[0034] In this embodiment, the metadata information to be migrated that has been converted twice, ie, the second conversion result, is stored in the target database.

[0035] By applying the technical solution of this embodiment, after receiving the metadata migration instruction, the first database type of the source database and the second database type of the target database are first identified. Based on the first database type of the source database and the second database type of the target database, a corresponding conversion strategy can be selected. Here, the conversion strategy includes a first conversion strategy corresponding to the front mapper and a second conversion strategy corresponding to the rear mapper. Among them, the first conversion strategy is generally used to convert the metadata of the source database from the first database type to the target data type, and the second conversion strategy is generally used to convert from the target data type to the second database type corresponding to the target database. According to the target operation control parameter, the metadata information to be migrated is extracted from the source database. Then, the front mapper is used to perform a first conversion on the extracted metadata information to be migrated according to the first conversion strategy, and the metadata information to be migrated is converted from the first database type to the target data type to obtain a first conversion result. Subsequently, the second conversion strategy is executed on the first conversion result through the rear mapper, and the first conversion result is converted from the target data type to the second database type to obtain the final second conversion result. The metadata information to be migrated that has undergone two conversions, that is, the second conversion result, is stored in the target database. The embodiment of the present application uses the target data type as an intermediate bridge, and performs mapping in two steps during metadata migration, thereby uncoupling the source database and the target database, effectively solving the problem of metadata migration between different database types, improving the scalability of the migration business, and facilitating the efficiency and accuracy of metadata migration. In addition, the entire migration process realizes the automatic conversion and storage of the metadata information to be migrated from the source database to the target database.

[0036] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another metadata migration method between databases is provided, such as Figure 2 As shown, the method includes:

[0037] Step 201 : In response to a metadata migration instruction, determine a first database type of a source database and a second database type of a target database.

[0038] Step 202: determine whether the first database type is consistent with the second database type.

[0039] Step 203: When the judgment result is consistent, determine that the first conversion strategy corresponding to the front mapper and the second conversion strategy corresponding to the rear mapper are both not to perform any operation.

[0040] Step 204, when the judgment result is inconsistent, query from the first conversion list whether there is a target conversion strategy between the first database type and the second database type; when the target conversion strategy exists in the first conversion list, determine the target conversion strategy as the first conversion strategy corresponding to the front mapper, and determine the second conversion strategy corresponding to the post mapper as not performing any operation; when the target conversion strategy does not exist in the first conversion list, query from the second conversion list the conversion strategy corresponding to the conversion from the first database type to the target data type as the first conversion strategy, and query from the third conversion list the conversion strategy corresponding to the conversion from the target data type to the second database type as the second conversion strategy.

[0041] In this embodiment, after obtaining the first database type and the second database type, it can be determined whether the first database type of the source database is consistent with the second database type of the target database.

[0042] If the judgment result shows that the two database types are consistent, then no data conversion is required because the metadata is already compatible in structure. In this case, the first conversion strategy corresponding to the front mapper and the second conversion strategy corresponding to the back mapper can both be determined as "do nothing". In other words, the metadata information to be migrated will be directly migrated from the source database to the target database without any intermediate conversion steps.

[0043] If the judgment result shows that the two database types are inconsistent, data conversion is required to ensure that the metadata is compatible with the target database. Specifically, first, it is possible to query from the predefined first conversion list whether there is a target conversion strategy directly for the two database types. The first conversion list may contain a part of known valid conversion strategies between database types. If there is a target conversion strategy in the first conversion list, it is determined as the first conversion strategy corresponding to the front mapper, and since direct conversion is sufficient, the second conversion strategy corresponding to the rear mapper is still "do nothing". If there is no target conversion strategy in the first conversion list (i.e., direct conversion is not feasible or unknown), an indirect conversion method is required. At this time, the target data type can be used as a bridge. The conversion strategy corresponding to the conversion from the first database type to the target data type is queried from the second conversion list, and it is determined as the first conversion strategy. Then, the conversion strategy corresponding to the conversion from the target data type to the second database type is queried from the third conversion list, and it is determined as the second conversion strategy. In this way, the metadata information to be migrated is first converted to the target data type through the first conversion strategy, and then converted to the second database type through the second conversion strategy. It should be noted that the second conversion list may include a conversion strategy for converting a first database type of an existing source database to a target data type, and the third conversion list may include a conversion strategy for converting from a target data type to a second database type of an existing target database. The embodiment of the present application uses this strategy selection mechanism to directly perform metadata migration when the first database type is the same as the second database type; directly apply the target conversion strategy when the first database type is different from the second database type but a target conversion strategy for directly converting from the first database type to the second database type is stored in the first conversion list; and determine a strategy for indirect conversion through the target data type when there is no corresponding target conversion strategy in the first conversion list. This allows metadata migration between various database types, regardless of whether they are directly compatible, and provides flexibility and scalability for metadata migration.

[0044] For example, Figure 3As shown, in response to the metadata migration instruction, further, the first database type T1 of the source database and the second database type T2 of the target database are determined. It is determined whether the first database type T1 is consistent with the second database type T2. If they are consistent, the first conversion strategy of the front mapper and the second conversion strategy of the rear mapper are directly determined as NOP (No Operation; no operation is performed). If they are inconsistent, it is determined from the first conversion list whether there is a target conversion strategy for directly converting T1 to T2. If it exists, the first conversion strategy of the front mapper is determined as the above target conversion strategy, and the second conversion strategy of the rear mapper is also determined as NOP. If it does not exist, the conversion strategy from T1 to the target data type is searched from the second conversion list as the first conversion strategy of the front mapper, and the conversion strategy from the target data type to T2 is searched from the third conversion list as the second conversion strategy of the rear mapper. After determining the first conversion strategy of the front mapper and the second conversion strategy of the rear mapper, the metadata information to be migrated can be processed by the front mapper to obtain the first conversion result, and then the first conversion result is processed by the rear mapper to finally obtain the second conversion result.

[0045] Step 205, according to the target operation control parameters, obtain the metadata information to be migrated from the source database, and execute the first conversion strategy on the metadata information to be migrated through the front mapper to obtain a first conversion result, and execute the second conversion strategy on the first conversion result through the post mapper to obtain a second conversion result.

[0046] In an embodiment of the present application, optionally, the "obtaining metadata information to be migrated from the source database according to target operation control parameters" includes: based on the first database type, calling a preset query interface, and returning a first target bridge and a target query statement corresponding to the first database type through the preset query interface; establishing a database connection with the source database through the first target bridge, and obtaining the metadata information to be migrated indicated by the target operation control parameters from the metadata table of the source database through the target query statement.

[0047] In this embodiment, the preset query interface is a general interface that is independent of the database type of a specific source database. The interface can pre-declare a set of methods that are used to query metadata from various types of source databases. Therefore, the preset query interface can be an upper-level code that is independent of the database type of the source database. For each source database that needs to be supported (such as MySQL, Oracle, SQL Server, etc.), a specific class that implements this interface is created in advance. The class can use the corresponding JDBC (Java Database Connectivity, bridge) driver to connect to the source database and execute a specific SQL (Structured Query Language, Structured Query Language) query to obtain metadata. Specifically, the first target bridge and target query statement corresponding to the first database type can be queried through the preset query interface. Then, the first target bridge can be used to establish a database connection with the source database, and the target query statement can be used to query the metadata information to be migrated indicated by the target operation control parameter from the metadata table of the source database. In this way, the embodiment of the present application makes it possible for the code that calls the preset query interface to not care about which source database is being interacted with, but only needs to call the method defined in the preset query interface. This not only improves the reusability and maintainability of the code of each method defined in the preset query interface, but also makes it easier to add a new source database.

[0048] Step 206: Store the second conversion result in the target database.

[0049] In an embodiment of the present application, optionally, step 206 includes: based on the second database type, calling a preset generator interface, and returning a second target bridge and a target generation statement corresponding to the second database type through the preset generator interface; establishing a database connection with the target database through the second target bridge, and generating a target DDL (data definition language) corresponding to the target database according to the second conversion result through the target generation statement, and storing the target DDL in the target database.

[0050] In this embodiment, the preset generator interface is a general interface that does not depend on the database type of the specific target database. The second target bridge and the target generation statement corresponding to the second database type can be queried through the preset generator interface. The second target bridge is similar to the first target bridge, but it is a bridge connected to the target database, and it is responsible for processing tasks such as connection with the target database. The target generation statement can be a DDL statement template or rule based on the second database type. Afterwards, the second target bridge is used to establish a physical connection with the target database, and a target DDL statement for the target database is generated based on the target generation statement and the second conversion result. These target DDL statements are used to create objects such as tables, indexes, views, etc. in the target database, and ensure that they are consistent with the objects in the source database in structure and data. Finally, the generated target DDL statement can be executed in the target database, thereby completing the migration of metadata. In this way, the embodiment of the present application makes it possible for the code that calls the preset generator interface to not care about which target database is being interacted with, and only needs to call the preset generator interface to determine the second target bridge and the target generation statement, making it easier to add a new target database, and the whole process can be more flexible and efficient.

[0051] In an embodiment of the present application, optionally, the step 205 of "executing the first conversion strategy on the metadata information to be migrated by the front mapper to obtain a first conversion result, and executing the second conversion strategy on the first conversion result by the back mapper to obtain a second conversion result" includes: for any source field in the metadata information to be migrated, determining the mapping rule corresponding to the any source field from the first conversion strategy, and parsing the first sub-rule and the second sub-rule from the mapping rule corresponding to the any source field, and converting the any source field based on the first sub-rule and the second sub-rule through the front mapper to obtain the first conversion result corresponding to the any source field, wherein the first sub-rule is used to indicate the conversion rule of the first information of the any source field, and the second sub-rule is used to indicate the conversion rule of the second information of the any source field, and the first information includes the any source field. a field name and / or field type of a source field, the second information includes the field length and / or field precision of any of the source fields; for any first conversion result, determine the mapping rule corresponding to any of the first conversion results from the second conversion strategy, and parse the third sub-rule and the fourth sub-rule from the mapping rule corresponding to any of the first conversion results, and through the post-mapper, convert any of the first conversion results based on the third sub-rule and the fourth sub-rule to obtain the second conversion result corresponding to any of the first conversion results, wherein the third sub-rule is used to indicate the conversion rule of the third information of any of the first conversion results, and the fourth sub-rule is used to indicate the conversion rule of the fourth information of any of the first conversion results, the third information includes the field name and / or field type of any of the first conversion results, and the fourth information includes the field length and / or field precision of any of the first conversion results.

[0052] In this embodiment, each source field in the metadata information to be migrated can be first identified and analyzed. For any source field, the mapping rule corresponding to the source field can be determined from the first conversion strategy. The mapping rule usually includes multiple sub-rules, which are used to guide the conversion of different aspects of field information. Specifically, the first sub-rule and the second sub-rule can be parsed from the mapping rule corresponding to the first conversion strategy, wherein the first sub-rule guides the conversion of the first information of the source field (such as field name, field type); the second sub-rule guides the conversion of the second information of the source field (such as field length, field precision). Apply the front mapper, and convert the source field based on the parsed first sub-rule and second sub-rule to obtain a first conversion result, that is, the first conversion result is the new representation of the source field after conversion according to the first conversion strategy. For each first conversion result, determine the corresponding mapping rule from the second conversion strategy. Specifically, the third sub-rule and the fourth sub-rule can be parsed from the corresponding mapping rules in the second conversion strategy, wherein the third sub-rule guides the conversion of the third information (such as field name, field type) of the first conversion result; the fourth sub-rule guides the conversion of the fourth information (such as field length, field precision) of the first conversion result. Applying the post-mapper, based on the parsed third sub-rule and fourth sub-rule, the first conversion result is further converted to obtain the second conversion result, that is, the second conversion result is the final representation of the first conversion result after conversion according to the second conversion strategy. The embodiment of the present application fully considers various situations such as field name changes, type changes, length precision changes, etc. during the metadata migration process, so that different change situations can be considered during the metadata migration process. At the same time, by accurately applying different conversion strategies and mapping rules, it is ensured that the source field can be accurately converted to the second conversion result during the migration process, and the integrity and accuracy of the field information are maintained.

[0053] In an embodiment of the present application, optionally, before step 201, the method further includes: obtaining default operation control parameters for the metadata migration task; determining whether there is a parameter configuration file for the metadata migration task, and when there is a parameter configuration file, updating the parameters with the same name in the default operation control parameters based on the parameter configuration file to obtain first operation control parameters; determining whether there is a command line parameter for the metadata migration task, and when there is a command line parameter, updating the parameters with the same name in the first operation control parameters based on the command line parameter to obtain second operation control parameters; determining whether there is a parameter setting script for the metadata migration task, and when there is a parameter setting script, obtaining third operation control parameters according to target setting parameters in the parameter setting script and the second operation control parameters, wherein the target setting parameters include sharding key setting parameters and / or partition setting parameters; and using the third operation control parameters as the target operation control parameters.

[0054] In this embodiment, during the design, all control factors that may change can also be sorted out and set as runtime control parameters. At the same time, in order to simplify user operations, the values ​​used in most cases of these parameters are set to their default values. Before starting the metadata migration task, these default operation control parameters can be first obtained. Next, it can be checked whether there is a parameter configuration file for the current metadata migration task. The parameter configuration file allows the user to customize the operation parameters for a specific task to meet specific performance or requirements. If there is a parameter configuration file, the parameter configuration file can be read and the parameter items with the same name as the default operation control parameters can be identified. For these parameter items with the same name, the values ​​in the parameter configuration file can be used to update the corresponding values ​​in the default operation control parameters, thereby obtaining the first operation control parameters. For example, the default operation control parameter indicates that the migrated metadata information includes a metadata index, but the current metadata migration task does not require a metadata index, then the user can set the parameter of the metadata index to "no need to crawl" through the parameter configuration file. In addition to the parameter configuration file, it can also be checked whether there are command line parameters. The command line parameters allow the user to temporarily modify the parameter value in the first operation control parameter when the task is started. After checking, if it is found that there are command line parameters, then these command line parameters can be parsed and it can be checked again whether there are items with the same name as the first operation control parameters. If yes, the value in the command line parameter can be used to update the corresponding value in the first operation control parameter, thereby obtaining the second operation control parameter. In some cases, the metadata migration task may require more complex parameter configuration, such as the setting of the shard key, the configuration of the partition, etc. These configurations are usually not achievable through simple parameter files or command line parameters. Therefore, after obtaining the second operation control parameter, it is also possible to check whether there is a parameter setting script for the current metadata migration task. The parameter setting script is usually an executable file or a script file, which contains logic for generating or modifying the operation control parameters. If there is a parameter setting script, the script can be executed, and the target setting parameters can be generated according to the logic in the script. These target setting parameters may include shard key setting parameters, partition setting parameters, etc. Then, the target setting parameters are merged or integrated with the second operation control parameters to generate the final third operation control parameters. After the above steps, the final third operation control parameters can be obtained. The third operation control parameters comprehensively consider the default configuration, configuration file, command line parameters, and various settings in the parameter setting script, and can fully reflect the user's needs for the operation control of the metadata migration task. The embodiment of the present application can avoid users from repeatedly setting parameters by setting default operation control parameters that meet the needs of most metadata migration tasks. At the same time, it combines various methods such as configuration files, command line parameters, and parameter setting scripts to provide flexible and powerful operation control parameter configuration capabilities for metadata migration tasks, which can meet the diverse needs of different users.

[0055] In an embodiment of the present application, optionally, the method further includes: when there is a metadata migration task and the metadata migration task is in an execution state, in response to a source database new addition instruction, obtaining a database identifier of a newly added source database and a conversion strategy to be added for converting from the database type of the newly added source database to the target data type, adding the database identifier of the newly added source database and the conversion strategy to be added to a first temporary list, wherein the query priority of the second conversion list is higher than that of the first temporary list; when there is no metadata migration task, updating the second conversion list based on the first temporary list; when there is a metadata migration task and the metadata migration task is in an execution state, in response to a target database new addition instruction, obtaining a database identifier of a newly added target database and a conversion strategy to be added for converting from the target data type to the database type of the newly added target database, adding the database identifier of the newly added target database and the conversion strategy to be added to a second temporary list, wherein the query priority of the third conversion list is higher than that of the second temporary list; when there is no metadata migration task, updating the third conversion list based on the second temporary list.

[0056] In this embodiment, when the metadata migration task is being executed, if a new instruction of the source database is received, it means that there is a new source database that may need to execute a new metadata migration task. Then, the database identifier of the newly added source database (such as database name, IP address, port, etc.) and the conversion strategy to be added from the database type of the source database to the target data type can be obtained. Then, the database identifier of the newly added source database and the corresponding conversion strategy to be added are added to a new first temporary list. It should be noted that the second conversion list is a long-term maintained list containing all known source database conversion strategies. When querying the conversion strategy, the second conversion list has a higher priority than the first temporary list. In other words, if a source database is to execute a metadata migration task, the first conversion strategy can be queried from the second conversion list first. When there is no corresponding first conversion strategy in the second conversion list, the first conversion strategy can be queried from the first temporary list. Subsequently, when the metadata migration task is completed, the first temporary list can be used to update the content of the second conversion list, so that the content of the first temporary list can be automatically refreshed to the second conversion list, and the first temporary list can be deleted. In this way, when the metadata migration task is executed next time, the first conversion strategy of these newly added source databases can be directly obtained from the second conversion list.

[0057] Similarly, when the metadata migration task is being executed, if a new instruction of the target database is received, it means that there is a new target database that may need to execute a new metadata migration task. Then, the database identifier of the newly added target database and the conversion strategy to be added from the target data type to the database type of the target database can be obtained. Then, the database identifier of the newly added target database and the corresponding conversion strategy to be added are added to a new second temporary list. It should be noted that the third conversion list is a long-term maintained list containing all known target database conversion strategies. When querying the conversion strategy, the third conversion list has a higher priority than the second temporary list. In other words, if a metadata migration task is executed to a certain target database, the second conversion strategy can be queried from the third conversion list first, and when there is no corresponding second conversion strategy in the third conversion list, the second conversion strategy can be queried from the second temporary list. Later, when the metadata migration task is completed, the content of the third conversion list can be updated using the second temporary list, so that the content in the second temporary list can be automatically refreshed to the third conversion list, and the second temporary list can be deleted, so that when the metadata migration task is executed next time, the second conversion strategies of these newly added target databases can be directly obtained from the third conversion list.

[0058] The embodiment of the present application provides a flexible and efficient method for processing new additions to the source database and the target database for metadata migration tasks. By combining the temporary list with the long-term maintained conversion list, the conversion strategy range of the original conversion list can be dynamically updated and expanded without affecting the execution of the current migration task, thereby improving the flexibility and scalability of metadata migration.

[0059] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a metadata migration device between databases, such as Figure 4 As shown, the device comprises:

[0060] A database type determination module, configured to determine a first database type of a source database and a second database type of a target database in response to a metadata migration instruction;

[0061] A conversion strategy determination module, used to determine a first conversion strategy corresponding to a front mapper and a second conversion strategy corresponding to a rear mapper based on the first database type and the second database type;

[0062] a conversion module, configured to obtain metadata information to be migrated from the source database according to target operation control parameters, and execute the first conversion strategy on the metadata information to be migrated through the front mapper to obtain a first conversion result, and execute the second conversion strategy on the first conversion result through the rear mapper to obtain a second conversion result;

[0063] A storage module is used to store the second conversion result in the target database.

[0064] Optionally, the conversion strategy determination module is used to:

[0065] Determining whether the first database type is consistent with the second database type;

[0066] When the judgment result is consistent, determine that the first conversion strategy corresponding to the front mapper and the second conversion strategy corresponding to the rear mapper are both not to perform any operation; when the judgment result is inconsistent, query from the first conversion list whether there is a target conversion strategy between the first database type and the second database type;

[0067] When the target conversion strategy exists in the first conversion list, the target conversion strategy is determined as the first conversion strategy corresponding to the front mapper, and the second conversion strategy corresponding to the post mapper is determined as not performing any operation; when the target conversion strategy does not exist in the first conversion list, the conversion strategy corresponding to the conversion from the first database type to the target data type is queried from the second conversion list as the first conversion strategy, and the conversion strategy corresponding to the conversion from the target data type to the second database type is queried from the third conversion list as the second conversion strategy.

[0068] Optionally, the conversion module is used to:

[0069] For any source field in the metadata information to be migrated, determine the mapping rule corresponding to the any source field from the first conversion strategy, and parse the first sub-rule and the second sub-rule from the mapping rule corresponding to the any source field, and convert the any source field based on the first sub-rule and the second sub-rule through the front mapper to obtain the first conversion result corresponding to the any source field, wherein the first sub-rule is used to indicate the conversion rule of the first information of the any source field, and the second sub-rule is used to indicate the conversion rule of the second information of the any source field, the first information includes the field name and / or field type of the any source field, and the second information includes the field length and / or field precision of the any source field;

[0070] For any first conversion result, a mapping rule corresponding to any first conversion result is determined from the second conversion strategy, and a third sub-rule and a fourth sub-rule are parsed from the mapping rule corresponding to any first conversion result, and the post-mapper converts any first conversion result based on the third sub-rule and the fourth sub-rule to obtain a second conversion result corresponding to any first conversion result, wherein the third sub-rule is used to indicate a conversion rule for third information of any first conversion result, and the fourth sub-rule is used to indicate a conversion rule for fourth information of any first conversion result, the third information includes a field name and / or field type of any first conversion result, and the fourth information includes a field length and / or field precision of any first conversion result.

[0071] Optionally, the device further comprises:

[0072] A parameter acquisition module, configured to acquire default operation control parameters of the metadata migration task before determining the first database type of the source database in response to the metadata migration instruction;

[0073] A parameter update module is used to determine whether there is a parameter configuration file for the metadata migration task, and when there is a parameter configuration file, update the same-named parameters in the default operation control parameters based on the parameter configuration file to obtain a first operation control parameter; determine whether there is a command line parameter for the metadata migration task, and when there is a command line parameter, update the same-named parameters in the first operation control parameters based on the command line parameter to obtain a second operation control parameter; determine whether there is a parameter setting script for the metadata migration task, and when there is a parameter setting script, obtain a third operation control parameter according to the target setting parameters in the parameter setting script and the second operation control parameter, wherein the target setting parameters include a shard key setting parameter and / or a partition setting parameter; and use the third operation control parameter as the target operation control parameter.

[0074] Optionally, the device further comprises:

[0075] A list updating module is used for, when there is a metadata migration task and the metadata migration task is in an execution state, in response to a source database new addition instruction, obtaining a database identifier of a newly added source database and a conversion strategy to be added for converting from the database type of the newly added source database to the target data type, adding the database identifier of the newly added source database and the conversion strategy to be added to a first temporary list, wherein the query priority of the second conversion list is higher than that of the first temporary list; when there is no metadata migration task, updating the second conversion list based on the first temporary list; when there is a metadata migration task and the metadata migration task is in an execution state, in response to a target database new addition instruction, obtaining a database identifier of a newly added target database and a conversion strategy to be added for converting from the target data type to the database type of the newly added target database, adding the database identifier of the newly added target database and the conversion strategy to be added to a second temporary list, wherein the query priority of the third conversion list is higher than that of the second temporary list; when there is no metadata migration task, updating the third conversion list based on the second temporary list.

[0076] Optionally, the conversion module is further used to:

[0077] Based on the first database type, calling a preset query interface, and returning a first target bridge and a target query statement corresponding to the first database type through the preset query interface;

[0078] A database connection is established with the source database through the first target bridge, and metadata information to be migrated indicated by the target operation control parameter is obtained from a metadata table of the source database through the target query statement.

[0079] Optionally, the storage module is used to:

[0080] Based on the second database type, calling a preset generator interface, and returning a second target bridge and a target generation statement corresponding to the second database type through the preset generator interface;

[0081] A database connection is established with the target database through the second target bridge, and a target DDL corresponding to the target database is generated according to the second conversion result through the target generation statement, and the target DDL is stored in the target database.

[0082] It should be noted that for other corresponding descriptions of the functional units involved in the metadata migration device between databases provided in the embodiment of the present application, reference can be made to Figures 1 to 3 The corresponding description in the method will not be repeated here.

[0083] The present application also provides a computer device, which may be a personal computer, a server, a network device, etc. Figure 5 As shown, the computer device includes a bus, a processor, a memory and a communication interface, and may also include an input and output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in each method embodiment are implemented.

[0084] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0085] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0086] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0087] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0088] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0089] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for metadata migration between databases, characterized in that: include: In response to the metadata migration instruction, determining a first database type of the source database and a second database type of the target database; Based on the first database type and the second database type, determining a first conversion strategy corresponding to a front mapper and a second conversion strategy corresponding to a rear mapper; According to the target operation control parameter, the metadata information to be migrated is obtained from the source database, and the first conversion strategy is executed on the metadata information to be migrated by the front mapper to obtain a first conversion result, and the second conversion strategy is executed on the first conversion result by the rear mapper to obtain a second conversion result; The second conversion result is stored in the target database.

2. The method according to claim 1, characterized in that The determining, based on the first database type and the second database type, a first conversion strategy corresponding to a front mapper and a second conversion strategy corresponding to a rear mapper includes: Determining whether the first database type is consistent with the second database type; When the judgment result is consistent, determining that the first conversion strategy corresponding to the front mapper and the second conversion strategy corresponding to the rear mapper are both not to perform any operation; When the judgment result is inconsistent, query from the first conversion list whether there is a target conversion strategy between the first database type and the second database type; when the target conversion strategy exists in the first conversion list, determine the target conversion strategy as the first conversion strategy corresponding to the front mapper, and determine the second conversion strategy corresponding to the post mapper as not performing any operation; when the target conversion strategy does not exist in the first conversion list, query from the second conversion list the conversion strategy corresponding to the conversion from the first database type to the target data type as the first conversion strategy, and query from the third conversion list the conversion strategy corresponding to the conversion from the target data type to the second database type as the second conversion strategy.

3. The method according to claim 2, characterized in that The step of executing the first conversion strategy on the metadata information to be migrated by the front mapper to obtain a first conversion result, and executing the second conversion strategy on the first conversion result by the rear mapper to obtain a second conversion result includes: For any source field in the metadata information to be migrated, determine the mapping rule corresponding to the any source field from the first conversion strategy, and parse the first sub-rule and the second sub-rule from the mapping rule corresponding to the any source field, and convert the any source field based on the first sub-rule and the second sub-rule through the front mapper to obtain the first conversion result corresponding to the any source field, wherein the first sub-rule is used to indicate the conversion rule of the first information of the any source field, and the second sub-rule is used to indicate the conversion rule of the second information of the any source field, the first information includes the field name and / or field type of the any source field, and the second information includes the field length and / or field precision of the any source field; For any first conversion result, a mapping rule corresponding to any first conversion result is determined from the second conversion strategy, and a third sub-rule and a fourth sub-rule are parsed from the mapping rule corresponding to any first conversion result, and the post-mapper converts any first conversion result based on the third sub-rule and the fourth sub-rule to obtain a second conversion result corresponding to any first conversion result, wherein the third sub-rule is used to indicate a conversion rule for third information of any first conversion result, and the fourth sub-rule is used to indicate a conversion rule for fourth information of any first conversion result, the third information includes a field name and / or field type of any first conversion result, and the fourth information includes a field length and / or field precision of any first conversion result.

4. The method according to any one of claims 1 to 3, characterized in that Before determining the first database type of the source database in response to the metadata migration instruction, the method further includes: Get the default run control parameters of the metadata migration task; Determine whether there is a parameter configuration file for the metadata migration task, and if there is a parameter configuration file, update the same-name parameter in the default operation control parameter based on the parameter configuration file to obtain a first operation control parameter; Determine whether the metadata migration task has a command line parameter, and if a command line parameter exists, update the parameter with the same name in the first operation control parameter based on the command line parameter to obtain a second operation control parameter; Determine whether there is a parameter setting script for the metadata migration task, and when there is a parameter setting script, obtain a third operation control parameter according to the target setting parameter in the parameter setting script and the second operation control parameter, wherein the target setting parameter includes a shard key setting parameter and / or a partition setting parameter; The third operation control parameter is used as the target operation control parameter.

5. The method according to claim 4, characterized in that The method further comprises: When there is a metadata migration task and the metadata migration task is in execution, in response to a source database addition instruction, obtain a database identifier of the newly added source database and a conversion strategy to be added for converting from the database type of the newly added source database to the target data type, and add the database identifier of the newly added source database and the conversion strategy to be added to a first temporary list, wherein the query priority of the second conversion list is higher than that of the first temporary list; when there is no metadata migration task, update the second conversion list based on the first temporary list; When there is a metadata migration task and the metadata migration task is in execution, in response to a target database addition instruction, the database identifier of the newly added target database and the to-be-added conversion strategy for converting from the target data type to the database type of the newly added target database are obtained, and the database identifier of the newly added target database and the to-be-added conversion strategy are added to a second temporary list, wherein the query priority of the third conversion list is higher than that of the second temporary list; when there is no metadata migration task, the third conversion list is updated based on the second temporary list.

6. The method according to claim 1, characterized in that The acquiring metadata information to be migrated from the source database according to the target operation control parameter includes: Based on the first database type, calling a preset query interface, and returning a first target bridge and a target query statement corresponding to the first database type through the preset query interface; A database connection is established with the source database through the first target bridge, and metadata information to be migrated indicated by the target operation control parameter is obtained from a metadata table of the source database through the target query statement.

7. The method according to claim 6, characterized in that The storing the second conversion result into the target database includes: Based on the second database type, calling a preset generator interface, and returning a second target bridge and a target generation statement corresponding to the second database type through the preset generator interface; A database connection is established with the target database through the second target bridge, and a target DDL corresponding to the target database is generated according to the second conversion result through the target generation statement, and the target DDL is stored in the target database.

8. A metadata migration device between databases, characterized in that: include: A database type determination module, configured to determine a first database type of a source database and a second database type of a target database in response to a metadata migration instruction; A conversion strategy determination module, used to determine a first conversion strategy corresponding to a front mapper and a second conversion strategy corresponding to a rear mapper based on the first database type and the second database type; a conversion module, configured to obtain metadata information to be migrated from the source database according to target operation control parameters, and execute the first conversion strategy on the metadata information to be migrated through the front mapper to obtain a first conversion result, and execute the second conversion strategy on the first conversion result through the rear mapper to obtain a second conversion result; A storage module is used to store the second conversion result in the target database.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images