Method and apparatus for metadata migration between databases, storage medium and device

Abstract

The application discloses a metadata migration method and device between databases, a storage medium and a computer device. The method comprises the following steps: in response to a metadata migration instruction, determining a first database type of a source database and a second database type of a target database; determining a first conversion strategy corresponding to a front-end mapper and a second conversion strategy corresponding to a rear-end mapper based on the first database type and the second database type; obtaining to-be-migrated metadata information from the source database according to a target operation control parameter, executing the first conversion strategy on the to-be-migrated metadata information through the front-end mapper to obtain a first conversion result, executing the second conversion strategy on the first conversion result through the rear-end mapper to obtain a second conversion result; and storing the second conversion result into the target database.

Description

Technical Field

[0001] This application relates to the field of database migration technology, and in particular to a method, apparatus, storage medium, and computer equipment for migrating metadata between databases. Background Technology

[0002] Migrating a database to a domestically developed or open-source database is a crucial step in achieving database autonomy and control, and database metadata migration is the most important first step in this process. This is because metadata contains vital information such as the database structure, table structure, indexes, views, stored procedures, triggers, and foreign key relationships—information that forms the foundation for database operation and management. Therefore, efficiently and flexibly completing metadata migration is fundamental to all subsequent work.

[0003] Currently, although there are many commercial or open-source database metadata migration tools, these tools generally only support migration between specific types of databases and lack scalability and flexibility. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, storage medium, and computer equipment for metadata migration between databases. By using the target data type as an intermediate bridge, the metadata migration is performed in two steps, decoupling the source database and the target database. This effectively solves the problem of metadata migration between different database types, improves the scalability of migration operations, and enhances the efficiency and accuracy of metadata migration. Furthermore, the entire migration process achieves automated conversion and storage of the metadata information to be migrated from the source database to the target database.

[0005] According to one aspect of this application, a method for metadata migration between databases is provided, comprising:

[0006] In response to metadata migration instructions, determine the first database type of the source database and the second database type of the target database;

[0007] Based on the first database type and the second database type, determine the first conversion strategy corresponding to the front mapper and the second conversion strategy corresponding to the back mapper;

[0008] Based on the target operation control parameters, the metadata information to be migrated is obtained from the source database, and the first conversion strategy is executed on the metadata information to be migrated through the front mapper to obtain a first conversion result. The second conversion strategy is executed on the first conversion result through the back mapper to obtain a second conversion result.

[0009] The second conversion result is stored in the target database.

[0010] According to another aspect of this application, a metadata migration apparatus between databases is provided, comprising:

[0011] The database type determination module is used to determine the first database type of the source database and the second database type of the target database in response to the metadata migration instruction.

[0012] The conversion strategy determination module is used to determine a first conversion strategy corresponding to the front mapper and a second conversion strategy corresponding to the back mapper based on the first database type and the second database type.

[0013] The conversion module is used to obtain the metadata information to be migrated from the source database according to the target operation control parameters, and to execute the first conversion strategy on the metadata information to be migrated through the front mapper to obtain a first conversion result, and to execute the second conversion strategy on the first conversion result through the back mapper to obtain a second conversion result;

[0014] A storage module is used to store the second conversion result in the target database.

[0015] According to another aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described method for migrating metadata between databases.

[0016] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described method for migrating metadata between databases.

[0017] Using the above technical solution, this application provides a method, apparatus, storage medium, and computer device for metadata migration between databases. Upon receiving a metadata migration instruction, the method first identifies a first database type of the source database and a second database type of the target database. Based on the first database type of the source database and the second database type of the target database, a corresponding conversion strategy can be selected. Here, the conversion strategy includes a first conversion strategy corresponding to the front-end mapper and a second conversion strategy corresponding to the back-end mapper. The first conversion strategy is typically used to convert the metadata of the source database from the first database type to the target data type, and the second conversion strategy is typically used to convert the metadata from the target data type to the second database type corresponding to the target database. According to the target operation control parameters, the metadata information to be migrated is extracted from the source database. Then, using the front-end mapper, the extracted metadata information to be migrated is converted for the first time according to the first conversion strategy, converting the metadata information to be migrated from the first database type to the target data type, obtaining a first conversion result. Subsequently, through the back-end mapper, the second conversion strategy is executed on the first conversion result, converting the first conversion result from the target data type to the second database type, obtaining a final second conversion result. The metadata information to be migrated after two conversions, i.e., the second conversion result, is stored in the target database. This application's embodiments utilize the target data type as an intermediate bridge, performing mapping in two steps during metadata migration. This decouples the source and target databases, effectively solving the problem of metadata migration between different database types, improving the scalability of migration operations, and enhancing the efficiency and accuracy of metadata migration. Furthermore, the entire migration process achieves automated conversion and storage of the metadata information to be migrated from the source database to the target database.

[0018] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 A flowchart illustrating a method for metadata migration between databases provided in an embodiment of this application is shown.

[0021] Figure 2 A flowchart illustrating another method for metadata migration between databases provided in an embodiment of this application is shown.

[0022] Figure 3 This illustration shows a flowchart of a process for determining a first conversion strategy and a second conversion strategy according to an embodiment of this application;

[0023] Figure 4 This illustration shows a schematic diagram of the structure of a metadata migration device between databases provided in an embodiment of this application;

[0024] Figure 5 A schematic diagram of the device structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation

[0025] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0026] This embodiment provides a method for migrating metadata between databases, such as Figure 1 As shown, the method includes:

[0027] Step 101: In response to the metadata migration instruction, determine the first database type of the source database and the second database type of the target database.

[0028] This application provides a method for metadata migration between databases, applicable to metadata migration between databases of the same type as well as between heterogeneous databases. This method can be implemented in the computing environment of the end performing the migration operation, such as a server or a developer's local machine. First, when a user wants to perform metadata migration between databases, they can trigger a metadata migration command. Upon receiving the metadata migration command, the server or local machine first identifies the first database type of the source database, i.e., determines the database type of the source database. Here, the source database can be a relational database, such as MySQL or Oracle, or a non-relational database, such as MongoDB or Cassandra. Simultaneously, it determines the second database type of the target database, i.e., the database type of the database receiving the migrated metadata. The target database can also be a relational database or a non-relational database.

[0029] Step 102: Based on the first database type and the second database type, determine the first conversion strategy corresponding to the front mapper and the second conversion strategy corresponding to the back mapper.

[0030] In this embodiment, based on the first database type of the source database and the second database type of the target database, a corresponding conversion strategy can be selected. Here, the conversion strategy includes a first conversion strategy corresponding to the front-end mapper and a second conversion strategy corresponding to the back-end mapper. The first conversion strategy is typically used to convert the metadata of the source database from the first database type to the target data type, and the second conversion strategy is typically used to convert the target data type to the second database type corresponding to the target database. The target data type can be determined according to requirements, serving as an intermediate bridge between the first database type corresponding to the source database and the second database type corresponding to the target database. The biggest advantage of this is decoupling. If a new source database is added, only a conversion strategy for the front-end mapper from the source database to the target data type needs to be added; similarly, if only a new target database is added, only a conversion strategy for the back-end mapper from the target data type to the target database needs to be added. This approach can handle metadata migration between multiple different source databases and target databases.

[0031] Step 103: Based on the target operation control parameters, obtain the metadata information to be migrated from the source database, and execute the first conversion strategy on the metadata information to be migrated through the front mapper to obtain a first conversion result. Then, execute the second conversion strategy on the first conversion result through the back mapper to obtain a second conversion result.

[0032] In this embodiment, metadata information to be migrated is extracted from the source database according to target runtime control parameters. Here, target runtime control parameters are parameters used to indicate the migration content and requirements corresponding to different metadata migration tasks. Specifically, target runtime control parameters may include control parameters such as the selection range of metadata tables, whether to migrate partitions, and whether to migrate foreign keys. Next, using a front-end mapper, the extracted metadata information to be migrated is transformed for the first time according to a first transformation strategy, converting the metadata information to be migrated from a first database type to a target data type, obtaining a first transformation result. Subsequently, using a back-end mapper, a second transformation strategy is executed on the first transformation result, converting the first transformation result from the target data type to a second database type, obtaining the final second transformation result. These steps ensure that the metadata information to be migrated is successfully converted from the first database type to the second database type.

[0033] Step 104: Store the second conversion result in the target database.

[0034] In this embodiment, the metadata information to be migrated, which has undergone two transformations, i.e. the second transformation result, is stored in the target database.

[0035] By applying the technical solution of this embodiment, upon receiving a metadata migration instruction, the first database type of the source database and the second database type of the target database are first identified. Based on the first database type of the source database and the second database type of the target database, a corresponding conversion strategy can be selected. Here, the conversion strategy includes a first conversion strategy corresponding to the front-end mapper and a second conversion strategy corresponding to the back-end mapper. The first conversion strategy is typically used to convert the metadata of the source database from the first database type to the target data type, and the second conversion strategy is typically used to convert the metadata from the target data type to the second database type corresponding to the target database. According to the target runtime control parameters, the metadata information to be migrated is extracted from the source database. Next, using the front-end mapper, the extracted metadata information to be migrated is converted for the first time according to the first conversion strategy, changing the metadata information from the first database type to the target data type, obtaining the first conversion result. Subsequently, through the back-end mapper, the second conversion strategy is executed on the first conversion result, changing the first conversion result from the target data type to the second database type, obtaining the final second conversion result. The metadata information to be migrated after two conversions, i.e., the second conversion result, is stored in the target database. This application's embodiments utilize the target data type as an intermediate bridge, performing mapping in two steps during metadata migration. This decouples the source and target databases, effectively solving the problem of metadata migration between different database types, improving the scalability of migration operations, and enhancing the efficiency and accuracy of metadata migration. Furthermore, the entire migration process achieves automated conversion and storage of the metadata information to be migrated from the source database to the target database.

[0036] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, another method for metadata migration between databases is provided, such as... Figure 2 As shown, the method includes:

[0037] Step 201: In response to the metadata migration instruction, determine the first database type of the source database and the second database type of the target database.

[0038] Step 202: Determine whether the first database type and the second database type are consistent.

[0039] Step 203: When the judgment result is consistent, it is determined that the first conversion strategy corresponding to the pre-mapper and the second conversion strategy corresponding to the post-mapper are both to perform no operation.

[0040] Step 204: When the judgment result is inconsistent, query the first conversion list to see if there is a target conversion strategy between the first database type and the second database type; when the target conversion strategy exists in the first conversion list, determine the target conversion strategy as the first conversion strategy corresponding to the front mapper, and determine the second conversion strategy corresponding to the back mapper as not performing any operation; when the target conversion strategy does not exist in the first conversion list, query the second conversion list to see the conversion strategy corresponding to the conversion from the first database type to the target data type as the first conversion strategy, and query the third conversion list to see the conversion strategy corresponding to the conversion from the target data type to the second database type as the second conversion strategy.

[0041] In this embodiment, after obtaining the first database type and the second database type, it can be determined whether the first database type of the source database is consistent with the second database type of the target database.

[0042] If the assessment results indicate that the two database types are consistent, then no data conversion is required because the metadata is structurally compatible. In this case, both the first conversion strategy corresponding to the front mapper and the second conversion strategy corresponding to the back mapper can be set to "do nothing." That is, the metadata information to be migrated will be directly migrated from the source database to the target database without any intermediate conversion steps.

[0043] If the assessment results indicate inconsistency between the two database types, data conversion is required to ensure metadata compatibility with the target database. Specifically, firstly, a predefined first conversion list can be consulted to determine if a direct conversion strategy exists between the two database types. This first conversion list may contain known effective conversion strategies between database types. If a target conversion strategy exists in the first conversion list, it is designated as the first conversion strategy for the front-end mapper. Since direct conversion is sufficient, the second conversion strategy for the back-end mapper remains "do nothing." If no target conversion strategy exists in the first conversion list (i.e., direct conversion is infeasible or unknown), an indirect conversion method is needed. In this case, the target data type can be used as a bridge. A conversion strategy for converting from the first database type to the target data type is queried from the second conversion list and designated as the first conversion strategy. Next, a conversion strategy for converting from the target data type to the second database type is queried from the third conversion list and designated as the second conversion strategy. Thus, the metadata information to be migrated is first converted to the target data type using the first conversion strategy, and then converted to the second database type using the second conversion strategy. It is important to note that the second conversion list may include conversion strategies for converting an existing source database of a first database type to a target data type, and the third conversion list may include conversion strategies for converting the target data type to an existing target database of a second database type. This embodiment of the application, through this strategy selection mechanism, directly performs metadata migration when the first and second database types are the same; when the first and second database types are different, but the first conversion list stores a target conversion strategy for directly converting from the first database type to the second database type, the target conversion strategy is directly applied; only when there is no corresponding target conversion strategy in the first conversion list is a strategy for indirect conversion through the target data type determined. This mechanism can handle metadata migration between various database types, regardless of their direct compatibility, providing flexibility and scalability for metadata migration.

[0044] For example, such as Figure 3As shown, in response to the metadata migration instruction, the first database type T1 of the source database and the second database type T2 of the target database are further determined. It is then determined whether the first database type T1 and the second database type T2 are consistent. If they are consistent, the first conversion strategy of the front-end mapper and the second conversion strategy of the back-end mapper are directly set to NOP (No Operation). If they are inconsistent, it is determined from the first conversion list whether there exists a target conversion strategy for directly converting T1 to T2. If it exists, the first conversion strategy of the front-end mapper is set as the target conversion strategy, and the second conversion strategy of the back-end mapper is also set to NOP. If it does not exist, a conversion strategy for converting from T1 to the target data type is searched from the second conversion list as the first conversion strategy of the front-end mapper, and a conversion strategy for converting from the target data type to T2 is searched from the third conversion list as the second conversion strategy of the back-end mapper. After determining the first conversion strategy of the front-end mapper and the second conversion strategy of the back-end mapper, the metadata information to be migrated can be processed by the front-end mapper to obtain the first conversion result. Then, the first conversion result is processed by the back-end mapper to finally obtain the second conversion result.

[0045] Step 205: Based on the target operation control parameters, obtain the metadata information to be migrated from the source database, and execute the first transformation strategy on the metadata information to be migrated through the front mapper to obtain a first transformation result. Then, execute the second transformation strategy on the first transformation result through the back mapper to obtain a second transformation result.

[0046] Optionally, in this embodiment, the step of "obtaining the metadata information to be migrated from the source database according to the target operation control parameters" includes: calling a preset query interface based on the first database type, and returning a first target bridge and a target query statement corresponding to the first database type through the preset query interface; establishing a database connection with the source database through the first target bridge, and obtaining the metadata information to be migrated indicated by the target operation control parameters from the metadata table of the source database through the target query statement.

[0047] In this embodiment, the preset query interface is a generic interface that is independent of the specific source database type. This interface can predeclare a set of methods for querying metadata from various types of source databases. Therefore, the preset query interface can be upper-level code independent of the source database type. For each source database that needs to be supported (such as MySQL, Oracle, SQL Server, etc.), a concrete class implementing this interface is created in advance. This class can use the corresponding JDBC (Java Database Connectivity, bridge) driver to connect to the source database and execute a specific SQL (Structured Query Language) query to obtain metadata. Specifically, the preset query interface can query the first target bridge and target query statement corresponding to the first database type. Then, a database connection can be established with the source database using the first target bridge, and the target query statement can be used to query the metadata information to be migrated, as indicated by the target runtime control parameters, from the metadata table of the source database. In this embodiment of the application, the code that calls the preset query interface does not need to care about which source database it is interacting with, but only needs to call the methods defined in the preset query interface. This not only improves the reusability and maintainability of the code of each method defined in the preset query interface, but also makes it easier to add new source databases.

[0048] Step 206: Store the second conversion result in the target database.

[0049] Optionally, in this embodiment, step 206 includes: based on the second database type, calling a preset generator interface, and returning a second target bridge and a target generation statement corresponding to the second database type through the preset generator interface; establishing a database connection with the target database through the second target bridge, and generating a target DDL (data definition language) corresponding to the target database according to the second conversion result through the target generation statement, and storing the target DDL in the target database.

[0050] In this embodiment, the preset generator interface is a generic interface that does not depend on a specific target database type. Through the preset generator interface, a second target bridge and target generation statements corresponding to the second database type can be queried. The second target bridge is similar to the first target bridge, but it is a bridge that connects to the target database and is responsible for handling tasks such as connecting to the target database. The target generation statements can be DDL statement templates or rules based on the second database type. Then, a physical connection is established with the target database using the second target bridge, and target DDL statements for the target database are generated based on the target generation statements and the second transformation results. These target DDL statements are used to create tables, indexes, views, and other objects in the target database, ensuring that they are consistent with the objects in the source database in terms of structure and data. Finally, the generated target DDL statements can be executed in the target database, thereby completing the metadata migration. This embodiment of the application makes it so that the code calling the preset generator interface does not need to care about which target database it is interacting with; it only needs to call the preset generator interface to determine the second target bridge and target generation statements, making it easier to add new target databases, and the entire process is more flexible and efficient.

[0051] In this embodiment of the application, optionally, step 205, "executing the first transformation strategy on the metadata information to be migrated through the pre-mapping mechanism to obtain a first transformation result, and executing the second transformation strategy on the first transformation result through the post-mapping mechanism to obtain a second transformation result," includes: for any source field in the metadata information to be migrated, determining the mapping rule corresponding to the any source field from the first transformation strategy, parsing the first sub-rule and the second sub-rule from the mapping rule corresponding to the any source field, and transforming the any source field based on the first sub-rule and the second sub-rule through the pre-mapping mechanism to obtain the first transformation result corresponding to the any source field, wherein the first sub-rule is used to indicate the transformation rule of the first information of the any source field, the second sub-rule is used to indicate the transformation rule of the second information of the any source field, and the first information includes the first sub-rule of the first source field. The second information includes the field name and / or field type of a source field, and the field length and / or field precision of any source field. For any first transformation result, the mapping rule corresponding to the first transformation result is determined from the second transformation strategy, and the third sub-rule and the fourth sub-rule are parsed from the mapping rule corresponding to the first transformation result. The first transformation result is then transformed based on the third sub-rule and the fourth sub-rule by the post-machiner to obtain the second transformation result corresponding to the first transformation result. The third sub-rule is used to indicate the transformation rule of the third information of the first transformation result, and the fourth sub-rule is used to indicate the transformation rule of the fourth information of the first transformation result. The third information includes the field name and / or field type of the first transformation result, and the fourth information includes the field length and / or field precision of the first transformation result.

[0052] In this embodiment, each source field in the metadata information to be migrated can be identified and analyzed first. For any given source field, a mapping rule corresponding to that source field can be determined from the first transformation strategy. This mapping rule typically contains multiple sub-rules, which guide the transformation of different aspects of the field information. Specifically, a first sub-rule and a second sub-rule can be parsed from the mapping rule corresponding to the first transformation strategy. The first sub-rule guides the transformation of the first information of the source field (such as field name and field type); the second sub-rule guides the transformation of the second information of the source field (such as field length and field precision). A pre-machining algorithm is applied to transform the source field based on the parsed first and second sub-rules, obtaining a first transformation result, which is the new representation of the source field after transformation according to the first transformation strategy. For each first transformation result, a corresponding mapping rule is determined from the second transformation strategy. Specifically, the third and fourth sub-rules can be parsed from the mapping rules corresponding to the second transformation strategy. The third sub-rule guides the transformation of the third information (such as field name and field type) of the first transformation result; the fourth sub-rule guides the transformation of the fourth information (such as field length and field precision) of the first transformation result. A post-mapping algorithm is applied to further transform the first transformation result based on the parsed third and fourth sub-rules, resulting in the second transformation result. That is, the second transformation result is the final representation of the first transformation result after transformation according to the second transformation strategy. This embodiment of the application fully considers various scenarios such as changes in field name, type, length, and precision during metadata migration, enabling the handling of different changes during the migration process. Simultaneously, by precisely applying different transformation strategies and mapping rules, it ensures that the source fields can be accurately transformed into the second transformation result during the migration process, maintaining the integrity and accuracy of the field information.

[0053] Optionally, before step 201, the method further includes: obtaining default runtime control parameters for the metadata migration task; determining whether the metadata migration task has a parameter configuration file, and if a parameter configuration file exists, updating the parameter with the same name in the default runtime control parameters based on the parameter configuration file to obtain a first runtime control parameter; determining whether the metadata migration task has command-line parameters, and if a command-line parameters exist, updating the parameter with the same name in the first runtime control parameters based on the command-line parameters to obtain a second runtime control parameter; determining whether the metadata migration task has a parameter setting script, and if a parameter setting script exists, obtaining a third runtime control parameter based on the target setting parameters in the parameter setting script and the second runtime control parameter, wherein the target setting parameters include shard key setting parameters and / or partition setting parameters; and using the third runtime control parameter as the target runtime control parameter.

[0054] In this embodiment, during the design phase, all potentially changeable control factors can be compiled and set as runtime control parameters. To simplify user operation, the default values ​​for these parameters are set for most common usage scenarios. Before starting the metadata migration task, these default runtime control parameters can be obtained. Next, it can be checked whether a parameter configuration file exists for the current metadata migration task. The parameter configuration file allows users to customize runtime parameters for specific tasks to meet specific performance or requirements. If a parameter configuration file exists, it can be read, and parameter items with the same name as the default runtime control parameters can be identified. For these parameter items with the same name, the values ​​in the parameter configuration file can be used to update the corresponding values ​​in the default runtime control parameters, thereby obtaining the first runtime control parameters. For example, the default runtime control parameters indicate that the migrated metadata information includes a metadata index, but the current metadata migration task does not require a metadata index. Therefore, the user can set the metadata index parameter to "not required" through the parameter configuration file. In addition to the parameter configuration file, it can also be checked whether command-line arguments exist. Command-line arguments allow users to temporarily modify parameter values ​​in the first runtime control parameters when the task starts. If command-line arguments exist, they can be parsed, and it can be checked again whether there are items with the same name as the first runtime control parameters. If available, the values ​​in the command-line arguments can be used to update the corresponding values ​​in the first run control parameters to obtain the second run control parameters. In some cases, metadata migration tasks may require more complex parameter configurations, such as shard key settings and partition configurations. These configurations are usually not achievable through simple parameter files or command-line arguments. Therefore, after obtaining the second run control parameters, it is also necessary to check if a parameter setting script exists for the current metadata migration task. The parameter setting script is usually an executable file or script file that contains the logic for generating or modifying run control parameters. If a parameter setting script exists, it can be executed to generate target setting parameters based on the logic in the script. These target setting parameters may include shard key settings, partition settings, etc. Next, the target setting parameters are merged or integrated with the second run control parameters to generate the final third run control parameters. After the above steps, the final third run control parameters can be obtained. The third run control parameters comprehensively consider the default configuration, configuration files, command-line arguments, and various settings in the parameter setting script, and can fully reflect the user's needs for the runtime control of the metadata migration task. This application embodiment avoids users having to repeatedly set parameters by setting default running control parameters that meet the needs of most metadata migration tasks. At the same time, by combining configuration files, command line parameters, and parameter setting scripts, it provides flexible and powerful running control parameter configuration capabilities for metadata migration tasks, which can meet the diverse needs of different users.

[0055] Optionally, in this embodiment, the method further includes: when a metadata migration task exists and the metadata migration task is in execution state, in response to a source database addition instruction, obtaining the database identifier of the new source database and a conversion strategy to be added from the database type of the new source database to the target data type, and adding the database identifier of the new source database and the conversion strategy to be added to a first temporary list, wherein the query priority of the second conversion list is higher than that of the first temporary list; when no metadata migration task exists, updating the second conversion list based on the first temporary list; when a metadata migration task exists and the metadata migration task is in execution state, in response to a target database addition instruction, obtaining the database identifier of the new target database and a conversion strategy to be added from the target data type to the database type of the new target database, and adding the database identifier of the new target database and the conversion strategy to be added to a second temporary list, wherein the query priority of the third conversion list is higher than that of the second temporary list; when no metadata migration task exists, updating the third conversion list based on the second temporary list.

[0056] In this embodiment, when a metadata migration task is being executed, if an instruction to add a source database is received, it indicates that a new source database may require a new metadata migration task. At this point, the database identifier (such as database name, IP address, port, etc.) of the new source database and the conversion strategy to be added for converting the database type from the source database to the target data type can be obtained. Then, the database identifier of the new source database and the corresponding conversion strategy to be added are added to a new first temporary list. It is important to note that the second conversion list is a long-term maintained list containing all known source database conversion strategies. When querying conversion strategies, the second conversion list has a higher priority than the first temporary list. That is, if a source database needs to perform a metadata migration task, the first conversion strategy can be queried first from the second conversion list first. If the second conversion list does not contain a corresponding first conversion strategy, then the first temporary list will be used to query for the first conversion strategy. After the metadata migration task is completed, the contents of the second transformation list can be updated using the first temporary list so that the contents of the first temporary list can be automatically refreshed to the second transformation list. The first temporary list can also be deleted so that when the metadata migration task is executed next time, the first transformation strategies of these newly added source databases can be directly obtained from the second transformation list.

[0057] Similarly, when a metadata migration task is in progress, if an instruction to add a target database is received, it indicates that a new target database may require a new metadata migration task. In this case, the database identifier of the new target database and the conversion strategy to be added for converting from the target data type to the target database's database type can be obtained. Then, the database identifier of the new target database and the corresponding conversion strategy to be added are added to a new second temporary list. It's important to note that the third conversion list is a long-term maintained list containing all known target database conversion strategies. When querying conversion strategies, the third conversion list has higher priority than the second temporary list. That is, if a metadata migration task is performed on a target database, the second conversion strategy is first queried from the third conversion list. If no corresponding second conversion strategy is found in the third conversion list, then the second temporary list is used to query for it. After the metadata migration task is completed, the contents of the third conversion list can be updated using the second temporary list so that the contents of the second temporary list are automatically reflected in the third conversion list. The second temporary list can then be deleted, so that the second conversion strategies for these newly added target databases can be directly obtained from the third conversion list in the next metadata migration task.

[0058] This application provides a flexible and efficient method for handling new additions to the source and target databases in metadata migration tasks. By combining temporary lists and long-term maintained transformation lists, the scope of transformation strategies in the original transformation list can be dynamically updated and expanded without affecting the execution of the current migration task, thereby improving the flexibility and scalability of metadata migration.

[0059] Furthermore, as Figure 1 To specifically implement the method, this application provides a metadata migration device between databases, such as... Figure 4 As shown, the device includes:

[0060] The database type determination module is used to determine the first database type of the source database and the second database type of the target database in response to the metadata migration instruction.

[0061] The conversion strategy determination module is used to determine a first conversion strategy corresponding to the front mapper and a second conversion strategy corresponding to the back mapper based on the first database type and the second database type.

[0062] The conversion module is used to obtain the metadata information to be migrated from the source database according to the target operation control parameters, and to execute the first conversion strategy on the metadata information to be migrated through the front mapper to obtain a first conversion result, and to execute the second conversion strategy on the first conversion result through the back mapper to obtain a second conversion result;

[0063] A storage module is used to store the second conversion result in the target database.

[0064] Optionally, the conversion strategy determination module is used to:

[0065] Determine whether the first database type and the second database type are consistent;

[0066] When the judgment result is consistent, it is determined that the first conversion strategy corresponding to the front mapper and the second conversion strategy corresponding to the back mapper are both not to perform any operation; when the judgment result is inconsistent, it is queried from the first conversion list whether there is a target conversion strategy between the first database type and the second database type;

[0067] When the target conversion strategy exists in the first conversion list, the target conversion strategy is determined as the first conversion strategy corresponding to the front mapper, and the second conversion strategy corresponding to the back mapper is determined to perform no operation; when the target conversion strategy does not exist in the first conversion list, the conversion strategy corresponding to the conversion from the first database type to the target data type is queried from the second conversion list as the first conversion strategy, and the conversion strategy corresponding to the conversion from the target data type to the second database type is queried from the third conversion list as the second conversion strategy.

[0068] Optionally, the conversion module is used to:

[0069] For any source field in the metadata information to be migrated, the mapping rule corresponding to the source field is determined from the first transformation strategy, and the first sub-rule and the second sub-rule are parsed from the mapping rule corresponding to the source field. The source field is transformed based on the first sub-rule and the second sub-rule through the pre-mapping, and the first transformation result corresponding to the source field is obtained. The first sub-rule is used to indicate the transformation rule of the first information of the source field, and the second sub-rule is used to indicate the transformation rule of the second information of the source field. The first information includes the field name and / or field type of the source field, and the second information includes the field length and / or field precision of the source field.

[0070] For any first transformation result, the mapping rule corresponding to the first transformation result is determined from the second transformation strategy, and the third sub-rule and the fourth sub-rule are parsed from the mapping rule corresponding to the first transformation result. Through the post-mapper, the first transformation result is transformed based on the third sub-rule and the fourth sub-rule to obtain the second transformation result corresponding to the first transformation result. The third sub-rule is used to indicate the transformation rule of the third information of the first transformation result, and the fourth sub-rule is used to indicate the transformation rule of the fourth information of the first transformation result. The third information includes the field name and / or field type of the first transformation result, and the fourth information includes the field length and / or field precision of the first transformation result.

[0071] Optionally, the device further includes:

[0072] The parameter acquisition module is used to acquire the default running control parameters of the metadata migration task before determining the first database type of the source database in response to the metadata migration instruction.

[0073] The parameter update module is used to determine whether the metadata migration task has a parameter configuration file, and if a parameter configuration file exists, to update the parameter with the same name in the default run control parameters based on the parameter configuration file to obtain the first run control parameter; to determine whether the metadata migration task has command line parameters, and if a command line parameters exist, to update the parameter with the same name in the first run control parameter based on the command line parameters to obtain the second run control parameter; to determine whether the metadata migration task has a parameter setting script, and if a parameter setting script exists, to obtain the third run control parameter according to the target setting parameter in the parameter setting script and the second run control parameter, wherein the target setting parameter includes shard key setting parameter and / or partition setting parameter; and to use the third run control parameter as the target run control parameter.

[0074] Optionally, the device further includes:

[0075] The list update module is used to, when a metadata migration task exists and the metadata migration task is in execution state, respond to a source database addition instruction, obtain the database identifier of the new source database and the conversion strategy to be added from the database type of the new source database to the target data type, and add the database identifier of the new source database and the conversion strategy to be added to a first temporary list, wherein the query priority of the second conversion list is higher than that of the first temporary list; when no metadata migration task exists, the second conversion list is updated based on the first temporary list; when a metadata migration task exists and the metadata migration task is in execution state, respond to a target database addition instruction, obtain the database identifier of the new target database and the conversion strategy to be added from the target data type to the database type of the new target database, and add the database identifier of the new target database and the conversion strategy to be added to a second temporary list, wherein the query priority of the third conversion list is higher than that of the second temporary list; when no metadata migration task exists, the third conversion list is updated based on the second temporary list.

[0076] Optionally, the conversion module is further configured to:

[0077] Based on the first database type, a preset query interface is invoked, and the first target bridge and target query statement corresponding to the first database type are returned through the preset query interface;

[0078] A database connection is established with the source database through the first target bridge, and the target query statement is used to obtain the target operation control parameter indication metadata information to be migrated from the metadata table of the source database.

[0079] Optionally, the storage module is used for:

[0080] Based on the second database type, the preset generator interface is called, and the second target bridge and target generation statement corresponding to the second database type are returned through the preset generator interface;

[0081] A database connection is established with the target database through the second target bridge, and the target DDL corresponding to the target database is generated according to the second transformation result through the target generation statement, and the target DDL is stored in the target database.

[0082] It should be noted that other corresponding descriptions of the functional units involved in the metadata migration device between databases provided in this application embodiment can be found in the following references. Figures 1 to 3 The corresponding descriptions in the methods will not be repeated here.

[0083] This application also provides a computer device, which may specifically be a personal computer, a server, a network device, etc. Figure 5 As shown, the computer device includes a bus, a processor, memory, and a communication interface, and may also include an input / output interface and a display device. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores location information. The network interface allows communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0084] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0085] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, having stored thereon a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0086] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0087] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0088] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for migrating metadata between databases, characterized in that, include: In response to metadata migration instructions, determine the first database type of the source database and the second database type of the target database; Based on the first database type and the second database type, determine the first conversion strategy corresponding to the front mapper and the second conversion strategy corresponding to the back mapper; Based on the target operation control parameters, the metadata information to be migrated is obtained from the source database, and the first conversion strategy is executed on the metadata information to be migrated through the front mapper to obtain a first conversion result. The second conversion strategy is executed on the first conversion result through the back mapper to obtain a second conversion result. The second conversion result is stored in the target database; The step of determining the first conversion strategy corresponding to the front-end mapper and the second conversion strategy corresponding to the back-end mapper based on the first database type and the second database type includes: Determine whether the first database type and the second database type are consistent; When the judgment result is consistent, it is determined that the first conversion strategy corresponding to the pre-mapper and the second conversion strategy corresponding to the post-mapper are both to perform no operation. When the determination result is inconsistent, the system queries the first conversion list to see if a target conversion strategy exists between the first database type and the second database type. If the target conversion strategy exists in the first conversion list, the target conversion strategy is determined as the first conversion strategy corresponding to the front-end mapper, and the second conversion strategy corresponding to the back-end mapper is determined to perform no operation. If the target conversion strategy does not exist in the first conversion list, the system queries the second conversion list to see if a conversion strategy corresponding to the conversion from the first database type to the target data type is used as the first conversion strategy, and queries the third conversion list to see if a conversion strategy corresponding to the conversion from the target data type to the second database type is used as the second conversion strategy.

2. The method according to claim 1, characterized in that, The process of executing the first transformation strategy on the metadata information to be migrated through the pre-machining to obtain a first transformation result, and then executing the second transformation strategy on the first transformation result through the post-machining to obtain a second transformation result, includes: For any source field in the metadata information to be migrated, the mapping rule corresponding to the source field is determined from the first transformation strategy, and the first sub-rule and the second sub-rule are parsed from the mapping rule corresponding to the source field. The source field is transformed based on the first sub-rule and the second sub-rule through the pre-mapping, and the first transformation result corresponding to the source field is obtained. The first sub-rule is used to indicate the transformation rule of the first information of the source field, and the second sub-rule is used to indicate the transformation rule of the second information of the source field. The first information includes the field name and / or field type of the source field, and the second information includes the field length and / or field precision of the source field. For any first transformation result, the mapping rule corresponding to the first transformation result is determined from the second transformation strategy, and the third sub-rule and the fourth sub-rule are parsed from the mapping rule corresponding to the first transformation result. Through the post-mapper, the first transformation result is transformed based on the third sub-rule and the fourth sub-rule to obtain the second transformation result corresponding to the first transformation result. The third sub-rule is used to indicate the transformation rule of the third information of the first transformation result, and the fourth sub-rule is used to indicate the transformation rule of the fourth information of the first transformation result. The third information includes the field name and / or field type of the first transformation result, and the fourth information includes the field length and / or field precision of the first transformation result.

3. The method according to claim 1 or 2, characterized in that, Before determining the first database type of the source database in response to the metadata migration instruction, the method further includes: Obtain the default runtime control parameters for the metadata migration task; Determine whether the metadata migration task has a parameter configuration file, and if a parameter configuration file exists, update the parameter with the same name in the default run control parameters based on the parameter configuration file to obtain the first run control parameter; Determine whether the metadata migration task has command-line parameters, and if it does, update the parameter with the same name in the first run control parameters based on the command-line parameters to obtain the second run control parameters; Determine whether the metadata migration task has a parameter setting script, and if it does, obtain the third running control parameters based on the target setting parameters in the parameter setting script and the second running control parameters, wherein the target setting parameters include shard key setting parameters and / or partition setting parameters; The third operation control parameter is used as the target operation control parameter.

4. The method according to claim 3, characterized in that, The method further includes: When a metadata migration task exists and is in execution, in response to a source database addition instruction, the database identifier of the new source database and the conversion strategy to be added from the database type of the new source database to the target data type are obtained. The database identifier of the new source database and the conversion strategy to be added are added to a first temporary list, wherein the query priority of the second conversion list is higher than that of the first temporary list. When there is no metadata migration task, the second conversion list is updated based on the first temporary list. When a metadata migration task exists and is in execution, in response to a target database addition instruction, the database identifier of the new target database and the conversion strategy to be added from the target data type to the database type of the new target database are obtained. The database identifier of the new target database and the conversion strategy to be added are added to a second temporary list, wherein the query priority of the third conversion list is higher than that of the second temporary list. When there is no metadata migration task, the third conversion list is updated based on the second temporary list.

5. The method according to claim 1, characterized in that, The step of obtaining the metadata information to be migrated from the source database according to the target operation control parameters includes: Based on the first database type, the preset query interface is invoked, and the first target bridge and target query statement corresponding to the first database type are returned through the preset query interface; A database connection is established with the source database through the first target bridge, and the target query statement is used to obtain the target operation control parameter indication metadata information to be migrated from the metadata table of the source database.

6. The method according to claim 5, characterized in that, The step of storing the second conversion result in the target database includes: Based on the second database type, the preset generator interface is called, and the second target bridge and target generation statement corresponding to the second database type are returned through the preset generator interface; A database connection is established with the target database through the second target bridge, and the target DDL corresponding to the target database is generated according to the second transformation result through the target generation statement, and the target DDL is stored in the target database.

7. A metadata migration device between databases, characterized in that, include: The database type determination module is used to determine the first database type of the source database and the second database type of the target database in response to the metadata migration instruction. The conversion strategy determination module is used to determine a first conversion strategy corresponding to the front-end mapper and a second conversion strategy corresponding to the back-end mapper based on the first database type and the second database type. The conversion module is used to obtain the metadata information to be migrated from the source database according to the target operation control parameters, and to execute the first conversion strategy on the metadata information to be migrated through the front mapper to obtain a first conversion result, and to execute the second conversion strategy on the first conversion result through the back mapper to obtain a second conversion result; A storage module is used to store the second conversion result in the target database; The conversion strategy determination module is used for: Determine whether the first database type and the second database type are consistent; When the judgment result is consistent, it is determined that the first conversion strategy corresponding to the front mapper and the second conversion strategy corresponding to the back mapper are both not to perform any operation; when the judgment result is inconsistent, it is queried from the first conversion list whether there is a target conversion strategy between the first database type and the second database type; When the target conversion strategy exists in the first conversion list, the target conversion strategy is determined as the first conversion strategy corresponding to the front mapper, and the second conversion strategy corresponding to the back mapper is determined to perform no operation; when the target conversion strategy does not exist in the first conversion list, the conversion strategy corresponding to the conversion from the first database type to the target data type is queried from the second conversion list as the first conversion strategy, and the conversion strategy corresponding to the conversion from the target data type to the second database type is queried from the third conversion list as the second conversion strategy.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

9. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

Images