A method, apparatus, device, medium, and product for migrating a database

Abstract

The present application discloses a method, device, equipment, medium and product for database migration, which relates to the technical field of data processing. The method includes parsing the received migration request to obtain the routing configuration information of the migration request and the receiving database; using the migration request to construct an abstract syntax tree corresponding to the migration request; determining the target database and the target shard table set from the routing configuration information; modifying the migration request based on the target database and the target shard table set to obtain a first migration request; using the abstract syntax tree to perform semantic conversion on the first migration request to obtain a second migration request; optimizing the second migration request to obtain a target migration request; and using the target migration request to migrate the data in the target shard table set to the receiving database. The present application can improve the stability and reliability of the database, thereby ensuring the normal operation of the migrated database.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a method, apparatus, device, medium and product for migrating a database. Background Art

[0002] Today, with the rapid development of informatization and digitization, enterprises' dependence on database systems is increasing. However, with the continuous expansion of business and the continuous update of technology, enterprises often face the urgent need to upgrade the database or change the database type. For example, upgrading from foreign database software to domestic database, or migrating from one type of database to another type of database. These operations are both necessary and extremely challenging for enterprises' information systems.

[0003] Directly migrating the database may cause a large number of modifications to the database access code in the application program. Not only is the workload large, but also compatibility problems are likely to occur, affecting the stability and reliability of the database, and posing a greater risk to the normal operation of the migrated database. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device, medium and product for migrating a database, which can improve the stability and reliability of the database, thereby ensuring the normal operation of the migrated database.

[0005] To achieve the above purpose, this application provides the following solutions:

[0006] In the first aspect, this application provides a method for migrating a database, including:

[0007] Parsing the received migration request to obtain the routing configuration information of the migration request and the receiving database;

[0008] Using the migration request to construct an abstract syntax tree corresponding to the migration request;

[0009] Determining the target database and the target shard table set from the routing configuration information;

[0010] Based on the target database and the target shard table set, changing the migration request to obtain a first migration request;

[0011] Using the abstract syntax tree to perform semantic conversion on the first migration request to obtain a second migration request;

[0012] Optimizing the second migration request to obtain a target migration request;

[0013] Using the target migration request to migrate the data in the target shard table set to the receiving database.

[0014] Optionally, constructing an abstract syntax tree corresponding to the migration request by using the migration request specifically includes:

[0015] Splitting the migration request to obtain a plurality of lexical units; wherein, the types of the lexical units at least include keyword type, identifier type, constant type, operator type, and delimiter type;

[0016] Using recursive descent analysis to construct the plurality of lexical units to obtain an abstract syntax tree corresponding to the migration request.

[0017] Optionally, determining a target database and a set of target sharded tables from the routing configuration information specifically includes:

[0018] If the routing configuration information is a database and sharded table configuration, splitting the migration request into a plurality of migration sub-requests;

[0019] Determining a target database from the routing configuration information;

[0020] Determining target sharded tables corresponding to each of the migration sub-requests from the target database;

[0021] Determining the priorities of the plurality of migration sub-requests;

[0022] Adding the target sharded tables to the set of target sharded tables in descending order of the priorities of the migration sub-requests corresponding to the target sharded tables.

[0023] Optionally, performing semantic conversion on the first migration request by using the abstract syntax tree to obtain a second migration request, specifically including:

[0024] Obtaining the target SQL dialect of the target database and the receiving SQL dialect of the receiving database;

[0025] Using the target SQL dialect and the receiving SQL dialect to determine a dialect difference comparison table; wherein, the dialect difference comparison table at least includes a data type difference sub-comparison table, a built-in function difference sub-comparison table, an operator semantic difference sub-comparison table, and a transaction control statement difference sub-comparison table;

[0026] Performing semantic conversion on the abstract syntax tree by using the dialect difference comparison table to obtain a target abstract syntax tree;

[0027] Performing semantic conversion on the first migration request by using the target abstract syntax tree to obtain a second migration request; wherein, the first migration request has the same format as the target SQL dialect, and the second migration request has the same format as the receiving SQL dialect.

[0028] Optionally, optimizing the second migration request to obtain a target migration request specifically includes:

[0029] Determine the common fields in the second migration request;

[0030] Add index information to the common fields without index information in the second migration request to obtain a third migration request;

[0031] Determine the fuzzy conditions in the third migration request;

[0032] Determine the exact conditions matching the fuzzy conditions;

[0033] Use the exact conditions to replace the fuzzy conditions in the third migration request to obtain a fourth migration request;

[0034] Determine the objective function in the fourth migration request;

[0035] Determine the objective constant corresponding to the objective function;

[0036] Use the objective constant to replace the objective function in the fourth migration request to obtain a fifth migration request;

[0037] Adjust the connection information in the fifth migration request to obtain a target migration request.

[0038] Optionally, the migration method of the database further includes:

[0039] Obtain the target data volume in the target database and the received data volume in the receiving database;

[0040] If the target data volume is the same as the received data volume, determine that the migration of the target database is completed;

[0041] If the target data volume is different from the received data volume, use the target migration request to migrate the data in the target shard table set to the receiving database, and obtain the current data volume in the receiving database;

[0042] If the target data volume is the same as the current data volume, determine that the migration of the target database is completed;

[0043] If the target data volume is different from the current data volume, obtain a handwritten SQL migration request, and use the handwritten SQL migration request to migrate the data in the target shard table set to the receiving database.

[0044] In a second aspect, the present application provides a database migration device, including:

[0045] A parsing unit, configured to parse the received migration request to obtain the routing configuration information and the receiving database of the migration request;

[0046] A construction unit, configured to use the migration request to construct an abstract syntax tree corresponding to the migration request;

[0047] A determination unit, configured to determine a target database and a set of target sharded tables from the routing configuration information;

[0048] An alteration unit, configured to alter the migration request based on the target database and the set of target sharded tables to obtain a first migration request;

[0049] A conversion unit, configured to semantically convert the first migration request using the abstract syntax tree to obtain a second migration request;

[0050] An optimization unit, configured to optimize the second migration request to obtain a target migration request;

[0051] A migration unit, configured to use the target migration request to migrate the data in the set of target sharded tables to the receiving database.

[0052] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the steps of the database migration method described in any one of the above.

[0053] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the database migration method described in any one of the above are implemented.

[0054] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the database migration method described in any one of the above are implemented.

[0055] In a sixth aspect, the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, the processor is configured to run a program or an instruction, and when the processor executes the program or the instruction, the steps of the database migration method described in any one of the above are implemented.

[0056] According to the specific embodiments provided by the present application, the present application discloses the following technical effects:

[0057] The present application provides a method, apparatus, device, medium and product for database migration. By systematically parsing the migration request and constructing a corresponding abstract syntax tree, it can accurately identify and process various configuration information during the migration process, including routing configuration, target database, target sharded table set, etc. Through this series of refined operations, the accuracy and efficiency of the migration request are ensured. At the same time, this method also further improves the flexibility and execution efficiency of the migration process by performing change, semantic conversion and optimization processing on the migration request. Finally, using the optimized target migration request, the conversion of different grammars and structures between databases realizes the accurate migration of the data in the target sharded table set to the receiving database, which can improve the stability and reliability of the database, thus ensuring the normal operation of the migrated database. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0059] Figure 1 It is a schematic flowchart of a method for database migration in an embodiment of the present application;

[0060] Figure 2 It is a schematic diagram of the functional modules of a database migration apparatus provided in an embodiment of the present application.

[0061] Figure 3 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0063] To make the above objects, features and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0064] In an exemplary embodiment, as Figure 1As shown, a method for migrating a database is provided. This method is executed by a computer device, which can be specifically executed by a computer device such as a terminal or a server alone, or jointly executed by a terminal and a server. In the embodiments of this application, it includes the following steps 101 to 107. Among them:

[0065] Step 101: Parse the received migration request to obtain the routing configuration information of the migration request and the receiving database.

[0066] The embodiments of this application can be applied to a database AI adaptation system platform. This platform has a multi-layer architecture: a multi-layer architecture including an interface layer, a conversion layer, and a data storage layer is designed, where:

[0067] The interface layer is responsible for receiving database operation requests from application programs, which were originally designed for foreign databases.

[0068] The conversion layer is the core part. With the help of an AI large model, it realizes the parsing, conversion, and adaptation of requests, and converts them into operations that can be correctly executed on domestic databases.

[0069] The data storage layer is a domestic database, which is used to store the migrated and converted data.

[0070] Model integration architecture: Integrate the AI large model into the system conversion layer to establish a model structure related to database conversion. The conversion layer receives the requests passed from the interface layer and calls the AI large model to perform analysis, conversion, adaptation, and other operations. For example, it includes model sub-modules for identifying different database operation types (such as query, insert, update, delete), and model sub-modules for converting different data types (such as character type, numeric type, date type, etc.) between foreign databases and domestic databases. According to the syntax rules of domestic databases, the AI large model converts operation statements in the style of foreign databases.

[0071] It can handle problems in aspects such as function usage and keyword differences in different databases. For example, certain specific string processing functions in some foreign databases have different expressions in domestic databases, and the large model can automatically replace them with the corresponding functions in domestic databases and optimize the statement structure to improve execution efficiency.

[0072] Step 102: Use the migration request to construct an abstract syntax tree corresponding to the migration request.

[0073] In the embodiments of this application, the migration request can be split into multiple atomic symbols that cannot be further divided, and an abstract syntax tree is constructed based on the multiple atomic symbols.

[0074] As an alternative implementation, the method of using the migration request in step 102 to construct an abstract syntax tree corresponding to the migration request may include:

[0075] Split the migration request to obtain a plurality of lexical units; wherein, the types of the lexical units at least include keyword type, identifier type, constant type, operator type, and delimiter type;

[0076] Use recursive descent analysis to construct the plurality of lexical units to obtain an abstract syntax tree corresponding to the migration request.

[0077] Among them, in this implementation, by splitting the migration request into multiple lexical units and clarifying the types of these lexical units (such as keywords, identifiers, constants, operators, and delimiters), the syntax structure of the migration request can be understood and analyzed more carefully. This careful splitting and classification provides a solid foundation for subsequent construction of the abstract syntax tree. By using recursive descent analysis, these lexical units can be efficiently utilized to gradually construct a complete abstract syntax tree according to the syntax rules. It not only improves the accuracy and efficiency of constructing the abstract syntax tree, but also makes the entire processing process of the migration request more transparent and controllable. Through the abstract syntax tree, the various components of the migration request and their interrelationships can be clearly seen, making it easier to discover and correct potential syntax errors or logical problems.

[0078] In the embodiments of the present application, a lexical unit (Token) may include lexical units of keyword type, identifier type, constant type, operator type, and delimiter type. Specifically:

[0079] Keyword type: Reserved words with specific semantics in SQL, such as SELECT, FROM, WHERE, AND, OR, etc., each representing different statement functions or logical relationships.

[0080] Identifier type: Used to represent table names, column names, etc., usually a string starting with a letter followed by letters, numbers, and underscores, such as table_name, column1, etc.

[0081] Constant type: Includes numeric constants (such as integer 123, decimal 3.14, etc.), string constants (character sequences enclosed in single quotes or double quotes, like 'abc', "def"), date and time constants (represented in a specific date and time format), etc.

[0082] Operator types: such as arithmetic operators (+, -, *, / , etc.), comparison operators (=, >, <, >=, <=, <>, etc.), logical operators (AND, OR, NOT, etc.), and other special operators (such as LIKE, BETWEEN, etc.).

[0083] Delimiter types: symbols such as parentheses (, ), commas ,, semicolons ;, etc., which are used to separate different syntactic components or represent specific syntactic structures.

[0084] In the embodiments of the present application, a syntax analyzer can process lexical units based on the recursive descent analysis method to construct a syntax tree:

[0085] The recursive descent analysis method is a top-down syntax analysis method. Starting from the top-level non-terminal (such as select_statement) according to the syntax rules, it gradually recursively calls functions to match the input lexical units and construct the nodes of the syntax tree. For example, for a SELECT statement, there will be a corresponding function to process the select_statement rule. In this function, the functions corresponding to sub-rules such as select_list, FROM, and WHERE are called in sequence. Each function performs corresponding processing according to the current lexical unit to be matched. If the match is successful, the corresponding syntax tree node is created and the analysis continues downward. If an unmatched situation is encountered, it indicates a syntax error. This method is simple and intuitive, and is easy to understand and implement.

[0086] Step 103: Determine the target database and the set of target sharded tables from the routing configuration information.

[0087] In the embodiments of the present application, a sharded table refers to a table with a large amount of original data that needs to be split into multiple tables according to a certain rule. In this way, each sharded table (i.e., the split data part) contains a part of the data of the original table, and all sharded tables together constitute the complete data set. The sharding dimension may be time, region, etc.

[0088] As an optional implementation manner, the method for step 103 to determine the target database and the set of target sharded tables from the routing configuration information may include:

[0089] If the routing configuration information is a database and table sharding configuration, split the migration request into multiple migration sub-requests;

[0090] Determine the target database from the routing configuration information;

[0091] Determine the target sharded tables corresponding to each migration sub-request from the target database;

[0092] Determine the priorities of the multiple migration sub-requests;

[0093] Add the target shard table to the target shard table set in descending order of the priorities of the migration sub-requests corresponding to the target shard table.

[0094] Among them, by implementing this implementation method, the migration request is split into multiple migration sub-requests, realizing the refined management and execution of large-scale migration tasks. This strategy not only improves the flexibility and scalability of the migration process, but also enables each migration sub-request to operate on a specific target database and target shard table, thereby effectively reducing resource consumption and potential conflicts during the migration process. At the same time, the target database is accurately determined from the routing configuration information, and the target shard tables corresponding to each migration sub-request are further determined in the target database, ensuring the accuracy and pertinence of the migration task. In addition, by determining the priorities of multiple migration sub-requests and adding the target shard tables to the target shard table set in the order of priorities, this implementation method also realizes the optimized scheduling of the migration task, enabling high-priority migration tasks to be processed first, thereby further improving the migration efficiency and user experience.

[0095] Step 104: Modify the migration request based on the target database and the target shard table set to obtain a first migration request.

[0096] In the embodiments of the present application, modifying the migration request may specifically be: replacing the original database or shard table information in the migration request with a suitable target database and the target shard table in the target shard table set, so that the obtained first migration request can be executed on the correct target database and the correct target shard table. The modification process may include replacing table names, adding shard key conditions, etc.

[0097] Step 105: Perform semantic conversion on the first migration request using the abstract syntax tree to obtain a second migration request.

[0098] As an alternative implementation method, the way of performing semantic conversion on the first migration request using the abstract syntax tree in step 105 to obtain a second migration request may include:

[0099] Obtain the target SQL dialect of the target database and the received SQL dialect of the received database;

[0100] Use the target SQL dialect and the received SQL dialect to determine a dialect difference comparison table; wherein, the dialect difference comparison table at least includes a data type difference sub-comparison table, a built-in function difference sub-comparison table, an operator semantics difference sub-comparison table, and a transaction control statement difference sub-comparison table;

[0101] Perform semantic conversion on the abstract syntax tree using the dialect difference comparison table to obtain a target abstract syntax tree;

[0102] Perform semantic conversion on the first migration request using the target abstract syntax tree to obtain a second migration request; wherein, the format of the first migration request is the same as that of the target SQL dialect, and the format of the second migration request is the same as that of the received SQL dialect.

[0103] Among them, implementing this implementation method, using the dialect difference comparison table to perform semantic conversion on the abstract syntax tree not only ensures the accuracy and consistency of the conversion process, but also effectively reduces the risk of migration errors or performance degradation caused by dialect differences. Through this conversion, we obtain a target abstract syntax tree that is compatible with both the target database and the received database. By using the target abstract syntax tree that is compatible with both the target database and the received database, the target database can be made compatible with the received database, thereby improving the accuracy of the second migration request.

[0104] In the embodiments of the present application, the dialect difference comparison table is a data mapping relationship established between the target SQL dialect and the received SQL dialect. Based on the dialect difference comparison table, accurate conversion between the target SQL dialect and the received SQL dialect can be achieved.

[0105] In the embodiments of the present application, the data type difference sub-comparison table: different databases may have different regulations on the same type, such as different names or precision ranges;

[0106] The built-in function difference sub-comparison table: differences in aspects such as name, number of parameters, and function implementation;

[0107] The operator semantic difference sub-comparison table: for example, the logical operator && represents the AND operation in some databases, but may not be supported in other databases;

[0108] The transaction control statement difference sub-comparison table: different writing methods for statements such as commit and rollback.

[0109] For example, make necessary adjustments to object names such as table names and column names in the target SQL dialect according to the naming specifications of the received SQL dialect, replace the functions used in the target SQL dialect with functions that are functionally equivalent in the received SQL dialect, and perform conversion on operators, etc. according to the semantics of the received SQL dialect. For example, convert specific date and time operators in the target SQL dialect to corresponding forms that can achieve the same function in the received SQL dialect, ensuring that the converted statement is semantically consistent with the source statement but meets the semantic requirements of the received SQL dialect.

[0110] Step 106, optimize the second migration request to obtain a target migration request.

[0111] In the embodiments of the present application, performance analysis can be performed on database operations through an AI large model, and the query plan and data access path can be optimized.

[0112] As an alternative implementation, the method of optimizing the second migration request to obtain a target migration request in step 106 may include:

[0113] Determine the frequently used fields in the second migration request;

[0114] Add index information to the frequently used fields without index information in the second migration request to obtain a third migration request;

[0115] Determine the fuzzy conditions in the third migration request;

[0116] Determine the exact conditions that match the fuzzy conditions;

[0117] Use the exact conditions to replace the fuzzy conditions in the third migration request to obtain a fourth migration request;

[0118] Determine the objective function in the fourth migration request;

[0119] Determine the objective constant corresponding to the objective function;

[0120] Use the objective constant to replace the objective function in the fourth migration request to obtain a fifth migration request;

[0121] Adjust the connection information in the fifth migration request to obtain a target migration request.

[0122] Among them, by implementing this implementation method, by adding indexes to frequently used fields, using exact conditions to replace fuzzy conditions, and using constants to replace functions, the unclear or difficult-to-understand fields in the migration request can be reduced, and the accuracy of the target migration request in the process of migrating data can be improved.

[0123] In the embodiments of the present application, analyze the frequently used fields in the second migration request, such as the fields in the WHERE clause and JOIN clause. If these fields do not have indexes, the database may perform a full table scan when executing a query, resulting in low performance. According to the query frequency and data volume, adding indexes to these key fields can significantly improve the query speed.

[0124] For example, for "SELECT * FROM products WHERE category_id = 1", if the category_id field does not have an index, the query may traverse the entire products table. After adding an index, the database can directly locate the records with category_id equal to 1, greatly reducing the query time.

[0125] In the embodiments of the present application, in order to ensure that the screening conditions of the migration request are as accurate as possible and avoid unnecessary data return, specific range conditions (i.e., precise conditions) can be used instead of fuzzy conditions, or IN can be used instead of multiple OR conditions. The OR conditions in the migration request can be determined as fuzzy conditions. The way to determine the precise conditions can be: convert the OR conditions into IN conditions corresponding to the OR conditions, and then the IN conditions can be determined as the precise conditions corresponding to the fuzzy conditions. The way to convert the OR conditions into IN conditions corresponding to the OR conditions can be achieved through a pre-trained condition conversion model or can be converted manually.

[0126] For example, "SELECT * FROM employees WHERE salary>5000 AND salary<10000" is more precise than "SELECT * FROM employees WHERE salary BETWEEN 0 AND 10000", reducing unnecessary data retrieval.

[0127] In the embodiments of the present application, in order to avoid using complex functions and expressions in the migration request (because this may cause index invalidation), consider applying the function to constants or using the function index provided by the database (if supported).

[0128] For example, for "SELECT * FROM users WHERE YEAR (birth_date) = 1990", applying the YEAR function to the birth_date field will cause index invalidation. It can be rewritten as "SELECT * FROM users WHERE birth_date BETWEEN '1990-01-01' AND '1990-12-31'", so that the index of the birth_date field can be utilized.

[0129] In the embodiments of the present application, the specific way to adjust the connection information in the fifth migration request can be:

[0130] 1. Select an appropriate connection type

[0131] Select the correct connection type (such as INNER JOIN, LEFT JOIN, RIGHT JOIN, etc.) according to the requirements of the fifth migration request. Ensure that the connection conditions are accurate and avoid generating a Cartesian product (i.e., a table connection without a correct connection condition, resulting in an abnormally large amount of data in the result set).

[0132] For example, when querying user orders and user information, if you only care about the user information of those with orders, using INNER JOIN is appropriate; if you need to include all user information, even those without orders, you should use LEFT JOIN.

[0133] 2. Join Order Optimization

[0134] When joining multiple tables, adjusting the join order of the tables may affect query performance. Generally, putting the table with a smaller amount of data in the front, or the table with the most restrictive filtering conditions in the front, can reduce the size of the intermediate result set, thereby improving the query speed.

[0135] For example, according to the index structure and data distribution characteristics of domestic databases, adjust the fifth migration request to improve the query speed. At the same time, cache and optimize frequently executed operations to reduce unnecessary database load and improve the overall performance of the system.

[0136] Step 107: Use the target migration request to migrate the data in the target sharded table set to the receiving database.

[0137] In the embodiments of the present application, if the request involves multiple sharded tables or multiple database nodes, it is necessary to perform a merging process on the data in the target sharded table set. The merging process may include operations such as sorting, grouping, and merging to ensure that the data finally migrated to the receiving database is correct. The merged data is encoded into the client protocol format of the receiving database, and the encoded result is transmitted to the client of the receiving database through the network to complete the execution process of the entire request.

[0138] As an optional implementation manner, after step 107, the following steps may also be executed:

[0139] Obtain the target data volume in the target database and the received data volume in the receiving database;

[0140] If the target data volume is the same as the received data volume, determine that the migration of the target database is completed;

[0141] If the target data volume is different from the received data volume, use the target migration request to migrate the data in the target sharded table set to the receiving database, and obtain the current data volume in the receiving database;

[0142] If the target data volume is the same as the current data volume, determine that the migration of the target database is completed;

[0143] If the target data volume is different from the current data volume, obtain a handwritten SQL migration request, and use the handwritten SQL migration request to migrate the data in the target sharded table set to the receiving database.

[0144] Among them, by implementing this implementation method, by obtaining the target data volume in the target database and the received data volume in the receiving database and comparing them, this implementation method provides an intuitive and effective mechanism for checking the migration completion degree. When the data volumes of the two are the same, it can be confirmed that the migration of the target database has been completed, which greatly improves the reliability and accuracy of the migration process.

[0145] For example, by establishing a data verification and synchronization mechanism, ensure the consistency of data during the conversion from a foreign database (target database) to a domestic database (receiving database);

[0146] Each migration request input by the user will leave a record. According to different request types, the data quantity in the domestic database will be queried regularly and compared with the quantity of the initiated request. For execution failures (different quantities) and execution of abnormal SQL, they will be saved to the exception log and can be executed again through manual repair or repair by the AI large model to ensure final consistency.

[0147] Use the AI large model to perform real-time monitoring and comparison of the data, monitor the execution process of the user request. During the execution process, if a system error occurs, capture the exception, call the AI large model to correct and repair the statement, and execute the SQL statement again. If it reports an error again, add it to the system log and perform a log alarm operation. The user can manually rewrite the correct SQL in the log management interface and add it to the big data model, and the same SQL will be executed normally and repaired successfully the next time, promptly discovering and handling possible problems such as data loss, duplication, or inconsistency. For example, after batch data migration, perform integrity checks and verification on key data.

[0148] Implementing the above steps 101 to 107 can improve the stability and reliability of the database, thus ensuring the normal operation of the migrated database. In addition, this application can also improve the accuracy and efficiency of constructing the abstract syntax tree. In addition, this application can also achieve optimized scheduling of the migration task. In addition, this application can also improve the accuracy of the second migration request. In addition, this application can also improve the accuracy of the target migration request during the data migration process. In addition, this application can also improve the reliability and accuracy of the migration process.

[0149] Based on the same inventive concept, an embodiment of the present application further provides a database migration apparatus for implementing the database migration method involved above. The solution provided by this apparatus for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the database migration apparatus provided below can refer to the limitations on the database migration method in the above text and will not be repeated here.

[0150] In an exemplary embodiment, as Figure 2 shown, a database migration apparatus is provided, including:

[0151] A parsing unit 201, configured to parse the received migration request to obtain the routing configuration information of the migration request and the receiving database;

[0152] A constructing unit 202, configured to use the migration request to construct an abstract syntax tree corresponding to the migration request;

[0153] A determining unit 203, configured to determine a target database and a target shard table set from the routing configuration information;

[0154] A changing unit 204, configured to change the migration request based on the target database and the target shard table set to obtain a first migration request;

[0155] A converting unit 205, configured to perform semantic conversion on the first migration request using the abstract syntax tree to obtain a second migration request;

[0156] An optimizing unit 206, configured to optimize the second migration request to obtain a target migration request;

[0157] A migrating unit 207, configured to use the target migration request to migrate the data in the target shard table set to the receiving database.

[0158] As an optional implementation manner, the specific manner in which the constructing unit 202 uses the migration request to construct an abstract syntax tree corresponding to the migration request may specifically be:

[0159] Split the migration request to obtain a plurality of lexical units; wherein, the types of the lexical units at least include keyword type, identifier type, constant type, operator type, and delimiter type;

[0160] Use recursive descent analysis to construct the plurality of lexical units to obtain an abstract syntax tree corresponding to the migration request.

[0161] Among them, in implementing this implementation method, by splitting the migration request into multiple lexical units and clarifying the types of these lexical units (such as keywords, identifiers, constants, operators, delimiters, etc.), the syntax structure of the migration request can be understood and analyzed more meticulously. This meticulous splitting and classification provide a solid foundation for constructing the abstract syntax tree subsequently. By using the recursive descent analysis method, these lexical units can be utilized efficiently to gradually construct a complete abstract syntax tree according to the syntax rules. This not only improves the accuracy and efficiency of constructing the abstract syntax tree but also makes the processing process of the entire migration request more transparent and controllable. Through the abstract syntax tree, each component of the migration request and their interrelationships can be clearly seen, making it easier to discover and correct potential syntax errors or logical problems.

[0162] As an optional implementation method, the way for the determination unit 203 to determine the target database and the set of target sharded tables from the routing configuration information can specifically be as follows:

[0163] If the routing configuration information is a database and table sharding configuration, split the migration request into multiple migration sub-requests;

[0164] Determine the target database from the routing configuration information;

[0165] Determine the target sharded table corresponding to each migration sub-request from the target database;

[0166] Determine the priorities of multiple migration sub-requests;

[0167] Add the target sharded tables to the set of target sharded tables in the order from high to low according to the priorities of the migration sub-requests corresponding to the target sharded tables.

[0168] Among them, in implementing this implementation method, by splitting the migration request into multiple migration sub-requests, the refined management and execution of large-scale migration tasks are realized. This strategy not only improves the flexibility and scalability of the migration process but also enables each migration sub-request to operate on a specific target database and target sharded table, thereby effectively reducing resource consumption and potential conflicts during the migration process. At the same time, accurately determining the target database from the routing configuration information and further determining the target sharded tables corresponding to each migration sub-request in the target database ensure the accuracy and pertinence of the migration task. In addition, by determining the priorities of multiple migration sub-requests and adding the target sharded tables to the set of target sharded tables in the order of priorities, this implementation method also realizes the optimized scheduling of the migration task, enabling high-priority migration tasks to be processed preferentially, thereby further improving the migration efficiency and user experience.

[0169] As an alternative implementation, the way that the conversion unit 205 performs semantic conversion on the first migration request using the abstract syntax tree to obtain a second migration request can specifically be as follows:

[0170] Obtain the target SQL dialect of the target database and the receiving SQL dialect of the receiving database;

[0171] Use the target SQL dialect and the receiving SQL dialect to determine a dialect difference comparison table; wherein, the dialect difference comparison table at least includes a data type difference sub - comparison table, a built - in function difference sub - comparison table, an operator semantics difference sub - comparison table, and a transaction control statement difference sub - comparison table;

[0172] Use the dialect difference comparison table to perform semantic conversion on the abstract syntax tree to obtain a target abstract syntax tree;

[0173] Use the target abstract syntax tree to perform semantic conversion on the first migration request to obtain a second migration request; wherein, the format of the first migration request is the same as that of the target SQL dialect, and the format of the second migration request is the same as that of the receiving SQL dialect.

[0174] Among them, when implementing this implementation, using the dialect difference comparison table to perform semantic conversion on the abstract syntax tree not only ensures the accuracy and consistency of the conversion process, but also effectively reduces the risk of migration errors or performance degradation caused by dialect differences. Through this conversion, we obtain a target abstract syntax tree that is compatible with both the target database and the receiving database, thereby improving the accuracy of the second migration request.

[0175] As an alternative implementation, the way that the optimization unit 206 optimizes the second migration request to obtain a target migration request can specifically be as follows:

[0176] Determine the common fields in the second migration request;

[0177] Add index information to the common fields without index information in the second migration request to obtain a third migration request;

[0178] Determine the fuzzy conditions in the third migration request;

[0179] Determine the exact conditions that match the fuzzy conditions;

[0180] Use the exact conditions to replace the fuzzy conditions in the third migration request to obtain a fourth migration request;

[0181] Determine the target function in the fourth migration request;

[0182] Determine the target constant corresponding to the target function;

[0183] Replace the target function in the fourth migration request with the said target constant to obtain a fifth migration request;

[0184] Adjust the connection information in the fifth migration request to obtain a target migration request.

[0185] Among them, by implementing this implementation method, by adding indexes to common fields, replacing fuzzy conditions with exact conditions, and replacing functions with constants, the unclear or difficult-to-understand fields in the migration request can be reduced, and the accuracy of the target migration request in the process of migrating data is improved.

[0186] As an alternative implementation method, the migration unit 207 is further configured to:

[0187] Obtain the target data volume in the target database and the received data volume in the receiving database;

[0188] If the target data volume is the same as the received data volume, it is determined that the migration of the target database is completed;

[0189] If the target data volume is different from the received data volume, use the target migration request to migrate the data in the target shard table set to the receiving database, and obtain the current data volume in the receiving database;

[0190] If the target data volume is the same as the current data volume, it is determined that the migration of the target database is completed;

[0191] If the target data volume is different from the current data volume, obtain a handwritten SQL migration request, and use the handwritten SQL migration request to migrate the data in the target shard table set to the receiving database.

[0192] Among them, by implementing this implementation method, by obtaining the target data volume in the target database and the received data volume in the receiving database and making a comparison, this implementation method provides an intuitive and effective mechanism for checking the migration completion degree. When the data volumes of the two are the same, it can be confirmed that the migration of the target database has been completed, which greatly improves the reliability and accuracy of the migration process.

[0193] Implementing the above implementation method can improve the stability and reliability of the database, thereby ensuring the normal operation of the migrated database. In addition, this application can also improve the accuracy and efficiency of constructing the abstract syntax tree. In addition, this application can also achieve optimized scheduling of the migration task. In addition, this application can also improve the accuracy of the second migration request. In addition, this application can also improve the accuracy of the target migration request in the process of migrating data. In addition, this application can also improve the reliability and accuracy of the migration process.

[0194] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as shown in Figure 3 . The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the migration data of the database. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a method for migrating a database.

[0195] Those skilled in the art can understand that Figure 3 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0196] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0197] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0198] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0199] In an exemplary embodiment, a chip is provided. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the steps in the above method embodiments, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0200] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0201] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0202] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0203] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0204] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0205] In this article, specific examples are used to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for migrating a database, characterized in that, The migration method of the database includes: Parsing the received migration request to obtain the routing configuration information of the migration request and the receiving database; Using the migration request to construct an abstract syntax tree corresponding to the migration request; Determining the target database and the target shard table set from the routing configuration information; Changing the migration request based on the target database and the target shard table set to obtain a first migration request; Using the abstract syntax tree to perform semantic conversion on the first migration request to obtain a second migration request; Optimizing the second migration request to obtain a target migration request; Using the target migration request to migrate the data in the target shard table set to the receiving database; Among them, the specific process of determining the target database and the target shard table set from the routing configuration information includes: If the routing configuration information is a database and table sharding configuration, splitting the migration request into multiple migration sub-requests; Determining the target database from the routing configuration information; Determining the target shard table corresponding to each migration sub-request from the target database; Determining the priorities of multiple migration sub-requests; Adding the target shard tables to the target shard table set in the order from high to low according to the priorities of the migration sub-requests corresponding to the target shard tables; And, the specific process of optimizing the second migration request to obtain a target migration request includes: Determining the common fields in the second migration request; among them, the common fields are the fields in the WHERE clause and the JOIN clause; Adding index information to the common fields without index information in the second migration request to obtain a third migration request; Determining the fuzzy conditions in the third migration request; among them, the fuzzy conditions are OR conditions; Determining the exact conditions matching the fuzzy conditions; Using the exact conditions to replace the fuzzy conditions in the third migration request to obtain a fourth migration request; Determining the target function in the fourth migration request; Determining the target constant corresponding to the target function; Using the target constant to replace the target function in the fourth migration request to obtain a fifth migration request; Adjusting the connection information in the fifth migration request to obtain a target migration request; Among them, the specific method of determining the exact conditions matching the fuzzy conditions is: Converting the OR condition into an IN condition through a pre-trained condition conversion model; the IN condition is an exact condition; And, the specific method of adjusting the connection information in the fifth migration request to obtain a target migration request is: Adjusting the connection information in the fifth migration request according to the correct connection type matching the fifth migration request to obtain an adjusted fifth migration request; Performing connection order optimization on the adjusted fifth migration request to obtain a target migration request; among them, the smaller the data volume of the target shard table corresponding to the target migration request, the higher the priority.

2. The method for migrating a database according to claim 1, wherein The specific process of using the migration request to construct an abstract syntax tree corresponding to the migration request includes: Split the migration request to obtain multiple lexical units; wherein, the types of the lexical units at least include keyword type, identifier type, constant type, operator type, and delimiter type; Use recursive descent analysis to construct the multiple lexical units to obtain an abstract syntax tree corresponding to the migration request.

3. The method for migrating a database according to claim 1, wherein The semantic conversion of the first migration request using the abstract syntax tree to obtain a second migration request specifically includes: Obtain the target SQL dialect of the target database and the receiving SQL dialect of the receiving database; Use the target SQL dialect and the receiving SQL dialect to determine a dialect difference comparison table; wherein, the dialect difference comparison table at least includes a data type difference sub - comparison table, a built - in function difference sub - comparison table, an operator semantic difference sub - comparison table, and a transaction control statement difference sub - comparison table; Use the dialect difference comparison table to perform semantic conversion on the abstract syntax tree to obtain a target abstract syntax tree; Use the target abstract syntax tree to perform semantic conversion on the first migration request to obtain a second migration request; wherein, the format of the first migration request is the same as that of the target SQL dialect, and the format of the second migration request is the same as that of the receiving SQL dialect.

4. The method for migrating a database according to any one of claims 1 to 3, characterized in that, The migration method of the database further includes: Obtain the target data volume in the target database and the receiving data volume in the receiving database; If the target data volume is the same as the receiving data volume, it is determined that the migration of the target database is completed; If the target data volume is different from the receiving data volume, use the target migration request to migrate the data in the target sharded table set to the receiving database, and obtain the current data volume in the receiving database; If the target data volume is the same as the current data volume, it is determined that the migration of the target database is completed; If the target data volume is different from the current data volume, obtain a handwritten SQL migration request, and use the handwritten SQL migration request to migrate the data in the target sharded table set to the receiving database.

5. A database migration device, characterized in that, The migration device of the database includes: A parsing unit, configured to parse the received migration request to obtain the routing configuration information of the migration request and the receiving database; A construction unit, configured to use the migration request to construct an abstract syntax tree corresponding to the migration request; A determination unit, configured to determine the target database and the target sharded table set from the routing configuration information; A change unit, configured to change the migration request based on the target database and the target sharded table set to obtain a first migration request; A conversion unit, configured to perform semantic conversion on the first migration request using the abstract syntax tree to obtain a second migration request; An optimization unit, configured to optimize the second migration request to obtain a target migration request; A migration unit, configured to use the target migration request to migrate the data in the target sharded table set to the receiving database; Wherein, the manner in which the determination unit determines the target database and the target sharded table set from the routing configuration information is specifically: If the routing configuration information is sharding configuration, split the migration request into multiple sub-migration requests; Determine the target database from the routing configuration information; Determine the target shard tables corresponding to the respective sub-migration requests from the target database; Determine the priorities of the multiple sub-migration requests; Add the target shard tables to the target shard table set in descending order of the priorities of the sub-migration requests corresponding to the target shard tables; Moreover, the manner in which the optimization unit optimizes the second migration request to obtain the target migration request is specifically as follows: Determine the common fields in the second migration request; where the common fields are the fields in the WHERE clause and the JOIN clause; Add index information to the common fields without index information in the second migration request to obtain a third migration request; Determine the fuzzy conditions in the third migration request; where the fuzzy conditions are OR conditions; Determine the exact conditions matching the fuzzy conditions; Use the exact conditions to replace the fuzzy conditions in the third migration request to obtain a fourth migration request; Determine the target functions in the fourth migration request; Determine the target constants corresponding to the target functions; Use the target constants to replace the target functions in the fourth migration request to obtain a fifth migration request; Adjust the connection information in the fifth migration request to obtain the target migration request; Among them, the manner in which the optimization unit determines the exact conditions matching the fuzzy conditions is specifically as follows: Convert the OR conditions into IN conditions through a pre-trained condition conversion model; the IN conditions are exact conditions; Moreover, the manner in which the optimization unit adjusts the connection information in the fifth migration request to obtain the target migration request is specifically as follows: Adjust the connection information in the fifth migration request according to the correct connection type matching the fifth migration request to obtain an adjusted fifth migration request; Optimize the connection order of the adjusted fifth migration request to obtain the target migration request; where the smaller the data volume of the target shard table corresponding to the target migration request, the higher the priority.

6. A computer device, comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the database migration method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the database migration method according to any one of claims 1-4.

8. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the database migration method according to any one of claims 1-4.

Images