Database migration method and device, equipment, medium and product

Abstract

The invention discloses a database migration method and device, equipment, a medium and a product, and relates to the technical field of data processing.The method comprises the steps that a received migration request is analyzed, and routing configuration information and a receiving database of the migration request are obtained; constructing an abstract syntax tree corresponding to the migration request by using the migration request; determining a target database and a target fragment table set from the routing configuration information; changing the migration request based on the target database and the target fragment table set to obtain a first migration request; performing semantic conversion on the first migration request by using the abstract syntax tree to obtain a second migration request; optimizing the second migration request to obtain a target migration request; and migrating the data in the target fragment table set to the receiving database by using the target migration request, so that the stability and the reliability of the database can be improved, and the normal operation of the migrated database is ensured.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a database migration method, device, equipment, medium and product. Background Art

[0002] With the rapid development of informatization and digitalization, enterprises are increasingly dependent on database systems. However, with the continuous expansion of business and continuous updating of technology, enterprises often face the urgent need to upgrade or replace database types. For example, upgrading from foreign database software to domestic databases, or migrating from one type of database to another, these operations are both necessary and extremely challenging for the information system of enterprises.

[0003] Direct database migration may require a lot of modifications to the database access code in the application. This is not only labor-intensive but also prone to compatibility issues, affecting the stability and reliability of the database and posing a greater risk to the normal operation of the migrated database. Summary of the invention

[0004] The purpose of this application is to provide a database migration method, device, equipment, medium and product, which can improve the stability and reliability of the database, thereby ensuring the normal operation of the migrated database.

[0005] To achieve the above objectives, this application provides the following solutions: In a first aspect, the present application provides a method for migrating a database, comprising: Parsing the received migration request to obtain routing configuration information and a receiving database of the migration request; Using the migration request, constructing an abstract syntax tree corresponding to the migration request; Determine a target database and a target shard table set from the routing configuration information; The migration request is modified based on the target database and the target shard table set to obtain a first migration request; Performing semantic conversion on the first migration request using the abstract syntax tree to obtain a second migration request; Optimizing the second migration request to obtain a target migration request; The target migration request is used to migrate the data in the target shard table set to the receiving database.

[0006] Optionally, using the migration request to construct an abstract syntax tree corresponding to the migration request specifically includes: Splitting the migration request to obtain a plurality of lexical units; wherein the types of the lexical units include at least a keyword type, an identifier type, a constant type, an operator type, and a delimiter type; The multiple lexical units are constructed using a recursive descent analysis method to obtain an abstract syntax tree corresponding to the migration request.

[0007] Optionally, determining a target database and a target shard table set from the routing configuration information specifically includes: If the routing configuration information is a sub-library and sub-table configuration, splitting the migration request into multiple migration sub-requests; Determining a target database from the routing configuration information; Determine, from the target database, target shard tables corresponding to the respective migration sub-requests; Prioritize multiple migration sub-requests; The target shard table is added to the target shard table set in descending order of priority of the migration sub-request corresponding to the target shard table.

[0008] Optionally, the using the abstract syntax tree to perform semantic conversion on the first migration request to obtain a second migration request specifically includes: Acquire a target SQL dialect of the target database and a receiving SQL dialect of the receiving database; Using the target SQL dialect and the received SQL dialect, determining a dialect difference comparison table; wherein the dialect difference comparison table includes at least a data type difference sub-comparison table, a built-in function difference sub-comparison table, an operator semantic difference sub-comparison table, and a transaction control statement difference sub-comparison table; Performing semantic conversion on the abstract syntax tree using the dialect difference comparison table to obtain a target abstract syntax tree; The first migration request is semantically converted using the target abstract syntax tree to obtain a second migration request; wherein the first migration request is in the same format as the target SQL dialect, and the second migration request is in the same format as the receiving SQL dialect.

[0009] Optionally, the optimizing the second migration request to obtain a target migration request specifically includes: determining common fields in the second migration request; adding index information to the common fields without index information in the second migration request to obtain a third migration request; determining an ambiguous condition in the third migration request; Determining a precise condition that matches the fuzzy condition; Using the precise condition to replace the fuzzy condition in the third migration request to obtain a fourth migration request; determining an objective function in the fourth migration request; Determining a target constant corresponding to the target function; replacing the target function in the fourth migration request with the target constant to obtain a fifth migration request; The connection information in the fifth migration request is adjusted to obtain a target migration request.

[0010] Optionally, the database migration method further includes: Acquire the target data volume in the target database and the received data volume in the receiving database; If the target data volume is the same as the received data volume, determining that the migration of the target database is completed; If the target data volume is different from the received data volume, using the target migration request, the data in the target shard table set is migrated to the receiving database, and the current data volume in the receiving database is obtained; If the target data volume is the same as the current data volume, determining that the migration of the target database is completed; If the target data volume is different from the current data volume, a handwritten SQL migration request is obtained, and the handwritten SQL migration request is used to migrate the data in the target shard table set to the receiving database.

[0011] In a second aspect, the present application provides a database migration device, comprising: A parsing unit, configured to parse the received migration request to obtain routing configuration information of the migration request and a receiving database; A construction unit, configured to use the migration request to construct an abstract syntax tree corresponding to the migration request; A determination unit, configured to determine a target database and a target shard table set from the routing configuration information; A changing unit, configured to change the migration request based on the target database and the target shard table set to obtain a first migration request; a conversion unit, configured to perform semantic conversion on the first migration request using the abstract syntax tree to obtain a second migration request; an optimization unit, configured to optimize the second migration request to obtain a target migration request; A migration unit is used to migrate the data in the target shard table set to the receiving database using the target migration request.

[0012] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-mentioned database migration methods.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described database migration methods.

[0014] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned database migration methods.

[0015] In a sixth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instructions. When the processor executes the program or instructions, the steps of the database migration method described in any one of the above are implemented.

[0016] According to the specific embodiments provided in this application, this application discloses the following technical effects: The present application provides a database migration method, device, equipment, medium and product, which can accurately identify and process various configuration information in the migration process, including routing configuration, target database and target shard table set, etc., by systematically parsing the migration request and constructing the corresponding abstract syntax tree. Through this series of refined operations, the accuracy and efficiency of the migration request are ensured. At the same time, the method further improves the flexibility and execution efficiency of the migration process by changing, semantically converting and optimizing the migration request. Finally, by using the optimized target migration request, the conversion of different syntaxes and structures between databases is realized, and the data in the target shard table set is accurately migrated to the receiving database, which can improve the stability and reliability of the database, thereby ensuring the normal operation of the migrated database. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a database migration method in an embodiment of the present application; Figure 2A schematic diagram of functional modules of a database migration device provided in one embodiment of the present application.

[0019] Figure 3 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0022] In an exemplary embodiment, Figure 1 As shown, a database migration method is provided, which is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method includes the following steps 101 to 107. Among them: Step 101: parse the received migration request to obtain routing configuration information and a receiving database of the migration request.

[0023] The embodiment of the present application can be applied to the database AI adaptation system platform. The platform is a multi-layer architecture: the design includes a multi-layer architecture including an interface layer, a conversion layer, and a data storage layer, wherein: The interface layer is responsible for receiving database operation requests from applications, which were originally designed for foreign databases.

[0024] The conversion layer is the core part, which uses the AI ​​big model to parse, convert and adapt the requests, and convert them into operations that can be correctly executed on domestic databases.

[0025] The data storage layer is a domestic database used to store migrated and converted data.

[0026] Model integration architecture: Integrate the AI ​​big model into the system conversion layer and establish a model structure related to database conversion. The conversion layer receives the request passed from the interface layer and calls the AI ​​big model for analysis and conversion adaptation. For example, it includes model submodules for identifying different database operation types (such as query, insert, update, delete), and model submodules for converting between foreign databases and domestic databases for different data types (such as character type, numeric type, date type, etc.). According to the grammatical rules of the domestic database, the AI ​​big model converts the operation statements in the style of the foreign database.

[0027] It can handle issues related to function usage and keyword differences between different databases. For example, some specific string processing functions in foreign databases have different expressions in domestic databases. The big model can automatically replace them with the corresponding functions in domestic databases, while optimizing the statement structure to improve execution efficiency.

[0028] Step 102: Use the migration request to construct an abstract syntax tree corresponding to the migration request.

[0029] In the embodiment of the present application, the migration request may be split into multiple indivisible atomic symbols, and an abstract syntax tree may be constructed based on the multiple atomic symbols.

[0030] As an optional implementation manner, the method of using the migration request in step 102 to construct an abstract syntax tree corresponding to the migration request may include: Splitting the migration request to obtain a plurality of lexical units; wherein the types of the lexical units include at least a keyword type, an identifier type, a constant type, an operator type, and a delimiter type; The multiple lexical units are constructed using a recursive descent analysis method to obtain an abstract syntax tree corresponding to the migration request.

[0031] Among them, by implementing this implementation method, by splitting the migration request into multiple lexical units and clarifying the types of these lexical units (such as keywords, identifiers, constants, operators and delimiters, etc.), the grammatical structure of the migration request can be understood and analyzed more carefully. This detailed splitting and classification provides a solid foundation for the subsequent construction of an abstract syntax tree. The use of the recursive descent analysis method can efficiently utilize these lexical units and gradually build a complete abstract syntax tree according to the grammatical rules. Not only does it improve the accuracy and efficiency of building an abstract syntax tree, it also makes the entire migration request processing process more transparent and controllable. Through the abstract syntax tree, the various components of the migration request and their interrelationships can be clearly seen, making it easier to discover and correct potential grammatical errors or logical problems.

[0032] In the embodiment of the present application, a lexical unit (Token) may include lexical units of keyword type, identifier type, constant type, operator type, and delimiter type, specifically: Keyword type: Reserved words such as SELECT, FROM, WHERE, AND, and OR have specific semantics in SQL, and each represents a different statement function or logical relationship.

[0033] Identifier type: used to represent table names, column names, etc., usually starting with a letter followed by a string of letters, numbers, and underscores, such as table_name, column1, etc.

[0034] Constant types: include numeric constants (such as the integer 123, the decimal 3.14, etc.), string constants (character sequences enclosed in single or double quotes, such as 'abc', "def"), date and time constants (following a specific date and time format), etc.

[0035] Operator type: such as arithmetic operators (+, -, *, / , etc.), comparison operators (=, >, <, >=, <=, <>, etc.), logical operators (AND, OR, NOT, etc.) and other special operators (such as LIKE, BETWEEN, etc.).

[0036] Delimiter type: Symbols such as brackets (,), commas, semicolons, etc. used to separate different grammatical components or represent specific grammatical structures. In the embodiment of the present application, a syntax tree can be constructed by processing lexical units using a syntax analyzer based on a recursive descent analysis method: Recursive descent analysis is a top-down syntax analysis method. It starts from the top-level non-terminal symbol (such as select_statement) according to the syntax rules, and gradually recursively calls functions to match the input lexical units to build the nodes of the syntax tree. For example, for a SELECT statement, there will be a corresponding function to process the select_statement rule. In this function, the functions corresponding to the sub-rules such as select_list, FROM, and WHERE are called in sequence. Each function performs corresponding processing according to the current lexical unit to be matched. If the match is successful, the corresponding syntax tree node is created and the analysis continues downward. If a match fails, it indicates a syntax error. This method is simple and intuitive, easy to understand and implement.

[0037] Step 103: determine the target database and the target shard table set from the routing configuration information.

[0038] In the embodiment of the present application, the shard table refers to those tables with a large amount of original data, which need to be split into multiple tables according to certain rules. In this way, each shard table (i.e., the data portion after segmentation) contains a part of the data of the original table, and all shard tables together constitute a complete data set. The sharding dimension may be time or region, etc.

[0039] As an optional implementation, the method of determining the target database and the target shard table set from the routing configuration information in step 103 may include: If the routing configuration information is a sub-library and sub-table configuration, splitting the migration request into multiple migration sub-requests; Determining a target database from the routing configuration information; Determine, from the target database, target shard tables corresponding to the respective migration sub-requests; Prioritize multiple migration sub-requests; The target shard table is added to the target shard table set in descending order of priority of the migration sub-request corresponding to the target shard table.

[0040] Among them, by implementing this implementation method, by splitting the migration request into multiple migration sub-requests, the refined management and execution of large-scale migration tasks are realized. This strategy not only improves the flexibility and scalability of the migration process, but also enables each migration sub-request to operate on a specific target database and target shard table, thereby effectively reducing resource consumption and potential conflicts in the migration process. At the same time, the target database is accurately determined from the routing configuration information, and the target shard table corresponding to each migration sub-request is further determined in the target database, ensuring the accuracy and pertinence of the migration task. In addition, by determining the priorities of multiple migration sub-requests and adding the target shard table to the target shard table set in order of priority, this implementation method also realizes the optimized scheduling of migration tasks, so that high-priority migration tasks can be processed first, thereby further improving migration efficiency and user experience.

[0041] Step 104: modify the migration request based on the target database and the target shard table set to obtain a first migration request.

[0042] In the embodiment of the present application, the migration request may be modified by replacing the original database or shard table information in the migration request with a suitable target database and a target shard table in the target shard table set, so that the first migration request obtained can be executed on the correct target database and the correct target shard table. The modification process may include replacing the table name, adding a shard key condition, etc.

[0043] Step 105: Use the abstract syntax tree to perform semantic conversion on the first migration request to obtain a second migration request.

[0044] As an optional implementation manner, step 105 may include: performing semantic conversion on the first migration request using the abstract syntax tree to obtain the second migration request by: Acquire a target SQL dialect of the target database and a receiving SQL dialect of the receiving database; Using the target SQL dialect and the received SQL dialect, determining a dialect difference comparison table; wherein the dialect difference comparison table includes at least a data type difference sub-comparison table, a built-in function difference sub-comparison table, an operator semantic difference sub-comparison table, and a transaction control statement difference sub-comparison table; Performing semantic conversion on the abstract syntax tree using the dialect difference comparison table to obtain a target abstract syntax tree; The first migration request is semantically converted using the target abstract syntax tree to obtain a second migration request; wherein the first migration request is in the same format as the target SQL dialect, and the second migration request is in the same format as the receiving SQL dialect.

[0045] Among them, implementing this implementation method and using the dialect difference comparison table to perform semantic conversion on the abstract syntax tree not only ensures the accuracy and consistency of the conversion process, but also effectively reduces the risk of migration errors or performance degradation caused by dialect differences. Through this conversion, we obtain a target abstract syntax tree that is compatible with both the target database and the receiving database. Through the target abstract syntax tree that is compatible with both the target database and the receiving database, the target database can be made compatible with the receiving database, thereby improving the accuracy of the second migration request.

[0046] In the embodiment of the present application, the dialect difference comparison table is a data mapping relationship established between the target SQL dialect and the receiving SQL dialect. Based on the dialect difference comparison table, accurate conversion between the target SQL dialect and the receiving SQL dialect can be achieved.

[0047] In the embodiment of the present application, the data type difference sub-comparison table: different databases may have different names or precision ranges for the same type; Comparison table of built-in function differences: differences in name, number of parameters, function implementation, etc. Operator semantic difference comparison table: For example, the logical operator && represents the AND operation in some databases, but may not be supported in other databases; Transaction control statement difference comparison table: different ways of writing statements such as commit and rollback.

[0048] For example, make necessary adjustments to the names of objects such as table names and column names in the target SQL dialect according to the naming conventions of the receiving SQL dialect, replace the functions used in the target SQL dialect with equivalent functions in the receiving SQL dialect, and convert operators according to the semantics of the receiving SQL dialect. For example, convert specific date and time operators in the target SQL dialect into corresponding forms that can achieve the same functions in the receiving SQL dialect, ensuring that the converted statements are semantically consistent with the source statements, but meet the semantic requirements of the receiving SQL dialect.

[0049] Step 106: Optimize the second migration request to obtain a target migration request.

[0050] In the embodiment of the present application, the AI ​​big model can be used to perform performance analysis on database operations and optimize query plans and data access paths.

[0051] As an optional implementation manner, step 106 optimizes the second migration request to obtain the target migration request, which may include: determining common fields in the second migration request; adding index information to the common fields without index information in the second migration request to obtain a third migration request; determining an ambiguous condition in the third migration request; Determining a precise condition that matches the fuzzy condition; Using the precise condition to replace the fuzzy condition in the third migration request to obtain a fourth migration request; determining an objective function in the fourth migration request; Determining a target constant corresponding to the target function; replacing the target function in the fourth migration request with the target constant to obtain a fifth migration request; The connection information in the fifth migration request is adjusted to obtain a target migration request.

[0052] Among them, implementing this implementation method, by adding indexes to commonly used fields, replacing fuzzy conditions with precise conditions, and replacing functions with constants, can reduce unclear or difficult-to-understand fields in migration requests and improve the accuracy of target migration requests during the data migration process.

[0053] In the embodiment of the present application, frequently used common fields in the second migration request are analyzed, such as fields in the WHERE clause and the JOIN clause. If these fields are not indexed, the database may perform a full table scan when executing the query, resulting in poor performance. Depending on the query frequency and data volume, adding indexes to these key fields can significantly improve the query speed.

[0054] For example, for "SELECT * FROM products WHERE category_id=1", if the category_id field has no index, the query may traverse the entire products table. After adding the index, the database can directly locate the record with category_id 1, greatly reducing the query time.

[0055] In an embodiment of the present application, in order to ensure that the screening conditions of the migration request are as accurate as possible and avoid unnecessary data returns, a specific range condition (i.e., a precise condition) can be used instead of a fuzzy condition, or IN can be used instead of multiple OR conditions. The OR condition in the migration request can be determined as a fuzzy condition. The precise condition can be determined by converting the OR condition into an IN condition corresponding to the OR condition, and then the IN condition can be determined as a precise condition corresponding to the fuzzy condition. The method of converting the OR condition into an IN condition corresponding to the OR condition can be achieved by a pre-trained condition conversion model, or it can be converted manually.

[0056] For example, "SELECT * FROM employees WHERE salary>5000 AND salary<10000" is more precise than "SELECT * FROM employees WHERE salary BETWEEN 0 AND 10000", which reduces unnecessary data retrieval.

[0057] In the embodiment of the present application, in order to avoid using complex functions and expressions in the migration request (because this may cause index failure), consider applying the function to the constant, or using the function index provided by the database (if supported).

[0058] For example, for "SELECT * FROM users WHERE YEAR (birth_date) = 1990", the YEAR function applied to the birth_date field will cause the index to fail. You can rewrite it as "SELECT * FROM users WHEREbirth_date BETWEEN '1990-01-01' AND '1990-12-31'", which can use the index of the birth_date field.

[0059] In the embodiment of the present application, the connection information in the fifth migration request may be adjusted in the following manner: 1. Choose the right connection type Select the correct join type (such as INNER JOIN, LEFT JOIN, RIGHT JOIN, etc.) according to the requirements of the fifth migration request. Ensure that the join conditions are correct to avoid Cartesian products (i.e., tables are joined without correct join conditions, resulting in an abnormally large amount of data in the result set).

[0060] For example, when querying user orders and user information, if you only care about the information of users who have orders, it is appropriate to use INNER JOIN; if you need to include all user information, even if there are no orders, you should use LEFT JOIN.

[0061] 2. Connection order optimization When multiple tables are connected, adjusting the table connection order may affect query performance. Generally, placing the table with smaller data volume at the front, or placing the table with the most stringent filtering conditions at the front, can reduce the size of the intermediate result set and thus improve query speed.

[0062] For example, according to the index structure and data distribution characteristics of domestic databases, the fifth migration request is adjusted to improve query speed. At the same time, frequently executed operations are cached and optimized to reduce unnecessary database load and improve the overall performance of the system.

[0063] Step 107: Use the target migration request to migrate the data in the target shard table set to the receiving database.

[0064] In an embodiment of the present application, if a request involves multiple shard tables or multiple database nodes, it is necessary to merge the data in the target shard table set. The merging process may include operations such as sorting, grouping, and merging to ensure that the data ultimately migrated to the receiving database is correct. The merged data is encoded into the client protocol format of the receiving database, and the encoded result is transmitted to the client of the receiving database via the network to complete the execution process of the entire request.

[0065] As an optional implementation, after step 107, the following steps may also be performed: Acquire the target data volume in the target database and the received data volume in the receiving database; If the target data volume is the same as the received data volume, determining that the migration of the target database is completed; If the target data volume is different from the received data volume, using the target migration request, the data in the target shard table set is migrated to the receiving database, and the current data volume in the receiving database is obtained; If the target data volume is the same as the current data volume, determining that the migration of the target database is completed; If the target data volume is different from the current data volume, a handwritten SQL migration request is obtained, and the handwritten SQL migration request is used to migrate the data in the target shard table set to the receiving database.

[0066] Among them, by implementing this implementation, by obtaining the target data volume in the target database and the received data volume in the receiving database and comparing them, this implementation provides an intuitive and effective migration completion check mechanism. When the data volumes of the two are the same, it can be confirmed that the migration of the target database has been completed, which greatly improves the reliability and accuracy of the migration process.

[0067] For example, by establishing a data verification and synchronization mechanism, the consistency of data during the conversion from a foreign database (target database) to a domestic database (receiving database) is ensured; Each migration request entered by the user will leave a trace. Depending on the type of request, the amount of data in the domestic database will be queried at regular intervals and compared with the number of requests initiated. For execution failures (different numbers) and abnormal SQL execution, the data will be saved in the abnormal log. It can be repaired manually or by AI large model repair and then executed again to ensure eventual consistency.

[0068] Use AI big models to monitor and compare data in real time, monitor the execution process of user requests, and if a system error occurs during the execution process, capture the exception, call the AI ​​big model to correct the statement, and execute the SQL statement again. If an error occurs again, add it to the system log and perform log alarm operations. Users can manually rewrite the correct SQL in the log management interface and add it to the big data model. The next time the same SQL is executed, it can be repaired normally, and possible data loss, duplication or inconsistency problems can be discovered and handled in a timely manner. For example, after batch data migration, integrity checks and verifications are performed on key data.

[0069] Implementing the above steps 101 to 107 can improve the stability and reliability of the database, thereby ensuring the normal operation of the database after migration. In addition, the present application can also improve the accuracy and efficiency of constructing an abstract syntax tree. In addition, the present application can also achieve optimized scheduling of migration tasks. In addition, the present application can also improve the accuracy of the second migration request. In addition, the present application can also improve the accuracy of the target migration request during the data migration process. In addition, the present application can also improve the reliability and accuracy of the migration process.

[0070] Based on the same inventive concept, the embodiment of the present application also provides a database migration device for implementing the database migration method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more database migration device embodiments provided below can refer to the limitations of the database migration method above, and will not be repeated here.

[0071] In an exemplary embodiment, Figure 2 As shown, a database migration device is provided, comprising: The parsing unit 201 is used to parse the received migration request to obtain the routing configuration information and the receiving database of the migration request; A construction unit 202, configured to use the migration request to construct an abstract syntax tree corresponding to the migration request; A determination unit 203, configured to determine a target database and a target shard table set from the routing configuration information; A changing unit 204 is used to change the migration request based on the target database and the target shard table set to obtain a first migration request; A conversion unit 205 is configured to perform semantic conversion on the first migration request using the abstract syntax tree to obtain a second migration request; An optimization unit 206, configured to optimize the second migration request to obtain a target migration request; The migration unit 207 is configured to use the target migration request to migrate the data in the target shard table set to the receiving database.

[0072] As an optional implementation manner, the construction unit 202 uses the migration request to construct an abstract syntax tree corresponding to the migration request in the following manner: Splitting the migration request to obtain a plurality of lexical units; wherein the types of the lexical units include at least a keyword type, an identifier type, a constant type, an operator type, and a delimiter type; The multiple lexical units are constructed using a recursive descent analysis method to obtain an abstract syntax tree corresponding to the migration request.

[0073] Among them, by implementing this implementation method, by splitting the migration request into multiple lexical units and clarifying the types of these lexical units (such as keywords, identifiers, constants, operators and delimiters, etc.), the grammatical structure of the migration request can be understood and analyzed more carefully. This detailed splitting and classification provides a solid foundation for the subsequent construction of an abstract syntax tree. The use of the recursive descent analysis method can efficiently utilize these lexical units and gradually build a complete abstract syntax tree according to the grammatical rules. Not only does it improve the accuracy and efficiency of building an abstract syntax tree, it also makes the entire migration request processing process more transparent and controllable. Through the abstract syntax tree, the various components of the migration request and their interrelationships can be clearly seen, making it easier to discover and correct potential grammatical errors or logical problems.

[0074] As an optional implementation manner, the determination unit 203 determines the target database and the target shard table set from the routing configuration information in the following manner: If the routing configuration information is a sub-library and sub-table configuration, splitting the migration request into multiple migration sub-requests; Determining a target database from the routing configuration information; Determine, from the target database, target shard tables corresponding to the respective migration sub-requests; Prioritize multiple migration sub-requests; The target shard table is added to the target shard table set in descending order of priority of the migration sub-request corresponding to the target shard table.

[0075] Among them, by implementing this implementation method, by splitting the migration request into multiple migration sub-requests, the refined management and execution of large-scale migration tasks are realized. This strategy not only improves the flexibility and scalability of the migration process, but also enables each migration sub-request to operate on a specific target database and target shard table, thereby effectively reducing resource consumption and potential conflicts in the migration process. At the same time, the target database is accurately determined from the routing configuration information, and the target shard table corresponding to each migration sub-request is further determined in the target database, ensuring the accuracy and pertinence of the migration task. In addition, by determining the priorities of multiple migration sub-requests and adding the target shard table to the target shard table set in order of priority, this implementation method also realizes the optimized scheduling of migration tasks, so that high-priority migration tasks can be processed first, thereby further improving migration efficiency and user experience.

[0076] As an optional implementation manner, the conversion unit 205 uses the abstract syntax tree to perform semantic conversion on the first migration request to obtain the second migration request in the following manner: Acquire a target SQL dialect of the target database and a receiving SQL dialect of the receiving database; Using the target SQL dialect and the received SQL dialect, determining a dialect difference comparison table; wherein the dialect difference comparison table includes at least a data type difference sub-comparison table, a built-in function difference sub-comparison table, an operator semantic difference sub-comparison table, and a transaction control statement difference sub-comparison table; Performing semantic conversion on the abstract syntax tree using the dialect difference comparison table to obtain a target abstract syntax tree; The first migration request is semantically converted using the target abstract syntax tree to obtain a second migration request; wherein the first migration request is in the same format as the target SQL dialect, and the second migration request is in the same format as the receiving SQL dialect.

[0077] Among them, implementing this implementation method and using the dialect difference comparison table to perform semantic conversion on the abstract syntax tree not only ensures the accuracy and consistency of the conversion process, but also effectively reduces the risk of migration errors or performance degradation caused by dialect differences. Through this conversion, we obtain a target abstract syntax tree that is compatible with both the target database and the receiving database, thereby improving the accuracy of the second migration request.

[0078] As an optional implementation manner, the optimization unit 206 optimizes the second migration request to obtain the target migration request in the following manner: determining common fields in the second migration request; adding index information to the common fields without index information in the second migration request to obtain a third migration request; determining an ambiguous condition in the third migration request; Determining a precise condition that matches the fuzzy condition; Using the precise condition to replace the fuzzy condition in the third migration request to obtain a fourth migration request; determining an objective function in the fourth migration request; Determining a target constant corresponding to the target function; replacing the target function in the fourth migration request with the target constant to obtain a fifth migration request; The connection information in the fifth migration request is adjusted to obtain a target migration request.

[0079] Among them, implementing this implementation method, by adding indexes to commonly used fields, replacing fuzzy conditions with precise conditions, and replacing functions with constants, can reduce unclear or difficult-to-understand fields in migration requests and improve the accuracy of target migration requests during the data migration process.

[0080] As an optional implementation manner, the migration unit 207 is further configured to: Acquire the target data volume in the target database and the received data volume in the receiving database; If the target data volume is the same as the received data volume, determining that the migration of the target database is completed; If the target data volume is different from the received data volume, using the target migration request, the data in the target shard table set is migrated to the receiving database, and the current data volume in the receiving database is obtained; If the target data volume is the same as the current data volume, determining that the migration of the target database is completed; If the target data volume is different from the current data volume, a handwritten SQL migration request is obtained, and the handwritten SQL migration request is used to migrate the data in the target shard table set to the receiving database.

[0081] Among them, by implementing this implementation, by obtaining the target data volume in the target database and the received data volume in the receiving database and comparing them, this implementation provides an intuitive and effective migration completion check mechanism. When the data volumes of the two are the same, it can be confirmed that the migration of the target database has been completed, which greatly improves the reliability and accuracy of the migration process.

[0082] The implementation of the above implementation method can improve the stability and reliability of the database, thereby ensuring the normal operation of the database after migration. In addition, the present application can also improve the accuracy and efficiency of constructing an abstract syntax tree. In addition, the present application can also achieve optimized scheduling of migration tasks. In addition, the present application can also improve the accuracy of the second migration request. In addition, the present application can also improve the accuracy of the target migration request during the data migration process. In addition, the present application can also improve the reliability and accuracy of the migration process.

[0083] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 3As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store database migration data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a database migration method is implemented.

[0084] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0085] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0086] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0087] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0088] In an exemplary embodiment, a chip is provided, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps in the above-mentioned method embodiments and achieve the same technical effects. To avoid repetition, they are not described here.

[0089] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0090] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0091] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0092] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0093] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A database migration method, characterized in that: The database migration method includes: Parsing the received migration request to obtain routing configuration information and a receiving database of the migration request; Using the migration request, constructing an abstract syntax tree corresponding to the migration request; Determine a target database and a target shard table set from the routing configuration information; Modify the migration request based on the target database and the target shard table set to obtain a first migration request; Performing semantic conversion on the first migration request using the abstract syntax tree to obtain a second migration request; Optimizing the second migration request to obtain a target migration request; The target migration request is used to migrate the data in the target shard table set to the receiving database.

2. The database migration method according to claim 1, characterized in that: The using the migration request to construct an abstract syntax tree corresponding to the migration request specifically includes: Splitting the migration request to obtain a plurality of lexical units; wherein the types of the lexical units include at least a keyword type, an identifier type, a constant type, an operator type, and a delimiter type; The multiple lexical units are constructed using a recursive descent analysis method to obtain an abstract syntax tree corresponding to the migration request.

3. The database migration method according to claim 1, characterized in that: The determining of the target database and the target shard table set from the routing configuration information specifically includes: If the routing configuration information is a sub-library and sub-table configuration, splitting the migration request into multiple migration sub-requests; Determining a target database from the routing configuration information; Determine, from the target database, target shard tables corresponding to the respective migration sub-requests; Prioritize multiple migration sub-requests; The target shard table is added to the target shard table set in descending order of priority of the migration sub-request corresponding to the target shard table.

4. The database migration method according to claim 3, characterized in that: The using the abstract syntax tree to perform semantic conversion on the first migration request to obtain a second migration request specifically includes: Acquire a target SQL dialect of the target database and a receiving SQL dialect of the receiving database; Using the target SQL dialect and the received SQL dialect, determining a dialect difference comparison table; wherein the dialect difference comparison table includes at least a data type difference sub-comparison table, a built-in function difference sub-comparison table, an operator semantic difference sub-comparison table, and a transaction control statement difference sub-comparison table; Performing semantic conversion on the abstract syntax tree using the dialect difference comparison table to obtain a target abstract syntax tree; The first migration request is semantically converted using the target abstract syntax tree to obtain a second migration request; wherein the first migration request is in the same format as the target SQL dialect, and the second migration request is in the same format as the receiving SQL dialect.

5. The database migration method according to claim 3, characterized in that: The optimizing the second migration request to obtain a target migration request specifically includes: determining common fields in the second migration request; adding index information to the common fields without index information in the second migration request to obtain a third migration request; determining an ambiguous condition in the third migration request; Determining a precise condition that matches the fuzzy condition; Using the precise condition to replace the fuzzy condition in the third migration request to obtain a fourth migration request; determining an objective function in the fourth migration request; Determining a target constant corresponding to the target function; replacing the target function in the fourth migration request with the target constant to obtain a fifth migration request; The connection information in the fifth migration request is adjusted to obtain a target migration request.

6. The database migration method according to any one of claims 1 to 5, characterized in that: The database migration method also includes: Acquire the target data volume in the target database and the received data volume in the receiving database; If the target data volume is the same as the received data volume, determining that the migration of the target database is completed; If the target data volume is different from the received data volume, using the target migration request, the data in the target shard table set is migrated to the receiving database, and the current data volume in the receiving database is obtained; If the target data volume is the same as the current data volume, determining that the migration of the target database is completed; If the target data volume is different from the current data volume, a handwritten SQL migration request is obtained, and the handwritten SQL migration request is used to migrate the data in the target shard table set to the receiving database.

7. A database migration device, characterized in that: The database migration device comprises: A parsing unit, configured to parse the received migration request to obtain routing configuration information of the migration request and a receiving database; A construction unit, configured to use the migration request to construct an abstract syntax tree corresponding to the migration request; A determination unit, configured to determine a target database and a target shard table set from the routing configuration information; A changing unit, configured to change the migration request based on the target database and the target shard table set to obtain a first migration request; a conversion unit, configured to perform semantic conversion on the first migration request using the abstract syntax tree to obtain a second migration request; an optimization unit, configured to optimize the second migration request to obtain a target migration request; A migration unit is used to migrate the data in the target shard table set to the receiving database using the target migration request.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the database migration method described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the database migration method described in any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the database migration method described in any one of claims 1 to 6 are implemented.

Images