Database migration and grammar transformation method, device and equipment

Abstract

The invention discloses a database migration and grammar transformation method, device and equipment, and the method comprises the steps: firstly responding to the login operation of a target user on a data architecture management platform, verifying whether the target user has the authority of database migration, and if yes, obtaining original database attribute information through the data architecture management platform; and according to the attribute information of the original database, establishing corresponding attribute information in the target database. The method comprises the following steps: firstly, acquiring an SQL execution statement, then synchronizing data in an original database to a target database through a data warehouse tool of a distributed system, then acquiring an interface and the corresponding SQL execution statement through a log pipeline recording platform, and performing grammar optimization on the SQL execution statement by utilizing a pre-trained large model to obtain an optimization result. Therefore, the efficiency and accuracy of database migration and grammar transformation can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and equipment for database migration and grammar transformation. Background Art

[0002] In recent years, data security and privacy protection issues have become increasingly prominent in the banking and other financial fields. In order to enhance security, reduce costs, improve data reliability, support business expansion and optimize data management, database migration is usually required. At the same time, due to the large differences in syntax and performance of different databases, applications often require more syntax modifications to complete database migration.

[0003] At present, there are two commonly used methods for data migration and syntax modification: one is to export the original database data first, then import it into the new database through the storage device, and then manually modify the database syntax and performance; the other is to use the migration tool that comes with the database to import the data from the source database to the destination database, and then use the SQL optimization tool to get the optimization suggestions and manually rewrite the SQL syntax. However, the data conversion process of these data migration methods is relatively complicated, and it is easy to cause data encoding confusion due to the inconsistent encoding formats of different databases. Moreover, different migration tools generally only support one database, which is not suitable for projects with multiple data migrations. In addition, for projects with a particularly large database inventory, the commonly used database migration tools have a slow migration speed and a long data export cycle, which will have a greater impact on the performance of the database in use. Moreover, due to the different syntaxes of different databases, the application involves a lot of syntax modification and SQL statement query performance modification after migration to different databases, which seriously affects the efficiency of database migration. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a method, device and equipment for database migration and grammar transformation, which can effectively improve the efficiency and accuracy of database migration and grammar transformation.

[0005] In a first aspect, an embodiment of the present application provides a database migration and grammar transformation method, including:

[0006] In response to a target user's login operation to the data architecture management platform, verifying whether the target user has permission to perform database migration;

[0007] If so, the original database attribute information is obtained through the data architecture management platform; and corresponding attribute information is established in the target database according to the original database attribute information;

[0008] Use the distributed system's data warehouse tool to synchronize the data in the original database to the destination database;

[0009] The interface and the corresponding SQL execution statement are obtained through the log flow recording platform, and the SQL execution statement is grammatically optimized using the pre-trained large model to obtain the optimization result.

[0010] Optionally, the acquiring original database attribute information through the data architecture management platform; and establishing corresponding attribute information in the destination database according to the source database attribute information, includes:

[0011] Obtaining table creation statements, views, and functions of the original database through the data architecture management platform;

[0012] The keywords of the table creation statements of the original database are replaced, and corresponding functions, sequences, views, and stored procedures are established in the target database according to the replaced table creation statements, views, and functions, while ensuring that the number of tables and the fields of each table in the target database are consistent.

[0013] Optionally, synchronizing the data in the original database to the destination database through a data warehouse tool of the distributed system includes:

[0014] Synchronize the data of the original database to the data warehouse tool of the distributed system;

[0015] The data in the data warehouse tool of the distributed system is synchronized to the target database, and the synchronization result is automatically verified.

[0016] Optionally, the interface and the corresponding SQL execution statement are obtained through the log flow recording platform, and the SQL execution statement is grammatically optimized using a pre-trained large model to obtain an optimization result, including:

[0017] Obtain the interface and the corresponding SQL execution statement through the log flow recording platform, and use the pre-trained large model to optimize the syntax and performance of the SQL execution statement to obtain the optimized SQL execution statement;

[0018] The optimized SQL execution statement is written back into the program, and function and performance tests are automatically performed according to the interface list. After the test passes, the syntax transformation of the SQL execution statement is completed to obtain the optimization result.

[0019] In a second aspect, the embodiment of the present application further provides a database migration and grammar transformation device, including:

[0020] A verification unit, configured to verify, in response to a target user's login operation to the data architecture management platform, whether the target user has the authority to perform database migration;

[0021] An establishing unit, configured to obtain original database attribute information through the data architecture management platform if it is verified that the target user has the authority to perform database migration; and to establish corresponding attribute information in the target database according to the source database attribute information;

[0022] The synchronization unit is used to synchronize the data in the original database to the destination database through the data warehouse tool of the distributed system;

[0023] The optimization unit is used to obtain the interface and the corresponding SQL execution statement through the log flow recording platform, and use the pre-trained large model to perform syntax optimization on the SQL execution statement to obtain the optimization result.

[0024] Optionally, the establishing unit includes:

[0025] An acquisition subunit is used to acquire table creation statements, views, and functions of the original database through the data architecture management platform;

[0026] A subunit is established to replace keywords in the table creation statements of the original database, and to establish corresponding functions, sequences, views, and stored procedures in the target database according to the replaced table creation statements, views, and functions, while ensuring that the number of tables and the fields of each table in the target database are consistent.

[0027] Optionally, the synchronization unit includes:

[0028] A first synchronization subunit is used to synchronize the data of the original database to the data warehouse tool of the distributed system;

[0029] The second synchronization subunit is used to synchronize the data in the data warehouse tool of the distributed system to the target database and automatically verify the synchronization result.

[0030] Optionally, the optimization unit includes:

[0031] The optimization subunit is used to obtain the interface and the corresponding SQL execution statement through the log flow recording platform, and optimize the syntax and performance of the SQL execution statement by using the pre-trained large model to obtain the optimized SQL execution statement;

[0032] The write-back subunit is used to write the optimized SQL execution statement back to the program, and automatically perform function and performance tests according to the interface list, and complete the syntax transformation of the SQL execution statement after the test passes to obtain the optimization result.

[0033] The embodiment of the present application also provides a database migration and grammar transformation device, including: a processor, a memory, and a system bus;

[0034] The processor and the memory are connected via the system bus;

[0035] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementations of the above-mentioned database migration and grammar transformation methods.

[0036] An embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned database migration and grammar transformation methods.

[0037] The embodiment of the present application provides a method, device and equipment for database migration and grammar transformation. First, in response to the target user's login operation on the data architecture management platform, it verifies whether the target user has the authority to perform database migration. If so, the original database attribute information is obtained through the data architecture management platform; and according to the original database attribute information, the corresponding attribute information is established in the target database. Then, the data in the original database is synchronized to the target database through the data warehouse tool of the distributed system. Then, the interface and the corresponding SQL execution statement are obtained through the log flow recording platform, and the SQL execution statement is grammatically optimized using the pre-trained large model to obtain the optimization result. Thereby, the efficiency and accuracy of database migration and grammar transformation can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 A flowchart of a database migration and grammar transformation method provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of the composition of a database migration and grammar transformation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In the banking and other financial fields, data security and privacy protection issues are becoming increasingly prominent. In order to enhance security, reduce costs, improve data reliability, support business expansion and optimize data management, database migration is usually required. For example, many large banks have gradually switched their internal databases from foreign databases to domestic databases. At the same time, because different databases have different syntax and performance, applications often need more syntax modifications to complete database migration.

[0042] At present, there are two commonly used methods for data migration and syntax modification. One is to export the original database data first, then import it into the new database through the storage device, and then manually modify the database syntax and performance; the other method is to use the migration tool to import the data from the source database to the destination database, and then use the SQL optimization tool to get the optimization suggestions and manually rewrite the SQL syntax. The overall data conversion process of these two methods is complicated. At the same time, the inconsistent encoding formats of different databases can easily lead to data encoding confusion; and different migration tools generally only support one database, which is not suitable for projects with multiple data migrations; because the data volume of the systems of large banks and other financial institutions is relatively large, the export data cycle is long, which will have a greater impact on the performance of the database in use; finally, due to the different syntax of different databases, migration to different databases involves a lot of syntax and SQL statement query performance modifications, which seriously affects the efficiency of database migration.

[0043] To solve the above defects, the embodiment of the present application provides a method for database migration and grammar transformation. First, in response to the target user's login operation on the data architecture management platform, it is verified whether the target user has the authority to perform database migration. If so, the original database attribute information is obtained through the data architecture management platform; and according to the original database attribute information, the corresponding attribute information is established in the target database. Then, through the data warehouse tool of the distributed system, the data in the original database is synchronized to the target database. Then, the interface and the corresponding SQL execution statement are obtained through the log flow recording platform, and the SQL execution statement is optimized for syntax using the pre-trained large model to obtain the optimization result. Thereby, the efficiency and accuracy of database migration and grammar transformation can be effectively improved.

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] First embodiment

[0046] See also Figure 1 , is a flow chart of a database migration and grammar transformation method provided in this embodiment, the method comprising the following steps:

[0047] S101: In response to a target user's login operation to a data architecture management platform, verify whether the target user has permission to perform database migration.

[0048] In this embodiment, any user who uses this embodiment to perform database migration and grammar transformation is defined as a target user. In order to effectively improve the efficiency and accuracy of database migration and grammar transformation, the target user first needs to log in (such as entering the system number to log in) to the data architecture management platform and select the system and target database type. Then, in response to the target user's login operation on the data architecture management platform, it is verified whether the user has the authority to migrate the system database. If not, it is directly rejected; if so, the subsequent step S102 is continued.

[0049] S102: If yes, the original database attribute information is obtained through the data architecture management platform; and corresponding attribute information is established in the target database according to the original database attribute information.

[0050] In this embodiment, if it is verified through step S101 that the target user has the authority to perform database migration, the attribute information such as the table creation statement, view, function, etc. of the original database can be obtained through the data architecture management platform. Then it is determined whether the target user can normally access the original database address, user name, and password. If it can be accessed normally, it is also necessary to determine whether the target user can normally access the target database through the user name and password. If it can be accessed normally, the keyword replacement of the table creation statement of the original database can be further performed according to the target database type selected by the target user, and according to the replaced table creation statement, view, and function, the table is created in the target database, and the corresponding function, sequence, view, stored procedure, etc. are established, and the default value of the table creation statement is set. At the same time, it is also necessary to count the number of tables in the target database and ensure that the number of tables in the target database and the fields of each table are consistent. If they are inconsistent, it is necessary to repeat the above steps until the number of tables in the target database and the fields of each table are consistent, and then end.

[0051] In this way, through a unified data architecture management system, the target user only needs to enter the system number to log in and after successful verification, the data structure of the source database can be migrated to the target database with one click, and the problem of incompatibility of views, functions, stored procedures, and keywords can be solved. It is convenient and fast and reduces the probability of migration errors.

[0052] S103: Synchronize the data in the original database to the target database through the data warehouse tool of the distributed system.

[0053] In this embodiment, after establishing the attribute information such as functions, sequences, views, stored procedures, etc. in the target database through step S102, the data of the original database can be further synchronized to the data warehouse tool of the distributed system, and then the data in the data warehouse tool of the distributed system can be synchronized to the target database, and the synchronization results can be automatically verified to execute the subsequent step S104.

[0054] Among them, it should be noted that the reason why the data warehouse tool of a distributed system is adopted in this application is that the data warehouse tool based on a distributed system can decompose the processing task of a large data set into multiple small tasks by utilizing the advantages of parallel computing and distributed storage of the distributed system, and execute them in parallel on multiple nodes in the cluster, thereby improving the speed of data processing and maintaining high efficiency; and it has strong scalability and can cope with large-scale data transmission; at the same time, the data warehouse tool based on a distributed system has high fault tolerance and flexibility, so that data can be efficiently moved between different databases.

[0055] Specifically, the specific implementation process of step S103 may include the following nine steps:

[0056] (1) Select the database, database user, and data table to be migrated. If no data table is selected, all data tables of the selected user in the database will be used by default.

[0057] (2) Synchronize the table structure to the data warehouse tool based on the distributed system. The system automatically uses the table structure of the original database to create a table in the data warehouse tool. After the system completes the table creation, it automatically counts the number of corresponding tables and whether the fields correspond.

[0058] (3) Synchronize data to the data warehouse tool and import the data in the original database into the database table of the data warehouse tool. The system automatically counts whether the data volume of the corresponding data is consistent. If it is consistent, the system continues and records the current time point.

[0059] (4) Select the target database, database type, and database user, and synchronize the corresponding table structure to the selected database, and count the number of corresponding tables and whether the fields correspond.

[0060] (5) Synchronize the data in the data warehouse tool database to the target database.

[0061] (6) Deactivate the original database, record the current time point, and then import the data generated from the migration time point to the current time point into the destination database.

[0062] (7) Compare the table name, data quantity, and table size of the source database with the table name, data quantity, and table size of the target database.

[0063] (8) Each table is sampled and checked according to the primary key.

[0064] (9) Check the data encoding of the target database and the original database. If the encoding is the same, compare the encoding of the target database and the Chinese fields. If there is no encoding problem, the data migration is successful.

[0065] In this way, database synchronization through the data warehouse tool database does not care about the type of the target database, and data can be quickly and completely synchronized from the source database to the target database, overcoming the difficulty that traditional migration tools can only migrate to a single target database, and has strong adaptability and versatility; at the same time, the data warehouse tool database based on the distributed system has the functions of deduplication and incremental data insertion, which can ensure the integrity and correctness of data during data synchronization, and avoid problems such as data duplication and data loss in traditional migration methods; at the same time, the data warehouse tool of the distributed system supports data transmission of large data volumes.

[0066] S104: Obtain the interface and the corresponding SQL execution statement through the log flow recording platform, and use the pre-trained large model to perform syntax optimization on the SQL execution statement to obtain the optimization result.

[0067] In this embodiment, after synchronizing the data in the original database to the destination database using the data warehouse tool of the distributed system through step S103, the interface and the corresponding SQL execution statement can be further obtained through the log flow recording platform, and the SQL execution statement can be optimized in syntax and performance using the pre-trained large model to obtain the optimized SQL execution statement, and then the optimized SQL execution statement is written back to the program, and functional and performance tests are automatically performed according to the interface list. After the test passes, the syntax transformation of the SQL execution statement is completed to obtain the optimization result.

[0068] Among them, it should be noted that the present application does not limit the composition structure and training method of the large model. For example, deep learning, random forest and neural network technology can be combined with various large model platforms to train SQL query accuracy and performance optimization. In addition, it should be noted that the reason why this application adopts a large model is that the large model has a strong transfer learning ability, can understand and parse complex SQL statements, optimize query plans, and process large-scale data sets. By training a large model, it can better understand and process the various characteristics and patterns of the SQL language. At the same time, the large model can learn more feature information by training a large amount of SQL data. When processing SQL queries, optimization and prediction tasks, the large model can understand and analyze data more accurately and give more accurate results.

[0069] Next, the training process of the large model will be introduced in detail:

[0070] First of all, before training a large model, data preparation and preprocessing are required. Specifically, the interface name, interface execution time, and SQL statements executed by the interface can be obtained through the log flow recording platform within banks and other financial institutions, and the data can be cleaned to remove duplicate, erroneous, or incomplete data. Then, the data is divided into SQL syntax training sets and SQL performance training sets according to the interface execution time. Perform lexical analysis on the SQL statement to obtain the select part, table part, where condition part, ORDER part, restriction condition part, join connection part, and subquery part in the SQL statement, and map them to corresponding relationships.

[0071] Then, based on the SQL grammar training set, you can use deep learning and random forest methods to combine to perform grammatical correctness training. In this case, you input the SQL grammar training set, use a large model for training, use cross-validation or early stopping to prevent overfitting, and use indicators such as precision, recall, and F1 score to evaluate the performance of the model. The main sequence steps can be divided into the following five steps:

[0072] (1) Extraction function

[0073] There are more than 200 original database functions sorted out, and the target database functions are counted respectively; the original database functions are selected into equivalent replacement function sets, partially compatible function sets, and unsupported function sets. For equivalent replacement functions, they are directly replaced, and for partially compatible functions, they are replaced according to the situation. For incompatible functions, custom database functions are used or implemented at the application layer.

[0074] (2) Sort out keywords

[0075] Sort out the commonly used keywords and reserved words in the original database, count the keywords and reserved words in the target database; replace and modify the columns in the original database that are named after the keywords and reserved words of the target database.

[0076] (3) Sorting out data types

[0077] Sort out the data types in the original database and the database types in the target database and compare them. Directly replace the database types that have the same function; transform the database types that have the same data type but inconsistent length, precision, etc.; rewrite the unsupported data types.

[0078] (4) Sorting out grammatical differences

[0079] Compare the syntax of the original database with that of the target database, mainly including DML (statements for operating data in the database) and DDL (statements for defining or modifying the database structure). For differences in syntax, perform SQL syntax checking, SQL compatibility assessment, and conversion, and then bring the converted and rewritten SQL into the application for testing and verification.

[0080] (5) Perform grammatical correctness training on a large model

[0081] Combine the functions, keywords, data types, and syntax differences sorted out in the above four steps with the large model framework, use the random forest algorithm to train the large model, and then virtually execute the replaced statements in the target database. Filter out the set of syntax that cannot be executed, manually troubleshoot and find solutions, then compare the results of SQL execution in the source database and the results of SQL execution in the target database and perform further training. Use indicators such as precision, recall, F1 score, and whether it is overfitting to evaluate the accuracy of the model.

[0082] Next, based on the SQL performance training set, performance training can be performed using a combination of neural networks and random forest methods. Among them, after completing the grammatical correctness training, the interface can be further used to compare the historical query time of the original database and the query time of the target database to filter out the slower SQL set for training. Use the execution plan to count the execution time of each step of the slow SQL and the data level corresponding to each table. The data level is divided into 0-100,000, 100,000-1 million, 1 million-10 million, etc. Get the SQL statement, source database execution time, target database execution time, main table data level, subquery data level, left join table data level, right join table data level, table index column and other information sets. It can be divided into the following four steps:

[0083] (1) Statistics of full table scan operations

[0084] First, obtain the database table name, field name, created index and index columns involved, and data level of each table that the SQL statement needs to access; then analyze the execution plan of the SQL statement, query the operations involving full table scans and statements with long subquery execution time, and divide the above statements into two categories: no index is set and index is set but not used correctly; then create indexes and rewrite indexes. If the SQL statement index is used improperly, adjust the column order in the composite index or add the index prefix column to the where condition to achieve the purpose of rewriting the index and no longer performing a full table scan. Then count the where conditions and use LIKE, >, !=, or, is null and other operator conditions to optimize and rewrite them respectively.

[0085] (2) Statistics table association operations

[0086] Count the table name and number of data entries in each associated table, and add indexes to the condition columns in the associated table. For associated tables with too much data, partition and store them in different partitions and add filtering conditions on partition columns to reduce the scope of scanning partitions. For subqueries, try to use left outer join or right outer join to optimize. Use caching strategies for commonly used functions.

[0087] (3) Optimize server settings

[0088] Optimize by adjusting the buffer pool size, database query cache, thread cache, log file size and number, and number of connections; perform regular maintenance, regularly organize the physical storage of tables, check and fix database errors; use tools to monitor and analyze query problems caused by optimizing server settings.

[0089] (4) Perform SQL optimization training on large models

[0090] Combine the optimization items sorted out in the above three steps with the large model framework, use the random forest algorithm to train the large model, and then virtually execute the replaced statements in the target database. Filter out the sets that cannot be executed or execute slowly after replacement, manually troubleshoot and find solutions, and conduct further training. Use indicators such as precision, recall, F1 score, and whether it is overfitting to evaluate the accuracy of the model.

[0091] In this way, a large amount of interface usage records, running time and other data can be captured through the log flow recording platform, providing good basic data for the training of large models; the SQL optimization method based on the large model can efficiently and accurately output the source database syntax as the target database syntax, and perform virtual execution on the system, optimizing SQL performance and realizing one-click migration of the database.

[0092] It can be seen that in the process of executing steps S101-S104, the original database table structure is first obtained through the data architecture management system and the system number input by the target user, and the database table, view, function, stored procedure, etc. that conform to the rules of the target database are automatically established in the target database. Secondly, the data of the original database is synchronized to the data warehouse tool of the distributed system, and then synchronized to the target database through the data warehouse tool and the synchronization result is automatically verified. Then, through the log flow recording platform, the corresponding interface and the corresponding SQL execution statement are captured, data cleaning and processing are performed, and the SQL statement is virtually executed in the target database to form a training set and trained on the large model. Finally, when the accuracy of the large model training reaches a certain level, the captured SQL is optimized for syntax and performance through the large model, and automatically tested according to the interface, and the problematic and poorly performing SQL is optimized. The problematic statements are analyzed and then the large model is trained. Finally, after the SQL output by the large model can pass the functional and performance tests, the database migration and syntax transformation are completed.

[0093] In summary, the database migration and grammar transformation method provided in this embodiment first responds to the target user's login operation on the data architecture management platform, verifies whether the target user has the authority to perform database migration, and if so, obtains the original database attribute information through the data architecture management platform; and establishes corresponding attribute information in the target database based on the original database attribute information. Then, the data in the original database is synchronized to the target database through the data warehouse tool of the distributed system, and then the interface and the corresponding SQL execution statement are obtained through the log flow recording platform, and the SQL execution statement is grammatically optimized using the pre-trained large model to obtain the optimization result. This can effectively improve the efficiency and accuracy of database migration and grammar transformation.

[0094] Second embodiment

[0095] This embodiment will introduce a database migration and grammar transformation device. For related content, please refer to the above method embodiment.

[0096] See also Figure 2 , is a schematic diagram of the composition of a database migration and grammar transformation device provided in this embodiment, the device includes:

[0097] The verification unit 201 is used to verify whether the target user has the authority to perform database migration in response to the target user's login operation on the data architecture management platform;

[0098] The establishing unit 202 is used to obtain the original database attribute information through the data architecture management platform if it is verified that the target user has the authority to perform database migration; and establish corresponding attribute information in the target database according to the source database attribute information;

[0099] A synchronization unit 203 is used to synchronize the data in the original database to the target database through a data warehouse tool of the distributed system;

[0100] The optimization unit 204 is used to obtain the interface and the corresponding SQL execution statement through the log flow recording platform, and use the pre-trained large model to perform syntax optimization on the SQL execution statement to obtain an optimization result.

[0101] In an implementation of this embodiment, the establishing unit 202 includes:

[0102] An acquisition subunit is used to acquire table creation statements, views, and functions of the original database through the data architecture management platform;

[0103] A subunit is established to replace keywords in the table creation statements of the original database, and to establish corresponding functions, sequences, views, and stored procedures in the target database according to the replaced table creation statements, views, and functions, while ensuring that the number of tables and the fields of each table in the target database are consistent.

[0104] In an implementation of this embodiment, the synchronization unit 202 includes:

[0105] A first synchronization subunit is used to synchronize the data of the original database to the data warehouse tool of the distributed system;

[0106] The second synchronization subunit is used to synchronize the data in the data warehouse tool of the distributed system to the target database and automatically verify the synchronization result.

[0107] In one implementation of this embodiment, the optimization unit 204 includes:

[0108] The optimization subunit is used to obtain the interface and the corresponding SQL execution statement through the log flow recording platform, and optimize the syntax and performance of the SQL execution statement by using the pre-trained large model to obtain the optimized SQL execution statement;

[0109] The write-back subunit is used to write the optimized SQL execution statement back to the program, and automatically perform function and performance tests according to the interface list, and complete the syntax transformation of the SQL execution statement after the test passes to obtain the optimization result.

[0110] In summary, the database migration and grammar transformation device provided in this embodiment first responds to the target user's login operation on the data architecture management platform, verifies whether the target user has the authority to perform database migration, and if so, obtains the original database attribute information through the data architecture management platform; and establishes corresponding attribute information in the target database based on the original database attribute information. Then, through the data warehouse tool of the distributed system, the data in the original database is synchronized to the target database, and then, the interface and the corresponding SQL execution statement are obtained through the log flow recording platform, and the SQL execution statement is grammatically optimized using the pre-trained large model to obtain the optimization result. This can effectively improve the efficiency and accuracy of database migration and grammar transformation.

[0111] Furthermore, the embodiment of the present application also provides a database migration and grammar transformation device, including: a processor, a memory, and a system bus;

[0112] The processor and the memory are connected via the system bus;

[0113] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementation methods of the above-mentioned database migration and grammar transformation methods.

[0114] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned database migration and grammar transformation methods.

[0115] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0116] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0117] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0118] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A database migration and grammar transformation method, characterized in that: include: In response to a target user's login operation to the data architecture management platform, verifying whether the target user has permission to perform database migration; If so, the original database attribute information is obtained through the data architecture management platform; and corresponding attribute information is established in the target database according to the original database attribute information; Use the distributed system's data warehouse tool to synchronize the data in the original database to the destination database; The interface and the corresponding SQL execution statement are obtained through the log flow recording platform, and the SQL execution statement is grammatically optimized using the pre-trained large model to obtain the optimization result.

2. The method according to claim 1, characterized in that The step of obtaining original database attribute information through the data architecture management platform and establishing corresponding attribute information in the target database according to the source database attribute information includes: Obtaining table creation statements, views, and functions of the original database through the data architecture management platform; The keywords of the table creation statements of the original database are replaced, and corresponding functions, sequences, views, and stored procedures are established in the target database according to the replaced table creation statements, views, and functions, while ensuring that the number of tables and the fields of each table in the target database are consistent.

3. The method according to claim 1, characterized in that: The method of synchronizing the data in the original database to the target database through the data warehouse tool of the distributed system includes: Synchronize the data of the original database to the data warehouse tool of the distributed system; The data in the data warehouse tool of the distributed system is synchronized to the target database, and the synchronization result is automatically verified.

4. The method according to claim 1, characterized in that: The interface and the corresponding SQL execution statement are obtained through the log flow recording platform, and the SQL execution statement is grammatically optimized using the pre-trained large model to obtain the optimization result, including: Obtain the interface and the corresponding SQL execution statement through the log flow recording platform, and use the pre-trained large model to optimize the syntax and performance of the SQL execution statement to obtain the optimized SQL execution statement; The optimized SQL execution statement is written back into the program, and function and performance tests are automatically performed according to the interface list. After the test passes, the syntax transformation of the SQL execution statement is completed to obtain the optimization result.

5. A database migration and grammar transformation device, characterized in that: include: A verification unit, configured to verify, in response to a target user's login operation to the data architecture management platform, whether the target user has the authority to perform database migration; An establishing unit, configured to obtain original database attribute information through the data architecture management platform if it is verified that the target user has the authority to perform database migration; and to establish corresponding attribute information in the target database according to the source database attribute information; The synchronization unit is used to synchronize the data in the original database to the destination database through the data warehouse tool of the distributed system; The optimization unit is used to obtain the interface and the corresponding SQL execution statement through the log flow recording platform, and use the pre-trained large model to perform syntax optimization on the SQL execution statement to obtain the optimization result.

6. The device according to claim 5, characterized in that The establishing unit comprises: An acquisition subunit is used to acquire table creation statements, views, and functions of the original database through the data architecture management platform; A subunit is established to replace keywords in the table creation statements of the original database, and to establish corresponding functions, sequences, views, and stored procedures in the target database according to the replaced table creation statements, views, and functions, while ensuring that the number of tables and the fields of each table in the target database are consistent.

7. The device according to claim 5, characterized in that The synchronization unit comprises: A first synchronization subunit is used to synchronize the data of the original database to the data warehouse tool of the distributed system; The second synchronization subunit is used to synchronize the data in the data warehouse tool of the distributed system to the target database and automatically verify the synchronization result.

8. The device according to claim 5, characterized in that The optimization unit comprises: The optimization subunit is used to obtain the interface and the corresponding SQL execution statement through the log flow recording platform, and optimize the syntax and performance of the SQL execution statement by using the pre-trained large model to obtain the optimized SQL execution statement; The write-back subunit is used to write the optimized SQL execution statement back to the program, and automatically perform function and performance tests according to the interface list, and complete the syntax transformation of the SQL execution statement after the test passes to obtain the optimization result.

9. A database migration and grammar transformation device, characterized in that: include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 4.

Images