Database adaptation method based on large language model

Abstract

The invention discloses a database adaptation method based on a large language model, and relates to the technical field of database adaptation. Obtaining training data, preprocessing the training data to obtain a training data set, and inputting the training data set into the open source large model to obtain a pre-training model; initializing a low-rank matrix, introducing the low-rank matrix as a fine tuning parameter into the pre-training model, freezing an original parameter of the pre-training model, and training the fine tuning parameter to obtain a target task model; and obtaining a project source code, inputting the project source code into the target task model to obtain a plurality of compatible codes, and deploying the compatible codes to the target database. The pre-training model learns differences and conversion relations between databases, generates codes compatible with a target database and performs database adaptation, so that the accuracy is improved, and stable operation of the system is guaranteed; the low-rank matrix fine tuning parameters are adopted, training resources and time cost are reduced, the adaptation workload is reduced, and manual intervention is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of database adaptation, and particularly relates to a database adaptation method based on a large language model. Background Art

[0002] In the field of cloud computing, the database, as a core component of the platform, is responsible for storing and managing persistent data. With the development of technology, domestic databases have increasingly significant advantages in terms of cost, security, and stability, and have gradually become the preferred choice for enterprises and organizations. However, due to significant differences in syntax compatibility and functional differences between different database systems, application programs face many challenges when migrating between different database systems.

[0003] Patent No.: CN118885461A, discloses a method for migrating a domain knowledge database of a multi-modal large model; by dividing data of the same format during the migration process into the same type of partition, then sorting the single data within the same type of partition, and identifying the corresponding transmission association interval from the transmission data characteristics of the previous group of single data, using this transmission association interval as the execution interval of the next group of single data, and then determining the relevant transmission association interval from the transmission process of the next group of single data and using it again as the execution interval of the next group of single data, and so on, gradually optimizing the transmission association interval, so that the entire database is not only stable enough during the migration process, but also has a fast enough transmission rate, achieving a better data migration effect.

[0004] For the above-mentioned existing technology, although the problems of migration efficiency and stability are solved during the existing database migration process, it is not applicable to domestic databases, resulting in increased migration costs, low adaptation efficiency, and low adaptation success rate when migrating data to domestic databases; When the cloud computing platform adapts to domestic databases, due to the syntax compatibility and functional differences of different databases, the data migration cost is high, the adaptation efficiency is low, the workload is large, and the adaptation success rate is low. Summary of the Invention

[0005] The purpose of the present invention is to address the problems of high data migration cost, low adaptation efficiency, large workload, and low adaptation success rate caused by syntax compatibility and functional differences of different databases when the cloud computing platform adapts to domestic databases, and to propose a database adaptation method based on a large language model.

[0006] In the first aspect of the implementation of the present invention, a database adaptation method based on a large language model is first proposed, and the method includes: Obtain training data and preprocess the training data to obtain a training data set, and input the training data set into an open-source large model to obtain a pre-trained model; the training data includes: an original database syntax document, a target database syntax document, and a common problem description document for database conversion. Initialize a low-rank matrix, introduce the low-rank matrix as a fine-tuning parameter into the pre-trained model, freeze the original parameters of the pre-trained model, and train the fine-tuning parameter to obtain a target task model. Obtain the project source code, input the project source code into the target task model to obtain multiple compatible codes, and deploy the compatible codes to the target database.

[0007] Optionally, the open-source large model is Tongyi Qianwen open-source large language model.

[0008] Optionally, before inputting the training data into the open-source large model to obtain a pre-trained model, the method further includes: Clean the training data to obtain target training data, and perform text recognition on the target training data through a text recognition formula to obtain a training data set. Text recognition formula:

[0009] where T represents the input text, w i is a word in the text, D is a preset dictionary, R is a rule function, w N is the Nth word in the text, and N is the number of words in the text.

[0010] Optionally, export the target task model into a loadable format; the loadable format includes: PyTorch format and ONNX format.

[0011] Optionally, inputting the project source code into the target task model to obtain compatible codes includes: Input the project source code into the target task model for inference to obtain an answer, and send the answer to the target user for correctness verification; If the verification is successful, identify the project source code to obtain multiple statements to be adapted, determine the syntax structure of the target database, and generate multiple compatible codes according to each statement to be adapted and the syntax structure; If the verification fails, perform inference again to obtain an answer and send the answer to the target user for correctness verification until the maximum number of replies is reached, then stop the inference.

[0012] Optionally, perform program semantic verification on the compatible codes, and if the verification is successful, generate a docker image and deploy it; the program semantic verification includes: syntax verification and function verification.

[0013] Optionally, each compatible code is simulated and run. If the execution fails, the compatible code is regenerated. If the execution is successful, the compatible code is used to generate a Docker image.

[0014] Advantages of the present invention: The present invention provides a database adaptation method based on a large language model. By obtaining training data and preprocessing the training data to obtain a training data set, the training data set is input into an open-source large model to obtain a pre-trained model; a low-rank matrix is initialized, and the low-rank matrix is introduced as a fine-tuning parameter into the pre-trained model, the original parameters of the pre-trained model are frozen, and the fine-tuning parameters are trained to obtain a target task model; project source code is obtained, and the project source code is input into the target task model to obtain multiple compatible codes, and the compatible codes are deployed to the target database. The pre-trained model learns the differences and conversion relationships between databases, generates codes compatible with the target database and performs database adaptation, improving accuracy and ensuring the stable operation of the system; the low-rank matrix is used as a fine-tuning parameter to reduce training resources and time costs, reduce the adaptation workload, and reduce manual intervention. Description of the Drawings

[0015] The present invention will be further described below with reference to the accompanying drawings.

[0016] Figure 1 It is a flowchart of a database adaptation method based on a large language model provided by an embodiment of the present invention; Figure 2 It is a flowchart of another database adaptation method based on a large language model provided by an embodiment of the present invention. Detailed Embodiments

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. The term "and / or" in this article is only a description of the associated relationship of the associated objects, indicating that there can be three relationships. For example, A and B can represent: A exists alone, A and B exist at the same time, and B exists alone. These three situations. In addition, the descriptions such as "first" and "second" in the present invention are only for the purpose of description, and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of these features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the fact that those skilled in the art can implement them. When the combination of technical solutions appears to be contradictory or unable to be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0018] Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0019] The embodiments of the present invention provide a database adaptation method based on a large language model. Refer to Figure 1 , Figure 1 which is a flowchart of a database adaptation method based on a large language model provided by the embodiments of the present invention. The method includes the following steps: S101, obtain training data, preprocess the training data to obtain a training data set, and input the training data set into an open-source large model to obtain a pre-trained model; S102, initialize a low-rank matrix, introduce the low-rank matrix as a fine-tuning parameter into the pre-trained model, freeze the original parameters of the pre-trained model, and train the fine-tuning parameter to obtain a target task model; S103, obtain project source code, input the project source code into the target task model to obtain multiple compatible codes, and deploy the compatible codes to the target database.

[0020] The training data includes: the original database syntax document, the target database syntax document, and the document for explaining common problems in database conversion; Based on the database adaptation method based on a large language model provided by the embodiments of the present invention, the pre-trained model learns the differences and conversion relationships between databases, generates codes compatible with the target database, and performs database adaptation, improving accuracy and ensuring the stable operation of the system; by using a low-rank matrix as a fine-tuning parameter, it reduces training resources and time costs, reduces the workload of adaptation, reduces manual intervention, and reduces costs. The model has generalization ability and adaptability, can handle various adaptation tasks, and has strong scalability. The whole process is efficient, shortening the migration cycle and improving the migration efficiency.

[0021] In one implementation, by obtaining training data including the original database syntax document, the target database syntax document, and the document for explaining common problems in database conversion, and preprocessing to form a training data set, the pre-trained model can learn the differences and conversion relationships between the source database and the target database in terms of syntax, function, etc. When the project source code is input into the target task model, the model can generate codes compatible with the target database based on the learned knowledge, thus effectively solving the adaptation problems caused by differences in syntax and function between different database systems, improving the accuracy of database adaptation, reducing errors and problems caused by improper adaptation, and ensuring the stable operation of the system after database migration.

[0022] In one implementation, a low-rank matrix is used as a fine-tuning parameter to introduce into the pre-trained model, and the original parameters are frozen and only the fine-tuning parameters are trained. Compared with retraining the entire pre-trained model, it greatly reduces the computing resources and time cost required for training. At the same time, the pre-trained model has learned general language knowledge and patterns from a large amount of data. On this basis, fine-tuning can enable it to quickly adapt to specific database adaptation tasks without having to train the model from scratch, further reducing the adaptation workload. In addition, the process of the model generating compatible code has a high degree of automation, reducing the workload of manually writing and modifying code, reducing the dependence on professional developers, and thus overall reducing the cost of database adaptation.

[0023] In one implementation, during the pre-training stage, the model is exposed to rich database-related document data, covering knowledge such as the syntax and common problems of different databases. This enables the model to have a certain generalization ability and be able to better understand and process various code and configuration fragments related to databases. During the fine-tuning process, the fine-tuning parameters introduced through the low-rank matrix further enhance the model's adaptability to specific database adaptation tasks, enabling it to generate code that meets the requirements of the target database more accurately. The combination of this generalization ability and adaptability enables the model to not only handle the adaptation tasks of the current project's source code but also, to a certain extent, cope with other similar database adaptation scenarios, having better scalability and generality.

[0024] In one implementation, the entire adaptation process from data collection, model training to code generation and deployment is orderly and efficient. After obtaining the training data and performing preprocessing, a pre-trained model is quickly obtained, and a target task model is obtained through fine-tuning. Then, the project's source code is input into the model to obtain multiple compatible codes, and finally, the compatible codes are deployed to the target database. This series of operations greatly shortens the time cycle of database migration compared with the traditional manual adaptation of code and configuration, and can quickly achieve the migration of the database from the source database to the target database, improving the migration efficiency.

[0025] In one implementation, see Figure 2 , Figure 2The flowchart of another database adaptation method based on large language models provided by the embodiments of the present invention; this process adopts a dual-path collaborative design: the left path focuses on the source code development and configuration management of the OpenStack database module: first, operate based on the MariaDB database environment used by each OpenStack module; then modify the database source code and adjust the configuration for each module; the system will call a dedicated inference interface (generated by the right path) to obtain code update prompts provided by artificial intelligence to assist in development; developers manually check and correct the generated source code and configuration to obtain the final database source code and configuration. The right path focuses on the construction and serviceization of domain knowledge-enhanced large models: the starting point of the process is the structured preprocessing of the target database document; then, using the preprocessed data, lightweight and efficient fine-tuning (LoRA technology) is performed based on open-source large language models; after training, a fine-tuned model with specific domain knowledge is generated; finally, the model is deployed as an inference service that can be called. The inference service output by the right path is directly embedded in the left path to provide intelligent coding assistance for developers.

[0026] In one embodiment, the open-source large model is Tongyi Qianwen open-source large language model.

[0027] In one implementation, the open-source large models include, for example: general base model RedPajama, Mistral7B, StableLM, multimodal and vertical domain models LLaVA, StarCoder, Dolphin, efficient fine-tuning and lightweight models LLaMA-Adapter, Qwen-7B-Chat, academic and research-oriented models Falcon-40B, Mixtral8x7B, etc.

[0028] In one embodiment, before inputting the training data into the open-source large model to obtain the pre-trained model, the method further includes: Clean the training data to obtain target training data, and perform text recognition on the target training data through a text recognition formula to obtain a training data set; Text recognition formula:

[0029] Among them, T represents the input text, w i is a word in the text, D is a preset dictionary, R is a rule function, w N is the Nth word in the text, and N is the number of words in the text.

[0030] In one implementation, the training data is cleaned to remove useless text and duplicate content and the format is unified, which can effectively improve the quality and consistency of the training data. High-quality training data helps reduce noise interference during the model learning process, enabling the model to more accurately learn key knowledge such as database syntax and conversion rules, thereby improving the performance and accuracy of the model. Text recognition of the cleaned target training data through a text recognition formula further ensures the accuracy and standardization of the training data, providing a reliable data basis for subsequent model training.

[0031] In one implementation, the text recognition formula performs word segmentation on the input text through a preset dictionary and rule function, and can accurately recognize and segment keywords, terms, etc. in the database document into words, sub-words or phrases. This rule-based word segmentation method is particularly suitable for highly structured content such as database documents that contain a large number of fixed terms or keywords, and helps the model better understand the semantics and syntax structure of the database document. When the model learns the training data set processed by text recognition, it can more accurately grasp the logic and rules of database operations, so that in the subsequent code analysis and conversion process, it can generate code that is more compatible with the target database more accurately.

[0032] In one implementation, by preprocessing and text recognition of the training data, converting the data into a format that the model can understand and accept, the efficiency of model training can be improved. The model can read and process the training data faster during the training process, reducing training errors and anomalies caused by non-standard or inconsistent data formats, thereby ensuring the stable progress of the model training process. In addition, a high-quality training data set helps the model converge faster, shortens the training time, reduces the training cost, and improves the efficiency and practicality of the entire database adaptation method.

[0033] In one embodiment, introducing a low-rank matrix as a fine-tuning parameter into a pre-trained model includes: Setting hyperparameters such as LoRA parameters and the rank of the LoRA matrix, initializing LoRA, and freezing the main parameters of the pre-trained model; Inserting low-rank matrices into each layer of the pre-trained model to obtain a LoRA model, and updating the optimizer and loss function of the LoRA model; the low-rank matrix includes a low-rank matrix A and a low-rank matrix B; Inputting the training data set into the LoRA model for training to obtain LoRA layer parameters, and combining the LoRA parameters with the main parameters to obtain target model parameters; Updating the training model according to the target model parameters to obtain a target task model.

[0034] In one implementation, by setting hyperparameters such as LoRA parameters and the rank of the LoRA matrix (hyperparameters such as the rank r of the LoRA matrix and the learning rate μ), and initializing LoRA, a good starting point can be provided for model fine-tuning. Freeze the main parameters of the pre-trained model and only train the introduced low-rank matrices A and B. In this way, on the basis of retaining the general features of the pre-trained model, it is possible to focus on learning specific knowledge related to database adaptation, thereby effectively improving the model's adaptation ability to a specific database. At the same time, since only the parameters of the low-rank matrix part are trained, compared with fine-tuning the entire model, the number of parameters to be trained is greatly reduced, the efficiency of model training is improved, and the training cost and time are reduced.

[0035] In one implementation, insert low-rank matrices into each layer of the pre-trained model to obtain the LoRA model, and update the optimizer and loss function. This fine-tuning method enables the model to better adapt to different database adaptation tasks. By adjusting hyperparameters such as the rank of the LoRA matrix, the degree and scope of the model's learning of new knowledge can be flexibly controlled, thereby enhancing the scalability of the model. For example, in the face of database adaptation requirements of different complexities, the performance of the model can be optimized by adjusting hyperparameters, enabling it to handle various situations more effectively and improving the generality and applicability of the model.

[0036] In one implementation, input the training dataset into the LoRA model for training. After obtaining the LoRA layer parameters, merge these parameters with the main parameters to obtain the target model parameters, and accordingly update the pre-trained model to obtain the target task model. This process not only makes full use of the existing knowledge of the pre-trained model, but also introduces specific domain knowledge through fine-tuning, making the final task target model more stable and efficient when dealing with database adaptation tasks. Through the LoRA fine-tuning technology, while the model learns new knowledge, it can maintain the stability of the original knowledge, avoiding fluctuations in model performance caused by large-scale parameter updates, thereby ensuring the reliability and accuracy of the model in practical applications.

[0037] In one implementation, in model fine-tuning, set the weight matrix in the model as W, the input vector as x, and the output of the training result before fine-tuning as. During LoRA fine-tuning, limit the weight update to the low-rank subspace to reduce the number of parameters. Use low-rank matrix factorization to represent the part of the weight change, which can be expressed as, where AB represents the product of two low-rank matrices. The adjusted output is:

[0038] For the input x, the output is:

[0039] Among them, is the output of the original model, and is the low-rank matrix part of LoRA; the original model is the pre-trained model.

[0040] When optimizing parameter training, fix the original parameters W and only train A and B. Taking the cross-entropy function as an example, assume the training data is, is the learning rate, the loss function is, and the objective function is

[0041] The gradient update method is as follows

[0042] Through LoRA fine-tuning, not only the parameter characteristics of the original model are retained, but also the newly added low-rank matrix is utilized to adapt to the new knowledge document, with low training cost and strong task adaptability.

[0043] In one embodiment, export the target task model into a loadable format; the loadable format includes: PyTorch format and ONNX format.

[0044] In one implementation, the loadable format reflects the interoperability and flexibility of cross-frame deployment of the model. By adapting to mainstream formats such as PyTorch, ONNX, and TensorFlow, the fine-tuned model can be efficiently loaded and run on different inference engines (such as PyTorch inference API, TensorRT) and platforms (training and inference integrated AI platform), taking into account the compatibility of model deployment and the optimization of inference performance.

[0045] In one embodiment, input the project source code into the target task model to obtain compatible code, including: Input the project source code into the target task model for inference to obtain an answer, and send the answer to the target user for correctness verification; If the verification is successful, identify multiple statements to be adapted from the project source code, determine the syntax structure of the target database, and generate multiple compatible codes according to each statement to be adapted and the syntax structure; If the verification fails, perform inference again to obtain an answer and send the answer to the target user for correctness verification until the maximum number of replies is reached, then stop the inference.

[0046] In one implementation, input the project source code into the target task model for inference, and send the obtained answer to the target user for correctness verification. This process can ensure that the generated code modification suggestions have high accuracy and reliability. Through the verification feedback of the user, the model can timely discover and correct possible errors or inaccuracies, thereby improving the success rate of code adaptation. This verification mechanism based on user feedback can effectively avoid system operation problems caused by the model generating incorrect code, and ensure the stability and reliability of the system after database migration.

[0047] In one implementation, if the verification fails, the inference is performed again to obtain an answer and the answer is sent to the target user for verification again until the maximum number of replies is reached. This mechanism endows the model with certain self - adaptation and error - correction capabilities. During multiple inferences, the model can continuously adjust and optimize the generated answer according to the user's feedback, gradually improving the accuracy and adaptability of the answer. Through this iterative inference and verification process, the model can better adapt to different project source codes and database adaptation requirements, and improve its robustness and adaptability in practical applications.

[0048] In one implementation, after successful verification, the project source code is identified to obtain multiple statements to be adapted, and the syntax structure of the target database is determined. Then, multiple compatible codes are generated based on each statement to be adapted and the syntax structure. This automated process greatly reduces the workload of manually writing and modifying code, improving the development efficiency. By automatically generating compatible codes by the model, developers can invest more time and energy in other key tasks, thereby reducing the overall labor cost and improving the development progress and quality of the project.

[0049] In one embodiment, program semantic verification is performed on the compatible code. If the verification is successful, a docker image is generated and deployed; the program semantic verification includes: syntax verification and function verification.

[0050] In one implementation, performing program semantic verification on the compatible code, including syntax verification and function verification, can comprehensively check the correctness and integrity of the code. Syntax verification ensures that the code conforms to the syntax rules of the target database, avoiding program operation failures caused by syntax errors. Function verification further confirms whether the logic and function of the code in actual operation meet the expectations and whether it can correctly interact with the target database. Through this dual - verification mechanism, code defects can be effectively reduced, the code quality can be improved, thereby ensuring the stable operation of the system after deployment and reducing the risk of system failures caused by code problems.

[0051] In one implementation, a docker image is generated and deployed after successful verification. This process not only improves the deployment efficiency but also enhances the reliability of the deployment. The docker image encapsulates the application program and its running environment, ensuring consistency and portability in different environments. In this way, the verified code can be quickly deployed to the production environment, reducing the uncertainty and risk during the deployment process, and improving the system's online speed and reliability. At the same time, the isolation of docker containers also ensures the security and stability of the application program, avoiding interference between different applications.

[0052] In one embodiment, each compatible code is simulated and run. If the execution fails, the compatible code is regenerated. If the execution is successful, the compatible code is used to generate a Docker image.

[0053] In one implementation, simulating and running each compatible code can simulate the running situation of the code in the target database environment before actual deployment. Through the simulation run, problems that may be encountered during the code execution, such as syntax errors, logical errors, or performance issues, can be discovered in a timely manner. If the execution fails, the compatible code is regenerated. This process can ensure that the finally generated code can be correctly executed in the target database, thereby improving the reliability and accuracy of code adaptation. This loop mechanism of simulation run and regeneration can effectively reduce the migration failure caused by code problems and improve the success rate of database adaptation.

[0054] In one implementation, if the simulation run is successfully executed, the compatible code is used to generate a Docker image. This process not only optimizes the code generation and deployment process, but also improves the deployment efficiency and consistency. The use of Docker images ensures the portability and consistency of the code in different environments, reducing deployment problems caused by environmental differences. In this way, the verified code can be quickly deployed to the production environment, reducing the uncertainty and risk during the deployment process, and improving the system's online speed and reliability.

[0055] The above has described in detail an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of the present invention application should still fall within the scope covered by the patent of the present invention.

Claims

1. A database adaptation method based on a large language model, characterized in that, The method includes: Obtain training data, preprocess the training data to obtain a training dataset, and input the training dataset into an open-source large model to obtain a pre-trained model; the training data includes: an original database syntax document, a target database syntax document, and a document explaining common problems in database conversion. Initialize a low-rank matrix, introduce the low-rank matrix as a fine-tuning parameter into the pre-trained model, freeze the original parameters of the pre-trained model, and train the fine-tuning parameters to obtain a target task model. Obtain the project source code, input the project source code into the target task model to obtain multiple compatible codes, and deploy the compatible codes to the target database.

2. The database adaptation method based on a large language model according to claim 1, wherein The open-source large model is Tongyi Qianwen open-source large language model.

3. The database adaptation method based on a large language model according to claim 1, wherein Before inputting the training data into the open-source large model to obtain a pre-trained model, the method further includes: Clean the training data to obtain target training data, and perform text recognition on the target training data through a text recognition formula to obtain a training dataset. Text recognition formula: ； Among them, T represents the input text, w i is a word in the text, D is a preset dictionary, R is a rule function, w N is the Nth word in the text, and N is the number of words in the text.

4. A database adaptation method based on a large language model according to claim 1, characterized in that Export the target task model into a loadable format; the loadable format includes: PyTorch format and ONNX format.

5. A database adaptation method based on a large language model according to claim 1, characterized in that Inputting the project source code into the target task model to obtain compatible codes includes: Input the project source code into the target task model for inference to obtain an answer, and send the answer to the target user for correctness verification. If the verification is successful, identify the project source code to obtain multiple statements to be adapted, determine the syntax structure of the target database, and generate multiple compatible codes according to each statement to be adapted and the syntax structure. If the verification fails, perform inference again to obtain an answer and send the answer to the target user for correctness verification, and stop inference until the maximum number of replies is reached.

6. The database adaptation method based on a large language model according to claim 1, wherein, Perform program semantic verification on the compatible codes. If the verification is successful, generate a docker image and deploy it; the program semantic verification includes: syntax verification and function verification.

7. A database adaptation method based on a large language model according to claim 6, characterized in that, Perform simulation running on each compatible code. If the execution fails, regenerate the compatible code. If the execution is successful, generate a docker image for the compatible code.

Images