A method for converting natural language statements into SQL statements based on fine-tuning large language models

By fine-tuning the large language model and feedback alignment and fine-tuning of the execution results, the problem of insensitive executability and execution results in natural language statements to SQL statements is solved, and the conversion effect is improved. It is suitable for intelligent data interaction systems and reduces database interactions threshold.

CN117194478BActive Publication Date: 2024-11-05BEIJING KNOWLEDGE ATLAS TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311181849.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-13
Publication Date
2024-11-05
Estimated Expiration
2043-09-13

AI Technical Summary

Technical Problem

In the prior art, when natural language statements are converted into SQL statements, the generated SQL statements are insensitive and the execution results are insufficient, and the cross-scene adaptability and flexibility are insufficient, resulting in limited accuracy.

Method used

By fine-tuning the large language model, using domain adaptation to fine-tuning and feedback alignment of execution results, introducing SQL executability and supervision information constraints on SQL execution results, optimizing the loss function of the large language model, and improving the executability and execution results of SQL statements accuracy.

Benefits of technology

It improves the overall effect of converting natural language statements to SQL statements, enhances the executability and correctness of the execution results of SQL statements, and is suitable for intelligent data interaction systems, lowering the threshold for database interaction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117194478B_ABST
    Figure CN117194478B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of natural language processing technology, and relates to a method for converting natural language statements into SQL statements based on a fine-tuned large language model, which comprises the following steps: 1) fine-tuning and training the large language model; 2) obtaining the natural language statement to be converted; 3) inputting the natural language statement into the fine-tuned and trained large language model to obtain the converted SQL statement. The present invention solves the problem that the large language model is insensitive to the executableness and execution results of the generated SQL statement, thereby improving the overall effect of converting the natural language statement into the SQL statement.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and relates to a method for converting natural language statements into SQL statements, and in particular to a method for converting natural language statements into SQL statements based on a fine-tuned large language model. Background Art

[0002] Deep learning methods were often used to convert natural language statements into SQL statements. However, since traditional deep learning methods do not have general natural language understanding capabilities, they require a lot of annotated corpus, have weak cross-scenario capabilities, and have low flexibility. After the model is deployed, there is basically no room for adjustment without retraining.

[0003] With the rapid development of artificial intelligence and big data technologies, the new generation of artificial intelligence technology represented by large language models has brought revolutionary breakthroughs. How to reconstruct and upgrade traditional products and businesses based on large language models has become a hot research topic.

[0004] Converting natural language statements to SQL statements is one of the hottest research directions in natural language processing. The diversity of human natural language expressions is very large, and traditional methods are difficult to deal with well. Large language models have powerful emergent capabilities, a universal understanding of natural language, and extremely high cross-domain capabilities and flexibility. Therefore, the method of converting natural language statements to SQL statements based on large language models has become a hot research direction.

[0005] For example, the invention with patent publication number CN115658729A proposes a method for converting natural language to SQL statements based on a pre-trained model, and its specific scheme is as follows: S1. Obtain the natural statement to be converted; S2. Input the natural statement into a pre-trained model for converting SQL queries to obtain the expected input representation and generate SQL output, and obtain the corresponding SQL statement. Among them, in the pre-trained model, natural language to SQL is represented as a multi-task learning problem. The pre-trained model includes an encoding layer, a decoding layer and an output layer. The encoding layer adopts a RoBERTa network structure, and the decoding layer adopts a bidirectional LSTM. This technical solution solves the problem that the previous method cannot fully obtain the encoded information in the data set using a single decoder by proposing a model that uses a bidirectional LSTM as an encoder.

[0006] However, the characteristic of converting natural language statements to SQL statements is that if a character is wrong in the generated SQL statement, the entire SQL statement is incorrect, and the information in the database table of each business scenario is quite different. The above technical solution has limitations in cross-scenario adaptability, as well as the executableness and sensitivity of the execution results of the generated SQL statements, which limits the accuracy.

[0007] Therefore, in view of the defects existing in the above-mentioned prior art, it is necessary to develop a new method for converting natural language statements into SQL statements. Summary of the invention

[0008] In order to overcome the defects of the prior art, the present invention proposes a method for converting natural language statements into SQL statements based on fine-tuning a large language model, which solves the problem that the large language model is insensitive to the executableness and execution results of the generated SQL statements, thereby improving the overall effect of converting natural language statements into SQL statements.

[0009] In order to achieve the above object, the present invention provides the following technical solutions:

[0010] A method for converting natural language statements into SQL statements based on fine-tuning a large language model, characterized by comprising the following steps:

[0011] 1) Fine-tune and train the large language model;

[0012] 2) Obtain the natural language sentence to be converted;

[0013] 3) Input the natural language sentence into the fine-tuned large language model to obtain the converted SQL sentence.

[0014] Preferably, the step 1) specifically includes:

[0015] 1.1) Create a training data set for converting natural language statements to SQL statements;

[0016] 1.2) Using the training dataset to fine-tune the large language model for domain adaptation;

[0017] 1.3) Using the training dataset, the large language model after domain adaptation fine-tuning is fine-tuned by execution result feedback alignment.

[0018] Preferably, in step 1.3), when fine-tuning the execution result feedback alignment, two supervisory information constraints, SQL executable and SQL execution result, are introduced. The labels of the two supervisory information are obtained by executing SQL statements, and the label values ​​are 0 or 1, wherein the label value of SQL executable is 0, indicating that the SQL execution fails, and 1 indicates that the SQL execution is successful; the label value of the SQL execution result is 0, indicating that the SQL execution result is wrong, and 1 indicates that the SQL execution result is correct. Therefore, the loss function of the large language model after domain adaptation fine-tuning is recorded as L final ,but,

[0019] L final =α 1-label_execute *β 1-label_result *L original

[0020] In the formula, α 1-label_execute represents the SQL executable feedback alignment item, label_execute represents the label value of SQL executable, β 1-label_result represents the SQL execution result feedback alignment item, label_result represents the label value of the SQL execution result, α and β represent the adjustable parameters of the SQL executable feedback alignment item and the SQL execution result feedback alignment item respectively, and L original is the original loss function of the large language model.

[0021] Preferably, α=2, β=1.5.

[0022] Preferably, in step 1.2), when performing domain adaptation fine-tuning, a full parameter fine-tuning method or a LoRA fine-tuning method is adopted.

[0023] Preferably, in the step 1.1), the natural language statement-to-SQL statement training data set is derived from an open source data set, and different open source data sets are processed into a unified format to obtain the training data set.

[0024] Preferably, the fine-tuned large language model is deployed on a cloud server.

[0025] Preferably, the large language model is ChatGLM, ChatGPT or GPT-4.

[0026] Compared with the prior art, the method for converting natural language statements into SQL statements based on fine-tuning a large language model of the present invention has one or more of the following beneficial technical effects:

[0027] 1. The present invention solves the problem that the large language model is insensitive to the executableness and execution results of the generated SQL statements by introducing SQL execution result feedback alignment fine-tuning, thereby improving the overall effect of converting natural language statements to SQL statements.

[0028] 2. The present invention introduces two supervisory information constraints, namely SQL executable and SQL execution result, to realize SQL execution result feedback alignment fine-tuning, so that the SQL statements generated by the large language model are close to the true value distribution while taking into account the SQL execution success rate and the accuracy of the SQL execution result.

[0029] 3. The present invention is applied to intelligent data interaction systems, which can lower the threshold for interactive use of databases. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The present invention is a flowchart of a method for converting natural language statements into SQL statements based on a fine-tuned large language model.

[0031] Figure 2This is a flowchart for fine-tuning and training a large language model in the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the accompanying drawings and embodiments, and the contents of the embodiments are not intended to limit the protection scope of the present invention.

[0033] In order to solve the problems existing in the conversion of existing natural language statements into SQL statements, this patent provides a method for converting natural language statements into SQL statements based on fine-tuning a large language model, which solves the problem that the large language model is insensitive to the executableness and execution results of the generated SQL statements, thereby improving the overall effect of converting natural language statements into SQL statements.

[0034] Figure 1 The flowchart of the method for converting natural language statements into SQL statements based on fine-tuning a large language model of the present invention is shown. Figure 1 As shown, the method for converting natural language statements into SQL statements based on fine-tuning a large language model of the present invention comprises the following steps:

[0035] 1. Fine-tune and train the large language model.

[0036] The large language model is an existing model in the industry, such as ChatGLM, ChatGPT or GPT-4, etc. By fine-tuning the large language model, the large language model can better understand natural language sentences and generate corresponding SQL statements.

[0037] In the present invention, if Figure 2 As shown in the figure, fine-tuning the large language model mainly includes the following steps:

[0038] 1. Create a training data set for converting natural language statements to SQL statements.

[0039] To fine-tune a large language model, you must first obtain a natural language sentence-to-SQL language training dataset.

[0040] In the present invention, the natural language statement to SQL statement training data set is derived from an open source data set, mainly including WikiSQL, Spider, etc. At the same time, in order to facilitate fine-tuning of a large language model, different open source data sets need to be processed into a unified format.

[0041] In the present invention, the format of the training data set is as follows:

[0042] {"input":"You are a text-to-SQL generator that directly outputs SQL statements based on the text without interpretation. The table name and table fields are as follows:\n#table('Representative','Years','State','Party','Lifespan')\nInput natural language: Which state does Jimmy Quillen represent?\nOutput SQL:","target":"SELECT State FROM table WHERE Representative=jimmy quillen"}

[0043] Among them, "input" represents the complete input text, including prompt and natural language statement to be converted, and "target" represents the output SQL statement.

[0044] 2. Use the training dataset to perform domain adaptation fine-tuning on the large language model.

[0045] Use the large language model to complete domain adaptation fine-tuning on the natural language sentence-to-SQL statement training dataset created above, so that the SQL statements generated by the large language model are close to the true value distribution.

[0046] Among them, the fine-tuning method can use the mainstream large language model fine-tuning method, such as the full parameter fine-tuning method, LoRA fine-tuning method, etc.

[0047] 3. Use the training dataset to perform result feedback alignment fine-tuning on the large language model after domain adaptation fine-tuning.

[0048] Although the large language model after domain adaptation fine-tuning can generate SQL statements, it is not sensitive to the executableness and execution results of the generated SQL statements, which affects the overall effect and accuracy of converting natural language statements to SQL statements. Therefore, in the present invention, it is also necessary to perform execution result feedback alignment fine-tuning.

[0049] In the present invention, when fine-tuning the execution result feedback alignment, two supervisory information constraints, namely SQL executability and SQL execution result, are introduced. These two supervisory information labels can be obtained by executing SQL statements, and the label values ​​are 0 and 1.

[0050] Among them, in the SQL executable monitoring information, a label value of 0 indicates that the SQL execution failed, and a label value of 1 indicates that the SQL execution was successful. Similarly, in the SQL execution result monitoring information, a label value of 0 indicates that the SQL execution result was wrong, and a label value of 1 indicates that the SQL execution result was correct.

[0051] The original loss function of the large language model is recorded as L original , we can use cross entropy to introduce two supervisory information constraints: SQL executable and SQL execution result. The final loss function is recorded as L final , which is defined as follows:

[0052] L final =α 1-label_execute *β 1-label_result *L original

[0053] Among them, α 1-label_execute represents the SQL executable feedback alignment item, label_execute represents the label value of SQL executable, β 1-label_result represents the SQL execution result feedback alignment item, label_result represents the label value of the SQL execution result, α and β represent the adjustable parameters of the SQL executable feedback alignment item and the SQL execution result feedback alignment item respectively, and the larger the value, the higher the dominance of the corresponding feedback alignment item on the overall loss function. In the present invention, the parameter setting can refer to α=2, β=1.5.

[0054] Therefore, the present invention solves the problem that the large language model is insensitive to the executableness and execution results of the generated SQL statements by introducing SQL execution result feedback alignment fine-tuning, thereby improving the overall effect of converting natural language statements to SQL statements. At the same time, the two supervisory information constraints of SQL executableness and SQL execution results are introduced to implement SQL execution result feedback alignment fine-tuning, so that the SQL statements generated by the large language model are close to the true value distribution, while taking into account the SQL execution success rate and the SQL execution result accuracy.

[0055] The large language model after fine-tuning training can be deployed on a cloud server for easy use.

[0056] 2. Obtain the natural language sentence to be converted.

[0057] Get the natural language sentence to be converted and record it as Q.

[0058] 3. Input the natural language sentence into the fine-tuned large language model to obtain the converted SQL sentence.

[0059] The natural language sentence to be converted is input into the fine-tuned large language model in the form of question and answer. The fine-tuned large language model will automatically generate the converted SQL statement, which can be recorded as A.

[0060] The natural language statement conversion SQL statement method based on fine-tuning a large language model of the present invention can be applied to an intelligent data interaction system, thereby lowering the threshold for database interaction use.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Those skilled in the art can modify or replace the technical solution of the present invention according to the idea of ​​the present invention without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A method for converting natural language statements into SQL statements based on fine-tuning a large language model, characterized in that: The following steps are involved: 1) Fine-tune and train large language models; 2) Obtain the natural language sentence to be converted; 3) Inputting the natural language sentence into the fine-tuned large language model to obtain the converted SQL sentence; The step 1) specifically includes: 1.1) Create a training dataset for converting natural language statements to SQL statements; 1.2) Using the training dataset to perform domain adaptation fine-tuning on the large language model; 1.3) Using the training dataset to perform execution result feedback alignment fine-tuning on the large language model after domain adaptation fine-tuning; In the step 1.3), when fine-tuning the execution result feedback alignment, two supervisory information constraints, SQL executable and SQL execution result, are introduced. The labels of these two supervisory information are obtained by executing SQL statements, and the label values ​​are 0 or 1. The label value of SQL executable is 0, indicating that the SQL execution fails, and 1 indicates that the SQL execution is successful; the label value of SQL execution result is 0, indicating that the SQL execution result is wrong, and 1 indicates that the SQL execution result is correct. Therefore, the loss function of the large language model after domain adaptation fine-tuning is recorded as ,but, In the formula, Indicates the SQL executable feedback alignment item. The tag value indicating the SQL executable. Indicates the SQL execution result feedback alignment item. The label value indicating the SQL execution result. They represent the adjustable parameters of the SQL executable feedback alignment item and the SQL execution result feedback alignment item. is the original loss function of the large language model.

2. The method for converting natural language statements into SQL statements based on fine-tuning a large language model according to claim 1, characterized in that: 。 3. The method for converting natural language statements into SQL statements based on fine-tuning a large language model according to claim 2, characterized in that: In the step 1.2), when performing domain adaptation fine-tuning, a full parameter fine-tuning method or a LoRA fine-tuning method is used.

4. The method for converting natural language statements into SQL statements based on fine-tuning a large language model according to claim 3, characterized in that: In the step 1.1), the natural language statement-to-SQL statement training data set is derived from an open source data set, and different open source data sets are processed into a unified format to obtain the training data set.

5. The method for converting natural language statements into SQL statements based on a fine-tuned large language model according to any one of claims 1 to 4, characterized in that: The large language model after fine-tuning training is deployed on the cloud server.

6. The method for converting natural language statements into SQL statements based on fine-tuning a large language model according to claim 5, characterized in that: The large language model is ChatGLM, ChatGPT or GPT-4.