High-quality context sample and self-correcting based large language model sql generation method
By acquiring a high-quality context sample set and combining entity linking, structure prediction, and self-correction techniques, the problem of generating erroneous SQL in Text-to-SQL tasks by large language models is solved, achieving higher accuracy and efficiency.
Patent Information
- Application Number
- CN202410624524.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-20
- Publication Date
- 2025-10-24
- Estimated Expiration
- 2044-05-20
AI Technical Summary
Existing large language models suffer from low-quality context samples in Text-to-SQL tasks, resulting in error-prone and difficult-to-correct SQL statements. Existing technologies also suffer from incomplete sample coverage, high overhead, and insufficient self-correction capabilities.
By acquiring a high-quality context sample set, the system generates SQL statement structures using entity links and structural prediction, and performs self-correction by combining syntax and semantic analysis, thereby improving the accuracy of SQL generation.
It improves the accuracy of SQL statement generation by large language models, reduces the overhead of context learning, and enhances the ability to identify and correct SQL errors.
Smart Images
Figure CN118568127B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of natural language processing and structured query statement generation, and more particularly to a large language model SQL generation method based on high-quality context samples and self-correction. BACKGROUND
[0002] The use of structured data and relational databases can help organize and manage massive amounts of data, conduct complex data analysis and mining, and SQL programming language is an important tool for interacting with relational databases. However, many end users do not have the ability to write SQL, so text-to-SQL generation becomes an important tool to solve this problem.
[0003] In recent years, with the emergence of large language models, SQL generation methods based on large language models have developed rapidly. Since the input and output forms of large language models are both text, the construction of input text content (referred to as prompt engineering) has an important impact on the performance of the Text-to-SQL task. Context learning is a kind of prompt engineering, just as humans learn new things from a few cases through analogical learning, context learning is based on the powerful natural language understanding ability of large language models, giving a small number of demonstration cases, and the large language model extracts and learns the hidden patterns in the demonstration cases, learns how to complete new tasks. More formally, context learning refers to estimating the likelihood of the potential answer using a large language model under the condition of giving demonstration samples (referred to as context samples).
[0004] Currently, the use of context learning for Text-to-SQL is relatively widespread, but due to the characteristics of context learning, the quality of the provided context samples largely determines the quality of the SQL generated by the large language model. If the quality of the provided context samples is low, the large language model will learn incorrect patterns and the accuracy of the generated SQL will decrease. Therefore, how to provide high-quality context samples, enhance the guiding effect of context samples on large language models, and let the large language model learn the correct patterns from them is the key to improving the performance of Text-to-SQL technology based on context learning.
[0005] Pourreza et al. in the published paper "DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction" propose a task decomposition based context learning Text-to-SQL technology that decomposes the Text-to-SQL task into multiple subtasks, each of which uses a hand-written fixed-sample context learning technology to call a large language model to complete the subtasks one by one and generate SQL, and finally uses a self-correction technology to let the large language model detect and correct possible errors in the SQL; this technology has the following shortcomings: since the samples used in the context learning in this technology are fixed and unchangeable, and the input text of the large language model has a length limit, these context samples cannot completely cover all scenarios in actual reasoning, covering various natural language problems and SQL structures; on the other hand, in order to cover as many scenarios as possible, it is necessary to squeeze as many context samples as possible into the input length limit, which will increase the overhead of using the large language model; in addition, the self-correction technology of this technology is relatively simple, limited by the knowledge acquired by the large language model during training, and may not be able to identify and correct some SQL errors.
[0006] Trummer in the published paper "CodexDB: Synthesizing code for query processing from natural language instructions using GPT-3 Codex" proposes a semantic similarity based context learning Text-to-SQL technology that first converts the input natural language into several processing steps to indicate how the large language model should generate the SQL statement, then retrieves several context samples with the highest semantic similarity in the context sample pool as examples, adds them to the prompt and inputs them to the large language model; then the generated SQL statement is executed on the database, if the execution is successful, it is output to the user, and the natural language-SQL pair is added to the context sample pool as a new context sample; this technology has the following shortcomings: due to the complexity and difference of database architecture, not all two samples with similar semantics on natural language query problems will have similar SQL statement structures. If the SQL statement structure of the context sample retrieved by this technology is not similar to the ideal SQL statement structure that should be generated, it means that the pattern to be learned by the large language model is wrong, which will mislead the generation of the large language model. In addition, this technology can only detect whether the generated SQL is incorrect, but has no ability to repair it. SUMMARY
[0007] The application provides a large language model SQL generation method based on high-quality context samples and self-correction to overcome the defects of low-quality context samples of large language models in the prior art, and the generated SQL statements are prone to errors and difficult to repair errors, and can improve the accuracy of large language model SQL generation, and improve the identification and correction capabilities of hidden errors in SQL.
[0008] To solve the above technical problems, the technical solutions of the application are as follows:
[0009] A large language model SQL generation method based on high-quality context samples and self-correction comprises the following steps:
[0010] S1: Data acquisition: acquiring a natural language query statement to be generated into SQL, and acquiring a context sample set based on a specified content database; the context sample set comprises a plurality of natural language query statements labeled with SQL;
[0011] S2: Entity linking generation: using a preset large language model to perform entity linking extraction on the natural language query statement to be generated into SQL and the context sample set, respectively;
[0012] S3: Structure prediction: inputting the context sample set and the corresponding entity linking into a large language model, and training the large language model to perform structure prediction of SQL statements, to obtain a large language model after structure training;
[0013] Inputting the natural language query statement to be generated into SQL and the corresponding entity linking into the large language model after structure training to perform structure prediction of SQL statements, to obtain a predicted SQL statement structure;
[0014] S4: Context sample retrieval: in the context sample set, retrieving a natural language query statement with a similarity greater than a preset first threshold value from the predicted SQL statement structure, as a structure-similar context sample set;
[0015] In the structure-similar context sample set, further retrieving a natural language statement with a semantic similarity greater than a preset second threshold value from the natural language query statement to be generated into SQL, as a high-quality context sample set;
[0016] S5: SQL generation: inputting the high-quality context sample set and the corresponding entity linking into a large language model, and training the large language model to perform content prediction of SQL statements, to obtain a large language model after content training;
[0017] Inputting the natural language query statement to be generated into SQL and the corresponding entity linking into the large language model after content training to perform content prediction of SQL statements, to generate a SQL prediction statement;
[0018] S6: SQL execution detection: perform execution detection on the SQL prediction statement, if executable, directly execute, if not executable, perform step S7;
[0019] S7: syntax semantic analysis: use a preset syntax analysis engine to detect syntax errors in the SQL prediction statement, and obtain syntax error information;
[0020] Use a preset semantic analysis engine to detect semantic errors in the SQL prediction statement, and obtain semantic error information;
[0021] S8: SQL guided self-correction: input the syntax error information and semantic error information into the large language model after content training, and guide the large language model after content training to correct errors in the SQL prediction statement, obtain the corrected SQL statement, and re-execute step S6 to re-perform SQL execution detection.
[0022] Preferably, in step S2, the entity linking includes generating table, column and value data required by the SQL statement.
[0023] Preferably, in step S3, the structure of the SQL statement includes any one of a simple structure, a non-nested complex structure and a nested complex structure.
[0024] Preferably, in step S4, it further includes: in the context sample set, all natural language query statements annotated with SQL are divided into several classes through clustering operation, and several samples are randomly extracted in each class, and all extracted samples are used as another high-quality context sample set.
[0025] Preferably, the ANTLR-based SQL lexical analyzer obtains the preset syntax analysis engine in step S7.
[0026] Preferably, in step S7, the syntax error information includes: syntax error category label, position of syntax error in the SQL prediction statement, and syntax error modification countermeasure;
[0027] The semantic error information includes: semantic error category label, position of semantic error in the SQL prediction statement, and semantic error modification countermeasure.
[0028] Preferably, in step S8, it further includes: inputting the syntax error information and semantic error information into a preset neural network analyzer, and guiding the neural network analyzer to correct errors in the SQL prediction statement, obtaining the corrected SQL statement, re-executing step S6, and re-performing SQL execution detection.
[0029] Preferably, the preset neural network analyzer is specifically a small and medium-sized language model fine-tuned on a Debug task dataset.
[0030] The application also provides a large language model SQL generation system based on high-quality context samples and self-correction, which applies the large language model SQL generation method based on high-quality context samples and self-correction.
[0031] The data acquisition unit is configured to acquire a natural language query statement to be generated into SQL and acquire a context sample set based on a specified content database; the context sample set includes a plurality of natural language query statements labeled with SQL;
[0032] The entity linking generation unit is configured to perform entity linking extraction on the natural language query statement to be generated into SQL and the context sample set respectively by using a preset large language model;
[0033] The structure prediction unit is configured to input the context sample set and the corresponding entity linking into a large language model and train the large language model to perform structure prediction of an SQL statement, so as to obtain a large language model after structure training;
[0034] The structure prediction unit is configured to input the context sample set and the corresponding entity linking into a large language model and train the large language model to perform structure prediction of an SQL statement, so as to obtain a large language model after structure training;
[0035] The context sample retrieval unit is configured to search, in the context sample set, a natural language query statement having a similarity greater than a preset first threshold with the predicted SQL statement structure, as a context sample set similar in structure;
[0036] The context sample retrieval unit is configured to search, in the context sample set, a natural language query statement having a similarity greater than a preset first threshold with the predicted SQL statement structure, as a context sample set similar in structure;
[0037] The SQL generation unit is configured to input the high-quality context sample set and the corresponding entity linking into a large language model and train the large language model to perform content prediction of an SQL statement, so as to obtain a large language model after content training;
[0038] The SQL generation unit is configured to input the high-quality context sample set and the corresponding entity linking into a large language model and train the large language model to perform content prediction of an SQL statement, so as to obtain a large language model after content training;
[0039] The SQL execution detection unit is configured to perform execution detection on the SQL prediction statement, and if the SQL prediction statement is executable, directly execute the SQL prediction statement; if the SQL prediction statement is not executable, perform the syntax semantic analysis unit and the SQL guide self-correction unit.
[0040] The syntax semantic analysis unit is configured to detect syntax errors in the SQL prediction statement by using a preset syntax analysis engine and obtain syntax error information;
[0041] The semantic analysis engine is configured to detect semantic errors in the SQL prediction statement and obtain semantic error information;
[0042] The SQL guided self-correction unit is configured to input the syntax error information and the semantic error information into the large language model trained by the content and guide the large language model trained by the content to correct errors in the SQL prediction statement, obtain a corrected SQL statement, and re-execute the SQL execution detection unit to re-perform SQL execution detection.
[0043] The application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the method.
[0044] Compared with the prior art, the technical scheme of the application has the following beneficial effects:
[0045] The application provides a large language model SQL generation method based on high-quality context samples and self-correction, which first uses a preset large language model to respectively perform entity linking extraction on a natural language query statement to be generated into SQL and a context sample set; then performs structure prediction training on the large language model and performs SQL structure prediction on the natural language query statement to be generated into SQL; then searches for context samples similar in structure and semantics to obtain a high-quality context sample set; then inputs the high-quality context sample set into the large language model to perform SQL content prediction training and generate a final SQL prediction statement.
[0046] The application proposes a new context learning technology, which can use a large amount of existing Text-to-SQL data sets to construct a context sample pool to enhance the coverage of various SQL generation scenarios, and can improve the guiding ability of context learning to the large language model by searching for more matching context samples as high-quality samples, thereby improving the accuracy of the large language model in generating SQL statements and reducing the cost of using the large language model for context learning.
[0047] In addition, the method of the application further includes SQL execution detection, if executable, directly execute; if not executable, detect error information by using a syntax and semantic analysis engine, and guide the large language model to perform guided self-correction on errors in the SQL prediction statement by using the error information, and then re-perform SQL execution detection.
[0048] The application provides a novel SQL repair technology, which effectively improves the identification and correction capabilities of a large language model for errors hidden in SQL. BRIEF DESCRIPTION OF DRAWINGS
[0049] Figure 1 A high-quality context sample and self-correction-based large language model SQL generation method flowchart provided in embodiment 1.
[0050] Figure 2 A high-quality context sample and self-correction-based large language model SQL generation method architecture diagram provided in embodiment 2.
[0051] Figure 3 A pre-processing diagram of a context sample provided in embodiment 2.
[0052] Figure 4 An entity linking generation and syntax structure prediction diagram provided in embodiment 2.
[0053] Figure 5 A diagram for retrieving context samples and generating SQL based on structure metrics provided in embodiment 2.
[0054] Figure 6 A SQL generation diagram provided in embodiment 2.
[0055] Figure 7 A diagram for guiding self-correction based on syntax semantic analysis provided in embodiment 2.
[0056] Figure 8 A mechanism diagram for guiding self-correction provided in embodiment 2.
[0057] Figure 9 A high-quality context sample and self-correction-based large language model SQL generation system structure diagram provided in embodiment 3. DETAILED DESCRIPTION
[0058] The drawings are only used for illustrative description and cannot be understood as a limitation of the patent;
[0059] In order to better illustrate the embodiments, some components in the drawings may be omitted, enlarged or reduced, and do not represent the actual product size;
[0060] For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.
[0061] The technical solutions of the application will be further described below in combination with the drawings and embodiments.
[0062] Embodiment 1
[0063] As Figure 1 shown, the embodiment provides a large language model SQL generation method based on high-quality context samples and self-correction, including the following steps:
[0064] S1: data acquisition: acquiring a natural language query statement to be generated SQL, and acquiring a context sample set based on a specified content database; the context sample set includes a plurality of natural language query statements annotated with SQL;
[0065] S2: entity linking generation: using a preset large language model to respectively perform entity linking extraction on the natural language query statement to be generated SQL and the context sample set;
[0066] S3: structure prediction: inputting the context sample set and its corresponding entity link into a large language model, and training the large language model to perform structure prediction of SQL statements, to obtain a large language model after structure training;
[0067] Input the natural language query statement to be generated SQL and its corresponding entity link into the large language model after structure training to perform structure prediction of SQL statements, to obtain a predicted SQL statement structure;
[0068] S4: context sample retrieval: in the context sample set, retrieve natural language query statements with a similarity greater than a preset first threshold to the predicted SQL statement structure, as a structure similar context sample set;
[0069] In the structure similar context sample set, further retrieve natural language statements with a semantic similarity greater than a preset second threshold to the natural language query statement to be generated SQL, as a high-quality context sample set;
[0070] S5: SQL generation: inputting the high-quality context sample set and its corresponding entity link into a large language model, and training the large language model to perform content prediction of SQL statements, to obtain a large language model after content training;
[0071] Input the natural language query statement to be generated SQL and its corresponding entity link into the large language model after content training to perform content prediction of SQL statements, to generate a SQL prediction statement;
[0072] S6: SQL execution detection: performing execution detection on the SQL prediction statement, if executable, directly execute; if not executable, execute step S7;
[0073] S7: syntax semantic analysis: using a preset syntax analysis engine to detect syntax errors in the SQL prediction statement, to obtain syntax error information;
[0074] The semantic error information is obtained by detecting a semantic error in the SQL prediction statement by using a preset semantic analysis engine.
[0075] S8: SQL guided self-correction: inputting the syntax error information and the semantic error information into the large language model after content training, guiding the large language model after content training to correct errors in the SQL prediction statement, obtaining a corrected SQL statement, and re-executing step S6 to re-perform SQL execution detection.
[0076] In the specific implementation process, the method mainly includes the following steps:
[0077] 1) Entity link generation: inputting a natural language query statement and corresponding database architecture information, and using a large language model to extract entity links;
[0078] 2) Syntax structure prediction: inputting a natural language query statement and entity links, and giving a plurality of artificially annotated context samples demonstrating how to perform syntax structure prediction, and a large language model will predict what structure the natural language query statement can generate according to the demonstration in these context samples;
[0079] 3) Context sample preprocessing and retrieval: given some context data composed of natural language, database architecture information and SQL, first, perform entity link generation on them, and thus obtain context samples for demonstrating how to perform SQL generation;
[0080] Then, according to the syntax structure of the natural language query statement to be generated, perform natural language semantic similarity retrieval in the structure-similar SQL generation demonstration context samples, select a plurality of context samples that are similar in both SQL syntax structure and natural language semantics, and these samples are used as high-quality context samples for guiding subsequent large language model to generate SQL statements;
[0081] 4) SQL generation: combining the high-quality context samples retrieved in the previous step, the natural language query statement to be generated and the entity links thereof, and inputting them into a large language model; since the SQL syntax structure has been predicted, and the high-quality context samples similar in both syntax structure and semantics have been retrieved, the large language model will imitate the SQL statement content in the high-quality context samples according to the description of the natural language query statement and the content of the entity links, generate the SQL statement content corresponding to the given natural language query statement, and obtain a SQL prediction statement;
[0082] 5) SQL execution detection: attempting to execute the SQL prediction statement on a database, if executable, jumping to step 8), otherwise, continuing with step 6);
[0083] 6) Syntax semantic analysis: use the syntax analysis engine to detect syntax errors in the SQL prediction statement, and use the semantic analysis engine to identify semantic errors related to the database content; the syntax analysis engine and the semantic analysis engine will detect errors hidden in the SQL prediction statement and output error information;
[0084] 7) Guided self-correction: input the incorrect SQL statement and the error information given by the syntax analysis engine and the semantic analysis engine in step 6), and the large language model will repair the SQL according to the error information. After the repair is completed, step 5) is re-executed to re-detect SQL execution;
[0085] 8) SQL output: the generated SQL is output to the place specified by the user;
[0086] 9) Generation end: repeat steps 1) ~ 8) several times until all natural language query statements to be generated SQL are converted to SQL statements, and the generation ends;
[0087] The method proposes a new context learning technology. A large number of existing Text-to-SQL datasets can be used to construct a context sample pool to enhance the coverage of various SQL generation scenarios. By retrieving more matching context samples as high-quality samples, the guiding ability of context learning to large language models can be improved, thereby improving the accuracy of large language models in generating SQL statements, while reducing the cost and cost of context learning using large language models.
[0088] In addition, the method proposes a new SQL repair technology, which effectively improves the identification and correction ability of large language models for errors hidden in SQL.
[0089] Embodiment 2
[0090] As shown in Figure 2 The embodiment provides a large language model SQL generation method based on high-quality context samples and self-correction, including the following steps:
[0091] S1: Data acquisition: acquire natural language query statements to be generated SQL, and acquire context sample set based on specified content database; the context sample set includes several natural language query statements annotated with SQL;
[0092] S2: Entity linking generation: use a pre-set large language model to perform entity linking extraction on the natural language query statements to be generated SQL and the context sample set, respectively;
[0093] S3: structure prediction: input the context sample set and its corresponding entity link into a large language model, and train the large language model for structure prediction of the SQL statement, to obtain the large language model after structure training;
[0094] The natural language query statement and its corresponding entity link are input into the large language model after structure training for structure prediction of the SQL statement, and the predicted SQL statement structure is obtained;
[0095] S4: context sample retrieval: in the context sample set, retrieve the natural language query statement with a similarity greater than a preset first threshold to the predicted SQL statement structure as a structure similar context sample set;
[0096] In the structure similar context sample set, further retrieve the natural language query statement with a semantic similarity greater than a preset second threshold to the natural language query statement to be generated as a high-quality context sample set;
[0097] S5: SQL generation: input the high-quality context sample set and its corresponding entity link into a large language model, and train the large language model for content prediction of the SQL statement, to obtain the large language model after content training;
[0098] The natural language query statement and its corresponding entity link are input into the large language model after content training for content prediction of the SQL statement, and the SQL prediction statement is generated;
[0099] S6: SQL execution detection: perform execution detection on the SQL prediction statement, if executable, directly execute; if not executable, execute step S7;
[0100] S7: syntax and semantic analysis: use a preset syntax analysis engine to detect syntax errors in the SQL prediction statement, and obtain syntax error information;
[0101] Use a preset semantic analysis engine to detect semantic errors in the SQL prediction statement, and obtain semantic error information;
[0102] S8: SQL guided self-correction: input the syntax error information and semantic error information into the large language model after content training, and guide the large language model after content training to correct the SQL prediction statement, to obtain the corrected SQL statement, and re-execute step S6 to perform SQL execution detection again;
[0103] In step S2, the entity link includes the table, column and value data required for generating the SQL statement;
[0104] In step S3, the structure of the SQL statement includes any one of a simple structure, a non-nested complex structure, and a nested complex structure;
[0105] The step S4 further includes: dividing all natural language query statements annotated with SQL in the context sample set into several categories through a clustering operation, randomly extracting several samples from each category, and using all the extracted samples as another high-quality context sample set;
[0106] In this embodiment, the ANTLR-based SQL lexical analyzer obtains the syntax analysis engine preset in step S7;
[0107] In step S7, the grammatical error information includes: a grammatical error category label, a location of the grammatical error in the predicted SQL statement, and a grammatical error correction strategy;
[0108] The semantic error information includes: semantic error category label, location of the semantic error in the SQL prediction statement, and semantic error correction countermeasures;
[0109] The step S8 further includes: inputting the syntax error information and the semantic error information into a preset neural network analyzer, guiding the neural network analyzer to correct errors in the SQL prediction statement, obtaining the corrected SQL statement, re-performing step S6, and re-performing SQL execution detection;
[0110] The preset neural network analyzer is specifically a small or medium-sized language model fine-tuned on the Debug task dataset.
[0111] In the specific implementation process, we first preprocess the SQL to generate a demonstration context sample, such as Figure 3 As shown, this embodiment uses a large language model to generate entity links and predict grammatical structures on a SQL dataset containing a large number of samples (such as training sets of Spider and WikiTQ). This generates a context sample set that includes natural language queries, SQL, and entity links. The predicted grammatical structures are then classified into simple structures, non-nested complex structures, and nested complex structures. Since the SQL statements are known in this step, the generated entity links and grammatical structures are considered correct (i.e., "writing the process by looking at the answer").
[0112] like Figure 4 As shown, given a natural language query question to be generated for the corresponding SQL and the database associated with it, a large language model is used to generate entity links. The entity links include the tables, columns, and value data required to generate SQL statements. Then, the generated entity links and natural language questions are used, and given a grammatical structure prediction context sample, the large language model is used to predict the grammatical structure.
[0113] Part of the context sample for structure prediction is as follows:
[0114] Sample 1) Q: "Find the name of the department that offers the highest total credits?"
[0115] schema_links: [course.dept_name, course.credits]
[0116] A: Let's think step by step. The SQL query for the question "Find the name of the department that offers the highest total credits?." needs the set tables = [course], so we don't need JOIN.
[0117] Plus, it doesn't require nested queries with (INTERSECT, UNION, EXCEPT, IN, NOT IN), and we need the answer to the questions = [""].
[0118] So, we don't need JOIN and don't need nested queries, then the SQL query can be classified as "EASY".
[0119] Label: "EASY";
[0120] Sample 2) Q: "What is the name of the instructor who advises the student with the greatest number of total credits?"
[0121] schema_links: [advisor.i_id = instructor.id,advisor.s_id =student.id,instructor.name,student.tot_cred ]
[0122] A: Let’s think step by step. The SQL query for the question "What isthe name of the instructor who advises the student with the greatest numberof total credits?" needs these tables = [advisor,instructor,student], so weneed JOIN.
[0123] Plus, it doesn't need nested queries with (INTERSECT, UNION, EXCEPT,IN, NOT IN), and we need the answer to the questions = [""].
[0124] So, we need JOIN and don't need nested queries, then the the SQLquery can be classified as "NON-NESTED".
[0125] Label: "NON-NESTED";
[0126] Sample 3) Q: "Find the id of instructors who taught a class in Fall 2009but not in Spring 2010."
[0127] schema_links: [teaches.id,teaches.semester,teaches.year,Fall,2009,Spring,2010]
[0128] A: Let's think step by step. The SQL query for the question "Find the id of instructors who taught a class in Fall 2009 but not in Spring 2010."needs these tables = [teaches], so we don't need JOIN.
[0129] Plus, it requires nested queries with (INTERSECT, UNION, EXCEPT, IN,NOT IN), and we need the answer to the questions = ["Find the id ofinstructors who taught a class in Spring 2010"].
[0130] So, we don't need JOIN and need nested queries, then the the SQLquery can be classified as "NESTED".
[0131] Label: "NESTED";
[0132] As Figure 5 shown, then based on the structure metric, the context sample retrieval is performed: in the context sample set, the natural language query statements similar to the predicted SQL statement structure are retrieved as the context sample set similar in structure; in the context sample set similar in structure, the natural language statements similar in semantics to the natural language query statement to be generated are further retrieved as the high-quality context sample set;
[0133] The range of similarity metric is 0~1, the closer to 0, the more dissimilar; according to actual observation, when the similarity is greater than 0.7, two samples can be considered very similar; and less than 0.3 can be considered very dissimilar; therefore, the first and second thresholds in this embodiment are both 0.7;
[0134] In this embodiment, as Figure 6As shown in the figure, since the grammatical structure predicted in the above steps is "simple", given the natural language query statement, the semantic detection model is used to sort the simple structure context samples in descending order according to semantic similarity, and the top n most similar context samples are selected; then, the selected high-quality context samples are input into the large language model together with the natural language query statement and entity linking, and the large language model generates an SQL prediction statement;
[0135] In this embodiment, when retrieving high-quality context samples, the search conditions can be replaced by using other measurement methods (such as searching based on the diversity measurement of context samples instead of the similarity measurement, or directly performing semantic similarity measurement on SQL after pre-generating SQL) to retrieve context samples;
[0136] For example, the diversity metric can be: in the context sample set, all natural language query statements annotated with SQL are divided into several categories through clustering operations (such as k-means algorithm), and several samples are randomly selected from each category, and all the samples are used as another high-quality context sample set.
[0137] Then, the generated SQL prediction statement is executed: an attempt is made to execute the SQL prediction statement on the database. If it is executable, it is executed directly.
[0138] If not executable, then Figure 7 As shown, a syntax analysis engine is constructed based on an ANTLR SQL lexical analyzer, and the syntax analysis engine is used to detect syntax errors in SQL prediction statements. After the syntax analysis engine completes lexical analysis of the SQL, it obtains the attributes of each element in the SQL (table name, column name, value). Therefore, based on the syntax analysis engine, the semantic analysis engine compares the identified table names, column names, and values with the information in the database and the values mentioned in the natural language question. If a table name, column name, or value does not exist, an error is raised at that location, and candidate table names, column names, and values that should be filled in at that location are provided. The syntax analysis engine and the semantic analysis engine will detect potential errors in the SQL prediction statement and output error information. The large language model guides self-correction based on the error information. In this embodiment, the error information includes a label corresponding to the error category, the location of the corresponding error in the SQL prediction statement, and a correction strategy for the corresponding error.
[0139] The specific repair process is as follows Figure 8As shown, two engine output analysis error information, and then natural language query statement, database information, wrong SQL statement and error information are self-corrected using a large language model to repair the SQL; in addition, the embodiment can also use a neural network analyzer to analyze and repair the generated SQL prediction statement, for example, using an additional small and medium-sized language model fine-tuned on the Debug task dataset to analyze and repair the SQL statement;
[0140] After the repair is completed, the SQL execution detection is performed again;
[0141] The method proposes a new context learning technology, which can use a large number of existing Text-to-SQL datasets to construct a context sample pool to enhance the coverage of various SQL generation scenarios, retrieve more matched context samples as high-quality samples, improve the guiding ability of context learning to large language models, and thus improve the accuracy of large language model generated SQL statements, while reducing the cost and cost of context learning using large language models;
[0142] In addition, the method proposes a new SQL repair technology, which effectively improves the identification and correction ability of large language models for errors hidden in SQL.
[0143] Embodiment 3
[0144] As Figure 9 shown, the embodiment provides a large language model SQL generation system based on high-quality context samples and self-correction, which applies the large language model SQL generation method based on high-quality context samples and self-correction described in embodiment 1 or 2, comprising:
[0145] The data acquisition unit 301 is configured to acquire a natural language query statement to be generated SQL, and acquire a context sample set based on a specified content database; the context sample set comprises a plurality of natural language query statements labeled with SQL;
[0146] The entity link generation unit 302 is configured to use a preset large language model to perform entity link extraction on the natural language query statement to be generated SQL and the context sample set, respectively;
[0147] The structure prediction unit 303 is configured to input the context sample set and its corresponding entity link into a large language model, and train the large language model to perform structure prediction of SQL statements, to obtain a large language model after structure training;
[0148] The natural language query statement to be generated SQL and its corresponding entity link are input into the large language model after structure training to perform structure prediction of SQL statements, to obtain a predicted SQL statement structure;
[0149] The context sample retrieval unit 304 retrieves, in the context sample set, a natural language query statement with a predicted SQL statement structure similarity greater than a preset first threshold as a structure similar context sample set;
[0150] In the structure similar context sample set, further retrieve a natural language statement with a semantic similarity greater than a preset second threshold with the natural language query statement to be generated as a high-quality context sample set;
[0151] The SQL generation unit 305 is configured to input the high-quality context sample set and its corresponding entity link into a large language model, and train the large language model for content prediction of the SQL statement, to obtain a content trained large language model;
[0152] The natural language query statement to be generated and its corresponding entity link are input into the content trained large language model for content prediction of the SQL statement, to generate a SQL prediction statement;
[0153] The SQL execution detection unit 306 is configured to perform execution detection on the SQL prediction statement, and if executable, directly execute; if not executable, perform the syntax semantic analysis unit and the SQL guided self-correction unit;
[0154] The syntax semantic analysis unit 307 is configured to detect syntax errors in the SQL prediction statement using a preset syntax analysis engine, to obtain syntax error information;
[0155] The semantic analysis engine is used to detect semantic errors in the SQL prediction statement, to obtain semantic error information;
[0156] The SQL guided self-correction unit 308 is configured to input the syntax error information and the semantic error information into the content trained large language model, and guide the content trained large language model to correct errors in the SQL prediction statement, to obtain a corrected SQL statement, and re-execute the SQL execution detection unit to re-perform SQL execution detection.
[0157] In the specific implementation process, first, the data acquisition unit 301 acquires a natural language query statement to be generated, and acquires a context sample set based on a specified content database;
[0158] The entity link generation unit 302 uses a preset large language model to perform entity link extraction on the natural language query statement to be generated and the context sample set, respectively;
[0159] Then the structure prediction unit 303 performs structure prediction training on the large language model, and performs SQL structure prediction on the natural language query sentence to be generated;
[0160] Then the context sample retrieval unit 304 retrieves context samples similar in structure and semantics to obtain a high-quality context sample set;
[0161] The SQL generation unit 305 inputs the high-quality context sample set into the large language model for SQL content prediction training, and generates a final SQL prediction statement;
[0162] Then the SQL execution detection unit 306 performs SQL execution detection, if executable, directly execute; if not executable, the syntax and semantic analysis unit 307 detects error information using a syntax and semantic analysis engine, the SQL guided self-correction unit 308 guides the large language model to guide self-correction of the SQL prediction statement error using the error information, and then re-performs SQL execution detection;
[0163] The system can use a large number of existing Text-to-SQL data sets to construct a context sample pool, to enhance the coverage of various SQL generation scenarios, and by retrieving more matching context samples as high-quality samples, the guiding ability of context learning to the large language model can be improved, and the accuracy of the large language model in generating SQL statements can be improved, while reducing the cost and cost of using the large language model for context learning;
[0164] In addition, the system also improves the recognition and correction ability of the large language model for the errors hidden in the SQL.
[0165] The same or similar reference numerals correspond to the same or similar components;
[0166] The terms describing the positional relationship in the drawings are only used for illustrative description, and cannot be understood as a limitation on the patent;
[0167] Obviously, the above embodiments of the application are only examples for clearly illustrating the application, and are not intended to limit the embodiments of the application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the embodiments need not and cannot be exhausted. Any modification, equivalent replacement and improvement made within the spirit and principle of the application shall be included in the protection scope of the claims of the application.
Claims
1. A large language model SQL generation method based on high-quality context samples and self-correction, characterized by, The method comprises the following steps: S1: data acquisition: acquiring a natural language query statement to be generated into an SQL, and acquiring a context sample set based on a specified content database; the context sample set comprises a plurality of natural language query statements annotated with SQLs; S2: entity linking generation: using a preset large language model to perform entity linking extraction on the natural language query statement to be generated into an SQL and the context sample set respectively; S3: structure prediction: inputting the context sample set and the corresponding entity linking into a large language model, and training the large language model to perform structure prediction of an SQL statement, to obtain a large language model after structure training; inputting the natural language query statement to be generated into an SQL and the corresponding entity linking into the large language model after structure training to perform structure prediction of an SQL statement, and obtaining a predicted SQL statement structure; S4: context sample retrieval: in the context sample set, retrieving a natural language query statement with a similarity greater than a preset first threshold from the predicted SQL statement structure as a structure-similar context sample set; in the structure-similar context sample set, further retrieving a natural language statement with a semantic similarity greater than a preset second threshold from the natural language query statement to be generated into an SQL as a high-quality context sample set; S5: SQL generation: inputting the high-quality context sample set and the corresponding entity linking into a large language model, and training the large language model to perform content prediction of an SQL statement, to obtain a large language model after content training; inputting the natural language query statement to be generated into an SQL and the corresponding entity linking into the large language model after content training to perform content prediction of an SQL statement, to generate an SQL prediction statement; S6: SQL execution detection: performing execution detection on the SQL prediction statement, and if executable, directly executing; if not executable, performing step S7; S7: syntax and semantic analysis: using a preset syntax analysis engine to detect syntax errors in the SQL prediction statement, to obtain syntax error information; using a preset semantic analysis engine to detect semantic errors in the SQL prediction statement, to obtain semantic error information; S8: SQL guided self-correction: inputting the syntax error information and the semantic error information into the large language model after content training, and guiding the large language model after content training to correct errors in the SQL prediction statement, to obtain a corrected SQL statement, and re-performing step S6 to perform SQL execution detection again.
2. The large language model SQL generation method based on high-quality context samples and self-correction according to claim 1, characterized in that, In step S2, entity linking includes table, column, and value data required for generating an SQL statement.
3. The large language model SQL generation method based on high-quality context samples and self-correction according to claim 1, characterized in that, In step S3, the structure of an SQL statement includes any one of a simple structure, a non-nested complex structure, and a nested complex structure.
4. The large language model SQL generation method based on high-quality context samples and self-correction according to claim 1, characterized in that, In step S4, further comprising: in the context sample set, dividing all natural language query statements annotated with SQLs into a plurality of classes through clustering operation, and randomly extracting a plurality of samples in each class, and taking all extracted samples as another high-quality context sample set.
5. The large language model SQL generation method based on high-quality context samples and self-correction according to claim 1, characterized in that, The preset syntax analysis engine in step S7 is obtained based on an ANTLR SQL lexical analyzer.
6. The large language model SQL generation method based on high-quality context samples and self-correction according to claim 1, characterized in that, The syntax error information includes a syntax error category label, a position of the syntax error in the SQL predicted statement, and a syntax error modification countermeasure. The semantic error information includes a semantic error category label, a position of the semantic error in the SQL predicted statement, and a semantic error modification countermeasure.
7. The large language model SQL generation method based on high-quality context samples and self-correction according to claim 1, characterized in that, In the step S8, the syntax error information and the semantic error information are input into a preset neural network analyzer, and the neural network analyzer is guided to correct errors in the SQL predicted statement to obtain a corrected SQL statement, and the step S6 is re-executed to perform SQL execution detection again.
8. The large language model SQL generation method based on high-quality context samples and self-correction according to claim 7, characterized in that, The preset neural network analyzer is specifically a small and medium-sized language model fine-tuned on a Debug task data set.
9. A large language model SQL generation system based on high-quality context samples and self-correction, applying the large language model SQL generation method based on high-quality context samples and self-correction in any one of claims 1-8, characterized in that, The method comprises the following steps: a data acquisition unit configured to acquire a natural language query statement to be generated into an SQL, and acquire a context sample set based on a specified content database; the context sample set comprises a plurality of natural language query statements annotated with an SQL; an entity linking generation unit configured to perform entity linking extraction on the natural language query statement to be generated into an SQL and the context sample set respectively by using a preset large language model; a structure prediction unit configured to input the context sample set and its corresponding entity linking into a large language model, and train the large language model to perform structure prediction of an SQL statement, to obtain a large language model trained in structure; input the natural language query statement to be generated into an SQL and its corresponding entity linking into the large language model trained in structure to perform structure prediction of an SQL statement, to obtain a predicted SQL statement structure; a context sample retrieval unit configured to retrieve, in the context sample set, a natural language query statement having a similarity greater than a preset first threshold with the predicted SQL statement structure, as a context sample set similar in structure; further retrieve, in the context sample set similar in structure, a natural language query statement having a semantic similarity greater than a preset second threshold with the natural language query statement to be generated into an SQL, as a high-quality context sample set; an SQL generation unit configured to input the high-quality context sample set and its corresponding entity linking into a large language model, and train the large language model to perform content prediction of an SQL statement, to obtain a large language model trained in content; input the natural language query statement to be generated into an SQL and its corresponding entity linking into the large language model trained in content to perform content prediction of an SQL statement, to generate an SQL predicted statement; an SQL execution detection unit configured to perform execution detection on the SQL predicted statement, and if the SQL predicted statement is executable, directly execute the SQL predicted statement; if the SQL predicted statement is not executable, execute a syntax and semantic analysis unit and an SQL guided self-correction unit; the syntax and semantic analysis unit is configured to detect syntax errors in the SQL predicted statement by using a preset syntax analysis engine, to obtain syntax error information; detect semantic errors in the SQL predicted statement by using a preset semantic analysis engine, to obtain semantic error information; and the SQL guided self-correction unit is configured to input the syntax error information and the semantic error information into a preset neural network analyzer, and guide the neural network analyzer to correct errors in the SQL predicted statement to obtain a corrected SQL statement. The SQL guiding self-correction unit is configured to input the syntax error information and the semantic error information into the large language model trained by the content, guide the large language model trained by the content to correct errors of the SQL predicted statement, obtain a corrected SQL statement, and re-execute the SQL execution detection unit to re-perform SQL execution detection.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the steps of the method as claimed in any one of claims 1-8.