Method and system for generating nl2sql training set based on large model and suitable for all industries
By constructing NL2SQL training sets based on large model, the problems of insufficient applicability and low construction efficiency in the existing technology are solved, and the generation of cross-industry applicability and high-quality training sets are realized, reducing costs and improving efficiency .
Patent Information
- Application Number
- CN202411156201.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-15
AI Technical Summary
Existing NL2SQL training sets are usually built based on specific fields or industries, and are difficult to adapt to cross-industry application needs. Manually constructing high-quality training sets is expensive and inefficient.
Using a large model-based method, the initial problem set is constructed through table building statements, manually filter common scenario problems, generate seed SQL with verification seeds, expand semantic similar problems, manually verify the expanded problems, generate SQL for the expansion problem, and By comparing the results with the database execution, the fine-tuning training set is finally built to ensure the high quality and accuracy of the training set.
It realizes high-quality NL2SQL training sets suitable for all industries, reduces data annotation costs, improves training set generation efficiency, and has good scalability and accuracy.
Smart Images

Figure CN118964579A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and system for generating an nl2sql training set based on a large model applicable to all industries. Background Art
[0002] With the development of artificial intelligence technology, the natural language to structured query language (NL2SQL) task has become a research hotspot. NL2SQL technology aims to convert natural language queries into SQL query statements in order to retrieve relevant information from the database.
[0003] However, the training of NL2SQL models requires a large amount of labeled data, that is, the pairing of natural language queries with corresponding SQL query statements. Existing NL2SQL training sets are often built based on specific fields or industries, and are difficult to adapt to cross-industry application needs. In addition, manually building large-scale, high-quality NL2SQL training sets is costly and inefficient. Summary of the invention
[0004] The purpose of the present invention is to provide a method and system for generating nl2sql training sets based on a large model applicable to all industries, so as to solve the problems raised in the above background technology.
[0005] To achieve the above object, the present invention provides the following technical solution: a method for generating an nl2sql training set based on a large model applicable to all industries, comprising the following steps:
[0006] Build an initial set of questions based on the table creation statements;
[0007] Manually screen common scenario issues;
[0008] Generate and verify seed SQL;
[0009] Semantic similarity question expansion;
[0010] Manual verification of the expanded issues;
[0011] Generate SQL to expand the question;
[0012] Database execution and result comparison;
[0013] Construct a fine-tuning training set.
[0014] Preferably, the specific operations of constructing the initial question set based on the table creation statement include:
[0015] In the process of building the initial question set, we construct questions by writing a general script, build a general prompt in the script, and call the large model interface. For different industry scenarios, we only need to change the table creation statement in the general prompt and some provided question examples or some provided table data to generate a question set for a specific scenario based on the table structure.
[0016] The specific operations for manually screening common scenario problems include:
[0017] In conjunction with relevant business personnel or customers, the problems are further screened, filtered, and transformed to select a set of seed questions that meet the customer's business scenarios. During the training process, the big model can understand the indicator calculation logic of common scenarios.
[0018] Preferably, the specific operations of generating and verifying the seed SQL include:
[0019] Write an automated script to automatically generate SQL statements corresponding to seed questions by requesting the large model interface, and output SQL statement files. The automated script greatly shortens the time for manual SQL writing, greatly improves the efficiency of building training sets, and manually verifies the generated SQL again;
[0020] The specific operations of semantic similarity question expansion include:
[0021] Conduct in-depth analysis of seed questions and extract key information points, such as the query subject, query conditions, and query purpose. Use large models to reorganize and transform key information points to generate new question statements. Use similarity calculation methods in natural language processing, such as cosine similarity or semantic embedding of the BERT model, to calculate the similarity between new questions and seed questions. For questions with too low or too high similarity, eliminate or merge them through code logic. Use manual review to check the screened questions one by one to ensure that each question is semantically consistent with the seed question and has a reasonable expression.
[0022] Preferably, the specific operations of manually checking the expanded questions include:
[0023] Carefully read each question to understand its query intent and expression, especially the expression of indicators or scenarios. Manually compare the question with the corresponding seed question to check whether they are semantically consistent. For questions with inconsistent semantics, in-depth analysis and discussion with business personnel or customers are required, and corrections are made according to actual conditions. Ambiguities are eliminated by modifying wording, adding contextual information, or clarifying query conditions to make them clearer and more specific. The questions and modification process are recorded so that the training set can be further optimized and improved later.
[0024] The specific operations for generating SQL to expand the problem include:
[0025] Use the big model to generate corresponding SQL statements for these questions, and use the big model's ability to understand natural language query intent to convert natural language questions into structured SQL statements; appropriately tune the big model by building prompts to make it more suitable for the needs of NL2SQL tasks, use the expanded questions as input, and use the tuned big model to generate corresponding SQL statements; during the generation process, pay attention to the grammatical correctness and logical rationality of the SQL statements to ensure that the generated SQL statements correctly reflect the query intent of the problem.
[0026] Preferably, the specific operations of the database execution and result comparison include:
[0027] Connect to an actual database and execute these SQL statements. The current step needs to ensure that all executed SQL statements can query the result data, otherwise the matching results will be inaccurate. The operator executes 300 seed SQL statements and records the query results, writing them into a file. The operator executes the SQL statements of 10 expanded questions corresponding to each seed question respectively, and compares the query results with the query results of the seed SQL. If the SQL query results of the expanded question are consistent with the corresponding seed SQL, it means that the SQL statement is accurate; if they are inconsistent, it means that there is a problem with the SQL statement and it needs to be corrected.
[0028] The specific operations for constructing a fine-tuning training set include:
[0029] Build a general script, fix the data file format, and write the constructed Excel file into a fine-tuning training file that meets the requirements of the large model through specific logic. The generated training set is not only large in number and rich in diversity, but each question corresponds to an accurate SQL statement. As the fine-tuning training set of the large model, it improves the performance and accuracy of the NL2SQL model and enhances the nl2sql capabilities of the large model.
[0030] A system for generating nl2sql training sets based on a large model applicable to all industries, the system comprising:
[0031] An initial question set building module, used to build an initial question set based on a table building statement;
[0032] Scenario problem screening module, used to manually screen common scenario problems;
[0033] Verification seed generation module, used to generate and verify seed SQL;
[0034] Question extension module, used to expand semantically similar questions;
[0035] Verification module, used to manually verify the expanded issues;
[0036] SQL generation module, used to generate SQL for expanding questions;
[0037] Execution and comparison module, used for database execution and result comparison;
[0038] The training set construction module is used to construct the fine-tuning training set.
[0039] Preferably, in the process of constructing the initial question set, the module for constructing the initial question set constructs the questions by writing a general script, constructs a general prompt in the script, and calls the large model interface. For different industry scenarios, it is only necessary to change the table building statement in the general prompt and some provided question examples or some provided table data to generate a question set for a specific scenario based on the table structure;
[0040] The scenario problem screening module, in conjunction with relevant business personnel or customers, further screens, filters, and transforms the problems to select a set of seed problems that meet the customer's business scenarios. During the training process, the big model can understand the indicator calculation logic of common scenarios.
[0041] Preferably, the seed generation module is verified, and an automated script is written to automatically generate SQL statements corresponding to the seed questions by requesting the large model interface, and output SQL statement files. Through the automated script, the time for manually writing SQL is greatly shortened, the efficiency of building the training set is greatly improved, and the generated SQL is manually verified again;
[0042] The question expansion module conducts in-depth analysis of seed questions, extracts key information points, such as the query subject, query conditions, and query purpose, and uses a large model to reorganize and transform key information points to generate new question expressions. It uses similarity calculation methods in natural language processing, such as cosine similarity or semantic embedding of the BERT model, to calculate the similarity between new questions and seed questions. For questions with too low or too high similarity, they are eliminated or merged through code logic. The screened questions are checked one by one through manual review to ensure that each question is semantically consistent with the seed question and has a reasonable expression.
[0043] Preferably, the verification module reads each question carefully to understand its query intent and expression, especially the expression of indicators or scenarios, and manually compares the question with the corresponding seed question to check whether they are semantically consistent. For semantically inconsistent questions, in-depth analysis and discussion with business personnel or customers are required, and corrections are made according to actual conditions; ambiguity is eliminated by modifying the wording, adding contextual information or clarifying the query conditions to make it clearer and more specific, and the questions and modification process are recorded so as to further optimize and improve the training set later;
[0044] The SQL generation module uses the big model to generate corresponding SQL statements for these questions, and uses the big model's ability to understand natural language query intent to convert natural language questions into structured SQL statements. By building prompts, the big model is properly tuned to make it more suitable for the needs of the NL2SQL task, and the expanded questions are used as input to generate corresponding SQL statements using the tuned big model. During the generation process, attention is paid to the grammatical correctness and logical rationality of the SQL statements to ensure that the generated SQL statements correctly reflect the query intent of the question.
[0045] Preferably, the execution and comparison module is connected to an actual database and executes these SQL statements. The current step needs to ensure that all executed SQL statements can query the result data, otherwise the matching results will be inaccurate; the operator executes 300 seed SQL statements and records the query results, and writes them into a file. The operator executes the SQL statements of 10 expanded questions corresponding to each seed question respectively, and compares the query results with the query results of the seed SQL. If the SQL query results of the expanded questions are consistent with the corresponding seed SQL, it means that the SQL statement is accurate; if they are inconsistent, it means that there is a problem with the SQL statement and it needs to be corrected;
[0046] The training set construction module builds a general script, fixes the data file format, and writes the constructed Excel file into a fine-tuning training file that meets the requirements of the large model through specific logic. The generated training set is not only large in number and rich in diversity, but each question corresponds to an accurate SQL statement. As the fine-tuning training set of the large model, it improves the performance and accuracy of the NL2SQL model and enhances the nl2sql capabilities of the large model.
[0047] Compared with the prior art, the present invention has the following beneficial effects:
[0048] The method and system for generating nl2sql training sets based on a large model, which are applicable to all industries, are based on common table creation statements and large model generation problems and are applicable to all industries requiring data query. Through manual screening, verification and correction, the generated training sets are ensured to be of high quality and accuracy. The automated and intelligent generation process reduces the data annotation cost and improves the training set generation efficiency. The number and proportion of problems and SQL are adjusted according to actual business scenarios, and the system has good scalability. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION
[0050] In order to make the purpose and technical solution of the present invention clearly and completely described, and the advantages more clearly understood, the embodiments of the present invention are further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, rather than all of the embodiments, and are only used to explain the embodiments of the present invention, and are not used to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] For example, see Figure 1 The present invention provides a technical solution: a method for generating nl2sql training sets based on a large model applicable to all industries, comprising the following steps:
[0052] Step 1: Build an initial set of questions based on table creation statements
[0053] First, commonly used database table creation statements can be collected from a single industry or multiple industries according to specific business needs. These table creation statements not only contain the structural information of the table, but also imply possible data query requirements. Using large models, such as BERT, GPT, etc., these table creation statements are deeply parsed to automatically generate natural language questions closely related to the table structure. For example, for an employee table containing fields such as "employee number", "name", and "salary", the large model may generate questions such as "query the name of the employee with the highest salary" or "find employees with a salary less than 5000". Through this step, we can get about 3,000 questions related to the table structure to form an initial question set. The present invention only provides a method, and the specific number of questions generated needs to be dynamically changed in combination with actual scenario business applications.
[0054] In the process of constructing the initial question set, the present invention proposes that the question construction can be performed by writing a general script. A general prompt is constructed in the script, and the large model interface is called. For different industry scenarios, only the table building statement in the general prompt and some provided question examples or some provided table data need to be changed to generate a question set for a specific scenario based on the table structure.
[0055] Step 2: Manually screen common scenario issues
[0056] Although the big model can generate a large number of questions, not all questions meet the actual query requirements or common scenarios. Therefore, the present invention requires manual screening of these 3,000 questions. During the screening process, the present invention proposes to focus on those questions that frequently appear in practical applications, are representative, and can comprehensively reflect various aspects of data query (such as selection, filtering, sorting, aggregation, etc.). The questions can be further screened, filtered, and transformed in conjunction with relevant business personnel or customers to screen out a set of seed questions that meet the customer's business scenarios, so that during the training process, the big model can more easily understand the indicator calculation logic of common scenarios. After this step, 300 (the number is for reference only) seed questions can be selected from the initial question set.
[0057] Step 3: Generate and verify seed SQL
[0058] Next, the present invention requires to use the big model again to generate corresponding SQL statements for these 300 seed questions. Since the big model has a high accuracy in understanding the natural language query intent, the generated SQL statements can correctly reflect the query requirements of the questions to a large extent. In the current step, the present invention proposes to write an automated script, automatically generate SQL statements corresponding to the seed questions by requesting the big model interface, and output the SQL statement file. Through the automated script, the time for manually writing SQL can be greatly shortened, greatly improving the efficiency of building the training set.
[0059] However, in order to ensure the absolute accuracy of the SQL statement, the present invention proposes to manually verify the generated SQL again. During the verification process, once an inaccurate SQL statement is found, the verification personnel are required to correct the SQL to ensure that each seed question corresponds to an accurate SQL statement. Only under the premise of ensuring the accuracy of the training set can the SQL generated by the large model be more accurate, avoiding the misleading of the large model due to inaccurate training data sets, or the same scene or indicator in the data set but inconsistent calculation logic before and after, resulting in a low accuracy rate of SQL generated by the large model.
[0060] Step 4: Expanding semantic similarity questions
[0061] After obtaining 300 carefully selected seed questions, the present invention requires further enriching the diversity of the training set by expanding with semantically similar questions. The core of this step is to use the large model's ability to understand natural language to generate new questions that are semantically similar to the seed questions but expressed differently.
[0062] To achieve this goal, the present invention proposes to first conduct an in-depth analysis of the seed question to extract key information points, such as the query subject, query conditions, query purpose, etc. Then, these key information points are reorganized and transformed using a large model to generate a new question statement.
[0063] In order to ensure the quality of the expanded questions, the present invention adopts a variety of strategies to screen the semantically similar questions generated by the large model. First, the present invention proposes to use the similarity calculation method in natural language processing, such as cosine similarity or semantic embedding of the BERT model, to calculate the similarity between the new question and the seed question. For questions with too low or too high similarity, they are eliminated or merged through code logic. Secondly, the present invention proposes to use manual review to check the screened questions one by one to ensure that each question is semantically consistent with the seed question and has a reasonable expression.
[0064] After this step of processing, the 300 seed questions were successfully expanded to 3,000 semantically similar questions, greatly enriching the diversity of the training set.
[0065] Step 5: Manually verify the expanded issues
[0066] After obtaining 3000 semantically similar questions, the present invention requires manual verification again to ensure the accuracy and rationality of these questions. This step is a crucial link in the training set construction process because it is directly related to the quality of the final training set.
[0067] During the manual verification process, the present invention requires that each question be carefully read to understand its query intent and expression, especially the expression of indicators or scenarios. Then, the present invention requires that these questions be manually compared with the corresponding seed questions to check whether they are semantically consistent. For semantically inconsistent questions, in-depth analysis and discussion with business personnel or customers are required, and corrections are made according to actual conditions.
[0068] In addition, the present invention will pay special attention to some issues that are vague or ambiguous. For these issues, the present invention requires attempts to eliminate ambiguity by modifying the wording, adding contextual information, or clarifying the query conditions to make them clearer and more specific. At the same time, these issues and the modification process are recorded so that the training set can be further optimized and improved later.
[0069] Through manual verification in this step, we can ensure that the expanded question is semantically consistent with the seed question and has a reasonable expression and clear query intent, which lays a solid foundation for the subsequent generation of accurate SQL statements.
[0070] Step 6: Generate SQL to expand the question
[0071] After confirming the accuracy and rationality of the expanded questions, the present invention proposes to use the big model to generate corresponding SQL statements for these questions. The key to this step is to use the big model's ability to understand the natural language query intent to convert natural language questions into structured SQL statements.
[0072] To achieve this goal, the present invention proposes to properly tune the large model by constructing prompts to make it more suitable for the needs of NL2SQL tasks. Then, the expanded question is used as input, and the corresponding SQL statement is generated using the tuned large model. During the generation process, special attention should be paid to the grammatical correctness and logical rationality of the SQL statement to ensure that the generated SQL statement can correctly reflect the query intent of the question.
[0073] In this step, you can manually perform a simple check on the SQL to ensure that there are no syntax problems. If manpower and time are limited, you can skip the check and leave it to the next step.
[0074] Step 7: Database execution and result comparison
[0075] In order to further verify the accuracy of the generated SQL statements, the present invention requires connecting to an actual database and executing these SQL statements. The current step needs to ensure that all executed SQL can query the result data, otherwise it will lead to inaccurate matching results. First, the operator executes 300 seed SQL statements and records the query results, writing them into a file. Then, the operator executes the SQL statements of the 10 expanded questions corresponding to each seed question respectively, and compares the query results with the query results of the seed SQL. If the SQL query results of the expanded question are consistent with the corresponding seed SQL, it means that the SQL statement is accurate; if not, it means that there is a problem with the SQL statement and it needs to be corrected.
[0076] In this step, the present invention also adopts an intelligent replacement strategy: once the SQL query result of the expanded question is found to be inconsistent with the corresponding seed SQL, the present invention requires that the SQL statement be immediately replaced with the corresponding seed SQL. The advantage of this is that even if there are errors in the SQL statement of the expanded question, the correct SQL statement can be guaranteed to be included in the training set.
[0077] Step 8: Build a fine-tuning training set
[0078] After strict screening and verification of the above steps, a high-quality training data document file containing 3,000 questions and corresponding SQL statements is finally obtained. However, the current file does not conform to the format of fine-tuning training. The present invention proposes to build a general script, fix the data file format, and write the above-constructed excel file into a fine-tuning training file that meets the requirements of the large model through specific logic. The generated training set is not only huge in number and rich in diversity, but also each question corresponds to an accurate SQL statement. Therefore, as a fine-tuning training set for a large model, it can further improve the performance and accuracy of the NL2SQL model and enhance the nl2sql capabilities of the large model.
[0079] The technical solution not only solves the time-consuming and laborious problem of manually constructing NL2SQL training sets, but also ensures the quality and quantity of the training sets by combining the automatic generation of large models with manual verification. At the same time, the technical solution also considers the expansion of semantic similarity problems and the comparison of database execution results, further improving the diversity and accuracy of the training sets. Therefore, the technical solution is of great significance to promoting the development and application of NL2SQL technology.
[0080] Embodiment 2
[0081] Based on the first embodiment, a system for generating nl2sql training sets based on a large model applicable to all industries is proposed, and the system includes:
[0082] The initial question set construction module is used to construct the initial question set based on the table creation statement. In the process of constructing the initial question set, questions are constructed by writing a general script, a general prompt is built in the script, and the large model interface is called. For different industry scenarios, only the table creation statement in the general prompt and some provided question examples or some provided table data need to be changed to generate a question set for a specific scenario based on the table structure.
[0083] The scenario problem screening module is used to manually screen common scenario problems. In conjunction with relevant business personnel or customers, the problems are further screened, filtered, and transformed to select a set of seed problems that meet the customer's business scenarios. During the training process, the big model can understand the indicator calculation logic of common scenarios.
[0084] The verification seed generation module is used to generate and verify the seed SQL; write an automated script to automatically generate the SQL statement corresponding to the seed question by requesting the large model interface, and output the SQL statement file. Through the automated script, the time for manual SQL writing is greatly shortened, the efficiency of building the training set is greatly improved, and the generated SQL is manually verified again.
[0085] The question expansion module is used to expand semantically similar questions. It conducts in-depth analysis on seed questions, extracts key information points, such as query subject, query conditions, and query purpose, and uses a large model to reorganize and transform key information points to generate new question expressions. It uses similarity calculation methods in natural language processing, such as cosine similarity or semantic embedding of the BERT model, to calculate the similarity between new questions and seed questions. For questions with too low or too high similarity, they are eliminated or merged through code logic. Manual review is used to check the screened questions one by one to ensure that each question is semantically consistent with the seed question and has a reasonable expression.
[0086] The verification module is used to manually verify the expanded questions. Each question is carefully read to understand its query intent and expression, especially the expression of indicators or scenarios. The question is manually compared with the corresponding seed question to check whether they are semantically consistent. For semantically inconsistent questions, in-depth analysis and discussion with business personnel or customers are required, and corrections are made according to actual conditions. Ambiguity is eliminated by modifying the wording, adding contextual information or clarifying the query conditions to make it clearer and more specific. The questions and modification process are recorded to facilitate further optimization and improvement of the training set.
[0087] The SQL generation module is used to generate SQL for expanded questions. The big model is used to generate corresponding SQL statements for these questions, and the big model's ability to understand natural language query intent is used to convert natural language questions into structured SQL statements. The big model is properly tuned by building prompts to make it more suitable for the needs of NL2SQL tasks. The expanded questions are used as input, and the corresponding SQL statements are generated using the tuned big model. During the generation process, attention is paid to the grammatical correctness and logical rationality of the SQL statements to ensure that the generated SQL statements correctly reflect the query intent of the questions.
[0088] The execution and comparison module is used for database execution and result comparison; it connects to an actual database and executes these SQL statements. The current step needs to ensure that all executed SQL statements can query the result data, otherwise the matching results will be inaccurate; the operator executes 300 seed SQL statements and records the query results, writing them into a file. The operator executes the SQL statements of 10 expanded questions corresponding to each seed question respectively, and compares the query results with the query results of the seed SQL. If the SQL query results of the expanded question are consistent with the corresponding seed SQL, it means that the SQL statement is accurate; if not, it means that there is a problem with the SQL statement and it needs to be corrected.
[0089] The training set construction module is used to construct a fine-tuning training set, build a general script, fix the data file format, and write the constructed Excel file into a fine-tuning training file that meets the requirements of the large model through specific logic. The generated training set is not only large in number and rich in diversity, but also each question corresponds to an accurate SQL statement. As the fine-tuning training set of the large model, it improves the performance and accuracy of the NL2SQL model and enhances the nl2sql capabilities of the large model.
[0090] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for generating nl2sql training sets based on a large model applicable to all industries, characterized by: The steps include: Build an initial set of questions based on the table creation statements; Manually screen common scenario issues; Generate and verify seed SQL; Semantic similarity question expansion; Manual verification of the expanded issues; Generate SQL to expand the question; Database execution and result comparison; Construct a fine-tuning training set.
2. A method for generating nl2sql training sets based on a large model applicable to all industries according to claim 1, characterized in that: The specific operations of constructing the initial question set based on the table creation statement include: In the process of building the initial question set, we construct questions by writing a general script, build a general prompt in the script, and call the large model interface. For different industry scenarios, we only need to change the table creation statement in the general prompt and some provided question examples or some provided table data to generate a question set for a specific scenario based on the table structure. The specific operations for manually screening common scenario problems include: In conjunction with relevant business personnel or customers, the problems are further screened, filtered, and transformed to select a set of seed questions that meet the customer's business scenarios. During the training process, the big model can understand the indicator calculation logic of common scenarios.
3. The method for generating nl2sql training sets based on a large model applicable to all industries according to claim 1, characterized in that: The specific operations for generating and verifying seed SQL include: Write an automated script to automatically generate SQL statements corresponding to seed questions by requesting the large model interface, and output SQL statement files. The automated script greatly shortens the time for manual SQL writing, greatly improves the efficiency of building training sets, and manually verifies the generated SQL again; The specific operations of semantic similarity question expansion include: Conduct in-depth analysis of seed questions and extract key information points, such as the query subject, query conditions, and query purpose. Use large models to reorganize and transform key information points to generate new question statements. Use similarity calculation methods in natural language processing, such as cosine similarity or semantic embedding of the BERT model, to calculate the similarity between new questions and seed questions. For questions with too low or too high similarity, eliminate or merge them through code logic. Use manual review to check the screened questions one by one to ensure that each question is semantically consistent with the seed question and has a reasonable expression.
4. The method for generating nl2sql training sets based on a large model applicable to all industries according to claim 1, characterized in that: The specific operations of manually checking the expanded questions include: Carefully read each question to understand its query intent and expression, especially the expression of indicators or scenarios. Manually compare the question with the corresponding seed question to check whether they are semantically consistent. For questions with inconsistent semantics, in-depth analysis and discussion with business personnel or customers are required, and corrections are made according to actual conditions. Ambiguities are eliminated by modifying wording, adding contextual information, or clarifying query conditions to make them clearer and more specific. The questions and modification process are recorded so that the training set can be further optimized and improved later. The specific operations for generating SQL to expand the problem include: Use the big model to generate corresponding SQL statements for these questions, and use the big model's ability to understand natural language query intent to convert natural language questions into structured SQL statements; appropriately tune the big model by building prompts to make it more suitable for the needs of NL2SQL tasks, use the expanded questions as input, and use the tuned big model to generate corresponding SQL statements; during the generation process, pay attention to the grammatical correctness and logical rationality of the SQL statements to ensure that the generated SQL statements correctly reflect the query intent of the problem.
5. The method for generating nl2sql training sets based on a large model applicable to all industries according to claim 1, characterized in that: The specific operations of database execution and result comparison include: Connect to an actual database and execute these SQL statements. The current step needs to ensure that all executed SQL statements can query the result data, otherwise the matching results will be inaccurate. The operator executes 300 seed SQL statements and records the query results, writing them into a file. The operator executes the SQL statements of 10 expanded questions corresponding to each seed question respectively, and compares the query results with the query results of the seed SQL. If the SQL query results of the expanded question are consistent with the corresponding seed SQL, it means that the SQL statement is accurate; if they are inconsistent, it means that there is a problem with the SQL statement and it needs to be corrected. The specific operations for constructing a fine-tuning training set include: Build a general script, fix the data file format, and write the constructed Excel file into a fine-tuning training file that meets the requirements of the large model through specific logic. The generated training set is not only large in number and rich in diversity, but each question corresponds to an accurate SQL statement. As the fine-tuning training set of the large model, it improves the performance and accuracy of the NL2SQL model and enhances the nl2sql capabilities of the large model.
6. A method for generating nl2sql training sets based on a large model applicable to all industries according to any one of claims 1 to 5, and a system for generating nl2sql training sets based on a large model applicable to all industries, characterized in that: The system comprises: An initial question set building module, used to build an initial question set based on a table building statement; Scenario problem screening module, used to manually screen common scenario problems; Verification seed generation module, used to generate and verify seed SQL; Question extension module, used to expand semantically similar questions; Verification module, used to manually verify the expanded issues; SQL generation module, used to generate SQL for expanding questions; Execution and comparison module, used for database execution and result comparison; The training set construction module is used to construct the fine-tuning training set.
7. A system for generating nl2sql training sets based on a large model applicable to all industries according to claim 6, characterized in that: The initial question set construction module constructs questions by writing a general script, building a general prompt in the script, and calling the large model interface. For different industry scenarios, you only need to change the table building statement in the general prompt and some provided question examples or some provided table data to generate a question set for a specific scenario based on the table structure. The scenario problem screening module, in conjunction with relevant business personnel or customers, further screens, filters, and transforms the problems to select a set of seed problems that meet the customer's business scenarios. During the training process, the big model can understand the indicator calculation logic of common scenarios.
8. The system for generating nl2sql training sets based on a large model applicable to all industries according to claim 6, characterized in that: Verify the seed generation module, write an automated script, automatically generate the SQL statements corresponding to the seed questions by requesting the large model interface, and output the SQL statement file. Through the automated script, the time for manual SQL writing is greatly shortened, the efficiency of building the training set is greatly improved, and the generated SQL is manually verified again; The question expansion module conducts in-depth analysis on seed questions, extracts key information points, such as the query subject, query conditions, and query purpose, and uses a large model to reorganize and transform key information points to generate new question statements; Use similarity calculation methods in natural language processing, such as cosine similarity or semantic embedding of the BERT model, to calculate the similarity between new questions and seed questions. For questions with too low or too high similarity, eliminate or merge them through code logic. Use manual review to check the screened questions one by one to ensure that each question is semantically consistent with the seed question and has a reasonable expression.
9. The system for generating nl2sql training sets based on a large model applicable to all industries according to claim 6, characterized in that: Verification module: Read each question carefully to understand its query intent and expression, especially the expression of indicators or scenarios. Manually compare the question with the corresponding seed question to check whether they are semantically consistent. For questions with inconsistent semantics, in-depth analysis and discussion with business personnel or customers are required, and corrections are made according to actual conditions. Ambiguities are eliminated by modifying wording, adding contextual information or clarifying query conditions to make them clearer and more specific. The questions and modification process are recorded so that the training set can be further optimized and improved later. The SQL generation module uses the big model to generate corresponding SQL statements for these questions, and uses the big model's ability to understand natural language query intent to convert natural language questions into structured SQL statements. By building prompts, the big model is properly tuned to make it more suitable for the needs of the NL2SQL task, and the expanded questions are used as input to generate corresponding SQL statements using the tuned big model. During the generation process, attention is paid to the grammatical correctness and logical rationality of the SQL statements to ensure that the generated SQL statements correctly reflect the query intent of the question.
10. The system for generating nl2sql training sets based on a large model applicable to all industries according to claim 6, characterized in that: The execution and comparison module connects to an actual database and executes these SQL statements. The current step needs to ensure that all executed SQL statements can query the result data, otherwise the matching results will be inaccurate. The operator executes 300 seed SQL statements and records the query results, writing them into a file. The operator executes the SQL statements of 10 expanded questions corresponding to each seed question respectively, and compares the query results with the query results of the seed SQL. If the SQL query results of the expanded question are consistent with the corresponding seed SQL, it means that the SQL statement is accurate; if not, it means that there is a problem with the SQL statement and it needs to be corrected. The training set construction module builds a general script, fixes the data file format, and writes the constructed Excel file into a fine-tuning training file that meets the requirements of the large model through specific logic. The generated training set is not only large in number and rich in diversity, but each question corresponds to an accurate SQL statement. As the fine-tuning training set of the large model, it improves the performance and accuracy of the NL2SQL model and enhances the nl2sql capabilities of the large model.
Citation Information
Patent Citations
Method for generating statement from natural language to SQL in business scene based on large model
CN117708162A
Training sample construction method and system combined with large language model, equipment and medium
CN117827847A
Language transformation method and device based on data enhancement and large language model
CN117931841A
Medical insurance intelligent query method and system based on NL2SQL
CN118132579A
NL2SQL data set construction method and device, equipment and medium
CN118503270A
Cited By
Method and device for realizing NL2SQL based on operation and maintenance industry large model
CN119961285A
Human-in-the-loop Text2SQL iterative data annotation method fused with large model
CN120030034A
Human-in-the-loop text2sql iterative data labeling method for large model fusion
CN120030034B
Method and system for constructing NL2SQL large model training set in operation and maintenance industry
CN120407605A
Large language model construction method and device based on NL2SQL
CN120448410A
