Data set generation method and device, electronic equipment and medium

Abstract

The invention provides a data set generation method and device, electronic equipment and a medium, and the method comprises the steps: determining a database table corresponding to a business scene, and generating table building statements based on natural language questions and answers in the business scene through a large model; generating a plurality of pieces of data for each database table in batches by utilizing the large model based on table building statements; based on the multiple pieces of data in each database table, generating problems described by a natural language and SQL query statements matched with the problems described by the natural language in batches; and performing at least one screening verification on the problems described by the natural language and the SQL query statements to generate a data set for converting the description of the problems described by the natural language into the SQL query statements, so that the diversity and reliability of the data set can be ensured, high-level SQL characteristics such as complex multi-table connection and nested sub-query can be processed, and the efficiency of the data set is improved. Through at least one screening verification, the accuracy of the data in the data set can be improved, and wide application in an actual service scene is facilitated.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, electronic device, and medium for generating a data set. Background Art

[0002] In the field of natural language processing, the task of converting natural language queries into Structured Query Language (SQL), or NL2SQL, has become a highly sought-after research area. In existing technologies, natural language processing based on neural network models relies on high-quality, large-scale NL2SQL datasets for model training, optimization, and testing. Traditional methods for constructing NL2SQL datasets rely primarily on manual annotation, where professionals familiar with SQL manually convert natural language questions into corresponding SQL queries. While this approach ensures data quality, it suffers from inefficiencies and high costs, making it difficult to meet the demands of large-scale datasets. Furthermore, manually annotated datasets often lack diversity and fail to capture the diverse and complex query scenarios encountered in real-world applications. Related technologies utilize existing question-answer pairs to automatically generate SQL queries. However, the SQL queries generated by this approach are often overly simplistic and fail to reflect the complex query requirements of the real world. Existing methods for automatically generating question-answer pairs often struggle to generate sufficiently realistic and diverse samples. This not only limits the performance of NL2SQL models but also hinders their widespread application in real-world business scenarios. Summary of the Invention

[0003] The present invention provides a dataset generation method, device, electronic device and medium to address the defect that traditional dataset generation methods use existing question-answer pairs to automatically generate SQL queries, which makes it difficult to produce sufficiently real and diverse samples, thereby limiting the performance of the NL2SQL model.

[0004] The present invention provides a data set generation method, comprising: Determine the database table corresponding to the business scenario, and use the big model to generate table creation statements based on the natural language questions and answers in the business scenario; Based on the table creation statement, the large model is used to batch generate multiple data for each database table; Using the large model to batch generate questions described in natural language and SQL query statements matching the questions described in natural language based on multiple data in each of the database tables; The problem described in the natural language and the SQL query statement matching the problem described in the natural language are screened and verified at least once to generate a data set of SQL query statements converted from the natural language problem description.

[0005] According to the dataset generation method provided by the present invention, the table creation statement includes a description of the table name, field descriptions and primary and foreign key associations, as well as multiple sample data insertion statements for each database table, wherein the sample data of the primary and foreign keys in the multiple sample data insertion statements for each database table are associated.

[0006] According to the data set generation method provided by the present invention, the batch generation of multiple data for each database table using the large model based on the table creation statement includes: Based on the table creation statement and data requirement prompts, multiple data entries are generated in batches for each table using the large model. The data requirement prompts include at least one of the following: data must comply with field interpretation and database attribute constraints, data must be diversified, data must be generated completely according to actual production data, primary and foreign key field data must correspond consistently, and data intersection between multiple database tables must be ensured; Execute the table creation statement in the database, and if an error is reported, modify the data based on the error information and the table creation statement through the large model; Verification information on whether the data meets actual production requirements is received, and based on the verification information, new data is modified or regenerated through the large model based on natural language questions and the table creation statement.

[0007] According to the dataset generation method provided by the present invention, the method of using the large model to batch generate questions described in natural language and SQL query statements matching the questions described in natural language based on multiple data in each database table includes: Randomly select a database table and obtain table information corresponding to the database table, wherein the table information includes at least one of a table name, a table name comment, a field name, a field comment, a field database attribute, and sample data; Generate, through the large model and based on the table information of the database table, a question described in natural language corresponding to the database table and an SQL query statement matching the question described in natural language; The data set table is replaced, and a corresponding natural language description question and an SQL query statement matching the natural language description question are generated for each data set table.

[0008] According to the dataset generation method provided by the present invention, the step of generating, by using a large model based on the table information of the database table, a question described in natural language corresponding to the database table and an SQL query statement matching the question described in natural language includes: By using different prompt words, different large models, sampling different table information and sampled data, natural language questions and their corresponding SQL query statements are generated synchronously; And / or, by using different prompt words, different large models, sampling different table information and sampling data, natural language questions are first generated and then corresponding SQL query statements are generated.

[0009] According to the dataset generation method provided by the present invention, the natural language description of the problem and the SQL query statement matching the natural language description of the problem are screened and verified at least once to generate a dataset of the natural language problem description converted into SQL query statements, including: Execute the SQL query statement query operation in the database, and filter out SQL query statements that generate error messages and SQL query statements that return null values.

[0010] According to the dataset generation method provided by the present invention, the natural language description of the problem and the SQL query statement matching the natural language description of the problem are screened and verified at least once to generate a dataset of the natural language problem description converted into SQL query statements, including: Based on the big model and the data format and page design requirements of the dataset file, a dataset verification system is constructed. Based on the dataset verification system, the problem described in natural language and the SQL query statement matching the problem described in natural language are optimized to generate a dataset that converts the natural language problem description into SQL query statements.

[0011] The present invention also provides a data set generating device, comprising: A table creation statement generation module is used to determine the database table corresponding to the business scenario and use the big model to generate the table creation statement based on the natural language questions and answers in the business scenario; A data generation module, configured to generate a plurality of data in batches for each database table using the large model based on the table creation statement; A matching generation module, configured to use the large model to batch generate questions described in natural language and SQL query statements matching the questions described in natural language based on multiple data in each of the database tables; The screening and verification module is used to perform at least one screening and verification on the problem described in the natural language and the SQL query statement matching the problem described in the natural language, and generate a data set of the natural language problem description converted into SQL query statements.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described data set generation methods when executing the program.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the above-mentioned methods for generating a data set.

[0014] The data set generation method, device, electronic device and medium provided by the present invention use a large model to generate table creation statements based on natural language questions and answers in business scenarios; based on the table creation statements, the large model is used to batch generate multiple data for each database table, which can ensure the diversity and reliability of the data set. By screening and verifying the questions described in natural language and the SQL query statements matching the questions described in natural language at least once, the accuracy of the data in the data set can be improved, which is conducive to its wide application in actual business scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 This is one of the flow charts of the data set generation method provided by an embodiment of the present invention; Figure 2 Schematic diagram of inquiry and response to the Claude-3.5-Sonnet large model provided by an embodiment of the present invention; Figure 3 This is a schematic diagram of questions and answers in a human resources department business scenario provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of prompt words for a Claude-3.5-Sonnet large model question provided by an embodiment of the present invention; Figure 5 This is a schematic diagram of prompt words for batch data generation provided by an embodiment of the present invention; Figure 6 This is a schematic diagram of prompt words verified by a large model in a program provided by an embodiment of the present invention; Figure 7 This is a schematic diagram of prompt words used for the first time by the system for generating a preliminary version provided by an embodiment of the present invention; Figure 8 This is a schematic diagram of a page of a manual verification system provided by an embodiment of the present invention; Figure 9 This is the second flow chart of the data set generation method provided by the embodiment of the present invention; Figure 10 Schematic diagram of the functional structure of a data set generating device provided by an embodiment of the present invention; Figure 11 It is a functional structure diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0018] Figure 1 A flowchart of a method for generating a data set according to an embodiment of the present invention is provided. Figure 1 As shown, the data set generation method provided by the embodiment of the present invention includes: Step 101: Determine the database table corresponding to the business scenario, and use the big model to generate a table creation statement based on the natural language questions and answers in the business scenario; In this embodiment of the present invention, the large model uses the Claude-3.5-Sonnet model. After the large model provides a database table, manual inspection and evaluation are performed to check whether the large model's response is reasonable and whether the provided database tables are the primary database tables required for the business scenario. If the manual inspection reveals any issues, such as excessive or redundant tables, too few tables, or inadequate requirements for the scenario, the large model can be re-edited based on the previous answer, or the previous question can be revised and then re-answered.

[0019] If only the business scenario is determined, the questions and answers are relatively open within the scope of the scenario, and there are no special human requirements, you can directly ask the big model which main database tables are needed to construct the reporting system under a certain business scenario. For example Figure 2 The query and answer of the Claude-3.5-Sonnet large model are shown. If there are clear questions that need to be answered in the business scenario, you can manually sort out the questions and answers, and then let the large model directly generate table creation statements based on the questions and answers. For example Figure 3 、 Figure 4 Answers to questions and prompts in the business scenario shown.

[0020] Step 102: Based on the table creation statement, use the large model to batch generate multiple data for each database table; Step 103: using the large model to batch generate questions described in natural language and SQL query statements matching the questions described in natural language based on multiple data in each of the database tables; Step 104: Perform at least one screening and verification on the problem described in the natural language and the SQL query statements that match the problem described in the natural language, and generate a data set of SQL query statements converted from the natural language problem description.

[0021] It should be noted that the large model in the embodiment of the present invention is a pre-trained large model based on natural language processing (NLP), including but not limited to Claude-3.5-Sonnet, ChatGPT, DeepSeek-Coder, etc.

[0022] Traditional dataset generation methods for converting natural language into SQL statements automatically generate SQL queries using existing question-answer pairs. However, these queries are often overly simplistic and fail to reflect the complex query requirements of the real world. However, when dealing with advanced SQL features such as complex multi-table joins and nested subqueries, existing automatic question-answer generation methods often struggle to produce sufficiently realistic and diverse samples. This not only limits model performance but also hinders the widespread application of this technology in real-world business scenarios.

[0023] The data set generation method provided by the embodiment of the present invention determines the database table corresponding to the business scenario, uses the big model to generate a table creation statement based on the natural language questions and answers in the business scenario; uses the big model to batch generate multiple data for each database table based on the table creation statement; uses the big model to batch generate natural language description questions and SQL query statements matching the natural language description questions based on the multiple data in each database table; performs at least one screening and verification on the natural language description questions and the SQL query statements matching the natural language description questions to generate a data set in which the natural language question description is converted into SQL query statements, uses the big model to generate table creation statements and database table data, and then generates natural language questions and corresponding SQL query statements based on the database table data. This can ensure the diversity and reliability of the data set, can handle complex advanced SQL features such as multi-table joins and nested subqueries, and can improve the accuracy of the data in the data set through at least one screening and verification, which is conducive to its wide application in actual business scenarios.

[0024] Based on any of the above embodiments, the table creation statement includes but is not limited to a description of the table name, field descriptions, and primary and foreign key associations, as well as multiple sample data insertion statements for each database table, wherein the sample data of the primary and foreign keys in the multiple sample data insertion statements for each database table are associated.

[0025] The embodiment of the present invention sets associations by using sample data of primary and foreign keys in multiple sample data insert statements, and is applicable to scenarios that process complex multi-table joins, nested subqueries, and other advanced SQL features.

[0026] In an embodiment of the present invention, after determining the table structure required for the business scenario, the large language model is allowed to generate a table creation statement that conforms to the PostgreSQL syntax. At the same time, the requirements for the table creation statement are given. The table creation statement must include: the table name and its description, field definition and description, primary and foreign key association relationships, and sample data insertion statements. It is necessary to ensure that the primary and foreign key relationships of the sample data are correct so that the associated query can return valid results. The prompt words are, for example: When building a banking financial institution supervision system, the following are the names of some essential main core tables: Financial Institution Basic Information Table (Financial_Institution_Info), Balance Sheet (Balance_Sheet), Income Statement (Income_Statement), Capital Adequacy Ratio Table (Capital_Adequacy_Ratio), Liquidity Ratio Table (Liquidity_Ratio), Non-Performing Loan Table (Non_Performing_Loans), Risk Exposure Table (Risk_Exposure), Related Party Transaction Table (Related_Party_Transactions), Regulatory Report Table (Regulatory_Reports), Compliance Record Table (Compliance_Records); Please provide the PostgreSQL database table creation statements for the above tables. The table creation statements should include a description of the table name, field descriptions, primary and foreign key associations, and several sample data insertion statements for each table. Note that the sample data for the primary and foreign keys should have intersections to ensure that there is data after the join association. The generated data should be as realistic as possible and meet the requirements of actual production use. The final result only returns the table creation statements that can be executed in the PostgreSQL database. No other explanations or explanations are required.

[0027] After the macro model generates table creation statements, the embodiment of the present invention performs manual inspection and modification to check whether the database table and its fields meet the requirements of the business scenario, whether new fields need to be added or deleted, whether the primary and foreign key designs of the tables are reasonable, whether the generated sample data conforms to the actual situation in real production, whether the table creation statements are generated as required, and whether any errors are reported during execution. If any problems are found, the macro model can be further modified and improved based on the specific issues.

[0028] Based on any of the above embodiments, the step of batch-generating multiple pieces of data for each database table using the large model based on the table creation statement includes: Step 201: Based on the table creation statement and data requirement prompts, multiple data entries are generated in batches for each table using the large model. The data requirement prompts include at least one of: data must comply with field interpretation and database attribute constraints; data must be diversified and generated completely according to actual production data; primary and foreign key field data must correspond consistently; and data intersection between multiple database tables must be ensured. Step 202: Execute the table creation statement in the database. If an error occurs, modify the data based on the error message and the table creation statement using the large model. Step 203: Receive verification information on whether the data meets actual production requirements, and modify or regenerate new data based on the verification information through the large model based on natural language questions and the table creation statement.

[0029] In the embodiment of the present invention, after the table is built, the large model is used to generate 10,000 data records for each table in batches based on the table building statement, and the requirements for generating data are given, such as the data must comply with the constraints of field interpretation and database attributes, the data must be diversified, and the data must be generated completely in accordance with the actual production data. The primary and foreign key field data must correspond to each other to ensure that the data has intersections after joining. Note that this step requires the best performance large model such as Claude-3.5-Sonnet, ChatGPT. For example, Figure 5 The prompt word shown.

[0030] After generating database table insert statements, execute them in the database. If any errors are reported, have the model modify them based on the error messages and SQL insert statements. After successful execution, manually check the database table data to ensure it meets actual production requirements. For example, check whether names and addresses are real, whether dates and times conform to the corresponding formats and current transaction dates and times, whether financial institution names are real, and whether profits and liabilities of financial institutions are consistent with real-world conditions. If any issues are found, have the model modify or regenerate new answers based on the questions and table creation statements. For example, generate 10,000 sample data insert statements for each table. Ensure that the sample data for primary and foreign keys overlap to ensure data exists after joins. The generated data should be as realistic as possible, meeting the requirements of actual production banking risk control scenarios. The final results only return table creation statements that can be executed in the PostgreSQL database, without any additional explanations or clarifications. The data should be in Chinese and should be diverse, rationalized, and authentic. For example, names should be real-life names, not names like Zhang San or Li Si.

[0031] The above table creation statements are for reports in the banking supervision business scenario system. Based on these table creation statements, please batch generate 10,000 data records and return them using SQL insert statements. The generated data must strictly follow the table name annotations and field name annotations, as well as database constraints. The data must be diverse, scientific, and reasonable, strictly consistent with real and actual production data. For example, the names and related-party transactions must be real bank or financial institution names, and the time period must be reasonable. For fields such as total assets, deposits, loans, and non-performing loan ratios, please note that the data is expressed in units of yuan, and the data must be within a reasonable range used in real-world banking production. For example, the unit is uniformly "yuan" and the data range is as follows: Net interest income: RMB 500 million to RMB 5 billion; Non-interest income: RMB 100 million to RMB 1 billion; Operating expenses: RMB 300 million to RMB 3 billion; Loan loss provisions: RMB 50 million to RMB 500 million; Net profit: 250 million to 2.5 billion yuan; These ranges are reasonable for large commercial banks and should be adjusted based on the institution's total assets or other indicators. Fields with primary and foreign key relationships maintain consistent data, ensuring table joins have intersections. Field data with calculation rules is generated strictly according to these rules. Finally, only the SQL query statement for inserting data into the PGSQL database, which generates a batch of 10,000 rows, is returned.

[0032] Based on any of the above embodiments, the step of using the large model to batch generate questions described in natural language and SQL query statements matching the questions described in natural language based on multiple data in each database table includes: Step 301: randomly select a database table and obtain table information corresponding to the database table, wherein the table information includes at least one of a table name, a table name comment, a field name, a field comment, a field database attribute, and sample data; Step 302: Generate, using the big model based on the table information of the database table, a question described in natural language corresponding to the database table and an SQL query statement matching the question described in natural language; Step 303: Replace the data set table, and generate a corresponding natural language description question and an SQL query statement matching the natural language description question for each data set table.

[0033] In an embodiment of the present invention, the step of generating, based on the table information of the database table, a question described in natural language corresponding to the database table and an SQL query statement matching the question described in natural language by using a large model includes: By using different prompt words, different large models, sampling different table information and sampled data, natural language questions and their corresponding SQL query statements are generated synchronously; And / or, by using different prompt words, different large models, sampling different table information and sampling data, natural language questions are first generated and then corresponding SQL query statements are generated.

[0034] In an embodiment of the present invention, after data generation is completed in accordance with actual production practices, database table information, namely, table names, table name annotations, field names, field annotations, field database attributes, and sampled data, is obtained. Natural language questions and SQL query statements are then generated based on this database information using a large model. To ensure the diversity and complexity of generated SQL query statements and questions, various prompt words, different models, sampling of different table information and sampled data, simultaneous question and SQL generation, question generation followed by SQL generation, and other diverse approaches are employed to meet varying levels of complexity. For example, the following prompt is used to ask questions for the Claude-3.5-Sonnet and Chatgpt models: [{'table_name': 'stock_basic', 'table_comment': None, 'columns': [{'column_name': 'ts_code', 'data_type': 'character varying', 'column_default': None, 'constraints': ['PRIMARY KEY'], 'comment': 'Stock Code'}, {'column_name': 'name', 'data_type': 'character varying', 'column_default': None, 'constraints': [], 'comment': 'Stock Name'}], 'sample_data': {'ts_code': '603012.SH', 'name': 'Chuangli Group'}}, {'table_name': 'stock_prices', 'table_comment': None, 'columns': [{'column_name': 'amount', 'data_type': 'double precision', 'column_default': None, 'constraints': [], 'comment': 'Transaction amount'}…].

[0035] Based on the database table information provided above, generate 100 natural language questions and corresponding SQL queries. Database table information includes the table name and description, the business description of the field data, the technical attributes and constraints of the field, and several sample data points. Each generated question should involve at least two tables, using at least one field from each table, and should include conditional filtering or aggregation operations whenever possible. Try asking questions from different angles, including questions about sampled data in sample_data, to increase the diversity of your questions and SQL. Whenever possible, combine multiple tables to generate question descriptions and corresponding complex SQL queries involving multi-table joins. Finally, ensure that the generated SQL fully corresponds to the natural language question, that field types match in JOIN operations, that it can be executed correctly in the PostgreSQL database, and that the natural language question is clear and easy to understand. Results should be returned in the format [{"question":"natural language question description","sql":"SQL query"},{...},{....}], with no additional explanation.

[0036] In this embodiment of the present invention, diverse prompt words are designed to guide the large model to generate complex SQL queries. For example, a SQL query involving multi-table joins is generated to query all orders of a user and their corresponding product information. A SQL query statement containing nested subqueries is generated to query users whose total order amount is greater than the average order amount. A SQL query containing aggregate functions is generated to query the total number of orders for each user. When generating samples, different prompt words are randomly selected to ensure that the generated SQL queries cover different complex scenarios.

[0037] Based on any of the above embodiments, performing at least one screening and verification on the problem described in the natural language and the SQL query statement matching the problem described in the natural language to generate a dataset of SQL query statements converted from the natural language problem description includes: Step 401: Execute the SQL query statement query operation in the database, and filter out SQL query statements that generate error messages and SQL query statements that return null values; In the embodiment of the present invention, after generating natural language questions and corresponding SQL query statements, the SQL query is executed in the database, and the SQL that generates error messages and the SQL that returns null values ​​are filtered out. The correct SQL and questions are screened out through preliminary verification, and then the large model is used to check and verify again, marking and filtering out problematic data samples. The prompt words for verification by the large model in the program are as follows: Figure 6 shown.

[0038] Step 402: Based on the large model and the data format and page design requirements of the data set file, a data set verification system is constructed. Based on the data set verification system, the problem described in natural language and the SQL query statement matching the problem described in natural language are optimized to generate a data set that converts the natural language problem description into SQL query statements.

[0039] In an embodiment of the present invention, a dataset verification system is constructed based on a large model combined with the data format and page design requirements of the dataset file, including: Step 601: Design a preliminary version of prompt words according to the page design requirements and the data set format; Step 602: Iteratively modify the prompt word, add descriptions of the annotation options, clarify the requirements of page layout and interaction design, and generate the final prompt word; Verification prompts include, for example, "Please check whether the following SQL statement meets the problem description and verify whether its logic is correct." "Please check whether the data returned by the following SQL statement meets the business scenario requirements."

[0040] Step 603: Input the final prompt word into the model to generate the initial version of the system code; Step 604: Run the initial version of the system code to test whether all functions are working properly. Step 605: Collect user feedback on the system page layout, interactive design, and functions; Step 606: Adjust the prompt words according to the feedback, input the optimized prompt words into the model, and generate a data set verification system.

[0041] In the embodiment of the present invention, based on the Claude-3.5-Sonnet model, combined with the data format of the dataset file generated and saved above, and according to the design requirements of the page, prompt words are designed, and iteratively modified and improved, and finally the model is used to generate a dataset verification system that meets the requirements. For example, the prompt words used for the first time in the initial version of the system are as follows: Figure 7 After several modifications and improvements, the final generated data set verification system page is as follows. Figure 8 shown.

[0042] After the initial screening and verification, the dataset samples are verified and annotated one by one based on the generated dataset verification system. The main verification contents include whether the natural language question can be converted into SQL query statements, whether the SQL query accurately reflects the query intent of the question and obtains the correct answer through execution, and whether there are any problems with the SQL association logic, where conditions, aggregation calculations, etc., and whether they correspond to the natural language description of the question.

[0043] like Figure 9As shown, the dataset generation method provided by the embodiment of the present invention specifically includes: Step 1: Determine the business scenario: Clarify business requirements, such as determining the target business scenario (e.g., banking regulation, e-commerce, logistics, etc.). Identify the problems that need to be solved in that scenario (e.g., querying user information, calculating order amounts, etc.). Organize questions and answers, listing the key questions in the business scenario and providing detailed answers for each, ensuring that all necessary information is included.

[0044] Step 2: Generate database table structure: Based on the questions and answers in the business scenario, use the large model to generate a database table structure containing multiple tables. Ensure that primary and foreign key relationships exist between tables.

[0045] Check whether the generated table structure meets business requirements and ensure that the fields and constraints in the table structure are reasonable.

[0046] Step 3: Generate table data: Generate table data that meets actual production requirements based on the table structure, ensuring that the data complies with the field constraints and data range in the business scenario.

[0047] Check whether the generated data meets actual production requirements (such as the rationality of fields such as name, address, and date).

[0048] Step 4: Get table information and extract data: Extract metadata of each table from the database, including table name, field name, field type, constraints, etc. Randomly extract a small amount of sample data from each table for subsequent generation of questions and SQL statements.

[0049] Step 5: Generate natural language questions and SQL query statements: Design diverse prompt words based on table information and sample data, ensuring that the prompt words cover information such as table name, field name, field comment, and sample data.

[0050] Use the large model to generate natural language questions and corresponding SQL query statements, ensuring that each question involves at least two tables and uses at least one field from each table.

[0051] Check whether the generated SQL statement meets the problem description and ensure that the SQL statement can be executed correctly on PostgreSQL.

[0052] Step 6: Execute SQL query statement: Execute the generated SQL query statements in the database, capture the execution results and possible error information, filter the data samples that meet the requirements, and filter out the SQL statements and data samples that are successfully executed and return a non-empty result set.

[0053] Step 7: Model verification: Input the valid SQL and its corresponding problems and execution results into the big model for verification, and obtain the verification results, including whether the SQL statement is correct and whether the returned data meets the problem description.

[0054] Based on the verification results, problematic SQL and problem descriptions are marked, problematic data samples are filtered out, and high-quality data is retained.

[0055] Step 8: Generate a dataset to verify the system: Design the system's page layout, including areas for displaying questions, displaying SQL statements, executing results, annotations, and editing. Implement system functionality, including displaying questions and SQL statements, executing SQL queries, and supporting manual annotation and correction. Deploy the system to a server or local environment for use by human reviewers.

[0056] Step 9: Manual verification of annotations: Verifiers use the system to verify data samples one by one, identify problems, and correct SQL statements. The verified high-quality NL2SQL dataset is saved to a file for subsequent use.

[0057] The embodiment of the present invention constructs information such as the table names of the main database tables required for specific business scenarios and descriptions and uses of related tables based on questions posed to a large model. Based on the table information, a large model with high performance, such as Claude-3.5-Sonnet and ChatGPT, is used to generate database table creation statements. After the table is successfully created, large models such as Claude-3.5-Sonnet and ChatGPT and optimized prompt words are used to batch generate real table data that conforms to actual production. The large model is used to ask natural language questions and generate corresponding SQL queries based on database table information, namely, table names, table name comments, field names, field comments, field database attributes, sampled data, etc. Different prompt words, different models, sampling of different table information and sampled data, synchronous generation of questions and SQL, generation of questions first and then SQL, and requirements of different SQL complexity are used to ensure the diversity and complexity of generated SQL and questions. The Claude-3.5-Sonnet large model is used to generate a data set verification system to facilitate manual verification.

[0058] The dataset generation method provided by the embodiment of the present invention constructs a high-quality dataset of more than 6,000 types, including single tables, multi-table associations, nested subqueries, aggregate queries, etc., in 6 business scenarios.

[0059] The data set generating device provided by the present invention is described below. The data set generating device described below and the data set generating method described above can refer to each other.

[0060] Figure 10 A schematic diagram of the structure of a data set generating device provided by an embodiment of the present invention is shown in FIG. Figure 10 As shown, the data set generation device provided by the embodiment of the present invention includes: A table creation statement generation module 1001 is used to determine a database table corresponding to a business scenario and generate a table creation statement based on the natural language questions and answers in the business scenario using a large model; A data generation module 1002 is configured to generate a plurality of data in batches for each database table using the large model based on the table creation statement; A matching generation module 1003 is configured to generate, in batches, questions described in natural language and SQL query statements matching the questions described in natural language based on multiple data items in each of the database tables using the large model; The screening and verification module 1004 is used to perform at least one screening and verification on the problem described in the natural language and the SQL query statement matching the problem described in the natural language, and generate a data set of SQL query statements converted from the natural language problem description.

[0061] The data set generation device provided by the embodiment of the present invention determines the database table corresponding to the business scenario, and uses a large model to generate a table creation statement based on the natural language questions and answers in the business scenario; based on the table creation statement, uses the large model to batch generate multiple data for each of the database tables; uses the large model to batch generate natural language description questions and SQL query statements matching the natural language description questions based on the multiple data in each of the database tables; performs at least one screening and verification on the natural language description questions and the SQL query statements matching the natural language description questions to generate a data set in which the natural language question description is converted into SQL query statements, uses the large model to generate table creation statements and database table data, and then generates natural language questions and corresponding SQL query statements based on the database table data. This can ensure the diversity and reliability of the data set, and can handle complex advanced SQL features such as multi-table joins and nested subqueries. The accuracy of the data in the data set can be improved through at least one screening and verification, which is conducive to its wide application in actual business scenarios.

[0062] Figure 11An example of a physical structure diagram of an electronic device is shown below. Figure 11 As shown, the electronic device may include: a processor 1110, a communication interface 1120, a memory 1130, and a communication bus 1140, wherein the processor 1110, the communication interface 1120, and the memory 1130 communicate with each other via the communication bus 1140. The memory 1130 includes a computer program, an operating system, and acquired data. The processor 1110 can call the logic instructions in the memory 1130 to execute a data set generation method, which includes: determining a database table corresponding to a business scenario, using a large model to generate a table creation statement based on the natural language questions and answers in the business scenario; based on the table creation statement, using the large model to batch generate multiple data for each database table; using the large model to batch generate natural language description questions and SQL query statements matching the natural language description questions based on the multiple data in each database table; performing at least one screening and verification on the natural language description questions and the SQL query statements matching the natural language description questions to generate a data set of natural language question descriptions converted into SQL query statements.

[0063] Furthermore, the logic instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the relevant art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0064] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the data set generation method provided by the above-mentioned methods, the method comprising: determining a database table corresponding to a business scenario, and using a large model to generate a table creation statement based on natural language questions and answers in the business scenario; based on the table creation statement, using the large model to batch generate multiple data for each of the database tables; using the large model to batch generate natural language described questions and SQL query statements matching the natural language described questions based on multiple data in each of the database tables; performing at least one screening and verification on the natural language described questions and the SQL query statements matching the natural language described questions to generate a data set in which the natural language question description is converted into SQL query statements.

[0065] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0066] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for generating a data set, characterized in that: include: Determine the database table corresponding to the business scenario, and use the big model to generate table creation statements based on the natural language questions and answers in the business scenario; Based on the table creation statement, the large model is used to batch generate multiple data for each database table; Using the large model to batch generate questions described in natural language and SQL query statements matching the questions described in natural language based on multiple data in each of the database tables; The problem described in the natural language and the SQL query statement matching the problem described in the natural language are screened and verified at least once to generate a data set of SQL query statements converted from the natural language problem description.

2. The data set generation method according to claim 1, characterized in that: The table creation statement includes a description of the table name, field descriptions, and primary and foreign key associations, as well as multiple sample data insertion statements for each database table, wherein the sample data of the primary and foreign keys in the multiple sample data insertion statements for each database table are associated.

3. The dataset generation method according to claim 1, wherein: The step of generating a plurality of data in batches for each database table using the large model based on the table creation statement includes: Based on the table creation statement and data requirement prompts, multiple data entries are generated in batches for each table using the large model. The data requirement prompts include at least one of the following: data must comply with field interpretation and database attribute constraints, data must be diversified, data must be generated completely according to actual production data, primary and foreign key field data must correspond consistently, and data intersection between multiple database tables must be ensured; Execute the table creation statement in the database, and if an error is reported, modify the data based on the error information and the table creation statement through the large model; Verification information on whether the data meets actual production requirements is received, and based on the verification information, new data is modified or regenerated through the large model based on natural language questions and the table creation statement.

4. The data set generation method according to claim 1, wherein: The step of using the large model to batch generate questions described in natural language and SQL query statements matching the questions described in natural language based on multiple data in each database table includes: Randomly select a database table and obtain table information corresponding to the database table, wherein the table information includes at least one of a table name, a table name comment, a field name, a field comment, a field database attribute, and sample data; Generate, through the large model and based on the table information of the database table, a question described in natural language corresponding to the database table and an SQL query statement matching the question described in natural language; The data set table is replaced, and a corresponding natural language description question and an SQL query statement matching the natural language description question are generated for each data set table.

5. The data set generation method according to claim 4, characterized in that: The step of generating, based on the table information of the database table and the table information of the database table, a question described in natural language corresponding to the database table and an SQL query statement matching the question described in natural language by the large model includes: By using different prompt words, different large models, sampling different table information and sampled data, natural language questions and their corresponding SQL query statements are generated synchronously; And / or, by using different prompt words, different large models, sampling different table information and sampling data, natural language questions are first generated and then corresponding SQL query statements are generated.

6. The data set generation method according to claim 1, characterized in that: Performing at least one screening and verification on the problem described in the natural language and the SQL query statements matching the problem described in the natural language to generate a data set of SQL query statements converted from the natural language problem description, including: Execute the SQL query statement query operation in the database, and filter out SQL query statements that generate error messages and SQL query statements that return null values.

7. The data set generation method according to claim 1, characterized in that: Performing at least one screening and verification on the problem described in the natural language and the SQL query statements matching the problem described in the natural language to generate a data set of SQL query statements converted from the natural language problem description, including: Based on the big model and the data format and page design requirements of the dataset file, a dataset verification system is constructed. Based on the dataset verification system, the problem described in natural language and the SQL query statement matching the problem described in natural language are optimized to generate a dataset that converts the natural language problem description into SQL query statements.

8. A data set generating device, characterized in that: include: A table creation statement generation module is used to determine the database table corresponding to the business scenario and use the big model to generate the table creation statement based on the natural language questions and answers in the business scenario; A data generation module, configured to generate a plurality of data in batches for each database table using the large model based on the table creation statement; A matching generation module, configured to use the large model to batch generate questions described in natural language and SQL query statements matching the questions described in natural language based on multiple data in each of the database tables; The screening and verification module is used to perform at least one screening and verification on the problem described in the natural language and the SQL query statement matching the problem described in the natural language, and generate a data set of the natural language problem description converted into SQL query statements.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the data set generation method according to any one of claims 1 to 7 is implemented.

10. A non-transitory readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating a data set according to any one of claims 1 to 7 is implemented.

Images