Method and device for querying data based on large model, electronic equipment and storage medium

Abstract

The invention discloses a method and device for querying data based on a large model, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the field of artificial intelligence such as large models and natural language processing. According to the specific implementation scheme, table information of a data table and a to-be-converted first natural language statement corresponding to the data table are obtained; based on the table information, utilizing a large language model to convert the first natural language statement to generate a first SQL statement; verifying the first SQL statement by using a large language model to obtain a verification result of the first SQL statement; in response to determining that the first SQL statement does not pass the verification according to the verification result, according to the verification result and the first SQL statement, utilizing the large language model to convert the first natural language statement again to generate a second SQL statement; in response to the fact that the second SQL statement passes verification, a training data set is obtained according to the first natural language statement and the second SQL statement, and the training data set is used for training a natural language to an SQL model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to artificial intelligence fields such as large models and natural language processing, and specifically to a method, device, electronic device and storage medium for generating a training data set based on a large model. Background Art

[0002] NL2SQL (Natural Language to Structured Query Language) is an artificial intelligence technology that essentially converts the user's natural language into a semantic representation that can be understood and executed by a computer. NLSQL aims to convert natural language queries into SQL for querying databases. Summary of the invention

[0003] The present application provides a method, device, electronic device and storage medium for generating a training data set based on a large model.

[0004] According to one aspect of the present application, a method for generating a training data set based on a large model is provided, comprising:

[0005] Acquire table information of a data table and a first natural language sentence to be converted corresponding to the data table;

[0006] Based on the table information, convert the first natural language statement using a large language model to generate a first structured query language SQL statement;

[0007] Using the large language model, verifying the first SQL statement to obtain a verification result of the first SQL statement;

[0008] In response to determining that the first SQL statement fails verification according to the verification result, re-converting the first natural language statement using the large language model according to the verification result and the first SQL statement to generate a second SQL statement;

[0009] In response to the second SQL statement passing verification, a training data set is obtained based on the first natural language statement and the second SQL statement; wherein the training data set is used to train a natural language to SQL model, and the training data set includes at least one statement pair, each statement pair includes a natural language statement and a SQL statement.

[0010] According to another aspect of the present application, a natural language to SQL model training method is provided, comprising:

[0011] Obtain a training data set; wherein the training data set is generated by the above method, and the training data set includes at least one statement pair, each statement pair includes a sample natural language statement and a sample SQL statement;

[0012] Converting the sample natural language statements using an initial natural language to SQL model to generate a predicted SQL statement;

[0013] The initial natural language to SQL model is trained according to the difference between the predicted SQL statement and the sample SQL statement to obtain a natural language to SQL model.

[0014] According to another aspect of the present application, a data query method is provided, comprising:

[0015] Obtaining natural language statements for data query;

[0016] Using a natural language to SQL model to convert the natural language statement to generate an SQL statement; wherein the natural language to SQL model is trained using the training data set generated by the above method;

[0017] Use SQL statements to query data and obtain query results.

[0018] According to another aspect of the present application, there is provided a device for generating a training data set based on a large model, comprising:

[0019] A first acquisition module, used to acquire table information of a data table and a first natural language sentence to be converted corresponding to the data table;

[0020] A generating module, configured to convert the first natural language statement based on the table information by using a large language model to generate a first structured query language SQL statement;

[0021] A verification module, used to verify the first SQL statement using the large language model to obtain a verification result of the first SQL statement;

[0022] The generating module is further configured to, in response to determining that the first SQL statement fails verification according to the verification result, re-convert the first natural language statement using the large language model according to the verification result and the first SQL statement to generate a second SQL statement;

[0023] A second acquisition module is used to acquire a training data set according to the first natural language statement and the second SQL statement in response to the second SQL statement passing the verification; wherein the training data set is used to train the natural language to SQL model, and the training data set includes at least one statement pair, each statement pair includes a natural language statement and a SQL statement.

[0024] According to another aspect of the present application, a training device for a natural language to SQL model is provided, comprising:

[0025] An acquisition module, used to acquire a training data set; wherein the training data set is generated by the above-mentioned device, and the training data set includes at least one statement pair, each statement pair includes a sample natural language statement and a sample SQL statement;

[0026] A generation module, used to convert the sample natural language statement using an initial natural language to SQL model to generate a predicted SQL statement;

[0027] The training module is used to train the initial natural language to SQL model according to the difference between the predicted SQL statement and the sample SQL statement to obtain a natural language to SQL model.

[0028] According to another aspect of the present application, a data query device is provided, comprising:

[0029] An acquisition module, used for acquiring natural language sentences for data query;

[0030] A generation module, used to convert the natural language statement using a natural language to SQL model to generate an SQL statement; wherein the natural language to SQL model is trained using a training data set generated by the above-mentioned device;

[0031] The query module is used to use SQL statements to query data and obtain query results.

[0032] According to another aspect of the present application, there is provided an electronic device, including:

[0033] at least one processor; and

[0034] a memory communicatively connected to the at least one processor; wherein,

[0035] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the method described in the above embodiment.

[0036] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to the above embodiment.

[0037] According to another aspect of the present application, a computer program product is provided, including a computer program, wherein the computer program implements the steps of the method described in the above embodiment when executed by a processor.

[0038] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0040] Figure 1 A flowchart of a method for generating a training data set based on a large model provided in an embodiment of the present application;

[0041] Figure 2 A flowchart of a method for generating a training data set based on a large model provided in another embodiment of the present application;

[0042] Figure 3 A flowchart of a method for generating a training data set based on a large model provided in another embodiment of the present application;

[0043] Figure 4 A schematic diagram of the structure of an apparatus for generating a training data set based on a large model provided in one embodiment of the present application;

[0044] Figure 5 It is a block diagram of an electronic device used to implement the method of generating a training data set based on a large model in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0046] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0047] The following describes the method, device, electronic device and storage medium for generating a training data set based on a large model in an embodiment of the present application with reference to the accompanying drawings.

[0048] In some embodiments, the NL2SQL model can be trained to convert the natural language sentences input by the user into SQL statements, query the data table in the database using the SQL statements, and return the query results to the user. In order to train a reliable NL2SQL model, a large number of high-quality data samples are usually required, especially to ensure the accuracy of the data, but the cost of manually writing SQL data samples is relatively high and the efficiency is relatively low.

[0049] Figure 1 A flowchart of a method for generating a training data set based on a large model provided in one embodiment of the present application.

[0050] The method for generating a training data set based on a large model in an embodiment of the present application can be executed by a device for generating a training data set based on a large model in an embodiment of the present application, and the device can be configured in an electronic device.

[0051] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0052] like Figure 1 As shown, the method for generating a training data set based on a large model includes:

[0053] Step 101: Obtain table information of a data table and a first natural language sentence to be converted corresponding to the data table.

[0054] In this application, the table information of a data table may include but is not limited to the structural description information of the data table, the foreign keys of the data table, the values ​​of some columns in the data table, the domain knowledge of the data table, and the additional description of the data table. Among them, the domain knowledge can be understood as the meaning of some fields in the data table in the field to which the data table belongs.

[0055] For the convenience of storage, some characters may be used to represent fields in data tables, but the computer may not accurately understand the meaning of these fields. Therefore, when converting natural language statements into SQL statements, the domain knowledge of the data table can be obtained, and then combined with the domain knowledge of the data table, the training data set for the business scenario can be obtained.

[0056] For example, the domain knowledge of a data table is "'["In_development"]' corresponds to 'under development'; '["online"]' corresponds to 'online'; '["cancelled"]' corresponds to 'cancelled'; '["w06Po"]' corresponds to 'guidance'; '["fesSa"]' corresponds to 'undertake'; '["BU0T7"]' corresponds to 'guidance+undertake'.

[0057] For example, an additional description of a data table is "the current user is AA, and now is September 2, 2024". Based on the additional description, information related to the user can be queried, such as how many employees are in the department where user AA is located.

[0058] In this application, the first natural language sentence to be converted corresponding to the data table can be understood as a natural language sentence used to query the data table. For example, the data table is a table of pending work orders for a power grid, and the first natural language sentence to be converted is "how many work orders have not been completed today".

[0059] Exemplarily, the first natural language sentence to be converted may be manually written or generated by a large language model, which is not limited in the present application.

[0060] Step 102: Based on the table information, the first natural language statement is converted using a large language model to generate a first SQL statement.

[0061] In the present application, prompt information can be generated based on the table information of the data table, the first natural language statement, etc., and the prompt information can be input into the large language model, and the large language model can be used to understand the structure of the data table based on the information of the data table, and based on the understood structure of the data table, the first natural language statement is converted to generate a first SQL statement. The prompt information is used to prompt the large language model to perform the NL2SQL task.

[0062] Exemplarily, the large language model can perform word segmentation on the first natural language statement, obtain each word segment of the first natural language statement, and understand the meaning of these word segments in the context, the relationship between the word segments, etc., and then identify the various parts of the first natural language statement corresponding to the SQL statement, such as SELECT, FROM, WHERE, GROUP BY and other clauses, and construct an SQL statement based on the identified SQL elements and the structure of the data table understood based on the table information, thereby converting the first natural language statement into a first SQL statement.

[0063] Optionally, SQL grammar rules may be obtained, and prompt information may be generated based on the table information, the SQL grammar rules and the first natural language statement. The first SQL statement may be generated based on the prompt information, which may improve the accuracy of the generated SQL statement.

[0064] Exemplarily, the prompt information for generating SQL statements may also include requirements for the output SQL statements, such as "1. Please directly output executable SQL statements, no need to output thinking and analysis; 2. Only output one SQL statement, if querying multiple information, it can also be implemented in one SQL; 3. When selecting columns, there is no need to use as to rename the columns."

[0065] Step 103: Use the large language model to verify the first SQL statement to obtain a verification result of the first SQL statement.

[0066] In the present application, after obtaining the first SQL statement, the large language model may be used to verify the first SQL statement, for example, to verify whether the syntax and logic of the first SQL statement are correct.

[0067] Exemplarily, prompt information is generated according to table information, SQL grammar rules, the first natural language statement, the first SQL statement, etc., and the prompt information is input into the large language model to verify the first SQL statement.

[0068] Exemplarily, the prompt information may also include requirements for the output results of the large language model, including requirements for the output format, output content, etc. For example, the requirement for the output results of the large language model is "You need to output the result in json format {"right": 0 or 1, whether the SQL statement to be verified is correct and can answer the user's query, "reason": the reason for the error of the SQL statement to be verified}".

[0069] Step 104 , in response to determining that the first SQL statement fails verification according to the verification result, the first natural language statement is reconverted using a large language model according to the verification result and the first SQL statement to generate a second SQL statement.

[0070] In the present application, if it is determined according to the verification result that the first SQL statement fails the verification, that is, the first SQL statement is incorrect, the first natural language statement can be reconverted using a large language model to generate a second SQL statement.

[0071] Exemplarily, prompt information can be generated based on table information, a first SQL statement, a first natural language statement, SQL grammar specifications, etc., and the prompt information is input into a large language model, and the first natural language statement is reconverted using the large language model to generate a second SQL statement.

[0072] Step 105 , in response to the second SQL statement passing verification, a training data set is obtained according to the first natural language statement and the second SQL statement.

[0073] The training data set may include at least one sentence pair, each sentence pair includes a natural language sentence and an SQL statement, and each sentence pair may be regarded as a training sample for model training of the NL2SQL model. Exemplarily, the NL2SQL model may be trained using the training data set in a supervised fine-tuning manner. Exemplarily, the large language model may be supervised fine-tuned using the training data set to obtain the NL2SQL model.

[0074] For example, in a database query scenario, a natural language sentence in text form input by a user may be obtained, and the NL2SQL model may be used to convert the natural language sentence input by the user into an SQL statement, and then the SQL statement may be executed to query the database and obtain the query result.

[0075] It is understandable that in a database query scenario, if the query statement input by the user is voice, the voice can be recognized and converted into text to obtain a natural language statement in text form, and then the NL2SQL model is used to convert the natural language statement into an SQL statement.

[0076] In the present application, the second SQL statement may be verified. If the second SQL statement passes the verification, the first natural language statement and the second SQL statement may be added to the training data set as a statement pair.

[0077] Exemplarily, in order to improve the speed and accuracy of model training, a statement pair consisting of a first natural language statement and a second SQL statement and table information of a data table can be added to a training data set as a training sample. When training the NL2SQL model, the table information and natural language statements in the training sample can be input into an initial NL2SQL model. Based on the differences between the SQL statements generated by the initial NL2SQL model and the SQL statements in the training sample, the initial NL2SQL model can be trained to obtain the NL2SQL model.

[0078] Exemplarily, if the second SQL statement fails the verification, a prompt message can be generated using the table information, the second SQL statement, the first natural language statement, the SQL grammar specification, etc., and the prompt message can be input into the large language model to regenerate the SQL statement using the large language model; or a prompt message can be generated using the table information, the first SQL statement, the first natural language statement, the SQL grammar specification, etc., and the SQL statement can be regenerated using the large language model, and the regenerated SQL statement can be continued to be verified; if the generated SQL statement still fails the verification when the preset number of times is reached, the first natural language statement will be discarded.

[0079] Exemplarily, if the first SQL statement passes verification, the first natural language statement and the first SQL statement can be added to the training data set as a statement pair, or the statement and the table information of the data table can be added to the training data set as a training sample.

[0080] In the embodiment of the present application, the first natural language statement to be converted is converted by using a large language model, a first SQL statement is generated, and the first SQL statement is verified. If the first SQL statement fails to pass the verification, the large language model is used to re-convert it according to the verification result and the first SQL statement. Therefore, when the generated SQL statement fails to pass the verification, the SQL statement is regenerated by reflecting on the verification result, which can improve the accuracy of the generated SQL statement, thereby improving the quality of the training data set. In addition, by using the large language model to generate SQL statements, the efficiency of obtaining SQL statements can be improved, and the training data set of the NL2SQL model can be constructed quickly and with high quality, which can also save costs. Then, using the high-quality training data set, a high-accuracy NL2SQL model can be trained. In database queries, the query statement in the natural language form input by the user can be converted into an SQL statement using the trained NL2SQL model, which can improve the accuracy of the SQL statement. Then, using the SQL statement to perform database queries can improve the accuracy of the query results.

[0081] Figure 2 A flowchart of a method for generating a training data set based on a large model provided in another embodiment of the present application.

[0082] like Figure 2 As shown, the method for generating a training data set based on a large model includes:

[0083] Step 201: Obtain table information of a data table and a first natural language sentence to be converted corresponding to the data table.

[0084] In the present application, step 201 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0085] Exemplarily, a large language model can be used to obtain a first natural language sentence, a second reference natural language sentence can be obtained, and a third prompt information can be generated based on the table information and the second reference natural language sentence. The third prompt information can be used to prompt the large language model to perform a natural language sentence generation task. The third prompt information is then input into the large language model. The large language model can output a natural language sentence based on the table information and reference the second reference natural language sentence. The output natural language sentence can be used as the first natural language sentence.

[0086] Exemplarily, the large language model can understand the structure of the data table according to the table information of the data table, and output the first natural language sentence based on the structure of the data table and by summarizing the rules of the second reference natural language sentence.

[0087] Therefore, by providing the second reference natural language sentence to the large language model, the large language model can obtain new natural language sentences based on some examples of natural language sentences, thereby improving the accuracy and acquisition efficiency of natural language sentences and saving costs.

[0088] Exemplarily, the large language model can generate a candidate natural language sentence based on the table information in combination with the second reference natural language sentence, and perform a grammar check on the candidate natural language sentence. If the candidate natural language sentence passes the grammar check, the candidate natural language sentence is determined to be the first natural language sentence.

[0089] Therefore, by performing grammar check on the natural language sentence generated by the large language model, the natural language sentence that passes the grammar check is used as the first natural language sentence, thereby improving the accuracy of the first natural language sentence.

[0090] Step 202: Based on the table information, the first natural language statement is converted using a large language model to generate a first SQL statement.

[0091] In the present application, step 202 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0092] Exemplarily, a second reference statement pair corresponding to the data table can be obtained, the second reference statement pair including a natural language statement and an SQL statement, and second prompt information can be generated based on the table information of the data table, the second reference statement pair and the first natural language statement. The second prompt information can be used to prompt the large language model to perform the NL2SQL task, and then the second prompt information is input into the large language model. The large language model can convert the first natural language statement based on the table information and with reference to the second reference statement pair to generate a first SQL statement.

[0093] Among them, the second reference statement pair corresponding to the data table can be understood as a reference statement pair that can be used to query the data table, the natural language statement in the second reference statement pair corresponds to the SQL statement, and the SQL statement in the second reference statement pair can be understood as the SQL statement required to query the data table based on the natural language statement in the second reference statement pair. In addition, the number of the second reference statement pairs can be one or more, and this application does not limit this.

[0094] Exemplarily, the large language model can understand the structure of the data table according to the table information, and based on the structure of the data table and referring to the rules for converting the natural language statements in the second reference statement pair into the SQL statements in the second reference statement pair, convert the first natural language statement into the first SQL statement.

[0095] Therefore, the large language model can convert the first natural language statement according to the table information and with reference to the second reference statement pair to generate the first SQL statement, thereby improving the accuracy of the generated SQL statement.

[0096] Step 203: Use the large language model to verify the first SQL statement to obtain a verification result of the first SQL statement.

[0097] In the present application, step 203 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0098] Step 204, in response to determining that the first SQL statement fails verification according to the verification result, the first natural language statement is reconverted using a large language model according to the verification result and the first SQL statement to generate a second SQL statement.

[0099] In the present application, step 204 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0100] Exemplarily, the verification result may include the cause of the error of the first SQL statement. A first prompt message may be generated based on the table information, the first natural language statement, the cause of the error and the first SQL statement. The first prompt message is input into the large language model. The large language model is used to reflect on the first natural language statement in combination with the table information and the cause of the error, and the first SQL statement is corrected to generate a second SQL statement.

[0101] Therefore, the large language model can reflect on the error cause of the first SQL statement and regenerate the SQL statement, which can not only improve the accuracy of the generated SQL statement, but also improve the efficiency of the regenerated SQL statement.

[0102] Since the values ​​of the same field in different data tables may be expressed differently, or there are some precautions in some SQL statements in some query situations, in order to further improve the accuracy of the generated SQL statements, illustratively, the SQL specification can be obtained, and a first prompt message can be generated based on the table information, the first natural language statement, the cause of the error, the first SQL statement and the SQL specification, and then the first prompt message is input into the large language model for processing to generate a third SQL statement.

[0103] The SQL specification may include but is not limited to time specifications in SQL statements, SQL style specifications, SQL requirements for target query situations, etc. The time specification may be understood as a processing method when time information appears in an SQL statement, and the SQL style specification may refer to style requirements for SQL statements.

[0104] For example, the SQL specification could be as follows:

[0105] 1. SQL time specification:

[0106] (1) If the column mentioned is a relative time and the time format is '%Y%m%d' or '%Y-%m-%d', use the curdate function, for example, "this quarter", use date_column between MAKEDATE(YEAR(CURDATE()),1)+INTERVAL QUARTER(CURDATE())QUARTER-INTERVAL 3MONTH and LAST_DAY(MAKEDATE(YEAR(CURDATE()),1)+INTERVAL QU ARTER(CURDATE())QUARTER-INTERVAL1DAY);

[0107] (2) If a specific year is mentioned, there is no need to use the curdate function. For example, for "23 Q2", use date_column between "2023-03-01" and "2023-06-30";

[0108] (3) Pay attention to the format of the time column. For example, "20230513" needs to be converted to the format '%Y%m%d'.

[0109] 2. SQL style specifications:

[0110] If multiple selects are required, involving splitting into multiple sub-tables, use the CTE structure with statement (SQL with a simple structure that only appears once does not need to use the CTE structure!!!), and multiple tables need to join two sub-tables or where conditions; they must be modified to legal SQL statements; do not use case when

[0111] 3. SQL requirements:

[0112] (1) When encountering a query for growth, abs can only be added to the denominator, not the numerator. If the denominator is a value that may be negative, such as income, you need to add the abs function to the denominator;

[0113] (2) There is no need to use having if there is no group by;

[0114] (3) If the query may result in multiple results, it is necessary to add auxiliary columns such as "time" and "id" to facilitate the distinction between multiple results.

[0115] Therefore, the large language model can also reflect on and regenerate SQL statements in combination with SQL specifications, thereby further improving the accuracy of the generated SQL statements.

[0116] In some embodiments, the two processes of verifying the first SQL statement and regenerating the SQL statement when the verification fails can be completed at one time, thereby improving the efficiency of generating SQL statements. Exemplarily, the large language model can verify the first SQL statement based on the table information and the first natural language statement in combination with the error cause, SQL specifications, etc. If the first SQL statement fails the verification, the first SQL statement is corrected based on the table information and the first natural language statement in combination with the error cause, SQL specifications, etc., to generate a second SQL statement.

[0117] For example, the requirement for the output result of the large language model in the prompt information used for verification can be "You need to output the result in json format {"right": 0 / 1, whether the SQL statement to be verified is correct and can answer the user's query, "reason": the reason for the error of the SQL statement to be verified, "rewirte_sql": if modification is required, output the modified SQL statement, otherwise it is empty}".

[0118] Step 205: In response to the second SQL statement passing verification, the first natural language statement and the second SQL statement are added as a statement pair to the training data set.

[0119] In the present application, step 205 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0120] Step 206 , obtaining a reference mask sentence pair corresponding to the data table, a first reference natural language sentence corresponding to the reference mask sentence pair, and a mask sentence pair to be completed.

[0121] The reference masked statement pair and the masked statement pair to be completed both include masked natural language statements and masked SQL statements. In addition, the missing information in the masked natural language statement and the missing information in the masked SQL statement may be the same or different, and this is not limited.

[0122] For example, the reference mask statement pair is "Templated query: **How ​​many work orders are left**; Templatedsql: select count(**) from order_table where status=**and timestamp=**".

[0123] In addition, the number of reference mask statement pairs can be one or more, and there is no limitation on this.

[0124] In the present application, the first reference natural language sentence corresponding to the reference masked sentence pair may refer to a sentence obtained by performing mask completion on the masked natural language sentence in the reference masked sentence pair.

[0125] For example, the reference masked statement pair is "Templated query: **How ​​many work orders are not yet completed**; Templated sql: select count(**)from order_table where status=**and timestamp=**", the first reference natural language statement is "Refiend query: How many work orders are not completed today", and the masked statement pair to be completed is "new Templated query: Which city has the most **; new Templated sql: select city_namefrom country order by**limit 1".

[0126] In the present application, the reference masked sentence pair may be pre-set, or may be obtained by masking an unmasked reference sentence pair. The present application does not limit the method for obtaining the reference masked sentence pair.

[0127] Exemplarily, a first reference statement pair corresponding to a data table can be obtained, and sensitive information in natural language statements and SQL statements in the first reference statement pair can be identified, such as column names, column values, time and other sensitive information, and then the sensitive information in natural language statements and SQL statements in the first reference statement pair can be masked to obtain a reference masked statement pair.

[0128] Therefore, by masking the sensitive information in the unmasked reference sentence pairs to obtain the reference masked sentence pairs, the efficiency of obtaining the reference masked sentence pairs can be improved.

[0129] It should be noted that the first reference sentence pair and the second reference sentence pair may be the same or different, and this application does not limit this.

[0130] Step 207 , based on the table information, the reference masked sentence pair and the first reference natural language sentence, the large language model is used to perform masked completion on the masked natural language sentence in the masked sentence pair to be completed, so as to generate a second natural language sentence.

[0131] In the present application, prompt information can be generated based on table information, reference mask sentence pairs, a first reference natural language sentence, etc., and the prompt information can be input into a large language model. The large language model can perform mask completion on the masked natural language sentence in the mask sentence pair to be completed based on the table information and using the reference mask sentence pair and the first reference natural language sentence as a reference to generate a second natural language sentence.

[0132] Here, mask completion is performed on the masked natural language sentence, which can be understood as completing the missing information at the mask position in the masked natural language sentence.

[0133] For example, the reference mask statement pair is "Templated query: **How ​​many work orders are not completed**; Templatedsql: select count(**)from order_table where status=**and timestamp=**", and the first reference natural language statement is "Refiend query: How many work orders are not completed today".

[0134] Step 208 , based on the table information and the masked SQL statement in the masked statement pair to be completed, the second natural language statement is converted using the large language model to generate a third SQL statement.

[0135] Exemplarily, prompt information can be generated according to the table information, the second natural language statement, etc., and the prompt information is input into the large language model. The large language model converts the second natural language statement according to the table information to generate a third SQL statement. The method of generating the third SQL statement using the large language model here is similar to the method of generating the first SQL statement using the large language model in the above embodiment, so it will not be repeated here.

[0136] Exemplarily, a prompt message may be generated based on the table information, the second natural language statement, and the masked SQL statement in the masked statement pair to be completed, and the prompt message may be input into the large language model, and the large language model may perform mask completion on the masked SQL statement in the masked statement pair to be completed based on the table information, the second natural language statement, etc., to generate a third SQL statement. Here, performing mask completion on the masked SQL statement may be understood as completing the missing information at the mask position in the masked SQL statement.

[0137] For example, the reference masked statement pair is "Templated query: **How ​​many work orders are left**; Templatedsql: select count(**)from order_table where status=**and timestamp=**", the first reference natural language statement is "Refiend query: How many work orders are not completed today", and the masked statement pair to be completed is "new Templated query: Which city has the most **; new Templated sql: select city_namefrom country order by**limit 1". The second natural language statement obtained by mask completion using the large language model is "new Refiend query: Which city has the most population", and the third SQL statement obtained is "new Refiendsql: select city_name from country order by population limit 1".

[0138] Step 209: Add the second natural language statement and the third SQL statement as a statement pair to the training data set.

[0139] Exemplarily, the second natural language statement and the third SQL statement may be taken as a statement pair, the statement pair may be taken as a training sample, and the training sample may be added to the training data set.

[0140] Exemplarily, the second natural language statement and the third SQL statement may be used as a statement pair, and the statement pair and the table information of the data table may be used as training samples, and then the training samples may be added to the training data set.

[0141] In this application, by referring to the masked sentence pairs, similar natural language sentences and SQL statements can be generated for the situation where the NL2SQL model performs poorly in bad cases, and the data diversity of the training data set is also ensured.

[0142] Therefore, the sentence pairs of natural language sentences and SQL statements obtained based on reflection and the sentence pairs of natural language sentences and SQL statements obtained based on reference mask sentence pairs can be merged to form a training data set.

[0143] It should be noted that the above-mentioned process of obtaining statement pairs of natural language statements and SQL statements based on reflection and the above-mentioned process of obtaining statement pairs of natural language statements and SQL statements based on reference mask statement pairs can be executed in parallel and independently, or the process of obtaining statement pairs of natural language statements and SQL statements based on reflection can be executed first, and then the process of obtaining statement pairs of natural language statements and SQL statements based on reference mask statement pairs can be executed. This application does not limit this.

[0144] In an embodiment of the present application, statement pairs of natural language statements and SQL statements can also be generated based on reference masked statement pairs using a large language model. A training data set can be obtained based on statement pairs of natural language statements and SQL statements obtained based on reflection and statement pairs of natural language statements and SQL statements obtained based on reference masked statement pairs, thereby enriching the training data set. Furthermore, by using masked reference masked pairs, a large amount of similar data can be conveniently and quickly generated, thereby improving the coverage of the training data and the robustness of the NL2SQL model.

[0145] Figure 3 A flowchart of a method for generating a training data set based on a large model provided in another embodiment of the present application.

[0146] like Figure 3 As shown, the method for generating a training data set based on a large model includes:

[0147] Step 301: Obtain table information of a data table and a first natural language sentence to be converted corresponding to the data table.

[0148] In the present application, step 301 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0149] Step 302: Based on the table information, the first natural language statement is converted using a large language model to generate a first SQL statement.

[0150] In the present application, step 302 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0151] Step 303: Use the large language model to verify the first SQL statement to obtain a verification result of the first SQL statement.

[0152] In the present application, step 303 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0153] Step 304, in response to determining that the first SQL statement fails verification according to the verification result, the first natural language statement is reconverted using a large language model according to the verification result and the first SQL statement to generate a second SQL statement.

[0154] In the present application, step 304 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0155] Step 305: In response to the second SQL statement passing verification, the first natural language statement and the second SQL statement are added as a statement pair to the training data set.

[0156] In the present application, step 305 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0157] Step 306, obtaining questions related to the target query condition corresponding to the data table.

[0158] Exemplarily, the target query condition may be, for example, a time condition, or other query conditions, etc., which are not limited thereto.

[0159] For example, the SQL statements generated by the NL2SQL model often have errors in time issues such as "last week" and "Q2", so the time-related issues corresponding to the data table can be obtained.

[0160] Step 307: Obtain an answer matching the question according to the question.

[0161] Exemplarily, the question may be matched with the answer in the answer library to obtain the answer matching the question from the answer library.

[0162] Exemplarily, question and answer pairs related to database queries may be pre-built, and answers matching the questions corresponding to the data table may be obtained by referring to the pre-built question and answer pairs through a large language model.

[0163] For example, the question could be "In the SQL statement, date_column is a time column in the format of %Y-%m-%d, such as "2024-05-26", so how do you use SQL to express the time condition of "Tuesday"? (Do not output the thinking process, just output the SQL statement that needs to follow where), relative time uses the CURDATE function", and the answer is "WHERE date_column = DATE_ADD(CURDATE(), INTERVAL(1-WEEKDAY(CURDATE()))DAY)".

[0164] For example, the question can be "In the SQL statement, date_column is a time column with the format of %Y%m%d, such as "20240526", so how to use SQL to express the time condition of "second quarter"? (Do not output the thinking process, just output the SQL statement that needs to be followed by where), relative time is all expressed using the CURDATE function", and the answer to this question can be "WHERE date_column between DATE_FORMAT(CUR DATE(),'%Y0401')and DATE_FORMAT(CURDATE(),'%Y0630".

[0165] Step 308: Add the question and the answer as a sentence pair to the training data set.

[0166] Exemplarily, the question and the answer may be taken as a sentence pair, the sentence pair may be taken as a training sample, and the training sample may be added to the training data set.

[0167] Therefore, the sentence pairs of natural language sentences and SQL statements obtained based on reflection can be combined with the sentence pairs obtained based on questions and answers to form a training data set.

[0168] It should be noted that the above-mentioned process of obtaining statement pairs of natural language statements and SQL statements based on reflection and the above-mentioned process of obtaining statement pairs of questions and answers can be executed independently in parallel, or the process of obtaining statement pairs of natural language statements and SQL statements based on reflection can be executed first, and then the process of obtaining statement pairs of questions and answers can be executed. This application does not limit this.

[0169] In an embodiment of the present application, a training data set can also be obtained based on questions and answers related to the target query conditions, statement pairs of natural language statements and SQL statements obtained based on reflection, and statement pairs obtained based on questions and answers related to the target query conditions, thereby enriching the training data set, and using the question and answer pairs related to the target query conditions to train the NL2SQL model to enhance the capabilities of the model.

[0170] In one embodiment of the present application, a sentence pair of a natural language statement and an SQL statement can be obtained based on reflection, a sentence pair of a natural language statement and an SQL statement can be obtained based on a reference mask sentence pair, questions and answers related to the target query condition can be obtained, and the sentence pairs of the natural language statement and the SQL statement obtained based on reflection and based on the reference mask sentence pair, and the question and answer pairs can be merged to form a training data set. The training data set not only covers common situations in business scenarios, but also covers edge cases where the model is prone to errors, which helps the model better cope with complex query tasks in practical applications.

[0171] In order to implement the above embodiment, the embodiment of the present application also proposes a training method for an NL2SQL model. The training method includes: obtaining a training data set, using an initial NL2SQL model to convert sample natural language sentences in the training sample data set to generate a predicted SQL statement, and training the initial NL2SQL model according to the difference between the predicted SQL statement and the sample SQL statement corresponding to the sample natural language statement to obtain the NL2SQL model.

[0172] The training data set may be generated using the above-mentioned method for generating a training data set based on a large model. The training data set may include at least one statement pair, and each statement pair includes a sample natural language statement and a sample SQL statement.

[0173] In the present application, prompt information can be generated according to a sample natural language sentence, and the prompt information can be input into an initial NL2SQL model to obtain a predicted SQL statement output by the initial NL2SQL model.

[0174] Exemplarily, the training data set may also include table information corresponding to each statement pair, and prompt information may be generated based on the sample natural language statements in the statement pair and the table information corresponding to the statement pair, and the prompt information may be input into the initial NL2SQL model to obtain the predicted SQL statement output by the initial NL2SQL model. The table information corresponding to the statement pair may refer to the table information of the data table to be queried by the statement pair.

[0175] In the embodiment of the present application, the NL2SQL model is trained using a high-quality training data set generated based on the above method for generating a training data set, thereby improving the accuracy of the NL2SQL model.

[0176] In order to implement the above embodiment, the embodiment of the present application also proposes a data query method. The data query method may include: obtaining a natural language statement for data query, converting the natural language statement using the NL2SQL model, generating an SQL statement, performing data query using the SQL statement, and obtaining a query result.

[0177] The NL2SQL model may be obtained by training with a training data set generated using the above-mentioned method for generating a training data set based on a large model.

[0178] The natural language sentence in this embodiment may be a natural language sentence input by a user, and may be a query sentence in natural language form for querying a data table. For example, a natural language sentence input by a user may be "how many work orders have not been processed today".

[0179] Exemplarily, the generated SQL statement may be executed to query the data table and obtain the query result.

[0180] In an embodiment of the present application, the NL2SQL model is used to convert natural language statements into SQL statements, and the SQL statements are used to perform data queries to obtain query results. Thus, data queries based on natural language statements can reduce the difficulty of data queries and improve the convenience of data queries.

[0181] In order to implement the above embodiment, the embodiment of the present application also proposes a device for generating a training data set based on a large model. Figure 4 A schematic diagram of the structure of an apparatus for generating a training data set based on a large model provided in one embodiment of the present application.

[0182] like Figure 4 As shown, the device 400 for generating a training data set based on a large model includes:

[0183] A first acquisition module 410 is used to acquire table information of a data table and a first natural language sentence to be converted corresponding to the data table;

[0184] A generating module 420, configured to convert the first natural language statement based on the table information by using a large language model to generate a first structured query language SQL statement;

[0185] A verification module 430, configured to verify the first SQL statement using the large language model to obtain a verification result of the first SQL statement;

[0186] The generating module 420 is further configured to, in response to determining that the first SQL statement fails verification according to the verification result, reconvert the first natural language statement using the large language model according to the verification result and the first SQL statement to generate a second SQL statement;

[0187] The second acquisition module 440 is used to acquire a training data set according to the first natural language statement and the second SQL statement in response to the second SQL statement passing the verification; wherein the training data set is used to train the natural language to SQL model, and the training data set includes at least one statement pair, each statement pair includes a natural language statement and a SQL statement.

[0188] Optionally, the second acquisition module 440 is used to:

[0189] Obtaining a reference masked statement pair corresponding to the data table, a first reference natural language statement corresponding to the reference masked statement pair, and a masked statement pair to be completed; wherein the reference masked statement pair and the masked statement pair to be completed both include a masked natural language statement and a masked SQL statement;

[0190] According to the table information, the reference masked sentence pair and the first reference natural language sentence, using the large language model, mask-complete the masked natural language sentence in the masked sentence pair to be completed to generate a second natural language sentence;

[0191] According to the table information and the masked SQL statement in the masked statement pair to be completed, convert the second natural language statement using the large language model to generate a third SQL statement;

[0192] In response to the second SQL statement passing verification, the first natural language statement and the second SQL statement are added as a statement pair to the training data set, and the second natural language statement and the third SQL statement are added as a statement pair to the training data set.

[0193] Optionally, the second acquisition module 440 is used to:

[0194] Obtaining a first reference sentence pair corresponding to the data table;

[0195] Identify sensitive information in the natural language statements and SQL statements in the first reference statement pair

[0196] Masking is performed on sensitive information in the natural language statement and the SQL statement in the first reference statement pair to obtain the reference masked statement pair.

[0197] Optionally, the second acquisition module 440 is used to:

[0198] Obtaining questions related to the target query condition corresponding to the data table;

[0199] According to the question, an answer matching the question is obtained;

[0200] In response to the second SQL statement passing verification, the first natural language statement and the second SQL statement are added as a statement pair to the training data set, and the question and the answer are added as a statement pair to the training data set.

[0201] Optionally, the generating module 420 is used to:

[0202] Generate first prompt information according to the table information, the first natural language statement, the error cause in the verification result and the first SQL statement;

[0203] The first prompt information is input into the large language model, and the first SQL statement is corrected by using the large language model according to the first natural language statement in combination with the table information and the error cause to generate the second SQL statement.

[0204] Optionally, the generating module 420 is used to:

[0205] Obtaining SQL specifications; wherein the SQL specifications include at least one of the following: time specifications in SQL statements, SQL style specifications, and SQL requirements for target query situations;

[0206] The first prompt information is generated according to the table information, the first natural language statement, the error cause, the first SQL statement and the SQL specification.

[0207] Optionally, the generating module 420 is used to:

[0208] Acquire a second reference statement pair corresponding to the data table, wherein the second reference statement pair includes a natural language statement and an SQL statement;

[0209] Generate second prompt information according to the table information, the second reference statement pair and the first natural language statement; wherein the second prompt information is used to prompt the large language model to perform a natural language to SQL task;

[0210] The second prompt information is input into the large language model, and the first natural language statement is converted by using the large language model according to the table information and the second reference statement pair to generate the first SQL statement.

[0211] Optionally, the first acquisition module 410 is used to:

[0212] Obtaining a second reference natural language sentence;

[0213] Generate third prompt information according to the table information and the second reference natural language sentence; wherein the third prompt information is used to prompt the large language model to perform a natural language sentence generation task;

[0214] The third prompt information is input into the large language model, and the first natural language sentence is acquired by using the large language model according to the table information and the second reference natural language sentence.

[0215] Optionally, the first acquisition module 410 is used to:

[0216] Using the large language model, generating a candidate natural language sentence according to the table information and the second reference natural language sentence;

[0217] A grammar check is performed on the candidate natural language sentence, and in response to the candidate natural language sentence passing the grammar check, the candidate natural language sentence is determined to be the first natural language sentence.

[0218] It should be noted that the explanation of the aforementioned method embodiment for generating a training data set based on a large model is also applicable to the device for generating a training data set based on a large model in this embodiment, so it will not be repeated here.

[0219] In an embodiment of the present application, a first natural language statement to be converted is converted by using a large language model to generate a first SQL statement, and the first SQL statement is verified. If the first SQL statement fails the verification, the large language model is used to re-convert the statement based on the verification result and the first SQL statement. Therefore, when the generated SQL statement fails the verification, the verification result is used to reflect on and regenerate the SQL statement, which can improve the accuracy of the generated SQL statement, thereby improving the quality of the training data set. In addition, using a large language model to generate SQL statements can improve the efficiency of obtaining SQL statements, achieve fast and high-quality construction of a training data set for the NL2SQL model, and save costs.

[0220] In order to implement the above embodiment, the present application also provides a training device for the NL2SQL model. The training device for the NL2SQL model includes:

[0221] An acquisition module, used to acquire a training data set; wherein the training data set is generated by using the method for generating training data in the above embodiment, and the training data set includes at least one statement pair, each statement pair includes a sample natural language statement and a sample SQL statement;

[0222] A generation module, used to convert the sample natural language statement using an initial natural language to SQL model to generate a predicted SQL statement;

[0223] The training module is used to train the initial natural language to SQL model according to the difference between the predicted SQL statement and the sample SQL statement to obtain a natural language to SQL model.

[0224] It should be noted that the explanation of the aforementioned NL2SQL model training method embodiment is also applicable to the NL2SQL model training device of this embodiment, so it will not be repeated here.

[0225] In order to implement the above embodiment, the present application also provides a data query device. The data query device includes:

[0226] An acquisition module, used for acquiring natural language sentences for data query;

[0227] A generation module, used to convert the natural language statement using a natural language to SQL model to generate an SQL statement; wherein the natural language to SQL model is obtained by training the training data set generated by the training data set generation method of the above embodiment;

[0228] The query module is used to use SQL statements to query data and obtain query results.

[0229] It should be noted that the explanation of the aforementioned data query method embodiment is also applicable to the data query device of this embodiment, so it will not be repeated here.

[0230] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0231] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0232] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 502 or a computer program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An I / O (Input / Output) interface 505 is also connected to the bus 504.

[0233] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0234] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as a method for generating a training data set based on a large model. For example, in some embodiments, the method for generating a training data set based on a large model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the method for generating a training data set based on a large model described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured in any other appropriate manner (eg, by means of firmware) to execute the method for generating a training data set based on a large model.

[0235] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0236] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0237] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0238] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0239] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0240] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0241] According to an embodiment of the present application, the present application also provides a computer program product. When the instruction processor in the computer program product is executed, the method for generating a training data set based on a large model proposed in the above embodiment of the present application is executed.

[0242] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0243] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for generating training data sets based on large models, include: Acquire table information of a data table and a first natural language sentence to be converted corresponding to the data table; Based on the table information, convert the first natural language statement using a large language model to generate a first structured query language SQL statement; Using the large language model, verifying the first SQL statement to obtain a verification result of the first SQL statement; In response to determining that the first SQL statement fails verification according to the verification result, re-converting the first natural language statement using the large language model according to the verification result and the first SQL statement to generate a second SQL statement; In response to the second SQL statement passing verification, a training data set is obtained based on the first natural language statement and the second SQL statement; wherein the training data set is used to train a natural language to SQL model, and the training data set includes at least one statement pair, each statement pair includes a natural language statement and a SQL statement.

2. The method according to claim 1, in, In response to the second SQL statement passing verification, obtaining a training data set according to the first natural language statement and the second SQL statement includes: Obtaining a reference masked statement pair corresponding to the data table, a first reference natural language statement corresponding to the reference masked statement pair, and a masked statement pair to be completed; wherein the reference masked statement pair and the masked statement pair to be completed both include a masked natural language statement and a masked SQL statement; According to the table information, the reference masked sentence pair and the first reference natural language sentence, using the large language model, mask-complete the masked natural language sentence in the masked sentence pair to be completed to generate a second natural language sentence; According to the table information and the masked SQL statement in the masked statement pair to be completed, convert the second natural language statement using the large language model to generate a third SQL statement; In response to the second SQL statement passing verification, the first natural language statement and the second SQL statement are added as a statement pair to the training data set, and the second natural language statement and the third SQL statement are added as a statement pair to the training data set.

3. The method according to claim 2, in, The obtaining of the reference mask statement pair corresponding to the data table includes: Obtaining a first reference sentence pair corresponding to the data table; Identify sensitive information in the natural language statements and SQL statements in the first reference statement pair Masking is performed on sensitive information in the natural language statement and the SQL statement in the first reference statement pair to obtain the reference masked statement pair.

4. The method according to claim 1, in, In response to the second SQL statement passing verification, obtaining a training data set according to the first natural language statement and the second SQL statement includes: Obtaining questions related to the target query condition corresponding to the data table; According to the question, an answer matching the question is obtained; In response to the second SQL statement passing verification, the first natural language statement and the second SQL statement are added as a statement pair to the training data set, and the question and the answer are added as a statement pair to the training data set.

5. The method according to claim 1, in, The step of re-converting the first natural language statement using the large language model according to the verification result and the first SQL statement to generate a second SQL statement includes: Generate first prompt information according to the table information, the first natural language statement, the error cause in the verification result and the first SQL statement; The first prompt information is input into the large language model, and the first SQL statement is corrected by using the large language model according to the first natural language statement in combination with the table information and the error cause to generate the second SQL statement.

6. The method according to claim 5, in, The generating of the first prompt information according to the table information, the first natural language statement, the error cause in the verification result and the first SQL statement includes: Obtaining SQL specifications; wherein the SQL specifications include at least one of the following: time specifications in SQL statements, SQL style specifications, and SQL requirements for target query situations; The first prompt information is generated according to the table information, the first natural language statement, the error cause, the first SQL statement and the SQL specification.

7. The method according to claim 1, in, The converting the first natural language statement based on the table information using a large language model to generate a first structured query language SQL statement includes: Acquire a second reference statement pair corresponding to the data table, wherein the second reference statement pair includes a natural language statement and an SQL statement; Generate second prompt information according to the table information, the second reference statement pair and the first natural language statement; wherein the second prompt information is used to prompt the large language model to perform a natural language to SQL task; The second prompt information is input into the large language model, and the first natural language statement is converted by using the large language model according to the table information and the second reference statement pair to generate the first SQL statement.

8. The method according to claim 1, in, The obtaining of the first natural language sentence comprises: Obtaining a second reference natural language sentence; Generate third prompt information according to the table information and the second reference natural language sentence; wherein the third prompt information is used to prompt the large language model to perform a natural language sentence generation task; The third prompt information is input into the large language model, and the first natural language sentence is acquired by using the large language model according to the table information and the second reference natural language sentence.

9. The method according to claim 8, in, The acquiring the first natural language sentence by using the large language model according to the table information in combination with the second reference natural language sentence includes: Using the large language model, generating a candidate natural language sentence according to the table information and the second reference natural language sentence; A grammar check is performed on the candidate natural language sentence, and in response to the candidate natural language sentence passing the grammar check, the candidate natural language sentence is determined to be the first natural language sentence.

10. A natural language to SQL model training method, include: Acquire a training data set; wherein the training data set is generated by the method according to any one of claims 1 to 9, and the training data set includes at least one statement pair, each statement pair including a sample natural language statement and a sample SQL statement; Converting the sample natural language statements using an initial natural language to SQL model to generate a predicted SQL statement; The initial natural language to SQL model is trained according to the difference between the predicted SQL statement and the sample SQL statement to obtain a natural language to SQL model.

11. A data query method, include: Obtaining natural language statements for data query; Convert the natural language statement using a natural language to SQL model to generate an SQL statement; wherein the natural language to SQL model is trained using a training data set generated by the method according to any one of claims 1 to 9; Use the SQL statement to query data and obtain query results.

12. A device for generating a training data set based on a large model, include: A first acquisition module, used to acquire table information of a data table and a first natural language sentence to be converted corresponding to the data table; A generating module, configured to convert the first natural language statement based on the table information by using a large language model to generate a first structured query language SQL statement; A verification module, used to verify the first SQL statement using the large language model to obtain a verification result of the first SQL statement; The generating module is further configured to, in response to determining that the first SQL statement fails verification according to the verification result, re-convert the first natural language statement using the large language model according to the verification result and the first SQL statement to generate a second SQL statement; A second acquisition module is used to acquire a training data set according to the first natural language statement and the second SQL statement in response to the second SQL statement passing the verification; wherein the training data set is used to train the natural language to SQL model, and the training data set includes at least one statement pair, each statement pair includes a natural language statement and a SQL statement.

13. The device according to claim 12, in, The second acquisition module is used to: Obtaining a reference masked statement pair corresponding to the data table, a first reference natural language statement corresponding to the reference masked statement pair, and a masked statement pair to be completed; wherein the reference masked statement pair and the masked statement pair to be completed both include a masked natural language statement and a masked SQL statement; According to the table information, the reference masked sentence pair and the first reference natural language sentence, using the large language model, mask-complete the masked natural language sentence in the masked sentence pair to be completed to generate a second natural language sentence; According to the table information and the masked SQL statement in the masked statement pair to be completed, convert the second natural language statement using the large language model to generate a third SQL statement; In response to the second SQL statement passing verification, the first natural language statement and the second SQL statement are added as a statement pair to the training data set, and the second natural language statement and the third SQL statement are added as a statement pair to the training data set.

14. The device according to claim 13, in, The second acquisition module is used to: Obtaining a first reference sentence pair corresponding to the data table; Identify sensitive information in the natural language statements and SQL statements in the first reference statement pair Masking is performed on sensitive information in the natural language statement and the SQL statement in the first reference statement pair to obtain the reference masked statement pair.

15. The device according to claim 12, in, The second acquisition module is used to: Obtaining questions related to the target query condition corresponding to the data table; According to the question, an answer matching the question is obtained; In response to the second SQL statement passing verification, the first natural language statement and the second SQL statement are added as a statement pair to the training data set, and the question and the answer are added as a statement pair to the training data set.

16. The device according to claim 12, in, The generating module is used for: Generate first prompt information according to the table information, the first natural language statement, the error cause in the verification result and the first SQL statement; The first prompt information is input into the large language model, and the first SQL statement is corrected by using the large language model according to the first natural language statement in combination with the table information and the error cause to generate the second SQL statement.

17. The device according to claim 16, in, The generating module is used for: Obtaining SQL specifications; wherein the SQL specifications include at least one of the following: time specifications in SQL statements, SQL style specifications, and SQL requirements for target query situations; The first prompt information is generated according to the table information, the first natural language statement, the error cause, the first SQL statement and the SQL specification.

18. The device according to claim 12, in, The generating module is used for: Acquire a second reference statement pair corresponding to the data table, wherein the second reference statement pair includes a natural language statement and an SQL statement; Generate second prompt information according to the table information, the second reference statement pair and the first natural language statement; wherein the second prompt information is used to prompt the large language model to perform a natural language to SQL task; The second prompt information is input into the large language model, and the first natural language statement is converted by using the large language model according to the table information and the second reference statement pair to generate the first SQL statement.

19. The device according to claim 12, in, The first acquisition module is used to: Obtaining a second reference natural language sentence; Generate third prompt information according to the table information and the second reference natural language sentence; wherein the third prompt information is used to prompt the large language model to perform a natural language sentence generation task; The third prompt information is input into the large language model, and the first natural language sentence is acquired by using the large language model according to the table information and the second reference natural language sentence.

20. The device according to claim 19, in, The first acquisition module is used to: Using the large language model, generating a candidate natural language sentence according to the table information and the second reference natural language sentence; A grammar check is performed on the candidate natural language sentence, and in response to the candidate natural language sentence passing the grammar check, the candidate natural language sentence is determined to be the first natural language sentence.

21. A training device for a natural language to SQL model, include: An acquisition module, configured to acquire a training data set; wherein the training data set is generated by using the apparatus according to any one of claims 12 to 20, and the training data set includes at least one statement pair, each statement pair including a sample natural language statement and a sample SQL statement; A generation module, used to convert the sample natural language statement using an initial natural language to SQL model to generate a predicted SQL statement; The training module is used to train the initial natural language to SQL model according to the difference between the predicted SQL statement and the sample SQL statement to obtain a natural language to SQL model.

22. A data query device, include: An acquisition module, used for acquiring natural language sentences for data query; A generation module, used to convert the natural language statement using a natural language to SQL model to generate an SQL statement; wherein the natural language to SQL model is trained using a training data set generated by the apparatus according to any one of claims 1 to 9; The query module is used to use the SQL statement to perform data query and obtain query results.

23. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1 to 9, or to execute the method of claim 10, or to execute the method of claim 11.

24. A non-transitory computer-readable storage medium storing computer instructions, in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9, or to execute the method according to claim 10, or to execute the method according to claim 11.

25. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 9, or implements the steps of the method according to claim 10, or implements the steps of the method according to claim 11.

Images