SQL (Structured Query Language) statement generation method and device based on large language model
By splicing the prompt text and inputting it into a large language model to generate SQL statements, the problems of low accuracy and low efficiency of SQL statement generation in the prior art are solved, and higher accuracy and efficiency are achieved.
Patent Information
- Application Number
- CN202411075308.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-12-13
AI Technical Summary
The existing SQL statement generation method based on large language models is low in accuracy and low in efficiency, especially when processing complex texts, error accumulation is prone to occur.
By splicing the text to be converted, the first prompt text and the second prompt text to form the input text, and input it into the large language model, candidate SQL statements and target data table names and field information are generated, and if it matches, it is determined as the target SQL statement, which improves Accuracy and reduce the number of model calls.
Improves the accuracy and efficiency of SQL statement generation, reduces error accumulation, and enhances the accuracy of matching data table names and field information.
Smart Images

Figure CN119127922A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of natural language processing, and in particular to a method and device for generating SQL statements based on a large language model. Background Art
[0002] With the rapid development of models based on Transformer decoder variants, large language models (LLMs) are gradually replacing traditional natural language models and have achieved good results in practical applications. For the text-to-SQL (Text2SQL) task, there are methods based on prompt engineering and methods based on fine-tuning. In actual scene applications, although the large language model trained through fine-tuning can generate relatively accurate structured query language (SQL) statements, it still cannot meet the requirements of practical applications.
[0003] In the related technologies, the Text2SQL optimization method based on Chain-of-thought (COT) can improve the accuracy. Currently, there are two main methods.
[0004] One is the traditional COT method, which is to pre-configure different transformation rules, and the large language model generates SQL statements corresponding to the text based on the transformation rules. However, due to the variety of text description methods, the pre-configured transformation rules cannot cover all situations, and errors are easy to occur in the process of converting and generating SQL statements.
[0005] The other is a simple-to-complex approach, which is to decompose the user's question text into multiple sub-texts, and generate the SQL statements corresponding to each sub-text in turn according to the complexity of each sub-text, and finally generate the SQL statement corresponding to the user's question text. This method requires calling a large language model once for each sub-text to generate the corresponding SQL statement, which is inefficient, and the SQL statement corresponding to each sub-text needs to be generated based on the SQL statement of the previous sub-text, which is prone to serious error accumulation.
[0006] Therefore, how to improve the accuracy of generated SQL statements becomes an urgent problem to be solved. Summary of the invention
[0007] The embodiments of the present application provide a method and device for generating SQL statements based on a large language model, so as to solve the problem of low accuracy of SQL statements generated based on a large language model in the prior art.
[0008] In a first aspect, the present application provides a method for generating SQL statements based on a large language model, the method comprising:
[0009] The text to be converted, the first prompt text and the second prompt text are concatenated to obtain an input text, wherein the first prompt text is used to prompt the large language model to generate an SQL statement based on the text to be converted, and the second prompt text is used to prompt the large language model to analyze and output the data table name and field information involved in the text to be converted in the data table;
[0010] Inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information;
[0011] If the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the candidate SQL statement is determined as the target SQL statement corresponding to the text to be converted.
[0012] In a second aspect, the present application provides a device for generating SQL statements based on a large language model, the device comprising:
[0013] a concatenation module, configured to concatenate the text to be converted, the first prompt text, and the second prompt text to obtain an input text, wherein the first prompt text is used to prompt the large language model to generate an SQL statement based on the text to be converted, and the second prompt text is used to prompt the large language model to analyze and output the data table name and field information involved in the text to be converted in the data table;
[0014] A generation module, used for inputting the input text into the large language model to obtain candidate SQL statements, target data table names and target field information;
[0015] The verification module is used to determine the candidate SQL statement as the target SQL statement corresponding to the text to be converted if the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information.
[0016] In a third aspect, the present application further provides an electronic device, comprising a processor, wherein the processor is configured to implement the steps of any of the above-described methods for generating SQL statements based on a large language model when executing a computer program stored in a memory.
[0017] In a fourth aspect, the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for generating SQL statements based on a large language model.
[0018] In the embodiment of the present application, the text to be converted, the first prompt text and the second prompt text are concatenated to obtain the input text, and the input text is input into the large language model, so that the large language model generates a candidate SQL statement based on the text to be converted according to the prompt of the first prompt text, and analyzes the data table name and field information involved in the text to be converted in the data table according to the prompt of the second prompt text to obtain the target data table name and target field information. If the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the candidate SQL statement is determined as the target SQL statement corresponding to the text to be converted, and by comparing the target data table name and the target field information analyzed based on the text to be converted with the first data table name and the first field information included in the generated candidate SQL statement, it is determined whether the generated candidate SQL statement is correct. If it matches, the candidate SQL statement is determined as the target SQL statement, thereby improving the accuracy of SQL statement generation, and there is no need to call the large language model multiple times, thereby improving the efficiency of SQL statement generation. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0020] Figure 1 A flowchart of a SQL statement generation process based on a large language model provided in an embodiment of the present application;
[0021] Figure 2 A schematic diagram of the structure of a SQL statement generation device based on a large language model provided in an embodiment of the present application;
[0022] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0023] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the embodiment of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.
[0024] For the Text2SQL task implemented by the large language model, the most direct method is to combine prompt words (prompt) and basic logic, and use the large language model to convert the user's input questions into SQL statements, so as to query the desired data in the database based on the SQL statement. The embodiment of the present application mainly implements the Text2SQL query based on COT, improves the accuracy of the generated SQL statement, and allows users to quickly query useful information in a large amount of data. Therefore, the embodiment of the present application provides a method and device for generating SQL statements based on a large language model, in which the text to be converted, the first prompt text and the second prompt text are spliced to obtain an input text, the first prompt text is used to prompt the large language model to generate an SQL statement based on the text to be converted, and the second prompt text is used to prompt the large language model to analyze and output the data table name and field information involved in the text to be converted in the data table; the input text is input into the large language model to obtain a candidate SQL statement, a target data table name and target field information; if the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the candidate SQL statement is determined as the target SQL statement corresponding to the text to be converted.
[0025] Figure 1 A flowchart of a SQL statement generation process based on a large language model provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the process includes the following steps:
[0026] S101: Concatenate the text to be converted, the first prompt text and the second prompt text to obtain an input text, wherein the first prompt text is used to prompt the large language model to generate an SQL statement based on the text to be converted, and the second prompt text is used to prompt the large language model to analyze and output the data table name and field information involved in the text to be converted in the data table.
[0027] The SQL statement generation method based on the large language model provided in the present application is applied to an electronic device, which may be a server, a PC, a mobile terminal, etc.
[0028] In order to improve the accuracy of SQL statement generation, in an embodiment of the present application, a first prompt text and a second prompt text may be pre-saved. The first prompt text is used to prompt the large language model to generate an SQL statement based on the text to be converted. The second prompt text is used to prompt the large language model to analyze and output the data table name and field information involved in the text to be converted in the data table. That is to say, the large language model generates an SQL statement for the text to be converted by means of a prompt, and at the same time analyzes the text to be converted to determine the data table name and field information involved in the data table of the database if the SQL statement for the text to be converted is generated. Among them, the field information may include field names and / or field values.
[0029] After obtaining the text to be converted, the text to be converted, the first prompt text and the second prompt text can be spliced to obtain the input text, and the subsequent large language model is processed based on the prompt words in the input text. Among them, the text to be converted can be the user of the electronic device, that is, the user input, or it can be sent by other electronic devices connected to the electronic device. When the user inputs the text to be converted to any electronic device, it can be input in any form. For example, the user can input the text to be converted in the form of text; the audio of the text to be converted can also be input in the form of audio. After the electronic device receives the audio, the text obtained by processing based on speech recognition technology can be used as the text to be converted; the picture where the text to be converted is located can also be input in the form of a picture. After the electronic device receives the picture, the text obtained by processing based on image recognition technology can be used as the text to be converted. It should be noted that technicians in this field can configure the method of obtaining the text to be converted as needed, and the embodiments of the present application do not limit this.
[0030] When the text to be converted, the first prompt text, and the second prompt text are spliced, the splicing can be performed in any order, and the embodiment of the present application does not limit the splicing order.
[0031] S102: Input the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information.
[0032] After obtaining the input text, the input text can be input into the large language model. Since the large language model has a powerful language analysis and processing capability, after receiving the input text, the large language model can perform corresponding processing based on the prompts in the input text to obtain the SQL statement of the text to be converted. Since there may be errors in the generated SQL statement, the SQL statement generated by the large language model can be referred to as a candidate SQL statement. Since the input text also includes the second prompt text, the large language model can also output the target data table name and target field information, which are obtained by the large language model after analyzing the text to be converted based on the input text.
[0033] In a possible implementation, the output format can also be specified in the input text, so that the large language model outputs the candidate SQL statements, target data table names, and target field information according to the format specified in the input text. Exemplarily, the input text can specify that the large language model outputs the target data table names and target field information in the JavaScript Object Notation (JSON) format. Exemplarily, the prompt words that specify the output format in the input text can include the following:
[0034] ##The output is in JSON format and contains two fields. The fields are as follows:
[0035] -sub,list format
[0036] -sql, string format
[0037] In the above prompts, "sub, list format" means that the content after sub is displayed in the form of a list, and the content recorded after sub can include the text to be converted, and the target data table name and target field information corresponding to the text to be converted. "sql, string format" means that the content after sql is displayed in the form of a string, and the content recorded after sql can be a candidate SQL statement.
[0038] For example, based on the above prompt words, the output data of the large language model can be in the following form:
[0039] {sub":["Query the number of drivers in each district of Qingdao";{"drivinglicense":{"xzqh"}"],\n"sql":"SELECT v.xzqh,COUNT(v.xh)FROM drivinglicense"}}
[0040] In the above example, "Query the number of drivers in each district of Qingdao" represents the text to be converted; "drivinglicense" represents the name of the target data table; "xzqh" represents the target field information; and "SELECT v.xzqh, COUNT(v.xh) FROM drivinglicense" represents the candidate SQL statement.
[0041] S103: If the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the candidate SQL statement is determined as the target SQL statement corresponding to the text to be converted.
[0042] Since the large language model generates candidate SQL statements directly based on the text, the first data table name and the first field information used in the candidate SQL statement may be wrong. In order to verify the candidate SQL statement generated by the large language model, in the embodiment of the present application, it is possible to check whether the first data table name and the first field information used in the candidate SQL statement are correct. Therefore, in the embodiment of the present application, the large language model can be used to analyze the semantics of the text to be converted to determine the target data table name and the target field information involved in the data table of the database if the text to be converted is converted into an SQL statement. If the first data table name and the first field information match the target data table name and the target field information, it means that the candidate SQL statement is correct, and the candidate SQL statement can be determined as the target SQL statement. The match can be understood as the first data table name and the first field information are consistent with the target data table name and the target field information, or it can be understood as the similarity is greater than the threshold. In other words, let the large language model use different methods to determine the data table name and field information involved. One way is to directly generate a candidate SQL statement by analyzing the text to be converted, thereby obtaining the first data table name and the first field information. Another way is to analyze the semantics of the text to be converted to determine the target data table name and target field information involved in the text to be converted. If the data table name and field information determined based on different methods match, it means that the correctness of the generated candidate SQL statement is very high, and the candidate SQL statement can be determined as the target SQL statement of the text to be converted.
[0043] If the first data table name and the first field information included in the candidate SQL statement do not match the target data table name and the target field information, the target data table name and the target field information can be used to update the first data table name and the first field information in the candidate SQL statement, and the updated candidate SQL statement can be determined as the target SQL statement corresponding to the text to be converted.
[0044] Specifically, assume that the text to be converted is "How many drivers were registered in Shinan District last month". The target data table name corresponding to the text to be converted determined by the large language model is "drivinglicense", and the target field information is "xzqh: Shinan District" and "cclzrq: last month", where "xzqh" and "cclzrq" are field names, and "Shinan District" and "last month" are field values. The candidate SQL statement is "SELECT COUNT(v.xh) FROM drivinglicense v WHERE v.fzjg IN ('Lu B', 'Lu U') AND v.gxsj >= ADD_MONTHS(TRUNC(SYSDATE, 'MM'), -1) AND v.gxsj < TRUNC(SYSDATE, 'MM') AND v.xzqh = 'Shinan District'". The first data table name in this candidate SQL statement is "drivinglicense", and the field names in the first field information are "gxsj" and "xzqh".
[0045] In the above example, the registration time needs to be judged according to cclzrq (initial license date), while gxsj (update time) is used in the candidate SQL statement, and it can be corrected through error detection and correction to obtain the target SQL statement corresponding to the text to be converted.
[0046] Since in the embodiments of the present application, the text to be converted, the first prompt text, and the second prompt text are concatenated to obtain the input text, and the input text is input into the large language model, so that the large language model generates a candidate SQL statement based on the text to be converted according to the prompt of the first prompt text, and analyzes the data table name and field information involved in the text to be converted in the data table according to the prompt of the second prompt text to obtain the target data table name and target field information. If the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the candidate SQL statement is determined as the target SQL statement corresponding to the text to be converted. By comparing whether the target data table name and target field information analyzed based on the text to be converted match the first data table name and the first field information included in the generated candidate SQL statement, it is determined whether the generated candidate SQL statement is correct. In the case of a match, the candidate SQL statement is determined as the target SQL statement, which improves the accuracy of SQL statement generation and does not require multiple calls to the large language model, thus improving the efficiency of SQL statement generation.
[0047] To further improve the accuracy of SQL statement generation, based on the above embodiments, in the embodiments of the present application, the inputting the input text into the large language model to obtain a candidate SQL statement, a target data table name, and target field information includes:
[0048] The third prompt text and the input text are input into the large language model to obtain candidate SQL statements, target data table name and target field information; the third prompt text is used to prompt the large language model to determine the target intent of the text to be converted, and determine the target splitting strategy corresponding to the target intent according to the correspondence between different intents and splitting strategies saved in advance, and process the text to be converted based on the target splitting strategy to obtain the target text, determine the candidate SQL statements of the target text and output them.
[0049] After research and statistics, it is found that the SQL statements generated for some intended texts to be converted are relatively complex. For example, the intention of some texts to be converted is to directly find data in the database, while the intention of some texts to be converted is to make a difference to the found data. For some texts to be converted that directly find data, the accuracy of generating SQL statements is high and the process of generating SQL statements is relatively simple, while for texts to be converted that need to be further processed, the accuracy of generating SQL statements is low and the generation process is relatively complex. Therefore, in the embodiment of the present application, a third prompt text can also be pre-configured, and the third prompt text is used to prompt the target intention of the text to be converted determined by the language model, and according to the correspondence between different intentions and splitting strategies saved in advance, the target splitting strategy corresponding to the target intention is determined, and the text to be converted is processed based on the target splitting strategy to obtain the target text, thereby determining the SQL statement of the target text and outputting it. The SQL statement of the target text can be used as the candidate SQL statement described in the above embodiment. That is to say, the large language model first analyzes the intention of the text to be converted, and then determines how to split the text to be converted based on the analyzed target intention, thereby obtaining the SQL statement of the target text after splitting. The target text may be each of the split texts or a certain text among all the split texts.
[0050] Exemplarily, according to the correspondence between different pre-saved intentions and splitting strategies, for the text to be converted of intention A and intention B, the corresponding splitting strategy may be "split the text to be converted into multiple clauses based on semantics"; for the text to be converted of intention C, the corresponding splitting strategy may be "do not split the text to be converted".
[0051] When using a large language model to generate candidate SQL statements, the third prompt text and the input text can be input into the large language model, and the large language model determines the target splitting strategy for the text to be converted based on the prompts in the third prompt text and the input text, so that the text to be converted is processed according to the target splitting strategy to obtain a processed target text, and the candidate SQL statements of the generated target text are determined.
[0052] Specifically, the third prompt text may be:
[0053] ##Think about the output results in the following way:
[0054] 1. Determine the user's input intention, including proportion, ranking, year-on-year, month-on-month, difference, and others;
[0055] 2. When the intention is proportion, ranking, year-on-year, month-on-month, or difference, output the SQL solution process and SQL statement, for example, split the text to be converted into multiple sub-texts;
[0056] 3. When the intent is other, output the text to be converted and the SQL statement entered by the user.
[0057] In order to further improve the accuracy of SQL statement generation, based on the above embodiments, in the embodiment of the present application, the process of obtaining the candidate SQL statement by the large language model includes:
[0058] If the target intent of the text to be converted is a preset intent, the large language model divides the text to be converted into multiple sub-texts; determines each sub-text as a target text, and determines the candidate SQL statement corresponding to each target text.
[0059] When the large language model generates candidate SQL statements, after determining the target intent of the text to be converted, it can determine whether the target intent is a preset intent. If it is determined that the target intent is the preset intent, the large language model can split the text to be converted into multiple sub-texts. In other words, the target splitting strategy corresponding to the target intent is "split the text to be converted into multiple sub-texts." Exemplarily, the preset intent can be the intention of proportion, ranking, year-on-year, month-on-month, difference, and others. All the sub-texts obtained by splitting can represent the solution process of generating a SQL statement. After obtaining each sub-text, the entire solution process is also known. The last sub-text in the solution process can be determined as the target text, and the candidate SQL statement corresponding to the target text can be determined.
[0060] Specifically, assuming that the text to be converted is "How many drivers are registered in Qingdao this month, and what is the month-on-month increase compared with the same period last month?", it can be determined that the target intent of the text to be converted is "month-on-month increase", which is the preset intent. The text to be converted can be split into multiple sub-texts, namely:
[0061] 1. Check how many drivers have registered in Qingdao this month;
[0062] 2. Check how many drivers were registered in Qingdao last month;
[0063] 3. How many drivers have been registered in Qingdao this month, and what is the month-on-month change compared with the same period last month?
[0064] The solution composed of the above three sub-texts is the solution process, then the third sub-text can be determined as the target text, and the candidate SQL statement corresponding to the target text can be determined.
[0065] In order to further improve the accuracy of SQL statement generation, based on the above embodiments, in the embodiment of the present application, the process of obtaining the candidate SQL statement by the large language model includes:
[0066] If the target intent of the text to be converted is a non-preset intent, the large language model determines the text to be converted as the target text, and determines a candidate SQL statement corresponding to the target text.
[0067] If the target intent of the text to be converted is determined to be a non-preset intent, the large language model can determine the text to be converted as the target text and determine the candidate SQL statement corresponding to the target text. In other words, when the target intent is a non-preset intent, the corresponding target splitting strategy is "not splitting the text to be converted".
[0068] The following describes a specific embodiment of the SQL statement generation process based on a large language model.
[0069] The text to be converted is: Analyze the number of driver registrations in each district of Qingdao last month and sort them by quantity.
[0070] The first prompt is: You are a MySQL generation tool, you need to provide the customer with MySQL statements that can be directly executed, and generate MySQL statements according to the following table structure definition and examples. The following is the definition of the database table structure: Table name: drivinglicense, driver table, fields are as follows: xm, string, name, …………
[0071] It should be noted that the first prompt text includes the structure of the data table of the database involved, and those skilled in the art can configure it as needed.
[0072] Second prompt text: Analyze and output the data table name and field information involved in the text to be converted in the data table.
[0073] The third prompt text: 1. Determine the intention of the user input, which includes proportion, ranking, year-on-year, quarter-on-quarter, difference, and others; 2. When the intention is proportion, ranking, year-on-year, quarter-on-quarter, difference, output the SQL solution process and SQL statement; 3. When the intention is others, output the user input and SQL statement.
[0074] The text obtained by concatenating the above text to be converted, the first prompt text, the second prompt text and the third prompt text is the input text.
[0075] In a possible implementation, the input text may also include other requirement texts, which are used to prompt the large language model with some special requirements. For example, the other requirement texts may be: when querying driver statistics information without a location description, query Qingdao driver information by default.
[0076] In order to further improve the accuracy of SQL statement generation, based on the above embodiments, in the embodiment of the present application, before inputting the input text into the large language model to obtain candidate SQL statements, target data table names and target field information, the method further includes:
[0077] Get SQL statement generation sample;
[0078] The step of inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information includes:
[0079] The input text and the SQL statement generation sample are input into the large language model, so that the large language model processes the text to be converted based on the processing idea of the SQL statement generation sample to obtain candidate SQL statements, target data table names and target field information.
[0080] In order to further improve the accuracy of SQL statement generation, in an embodiment of the present application, a SQL statement generation sample can also be obtained, and the SQL statement generation sample can be input into the large language model, so that the large language model can refer to the processing ideas in the SQL statement generation sample when determining the target SQL statement of the text to be converted, thereby more accurately generating the target SQL statement of the text to be converted. Among them, the SQL statement generation sample can be pre-written and saved by relevant staff. The SQL statement generation sample can be one or more.
[0081] After obtaining the SQL statement generation sample, the input text and the SQL statement generation sample can be input into the large language model, so that the large language model processes the conversion text based on the processing idea of the SQL statement generation sample to obtain the candidate SQL statement, the target data table name and the target field information.
[0082] Inputting SQL statement generation samples and input text into the large language model can not only allow the large language model to better understand the output format, but also allow the large language model to learn processing ideas from similar SQL statement generation samples, thereby obtaining more accurate SQL statements.
[0083] In order to further improve the accuracy of SQL statement generation, based on the above embodiments, in the embodiment of the present application, the step of obtaining SQL statement generation samples includes:
[0084] Determining a first similarity between the text to be converted and each pre-stored sample question;
[0085] Determine a first preset number of target sample questions according to each first similarity and a first preset threshold;
[0086] The target sample problem and the processing result saved for the target sample problem are determined as the SQL statement generation sample.
[0087] In order to further improve the accuracy of SQL statement generation, in an embodiment of the present application, a sample set may be pre-saved, wherein a plurality of candidate SQL statement generation samples are saved in the sample set, and any candidate SQL statement consists of a sample question and a processing result. When obtaining SQL statement generation samples, a candidate SQL statement generation sample similar to the text to be converted may be searched in the sample set as a SQL statement generation sample.
[0088] In the embodiment of the present application, the first similarity between the text to be converted and each sample question can be determined. Exemplarily, the cosine similarity between the feature vector of the text to be converted and the feature vector of the sample question can be calculated, and the cosine similarity can be used as the first similarity. It should be noted that those skilled in the art can configure the method for determining the first similarity as needed, and the embodiment of the present application is not limited to this.
[0089] After obtaining each first similarity, a first preset number of target sample problems can be determined according to each first similarity and a preset threshold. For example, the preset threshold can be any value such as 0.8, 0.95, etc. Specifically, the first preset number of sample problems whose first similarity is higher than the first preset threshold can be determined as target sample problems. The first preset number can be any integer such as 1, 5, 8, 10, etc.
[0090] After the target sample problem is determined, the determined target sample problem and the processing result saved for the target sample problem can be determined as a SQL statement generation sample.
[0091] In order to further improve the accuracy of SQL statement generation, based on the above embodiments, in the embodiment of the present application, after obtaining the candidate SQL statement, the target data table name and the target field information, before determining whether the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the method further includes:
[0092] Obtaining the data table involved in generating the target SQL statement corresponding to the text to be converted;
[0093] If the first field information exists in the data table, the subsequent step of determining whether the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information is continued.
[0094] Since the acquired text to be converted may not use the actual field names and field values stored in the data table of the database, for example, the status of a motor vehicle includes overdue inspection, overdue review, reaching the scrap standard, reaching the scrap standard and the license plate and certificate are invalid, and the text to be converted may be "How many overdue but not scrapped cars are there in Laoshan District". If all fields of sufficient status are trained in advance, "overdue but not scrapped" can be more accurately mapped to the two states of "reaching the scrap standard" and "reaching the scrap standard and the license plate and certificate are invalid". If the data is insufficient, the first field information in the candidate SQL statement obtained based on the text to be converted may be "overdue but not scrapped". The candidate SQL statement containing the first field information will inevitably fail to accurately find data in the database.
[0095] Therefore, in an embodiment of the present application, in order to further improve the accuracy of SQL statement generation, after obtaining the candidate SQL statement, the target data table name and the target field information, before determining whether the first data table name and the first field information match the target data table name and the target field information, the data table involved in generating the target SQL statement corresponding to the text to be converted can be obtained. If the first field information in the candidate SQL statement exists in the data table, it means that the use of the first field information in the candidate SQL statement is relatively appropriate, and the subsequent steps of determining whether the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information can be continued.
[0096] In order to further improve the accuracy of SQL statement generation, based on the above embodiments, in the embodiment of the present application, the method further includes:
[0097] If the first field information does not exist in the data table, obtaining each second field information in the data table;
[0098] Determine a second similarity between the first data table name and each second field information;
[0099] Determine and output a second preset number of candidate field information according to each of the second similarities and a second preset threshold;
[0100] Receive third field information selected from the candidate field information, use the third field information to update the first field information in the candidate SQL statement, and determine the third field information as the target field information.
[0101] If the first field information does not exist in the data table, it means that the text to be converted may be a more colloquial expression, or the large language model incorrectly uses the first field information when generating candidate SQL statements. In an embodiment of the present application, each second field information in the data table can be obtained, and the second similarity between the first field information and each second field information can be determined respectively. It should be noted that when determining the second similarity, a technician in this field can select a similarity determination method as needed, and the embodiment of the present application does not limit this.
[0102] After determining each second similarity, a second preset number of candidate field information can be determined according to each second similarity and a second preset threshold. After determining the candidate field information, the candidate field information can be output so that the user can select appropriate third field information from the candidate field information. Exemplarily, the output content can be as follows: Are you asking whether it has reached the scrap standard or whether the scrap standard bulletin board certificate has been invalidated?
[0103] After receiving the third field information selected in the candidate field information, the first field information in the candidate SQL statement is updated using the third field information, and the third field information is determined as the target field information. In other words, the third field information comes from the data table, and the third field information is selected by the user. It can be considered that the accuracy of the third field information is extremely high, and the third field information can be directly used to replace the first field information in the large language model. The third field information is determined as the target field information to ensure that the first field information matches the target field information, so as to ensure that the candidate SQL statement with the field information replaced is subsequently determined as the target SQL statement.
[0104] In the embodiments of the present application, the field information of the data table is generally fixed, and better results can be obtained through error detection and correction. A field may have many field values, and the expressions are diverse. When the training data is insufficient, such as when a lot of field information is not involved, accurate field information cannot be obtained. The verification of the field information can have a certain effect when interacting with the user, and the SQL statement generation effect can be improved through friendly interaction with the user.
[0105] In order to further improve the accuracy of SQL statement generation, based on the above embodiments, in the embodiment of the present application, the fine-tuning training process of the large language model includes:
[0106] Obtaining a training set, wherein the training set includes sample text, and sample SQL statements corresponding to the sample text, standard data table names, and standard field information;
[0107] For each sample text, the sample text, the corresponding sample SQL statement, the standard data table name, the standard field information, the first prompt text and the second prompt text are concatenated to obtain training data;
[0108] Fine-tune the pre-trained large language model based on the training set to obtain the large language model.
[0109] Although the large language model has powerful language analysis capabilities, the large language model is trained based on general knowledge. The large language model may not be accurate in determining the data table name and field information involved in the text to be converted. Therefore, in order to improve the accuracy of SQL statement generation, in the embodiment of the present application, the general large language model can be fine-tuned. In other words, the large language model used in the process of generating SQL statements based on the large language model is obtained after fine-tuning training.
[0110] In order to fine-tune the large language model, in an embodiment of the present application, a training set is pre-configured, which includes sample text, and sample SQL statements corresponding to the sample text, standard data table names, and standard field information.
[0111] After obtaining the training set, for each sample text, the sample text, the corresponding sample SQL statement, the standard data table name, the standard field information, the first prompt text and the second prompt text can be spliced to obtain training data. In other words, the format and content of the training data during fine-tuning training are similar to the input data during model application, so as to maximize the accuracy of the SQL statement generated by the large language model.
[0112] After obtaining the training data corresponding to all sample texts, the large language model can be fine-tuned based on the training data to obtain a fine-tuned large language model, which is used to generate SQL statements. How to fine-tune the large language model based on the training data is a prior art and will not be described in detail in the embodiments of this application.
[0113] In the embodiment of the present application, the sample texts and the sample SQL statements, standard data table names and standard field information corresponding to the sample texts included in the training set may be manually pre-written or may be written using a large language model. For example, the large language model may analyze the intent of the sample texts based on the following prompt words and determine the splitting strategy of the sample texts:
[0114] #You are a language expert. Please judge whether the user question needs to be split and give the split results based on user needs.
[0115] ##Think about the output results in the following way:
[0116] 1. Determine the user's input intention, including proportion, ranking, year-on-year, difference, and others;
[0117] 2. When the intent is percentage, ranking, year-on-year comparison, or difference, the user's question must be split and the split sentences must be output one by one;
[0118] 3. When the intent is other, the user question does not need to be split, and only the original user question is output;
[0119] ##Output in json format, which includes three fields:
[0120] 1. Intent, list format
[0121] 2. Whether to split, string format
[0122] 3. Split result, string format
[0123] ##Example:
[0124] -Input: How many drivers have been registered in Qingdao this month, and what is the month-on-month comparison with the same period last month?
[0125] Output: {"Intent":["Month-on-month"],
[0126] "Whether to split": "Yes",
[0127] "Splitting result":"1. Check how many drivers were registered in Qingdao this month; \\n 2. Check how many drivers were registered in Qingdao last month; \\n 3. How many drivers were registered in Qingdao this month, and what was the month-on-month change compared with the same period last month?"}
[0128] After obtaining the above sub-texts, manual verification can be performed to identify the standard data table name and standard field information corresponding to each sub-text.
[0129] In a possible implementation, when verifying the first data table name, the first field information and the target data table name, the target field information, the data table name and the field information of each clause in the SQL statement can be verified. In an embodiment of the present application, the candidate SQL statement can be segmented according to preset keywords to obtain multiple SQL clauses. Among them, the preset keywords can be select, from, where, group, order by, etc. After obtaining each SQL clause, the first data table name and the first field information of each SQL clause can be determined respectively. When determining the target data table name and target field information, the large language model can also determine the target data table name and target field information corresponding to each keyword based on the above-mentioned preset keywords. When determining whether the data table name and field information match, a comparison can be made for each SQL clause to further improve the effect of SQL statement generation.
[0130] In the process of model application, we hope that the large language model can determine the target data table name and target field information according to certain ideas. In the fine-tuning training stage of the model, we can write similar SQL statements to generate samples and perform fine-tuning training on the large language model.
[0131] The following describes the SQL statement generation process based on a large language model in conjunction with a specific embodiment. The input and output data of the large language model may be as follows:
[0132] You are a MySQL generation tool and need to provide customers with MySQL statements that can be directly executed. Generate MySQL statements based on the following table structure definition and examples.
[0133] #The following is the definition of the database table structure:
[0134] ##Table name: drivinglicense, driver table, fields are as follows:
[0135] xm, string, name
[0136] …
[0137] …
[0138] #When querying driver statistics without a location description, query Qingdao driver information by default
[0139] ##Think about the output results in the following way:
[0140] 1. Determine the user's input intention, including proportion, ranking, year-on-year, month-on-month, difference, and other
[0141] 2. When the intention is proportion, ranking, year-on-year, month-on-month, or difference, output the SQL solution process and SQL
[0142] 3. When the intention is otherwise, output the user input and the SQL statement
[0143] ## The output is in JSON format and contains two fields as follows:
[0144] - intent, in list format
[0145] - sub, in list format
[0146] - sql, in string format
[0147] # Text to be converted: "Analyze the number of driver registrations in each district of Qingdao last month and sort them by the number"
[0148] # Output: "{{\"intent\":[\"Ranking\"],\"sub\":[\"1. Query the number of drivers in each district of Qingdao; {\\\"drivinglicense\\\":{\\\"xzqh\\\"}}\",\"2. Analyze the number of drivers in each district of Qingdao and sort them by the number; {\\\"drivinglicense\\\":{\\\"xzqh\\\"}}\"],\n\"sql\":\"SELECT v.xzqh,COUNT(v.xh)FROMdrivinglicense v WHERE v.fzjg IN('Lu B','Lu U')AND NOT REGEXP_LIKE(v.zt,'C|E|F|G')GROUP BY v.xzqh ORDER BY COUNT(v.xh)desc\"}}"
[0149] In the embodiment of the present application, the output specification is unified. In the output data, tasks such as intention, clause splitting, table structure judgment, and SQL statement are integrated, and there is a logical relationship between tasks. This not only avoids multiple calls to the large language model, reduces the process, and saves time, but also can improve the reasoning process through logical reasoning tasks, making the large language model more understand the task process and improving the accuracy rate.
[0150] The embodiment of the present application forms a standardized COT method for the Text2SQL task, simplifies the task process, and can improve the generation accuracy.
[0151] Based on the same inventive concept, the embodiment of the present application also provides a SQL statement generation device based on a large language model, Figure 2 which is a schematic structural diagram of a SQL statement generation device based on a large language model provided by the embodiment of the present application. The device includes:
[0152] A concatenation module 201 is used to concatenate the text to be converted, the first prompt text and the second prompt text to obtain an input text, wherein the first prompt text is used to prompt the large language model to generate an SQL statement based on the text to be converted, and the second prompt text is used to prompt the large language model to analyze and output the data table name and field information involved in the text to be converted in the data table;
[0153] A generating module 202 is used to input the input text into the large language model to obtain candidate SQL statements, target data table names and target field information;
[0154] The verification module 203 is configured to determine the candidate SQL statement as the target SQL statement corresponding to the text to be converted if the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information.
[0155] In a possible implementation, the generation module 202 is specifically used to input the third prompt text and the input text into the large language model to obtain candidate SQL statements, target data table name and target field information; the third prompt text is used to prompt the large language model to determine the target intent of the text to be converted, and determine the target splitting strategy corresponding to the target intent according to the correspondence between different intents and splitting strategies saved in advance, and process the text to be converted based on the target splitting strategy to obtain the target text, determine the candidate SQL statements of the target text and output them.
[0156] In a possible implementation, the generation module 202 is specifically used to split the text to be converted into multiple sub-texts by the large language model if the target intent of the text to be converted is a preset intent; determine the last sub-text in the solution process as the target text, and determine the candidate SQL statement corresponding to the target text.
[0157] In a possible implementation, the generation module 202 is specifically configured to, if the target intent of the text to be converted is a non-preset intent, determine the text to be converted as the target text by the large language model, and determine a candidate SQL statement corresponding to the target text.
[0158] In a possible implementation, the device further includes:
[0159] An acquisition module 204 is used to acquire an SQL statement generation sample;
[0160] The generation module 202 is specifically used to input the input text and the SQL statement generation sample into the large language model, so that the large language model processes the text to be converted based on the processing idea of the SQL statement generation sample to obtain candidate SQL statements, target data table names and target field information.
[0161] In a possible implementation, the acquisition module 204 is specifically used to determine a first similarity between the text to be converted and each pre-saved sample question; determine a first preset number of target sample questions based on each first similarity and a first preset threshold; and determine the target sample questions and the processing results saved for the target sample questions as samples for generating SQL statements.
[0162] In a possible implementation, the acquisition module 204 is further used to acquire a data table involved in generating a target SQL statement corresponding to the text to be converted;
[0163] The verification module 203 is also used to continue to execute the subsequent step of determining whether the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information if the first field information exists in the data table.
[0164] In a possible implementation, the verification module 203 is also used to obtain each second field information in the data table if the first field information does not exist in the data table; determine the second similarity between the first data table name and each second field information; determine and output a second preset number of candidate field information based on each second similarity and a second preset threshold; receive the third field information selected from the candidate field information, use the third field information to update the first field information in the candidate SQL statement, and determine the third field information as the target field information.
[0165] In a possible implementation, the acquisition module 204 is further used to acquire a training set, wherein the training set includes sample texts, and sample SQL statements, standard data table names, and standard field information corresponding to the sample texts; for each sample text, the sample text, the corresponding sample SQL statement, the standard data table name, the standard field information, the first prompt text, and the second prompt text are concatenated to obtain training data;
[0166] The training module 205 is used to perform fine-tuning training on the pre-trained large language model based on the training data to obtain the large language model.
[0167] Based on the same inventive concept, an embodiment of the present application further provides an electronic device, Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, it includes: a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304;
[0168] The memory 303 stores a computer program. When the program is executed by the processor 301, the processor 301 performs the following steps:
[0169] The text to be converted, the first prompt text and the second prompt text are concatenated to obtain an input text, wherein the first prompt text is used to prompt the large language model to generate an SQL statement based on the text to be converted, and the second prompt text is used to prompt the large language model to analyze and output the data table name and field information involved in the text to be converted in the data table;
[0170] Inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information;
[0171] If the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the candidate SQL statement is determined as the target SQL statement corresponding to the text to be converted.
[0172] In a possible implementation, the inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information includes:
[0173] The third prompt text and the input text are input into the large language model to obtain candidate SQL statements, target data table name and target field information; the third prompt text is used to prompt the large language model to determine the target intent of the text to be converted, and determine the target splitting strategy corresponding to the target intent according to the correspondence between different intents and splitting strategies saved in advance, and process the text to be converted based on the target splitting strategy to obtain the target text, determine the candidate SQL statements of the target text and output them.
[0174] In a possible implementation, the process of obtaining the candidate SQL statement by the large language model includes:
[0175] If the target intent of the text to be converted is a preset intent, the large language model splits the text to be converted into multiple sub-texts; determines the last sub-text in the solution process as the target text, and determines the candidate SQL statement corresponding to the target text.
[0176] In a possible implementation, the process of obtaining the candidate SQL statement by the large language model includes:
[0177] If the target intent of the text to be converted is a non-preset intent, the large language model determines the text to be converted as the target text, and determines a candidate SQL statement corresponding to the target text.
[0178] In a possible implementation manner, before inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information, the method further includes:
[0179] Get SQL statement generation sample;
[0180] The step of inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information includes:
[0181] The input text and the SQL statement generation sample are input into the large language model, so that the large language model processes the text to be converted based on the processing idea of the SQL statement generation sample to obtain candidate SQL statements, target data table names and target field information.
[0182] In a possible implementation, obtaining a SQL statement generation sample includes:
[0183] Determining a first similarity between the text to be converted and each pre-stored sample question;
[0184] Determining a first preset number of target sample questions according to each first similarity and a first preset threshold;
[0185] The target sample problem and the processing result saved for the target sample problem are determined as the SQL statement generation sample.
[0186] In a possible implementation, after obtaining the candidate SQL statement, the target data table name, and the target field information, and before determining whether the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the method further includes:
[0187] Obtaining the data table involved in generating the target SQL statement corresponding to the text to be converted;
[0188] If the first field information exists in the data table, the subsequent step of determining whether the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information is continued.
[0189] In a possible implementation, the method further includes:
[0190] If the first field information does not exist in the data table, obtaining each second field information in the data table;
[0191] Determine a second similarity between the first data table name and each second field information;
[0192] Determine and output a second preset number of candidate field information according to each of the second similarities and a second preset threshold;
[0193] Receive third field information selected from the candidate field information, use the third field information to update the first field information in the candidate SQL statement, and determine the third field information as the target field information.
[0194] In a possible implementation, the fine-tuning training process of the large language model includes:
[0195] Obtaining a training set, wherein the training set includes sample text, and sample SQL statements corresponding to the sample text, standard data table names, and standard field information;
[0196] For each sample text, the sample text, the corresponding sample SQL statement, the standard data table name, the standard field information, the first prompt text and the second prompt text are concatenated to obtain training data;
[0197] Fine-tune the pre-trained large language model based on the training data to obtain the large language model.
[0198] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface 302 is used for communication between the above electronic device and other devices. The memory may include a random access memory (RAM) and may also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0199] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.
[0200] On the basis of the above embodiments, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program executable by a processor is stored. When the program runs on the processor, the processor implements the following steps when executing:
[0201] The text to be converted, the first prompt text and the second prompt text are concatenated to obtain an input text, wherein the first prompt text is used to prompt the large language model to generate an SQL statement based on the text to be converted, and the second prompt text is used to prompt the large language model to analyze and output the data table name and field information involved in the text to be converted in the data table;
[0202] Inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information;
[0203] If the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the candidate SQL statement is determined as the target SQL statement corresponding to the text to be converted.
[0204] In a possible implementation, the inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information includes:
[0205] The third prompt text and the input text are input into the large language model to obtain candidate SQL statements, target data table name and target field information; the third prompt text is used to prompt the large language model to determine the target intent of the text to be converted, and determine the target splitting strategy corresponding to the target intent according to the correspondence between different intents and splitting strategies saved in advance, and process the text to be converted based on the target splitting strategy to obtain the target text, determine the candidate SQL statements of the target text and output them.
[0206] In a possible implementation, the process of obtaining the candidate SQL statement by the large language model includes:
[0207] If the target intent of the text to be converted is a preset intent, the large language model splits the text to be converted into multiple sub-texts; determines the last sub-text in the solution process as the target text, and determines the candidate SQL statement corresponding to the target text.
[0208] In a possible implementation, the process of obtaining the candidate SQL statement by the large language model includes:
[0209] If the target intent of the text to be converted is a non-preset intent, the large language model determines the text to be converted as the target text, and determines a candidate SQL statement corresponding to the target text.
[0210] In a possible implementation manner, before inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information, the method further includes:
[0211] Get SQL statement generation sample;
[0212] The step of inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information includes:
[0213] The input text and the SQL statement generation sample are input into the large language model, so that the large language model processes the text to be converted based on the processing idea of the SQL statement generation sample to obtain candidate SQL statements, target data table names and target field information.
[0214] In a possible implementation, obtaining a SQL statement generation sample includes:
[0215] Determining a first similarity between the text to be converted and each pre-stored sample question;
[0216] Determining a first preset number of target sample questions according to each first similarity and a first preset threshold;
[0217] The target sample problem and the processing result saved for the target sample problem are determined as the SQL statement generation sample.
[0218] In a possible implementation, after obtaining the candidate SQL statement, the target data table name, and the target field information, and before determining whether the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the method further includes:
[0219] Obtaining the data table involved in generating the target SQL statement corresponding to the text to be converted;
[0220] If the first field information exists in the data table, the subsequent step of determining whether the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information is continued.
[0221] In a possible implementation, the method further includes:
[0222] If the first field information does not exist in the data table, obtaining each second field information in the data table;
[0223] Determine a second similarity between the first data table name and each second field information;
[0224] Determine and output a second preset number of candidate field information according to each of the second similarities and a second preset threshold;
[0225] Receive third field information selected from the candidate field information, use the third field information to update the first field information in the candidate SQL statement, and determine the third field information as the target field information.
[0226] In a possible implementation, the fine-tuning training process of the large language model includes:
[0227] Obtaining a training set, wherein the training set includes sample text, and sample SQL statements corresponding to the sample text, standard data table names, and standard field information;
[0228] For each sample text, the sample text, the corresponding sample SQL statement, the standard data table name, the standard field information, the first prompt text and the second prompt text are concatenated to obtain training data;
[0229] Fine-tune the pre-trained large language model based on the training data to obtain the large language model.
[0230] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0231] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0232] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0233] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0234] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A method for generating SQL statements based on a large language model, characterized in that: The method comprises: The text to be converted, the first prompt text and the second prompt text are concatenated to obtain an input text, wherein the first prompt text is used to prompt the large language model to generate an SQL statement based on the text to be converted, and the second prompt text is used to prompt the large language model to analyze and output the data table name and field information involved in the text to be converted in the data table; Inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information; If the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the candidate SQL statement is determined as the target SQL statement corresponding to the text to be converted.
2. The method according to claim 1, characterized in that The step of inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information includes: The third prompt text and the input text are input into the large language model to obtain candidate SQL statements, target data table name and target field information; the third prompt text is used to prompt the large language model to determine the target intent of the text to be converted, and determine the target splitting strategy corresponding to the target intent according to the correspondence between different intents and splitting strategies saved in advance, and process the text to be converted based on the target splitting strategy to obtain the target text, determine the candidate SQL statements of the target text and output them.
3. The method according to claim 2, characterized in that The process of obtaining the candidate SQL statement by the large language model includes: If the target intent of the text to be converted is a preset intent, the large language model splits the text to be converted into multiple sub-texts; determines the last sub-text in the solution process as the target text, and determines the candidate SQL statement corresponding to the target text.
4. The method according to claim 2, characterized in that: The process of obtaining the candidate SQL statement by the large language model includes: If the target intent of the text to be converted is a non-preset intent, the large language model determines the text to be converted as the target text, and determines a candidate SQL statement corresponding to the target text.
5. The method according to claim 1, characterized in that Before inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information, the method further includes: Get SQL statement generation sample; The step of inputting the input text into the large language model to obtain candidate SQL statements, target data table names, and target field information includes: The input text and the SQL statement generation sample are input into the large language model, so that the large language model processes the text to be converted based on the processing idea of the SQL statement generation sample to obtain candidate SQL statements, target data table names and target field information.
6. The method according to claim 5, characterized in that The obtaining of SQL statement generation sample includes: Determining a first similarity between the text to be converted and each pre-stored sample question; Determining a first preset number of target sample questions according to each first similarity and a first preset threshold; The target sample problem and the processing result saved for the target sample problem are determined as the SQL statement generation sample.
7. The method according to claim 1, characterized in that After obtaining the candidate SQL statement, the target data table name and the target field information, and before determining whether the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information, the method further includes: Obtaining the data table involved in generating the target SQL statement corresponding to the text to be converted; If the first field information exists in the data table, the subsequent step of determining whether the first data table name and the first field information included in the candidate SQL statement match the target data table name and the target field information is continued.
8. The method according to claim 7, characterized in that The method further comprises: If the first field information does not exist in the data table, obtaining each second field information in the data table; Determine a second similarity between the first data table name and each second field information; Determine and output a second preset number of candidate field information according to each of the second similarities and a second preset threshold; Receive third field information selected from the candidate field information, use the third field information to update the first field information in the candidate SQL statement, and determine the third field information as the target field information.
9. The method according to claim 1, characterized in that: The fine-tuning training process of the large language model includes: Obtaining a training set, wherein the training set includes sample text, and sample SQL statements corresponding to the sample text, standard data table names, and standard field information; For each sample text, the sample text, the corresponding sample SQL statement, the standard data table name, the standard field information, the first prompt text and the second prompt text are concatenated to obtain training data; Fine-tune the pre-trained large language model based on the training data to obtain the large language model.
10. An electronic device, characterized in that: The electronic device includes a processor, and the processor is used to implement the steps of the SQL statement generation method based on a large language model as described in any one of claims 1 to 9 when executing a computer program stored in a memory.
Citation Information
Patent Citations
NL2SQL method based on LLM multi-table adaptive selection
CN117112732A
Data processing method, system and device, storage medium and program product
CN118132684A
Method and system for constructing language transformation assistant based on generative model
CN118332000A
Object Oriented System and Method for Optimizing the Execution of Marketing Segmentations
US20080027788A1
Cited By
SQL (Structured Query Language) statement acquisition method, medium and equipment
CN119862199A
SQL statement acquisition method, medium and device
CN119862199B
SQL statement generation method and device, electronic equipment and storage medium
CN120470021A
SQL statement generation method, device, electronic device and storage medium
CN120470021B
