Method and device for converting natural language into SQL (Structured Query Language) statement based on large model

Abstract

The invention discloses a method and a device for converting a natural language into an SQL (Structured Query Language) statement based on a large model. The method comprises the following steps: S1, performing data governance on an original data table; s2, constructing indexes for all the values in the database, selecting correlation values from the database based on the user problems, calculating the matching degree between the user problems and the correlation values based on a longest common substring matching algorithm, and selecting the correlation value with the highest matching degree; s3, rewriting the problem of the user; s4, performing word segmentation matching on the user question; s5, storing the accumulated seed question and answer statements into a vector database, performing weight assignment on different keywords in the user question, matching the user question with the accumulated seed question and answer statements, and selecting SQL question and answer statements; and S6, optimizing the DDL statement to construct a Prompt statement, and calling the large model to generate an SQL statement. According to the method, the quality of the SQL statements generated by the large model is greatly improved, and the flexibility of querying the statements by the user and the accuracy of generating the SQL statements are greatly improved.

Description

Technical Field

[0001] The present invention relates to the fields of machine learning and natural language processing, and in particular to a method and device for converting natural language into SQL statements based on a large model. Background Art

[0002] Currently, there are two commonly used natural language to SQL statement models: sequence model and pre-trained language model, which have different advantages and disadvantages, as follows:

[0003] Sequence model: Seq2Seq (Sequence-to-Sequence) model is widely used in natural language to SQL statement (Text-to-SQL) tasks. Its basic idea is to take natural language query as input sequence and generate corresponding SQL query as output sequence. Seq2Seq model usually consists of encoder and decoder. A common model architecture is Figure 1 As shown; in the figure, x1 to x N As input, a natural language query sequence, h1 to h N is the encoder; node c is the intermediate output state; y′1 to y′ m is the generated SQL statement sequence, h′1 to h′ m For the decoder.

[0004] Pre-trained language model: With the rise of pre-trained language models, BERT, GPT and other pre-trained language models have been introduced into Text-to-SQL tasks, significantly improving the performance of the model. Pre-trained language models are usually pre-trained on large-scale unlabeled text data, and then fine-tuned on specific tasks (such as Text-to-SQL), which can better understand and generate natural language and SQL statements. Figure 2 This is a schematic diagram of the GPT model architecture. N is the input query sequence; Trm node is the decoder of Transformer, and the GPT model architecture is composed of Trm node stacking architecture; T1 to T N is the output SQL statement sequence. The high performance of Transformer is due to its self-attention mechanism.

[0005] Among them, the essential idea of ​​the Attention mechanism can be abstracted into the following formula:

[0006]

[0007] In the field of NLP, Similarity(Q,K i ) i It can be understood as a word in a sentence, and Q is the entire sentence. The similarity of all words in a certain word is calculated. The three common calculation methods are as follows:

[0008]

[0009] Although both the sequence model and the pre-trained language model can convert SQL statements, there are still some problems, as follows:

[0010] 1. Seq2Seq models. Seq2Seq models are usually trained based on large-scale data, but in actual applications, queries may involve knowledge and background in specific fields. The model may not be able to fully understand and utilize this background knowledge to generate correct SQL queries; Seq2Seq models often show poor performance when processing complex queries (such as multi-table joins, nested queries, subqueries, etc.). Complex queries usually require the model to have higher logical reasoning capabilities and deeper semantic understanding.

[0011] 2. Pre-trained language model. Compared with the Seq2Seq model, the pre-trained language model has been greatly upgraded in terms of model architecture complexity and model parameter quantity, and the accuracy of SQL statement generation has also been significantly improved. However, the sequence input length supported by the pre-trained language model is generally only 512. When there are large wide tables or joint queries of multiple tables in the database, the length of the input information is often far more than 512, which seriously restricts the query effect.

[0012] In summary, the current common methods of converting natural language into SQL still have certain problems. Based on this, the inventors have developed a method and device for converting natural language into SQL statements based on a large model. Summary of the invention

[0013] In order to solve the above technical problems, the present invention provides a method and device for converting natural language into SQL statements based on a big model, which improves the quality of SQL statements generated by the big model and greatly improves the flexibility of user query statements and the accuracy of SQL statement generation.

[0014] In a first aspect, the present invention provides a method for converting natural language into SQL statements based on a large model, which adopts the following technical solution:

[0015] A method for converting natural language into SQL statements based on a large model, comprising the following steps:

[0016] S1. Perform data governance on the original data table;

[0017] Rewrite the original data table to generate a rewritten data table, and merge multiple tables into a wide table;

[0018] S2. Build an index for all values ​​in the database and select TOP-N relevant values ​​from the database based on the user question, then calculate the matching degree between the user question and the TOP-N relevant values ​​based on the longest common substring matching algorithm, and select the relevant value with the highest matching degree;

[0019] S3. Rewrite the user's question;

[0020] Rewrite user questions with the help of domain dictionaries, and rewrite abbreviations, abbreviations and aliases in the fields involved in user questions into professional full names;

[0021] When user questions involve time content, the time content is uniformly converted into UNIX timestamp;

[0022] S4. Perform word segmentation matching on user questions;

[0023] Match the user question with all values ​​stored in the database based on steps S2 and S3 respectively;

[0024] S5. Store the accumulated seed question and answer sentences into the vector database and build a vector matching model. The vector matching model is a twin network. The twin network includes two left and right input channels, and the two input channels share the same BERT model. Weights are assigned to different keywords in the user's questions. Then, the user's questions are matched with the accumulated seed question and answer sentences, and the TOP-S SQL question and answer sentences are selected.

[0025] S6. Optimize DDL statements to construct prompt statements and call the large model to generate SQL statements.

[0026] Preferably, constructing the Prompt statement in S6 comprises the following steps:

[0027] S61, displaying the rewritten data table in step S1 using the adapted DDL statement;

[0028] S62, after removing duplicates from each column in the adapted DDL statement display in S61, the top-M columns are selected as examples and written into the DDL statement;

[0029] S63, bringing the table information and column information of the related value with the highest matching degree matched in step S2 into the Prompt statement;

[0030] S64. Assemble the TOP-S SQL question-answer statements matched in step S5 into a Prompt statement as a few-shot, and input the Prompt statement into the big model to generate an SQL statement.

[0031] Preferably, the original business table in S1 includes a table name, a table comment, a column name and a column comment.

[0032] Preferably, the index in S2 is a BM25 index.

[0033] Preferably, the BM25 index includes term frequency, inverse document frequency and document length.

[0034] Preferably, the accumulated seed question-and-answer sentences include manually edited sentences and actual question-and-answer stage sentences.

[0035] In a second aspect, the present invention provides a device for converting natural language into SQL statements based on a large model, which adopts the following technical solution:

[0036] A device for converting natural language into SQL statements based on a large model, comprising the following modules:

[0037] The table management module is used to manage the original data table; rewrite the original data table to generate a rewritten data table, and merge multiple tables into a wide table;

[0038] Index building module, used to build indexes for all values ​​in the database;

[0039] A screening module is used to select TOP-N relevant values ​​from the database based on the user question, and then calculate the matching degree between the user question and the TOP-N relevant values ​​based on the longest common substring matching algorithm, and select the relevant value with the highest matching degree;

[0040] The rewriting module is used to rewrite user questions. With the help of the domain dictionary, the user questions are rewritten to rewrite the abbreviations, abbreviations and aliases in the fields involved in the user questions into professional full names. When the user questions involve time content, the time content is uniformly converted into UNIX timestamps.

[0041] The word segmentation matching module is used to perform word segmentation matching on user questions and match user questions with all values ​​stored in the database;

[0042] The vector matching module is used to store the accumulated seed question and answer sentences into the vector database and build a vector matching model. The vector matching model is a twin network. The twin network includes two left and right input channels, and the two input channels share the same BERT model. It assigns weights to different keywords in user questions, then matches user questions with the accumulated seed question and answer sentences, and selects the TOP-S SQL question and answer sentences.

[0043] The SQL statement generation module is used to optimize DDL statements, build Prompt statements, and call large models to generate SQL statements.

[0044] In a third aspect, the present application provides an electronic device, which adopts the following technical solution:

[0045] An electronic device, comprising:

[0046] one or more processors;

[0047] Memory;

[0048] One or more of the applications are stored in the memory and configured to be executed by one or more of the processors, and one or more of the applications are configured to: execute a method for converting natural language to SQL statements based on a large model according to any of the above items.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0050] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements any of the above-mentioned methods for converting natural language into SQL statements based on a large model.

[0051] In summary, the present invention includes the following beneficial technical effects:

[0052] 1. The present invention greatly improves the quality of SQL statements generated by large models and greatly improves the flexibility of user query statements and the accuracy of SQL statement generation.

[0053] 2. Compared with the Seq2Seq model, the Seq2Seq model cannot fully understand and utilize background knowledge, while the large model can generate SQL statements for the corresponding table based on the background information of the Prompt, which the Seq2Seq model cannot do.

[0054] 3. Compared with the pre-trained model, the input length of the pre-trained model is generally limited to 512, while the large models can almost all support a length of at least 8K, and some can even support up to 128K. The length extension of Prompt means that it is now possible to upgrade from a single-table simple question-and-answer scenario to a multi-table joint question-and-answer scenario.

[0055] 4. The present invention first designs a pattern matching method, which can quickly standardize the expressions such as idioms and abbreviations input by users; secondly, it designs a time conversion method, which converts the time word slots into timestamps that can be understood by the large model through two steps of time word slot extraction and time conversion. By combining these two methods, the flexibility of user query statements and the accuracy of SQL statement generation are greatly improved.

[0056] 5. The present invention rewrites the table name, table comment, column name, column comment and adjusts the column content that is prone to errors in the large model, and adds new columns to assist the large model in understanding user problems. Finally, for multiple related tables, a wide table management method of table merging is introduced to eliminate semantically overlapping fields and shorten the length of the prompt, paving the way for subsequent large model to generate high-quality SQL. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic diagram of a common model architecture in the background technology of the present invention.

[0058] Figure 2 It is a schematic diagram of the GPT model architecture in the background technology of the present invention.

[0059] Figure 3 It is a partial example of the domain dictionary in step S3 in the embodiment of the present invention.

[0060] Figure 4 This is an example of model fine-tuning data in step S3 in an embodiment of the present invention.

[0061] Figure 5 is a schematic diagram of a vector matching model in step S5 in an embodiment of the present invention.

[0062] Figure 6 It is a schematic diagram of the transformation through the vector matching model in step S5 in an embodiment of the present invention.

[0063] Figure 7 It is a schematic diagram of components of the Prompt statement in step S6 in the embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0065] The embodiment of the present invention discloses a method for converting natural language into SQL statements based on a large model.

[0066] The method for converting natural language to SQL statements based on a large model includes the following steps:

[0067] S1. Perform data governance on the original data table;

[0068] Rewrite the original data table to generate a rewritten data table, and merge multiple tables into a wide table;

[0069] The original data table includes table name, table comment, column name and column comment.

[0070] Specifically, the original data table will be transformed to make the "semantics more clear", such as rewriting the table name, table comments, column name, column comments and other content related to natural language processing; at the same time, with the help of data governance experience, multiple tables will be merged into wide tables.

[0071] On the one hand, it makes the organization of the original data table more reasonable, and on the other hand, it can further improve the accuracy of SQL generation.

[0072] Preferably, a corresponding example is given below:

[0073] Take the following table as an example to perform semantic rewriting of English column names. This is an "institution information table". The original English column names in the table are relatively simple, such as "institution code" is defined as "id" and "institution name" is defined as "name". Using "id" and "name" as English column names will obviously lose important information and is not conducive to the semantic understanding of subsequent large models. The present invention first performs semantic rewriting on the original English column names, and the results are as follows:

[0074] Chinese column name English listing Modified to Institution Code id institution_id Institution Name name institution_name Whether to cancel isredo is_revoked Institutional Level level institution_level Affiliations frommec parent_institution ... ... ...

[0075] Obviously, the semantic information of the rewritten English column names is clearer.

[0076] Take the following table as an example to rewrite the column content. This is a "student basic information table". When using it, users can ask questions such as "how many domestic and foreign students are there", "what is the age distribution of male students from abroad", etc. When the present invention manages table data, it will adjust the column content that is prone to errors. For example, in this scenario, a new column "Is it Chinese?" is added, and the value is a Boolean type of "yes / no".

[0077] Chinese English listing Modified to Student ID student_id student_id Name name name Education level educational_level_name educational_level gender gender_name gender place of birth birthplace_name place_of_birth Place of origin native_place_name native_place Country / Region Name region_name national_region_name ... ... ... Are you Chinese? is_chinese is_chinese

[0078] In addition, the present invention can also associate multiple tables into a wide table, such as "student basic information table", "student admission information table" and other related tables can be merged into a wide table to reduce redundant fields and simplify the token length for the subsequent assembly of prompts for large models.

[0079] S2. Build an index for all values ​​in the database and select TOP-N relevant values ​​from the database based on the user question, then calculate the matching degree between the user question and the TOP-N relevant values ​​based on the longest common substring matching algorithm, and select the relevant value with the highest matching degree;

[0080] Among them, the constructed index is the BM25 index, and the BM25 index takes into account factors such as word frequency, inverse document frequency, and document length.

[0081] Specifically, an index based on the full-text search service is created for the table content. In order to solve the problem of slow query performance, a "coarse-to-fine" matching method is used. The essence of this method is to use the index for a fast but coarse-grained initial search, and then use the LCS algorithm for a fine-grained matching process.

[0082] Specifically, Lucene is first used to build a BM25 index for all values ​​stored in each database. The BM25 algorithm takes into account factors such as word frequency, inverse document frequency, and document length, and can provide a more accurate relevance score. This indexing method not only improves the accuracy of the search, but also significantly improves the retrieval efficiency of large-scale data sets. When a user question is received, the BM25 index first extracts the TOP-N potential relevant values ​​from the entire database based on the user question, where N in TOP-N is a natural number, and N can be preferably 50. Then, the longest common substring matching algorithm is used to calculate the degree of match between the user question and the TOP-N relevant values, and selects the relevant value with the highest degree of match.

[0083] The integration of the BM25 index significantly improves search speed across a wide range of databases, drastically reducing the number of longest common substring matching algorithm calls from potentially millions to just hundreds.

[0084] Preferably, a corresponding example is given below:

[0085] For example, if a user enters the query "Does the school have a classmate named Han Meimei?", the sentence after word segmentation is "School|Does|have|a|classmate|named|Han Meimei|?", the full-text search service can first match the TOP-N related column values ​​in milliseconds, such as "Han Meimei", "Han Mei", "Han Xuemei", "South Korea", etc. Then, the longest common substring matching algorithm can match the column value "Han Meimei" with 100% similarity. Then the table name and column name to which "Han Meimei" belongs can be obtained at the same time, which lays the foundation for the subsequent assembly of prompts for the SQL large model.

[0086] S3. Rewrite the user's question;

[0087] Rewrite user questions with the help of domain dictionaries, and rewrite abbreviations, abbreviations and aliases in the fields involved in user questions into professional full names;

[0088] When user questions involve time content, the time content is uniformly converted into UNIX timestamp;

[0089] Specifically, when constructing a user question rewriting model, facing various user questions, the first thing to consider is how to unify the different ways of asking the same question into a more semantically clear expression. This embodiment adopts a method combining pattern matching with a large model.

[0090] In terms of pattern matching, with the help of the question-answering system platform, you can quickly upload the domain dictionary, which supports the .xlsx format. You can also use the domain dictionary to rewrite user questions, and rewrite the abbreviations, abbreviations, and aliases in the fields involved in the user questions into professional full names.

[0091] Reference Figure 3 , the following is a corresponding example:

[0092] For example, the user inputs the query statement "query the number of students in the School of Telecommunications", but the name of the school actually stored in the table is "School of Electronic and Information Engineering"; the SQL big model itself does not know that "School of Telecommunications" is the abbreviation of "School of Electronic and Information Engineering". At this time, the query statement can be rewritten with the help of the pattern matching dictionary.

[0093] In terms of fine-tuning of the big model, this embodiment has upgraded the fine-tuning capability for scenarios involving time. For example, when a user inputs the sentence "How many doctoral students have graduated from the last three months to now", the big model cannot understand time concepts such as "the last three months" and "now". This embodiment uniformly converts time processing into UNIX timestamps. The steps of converting to UNIX timestamps are divided into two steps: first, use the big model to extract time word slots, namely "the last three months" and "now"; second, use the big model to generate Java program code for time conversion.

[0094] Please refer to the model fine-tuning data Figure 4 .

[0095] The input sentence is the extracted time word slot, and the output part is part of the Java 8 code, which is embedded in other codes to form a complete time conversion code.

[0096] S4. Perform word segmentation matching on user questions;

[0097] Match the user question with all values ​​stored in the database based on steps S2 and S3 respectively;

[0098] Rewrite the user's question and match the word segmentation. Step S4 is the application of step S2 and step S3. After the question rewriting in step S2, the query sentence input by the user can be converted into a semantically clearer expression that is more suitable for the large model. Take a comprehensive question as an example. The user inputs "How many times has Han Meimei from the School of Telecommunications used her campus card in the past three months?"

[0099] First, perform step 202 to quickly match the table content, and you can get that the English table name of "Han Meimei" is "student_information" and the English column name is "name".

[0100] Secondly, step 203 is performed to extract and convert the time word slots, and the original question will be rewritten as "How many times did Han Meimei from the School of Electronic and Information Engineering use her campus card from 1711900800000 to 1719763200000?" This will obviously significantly improve the accuracy of SQL statement generation.

[0101] S5. Store the accumulated seed question and answer statements into the vector database and build a vector matching model. The vector matching model is a twin network. The twin network includes two left and right input channels, and the two input channels share the same BERT-like model. Weights are assigned to different keywords in user questions, and then user questions are matched with the accumulated seed question and answer statements, and TOP-S SQL question and answer statements are selected.

[0102] The accumulated seed question-and-answer sentences include manually edited sentences and actual question-and-answer stage sentences.

[0103] Specifically, with the help of the existing similarity vector retrieval capabilities of the question-and-answer platform, the accumulated seed question-and-answer sentences are stored in the vector database. The vector database can quickly return results in milliseconds, laying the foundation for the subsequent few-shot examples of large-model prompt sentences.

[0104] The accumulated seed question-answering sentences are divided into two parts: manually edited sentences in the initial stage and sentences in the actual question-answering stage. It uses a fine-tuned vector matching model for high-precision SQL data retrieval. The vector matching model architecture belongs to a twin network. Please refer to the schematic diagram Figure 5 .

[0105] The twin network structure has two input channels, left and right, which share the same BERT-like model. Its loss function can be cosine similarity, Euclidean distance L2, etc. In the SQL scenario, the fine-tuned vectorized model can assign weights to keywords in the sentence, increase the weight of keywords, and reduce the weight of auxiliary words, thereby greatly widening the semantic similarity between similar sentences and dissimilar sentences.

[0106] For example, the initial seed question and the actual question and answer statements are both vectorized through the vectorization model, and then matched with the vectorized statements in the vector database, and the TOP-S SQL question and answer statements are selected.

[0107] The value S in TOP-S is a natural number, and in this embodiment, S is preferably 1.

[0108] Specific reference Figure 6 .

[0109] S6. Optimize DDL statements to construct prompt statements and call the large model to generate SQL statements.

[0110] The input of the big model is called a Prompt statement. In order for the big model to generate more accurate SQL statements, the Prompt statement needs to contain the required key information. However, longer Prompt statements are not necessarily better. Too long Prompt statements can cause the big model to "forget information", which in turn affects the quality of SQL statement generation. Therefore, generating high-quality Prompt statements is an important step.

[0111] Specifically, S6 includes the following steps:

[0112] S61, displaying the rewritten data table in step S1 using the adapted DDL statement;

[0113] S62. De-duplicate the contents of each column in the adapted DDL statement display in S61, and remove the TOP-M columns as an example and write them into the DDL statement; wherein M in TOP-M is a natural number, and in this embodiment, M is preferably 3.

[0114] S63, bringing the table information and column information matched in step S2 into the Prompt statement;

[0115] S64. Assemble the TOP-S SQL question-answer statements matched in step S5 into a Prompt statement as a few-shot, and input the Prompt statement into the big model to generate an SQL statement.

[0116] The value S in TOP-S is a natural number, and in this embodiment, S is preferably 1.

[0117] Specifically, the Prompt statement consists of 5 parts, refer to Figure 7 , the 5 parts are:

[0118] 1. Role description: Write the large model role and task information here, for example, "You are a SQL expert, please pay full attention to the information in the table below, understand the column names and the corresponding explanations and examples of values, and generate SQL statements";

[0119] 2. Table information: Fill in the changed DDL statement information here;

[0120] 3. Example: Here is a few-shot example, selecting the most similar N SQL question-answer pairs from the vector library;

[0121] 4. Matched table information: The table column information here helps the large model to correctly select the corresponding table and column when generating SQL statements;

[0122] 5. Requirements: Write the output requirements for the large model here, such as "only need to output standard SQL statements".

[0123] In summary, inputting optimized Prompt statements into the big model can greatly improve the quality of SQL statements generated by the big model.

[0124] The embodiment of the present application also discloses a device for converting natural language into SQL statements based on a large model, and the device includes the following modules:

[0125] The table management module is used to manage the original data table; rewrite the original data table to generate a rewritten data table, and merge multiple tables into a wide table;

[0126] Index building module, used to build indexes for all values ​​in the database;

[0127] A screening module is used to select relevant values ​​from the database based on the user's question, and then calculate the matching degree between the user's question and the TOP-N relevant values ​​based on the longest common substring matching algorithm, and select the relevant value with the highest matching degree;

[0128] The rewriting module is used to rewrite user questions. With the help of the domain dictionary, the user questions are rewritten to rewrite the abbreviations, abbreviations and aliases in the fields involved in the user questions into professional full names. When the user questions involve time content, the time content is uniformly converted into UNIX timestamps.

[0129] The word segmentation matching module is used to perform word segmentation matching on user questions and match user questions with all values ​​stored in the database;

[0130] The vector matching module is used to store the accumulated seed question and answer sentences into the vector database and build a vector matching model. The vector matching model is a twin network. The twin network includes two left and right input channels. The two input channels share the same BERT-like model. It assigns weights to different keywords in user questions and then matches user questions with the accumulated seed question and answer sentences.

[0131] The SQL statement generation module is used to optimize DDL statements, build Prompt statements, and call large models to generate SQL statements.

[0132] The SQL statement generation module includes the following units:

[0133] A DDL statement display unit, used to display the rewritten data table generated by the data table management module using the adapted DDL statement;

[0134] A deduplication unit is used to deduplicate the rewritten data table displayed by the DDL statement display unit in the form of DDL statements;

[0135] A writing unit, used for writing the TOP-M columns of the rewritten data table displayed in the DDL statement after deduplication into the DDL statement as an example;

[0136] The import unit is used to import the table information and column information of the related value with the highest matching degree matched by the screening module into the Prompt statement;

[0137] The prompt statement assembly unit is used to assemble the TOP-S SQL question-answering statements selected by the vector matching module into the prompt statement as a few-shot, and input the prompt statement into the large model to generate an SQL statement.

[0138] An embodiment of the present invention further discloses an electronic device, which includes: a processor and a memory. The processor and the memory are connected, such as through a bus. Optionally, the electronic device may also include a transceiver. It should be noted that in actual applications, the transceiver is not limited to one, and the structure of the electronic device does not constitute a limitation on the embodiment of the present invention.

[0139] The processor may be a CPU central processing unit, a general processor, a DSP data signal processor, an ASIC application specific integrated circuit, an FPGA field programmable gate array or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0140] The bus may include a path to transmit information between the above components. The bus may be a PCI peripheral component interconnect standard bus or an EISA extended industry standard architecture bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0141] The memory can be a ROM read-only memory or other types of static storage devices that can store static information and instructions, a RAM random access memory or other types of dynamic storage devices that can store information and instructions, or an EEPROM electrically erasable programmable read-only memory, a CD-ROM read-only disc or other optical disc storage, an optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0142] The memory is used to store the application code for executing the solution of the present invention, and the execution is controlled by the processor. The processor is used to execute the application code stored in the memory to implement the content shown in the method for converting natural language to SQL statements based on a large model disclosed in the above embodiment.

[0143] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc.

[0144] An embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content of the method for converting natural language into SQL statements based on a large model disclosed in the above embodiment.

[0145] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for converting natural language into SQL statements based on a large model, characterized in that: The following steps are involved: S1. Perform data governance on the original data table; Rewrite the original data table to generate a rewritten data table, and merge multiple tables into a wide table; S2. Build an index for all values ​​in the database and select TOP-N relevant values ​​from the database based on the user question, then calculate the matching degree between the user question and the TOP-N relevant values ​​based on the longest common substring matching algorithm, and select the relevant value with the highest matching degree; S3. Rewrite the user's question; Rewrite user questions with the help of domain dictionaries, and rewrite abbreviations, abbreviations and aliases in the fields involved in user questions into professional full names; When user questions involve time content, the time content is uniformly converted into UNIX timestamp; S4. Perform word segmentation matching on user questions; Match the user question with all values ​​stored in the database based on steps S2 and S3 respectively; S5. Store the accumulated seed question and answer sentences into the vector database and build a vector matching model. The vector matching model is a twin network. The twin network includes two left and right input channels, and the two input channels share the same BERT model. Weights are assigned to different keywords in the user's questions. Then, the user's questions are matched with the accumulated seed question and answer sentences, and the TOP-S SQL question and answer sentences are selected. S6. Optimize DDL statements to construct prompt statements and call the large model to generate SQL statements.

2. According to the method of converting natural language into SQL statements based on a large model according to claim 1, it is characterized in that: Constructing the Prompt statement in S6 includes the following steps: S61, displaying the rewritten data table in step S1 using the adapted DDL statement; S62, after removing duplicates from each column in the adapted DDL statement display in S61, the top-M columns are selected as examples and written into the DDL statement; S63, bringing the table information and column information of the related value with the highest matching degree matched in step S2 into the Prompt statement; S64. Assemble the TOP-S SQL question-answer statements matched in step S5 into a Prompt statement as a few-shot, and input the Prompt statement into the big model to generate an SQL statement.

3. The method for converting natural language to SQL statements based on a large model according to claim 1, characterized in that: The original business table in S1 includes a table name, a table comment, a column name, and a column comment.

4. The method for converting natural language to SQL statements based on a large model according to claim 1, characterized in that: The index in S2 is a BM25 index.

5. The method for converting natural language to SQL statements based on a large model according to claim 4, characterized in that: The BM25 index includes term frequency, inverse document frequency and document length.

6. The method for converting natural language to SQL statements based on a large model according to claim 1, characterized in that: The seed question-answering sentences accumulated in S5 include manually edited sentences and actual question-answering stage sentences.

7. A device for converting natural language to SQL statements based on a large model, characterized in that: Includes the following modules: The table management module is used to manage the original data table; rewrite the original data table to generate a rewritten data table, and merge multiple tables into a wide table; Index building module, used to build indexes for all values ​​in the database; A screening module is used to select TOP-N relevant values ​​from the database based on the user question, and then calculate the matching degree between the user question and the TOP-N relevant values ​​based on the longest common substring matching algorithm, and select the relevant value with the highest matching degree; The rewriting module is used to rewrite user questions. With the help of the domain dictionary, the user questions are rewritten to rewrite the abbreviations, abbreviations and aliases in the fields involved in the user questions into professional full names. When the user questions involve time content, the time content is uniformly converted into UNIX timestamps. The word segmentation matching module is used to perform word segmentation matching on user questions and match user questions with all values ​​stored in the database; The vector matching module is used to store the accumulated seed question and answer sentences into the vector database and build a vector matching model. The vector matching model is a twin network. The twin network includes two left and right input channels, and the two input channels share the same BERT-like model. It assigns weights to different keywords in user questions, then matches user questions with the accumulated seed question and answer sentences, and selects the TOP-S SQL question and answer sentences. The SQL statement generation module is used to optimize DDL statements, build Prompt statements, and call large models to generate SQL statements.

8. An electronic device, characterized in that: It includes: One or more processors; Memory; one or more applications; One or more of the applications are stored in the memory and configured to be executed by one or more of the processors, and one or more of the applications are configured to: execute a method for converting natural language to SQL statements based on a large model according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for converting natural language into SQL statements based on a large model as described in any one of claims 1 to 6 is implemented.

Images