A method and device for converting natural language into SQL statements based on a large model

Abstract

The application discloses a kind of method and device for natural language conversion SQL sentence based on large model, the method includes: S1, data governance is carried out to original data table;S2, all values in database are indexed and relevant values are selected from database based on user question, the matching degree between user question and relevant value is calculated based on the longest common substring matching algorithm, and the relevant value with the highest matching degree is selected;S3, rewriting is carried out to user question;S4, word segmentation matching is carried out to user question;S5, the seed question and answer sentence that has been accumulated is stored in vector database, the weight assignment is carried out to different key words in user question, user question is matched with the seed question and answer sentence that has been accumulated, and SQL question and answer sentence is selected;S6, Prompt sentence is constructed by optimizing DDL sentence, and SQL sentence is generated by calling large model.The quality of SQL sentence generated by large model is greatly improved, and the flexibility of user query sentence and the accuracy of SQL sentence generation are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning and natural language processing, and in particular to a method and device for converting natural language into SQL statements based on a large model. BACKGROUND

[0002] The current commonly used natural language to SQL statement model is divided into two types, namely sequence model and pre-training language model, which have different advantages and disadvantages, as follows:

[0003] Sequence model: Seq2Seq (Sequence-to-Sequence) model has been widely used in natural language to SQL (Text-to-SQL) task, and its basic idea is to take natural language query as input sequence and generate corresponding SQL query as output sequence. Seq2Seq model is usually composed of an encoder and a decoder, and a common model architecture is as shown in Figure 1 The input natural language query sequence is x1 to x N The encoder is h1 to h N The intermediate output state is node c; the generated SQL statement sequence is y'1 to y' m The decoder is h'1 to h' m .

[0004] Pre-training language model: with the rise of pre-training language models, BERT and GPT pre-training language models are introduced into the Text-to-SQL task, which significantly improves the performance of the model. Pre-training language models are usually pre-trained on large amounts of unlabeled text data, and then fine-tuned on specific tasks (such as Text-to-SQL), which can better understand and generate natural language and SQL statements. Figure 2 The GPT model architecture is shown in the figure, and the input query sequence is E1 to E N The decoder of the Transformer is the Trm node, and the GPT model architecture is stacked with the Trm node; the output SQL statement sequence is T1 to T N .

[0005] The essence of the Attention mechanism can be abstracted as follows:

[0006]

[0007] In the field of NLP, Similarity(Q,K) i K in ) i This can be understood as a single word in a sentence, while Q represents the entire sentence. The similarity is calculated for all words within a single word, and there are three common calculation methods with the following formulas:

[0008]

[0009] Although both sequence models and pre-trained language models can convert SQL statements, they still have certain problems, as follows:

[0010] 1. Regarding the Seq2Seq model: Seq2Seq models are typically trained on large-scale datasets, but in real-world applications, queries may involve domain-specific knowledge and background. The model may not fully understand and utilize this background knowledge to generate correct SQL queries; Seq2Seq models often exhibit poor performance when handling complex queries (such as multi-table joins, nested queries, subqueries, etc.). Complex queries usually require models with higher logical reasoning capabilities and deeper semantic understanding.

[0011] 2. Pre-trained Language Models. Compared to Seq2Seq models, pre-trained language models offer significant upgrades in both model architecture complexity and parameter count, and also demonstrate a marked improvement in the accuracy of generated SQL statements. However, pre-trained language models typically only support sequence input lengths of 512. When dealing with large tables or joins between multiple tables in a database, the length of the input information often far exceeds 512, severely limiting query performance.

[0012] In summary, current methods for converting natural language into SQL still have certain problems. Based on this, the inventors have developed a method and device for converting natural language into SQL statements based on a large model. Summary of the Invention

[0013] To address the aforementioned technical problems, this invention provides a method and apparatus for converting natural language to SQL statements based on a large model. This method improves the quality of SQL statements generated by the large model and significantly enhances the flexibility of user queries and the accuracy of SQL statement generation.

[0014] Firstly, the present invention provides a method for converting natural language to SQL statements based on a large model, which adopts the following technical solution:

[0015] A method for converting natural language to SQL statements based on a large model includes the following steps:

[0016] S1. Perform data governance on the original data table;

[0017] Rewriting the original data table to generate a rewritten data table, and merging multiple tables into a wide table;

[0018] S2, constructing an index for all values in the database and selecting TOP-N related values from the database based on the user question, and then calculating the matching degree between the user question and the TOP-N related values based on the longest common substring matching algorithm, and selecting the related value with the highest matching degree;

[0019] S3, rewriting the user question;

[0020] Rewriting the user question with the help of the field dictionary, rewriting the abbreviations, abbreviations and alternative names in the field involved in the user question into professional full names;

[0021] When the user question involves time content, the time content is uniformly converted into UNIX timestamp;

[0022] S4, segmenting and matching the user question;

[0023] Match the user question with all values stored in the database based on steps S2 and S3 respectively;

[0024] S5, store the accumulated seed question and answer sentences in the vector database, and build a vector matching model, the vector matching model is a twin network, the twin network includes left and right two input channels, and the two input channels share the same BERT model, the different keywords in the user question are weighted, then the user question and the accumulated seed question and answer sentences are matched, and TOP-S SQL question and answer sentences are selected;

[0025] S6, optimize the DDL statement to build the Prompt statement, and call the large model to generate the SQL statement.

[0026] Preferably, the Prompt statement in S6 includes the following steps:

[0027] S61, the rewritten data table in step S1 is displayed using the adapted DDL statement;

[0028] S62, the TOP-M column values of each column content in the adapted DDL statement display in S61 are taken as examples after deduplication and written into the DDL statement;

[0029] S63, the table information and column information of the related value with the highest matching degree matched in step S2 are brought into the Prompt statement;

[0030] S64, assemble the matched TOP-S SQL question and answer statements in step S5 as few-shot into a Prompt statement, and input the Prompt statement into the large model to generate an SQL statement.

[0031] Preferably, the original business table in S1 includes table name, table annotation, column name and column annotation.

[0032] Preferably, the index in S2 is a BM25 index.

[0033] Preferably, the BM25 index includes word frequency, inverse document frequency and document length.

[0034] Preferably, the accumulated seed question and answer statements include artificially sorted statements and actual question and answer stage statements.

[0035] In a second aspect, the present application provides a device for converting natural language to SQL statements based on a large model, which adopts the following technical scheme:

[0036] A device for converting natural language to SQL statements based on a large model, comprising the following modules:

[0037] A table management module for data management of an original data table; rewriting the original data table to generate a rewritten data table, and merging multiple tables into a wide table;

[0038] An index construction module for constructing indexes for all values in the database;

[0039] A screening module for selecting TOP-N related values from the database based on a user question, then calculating the matching degree between the user question and the TOP-N related values based on the longest common substring matching algorithm, and selecting the related value with the highest matching degree;

[0040] A rewriting module for rewriting a user question; rewriting the user question by means of a domain dictionary, rewriting the abbreviations, abbreviations and alias words in the domain involved in the user question into professional full name words; when the user question involves time content, converting the time content into UNIX timestamp;

[0041] A word segmentation and matching module for word segmentation and matching of a user question; matching the user question with all values stored in the database;

[0042] The vector matching module is used to store the accumulated seed question-and-answer statements into the vector database and build a vector matching model. The vector matching model is a Siamese network, which includes two input channels, left and right, and the two input channels share the same BERT class model. It assigns weights to different keywords in the user question, and then matches the user question with the accumulated seed question-and-answer statements and selects the top-S SQL question-and-answer statements.

[0043] The SQL statement generation module is used to optimize DDL statements, construct Prompt statements, and call large models to generate SQL statements.

[0044] Thirdly, this application provides an electronic device that adopts the following technical solution:

[0045] An electronic device comprising:

[0046] One or more processors;

[0047] Memory;

[0048] One or more of the applications are stored in the memory and configured to be executed by one or more of the processors, and the one or more applications are configured to execute a method for converting natural language to SQL statements based on a large model as described in any of the preceding claims.

[0049] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution:

[0050] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for converting natural language to SQL statements based on a large model as described above.

[0051] In summary, the present invention has the following beneficial technical effects:

[0052] 1. This invention greatly improves the quality of SQL statements generated from large models and significantly enhances the flexibility of user query statements and the accuracy of SQL statement generation.

[0053] 2. Compared with the Seq2Seq model, the Seq2Seq model cannot fully understand and utilize background knowledge, while the large model can generate SQL statements for the corresponding tables based on the background information of the Prompt, which is something that the Seq2Seq model cannot do.

[0054] 3. Compared with the pre-training model, the pre-training model input length is generally limited to 512, while the large model can support at least 8K length at present, and some can even support up to 128K. The length expansion of Prompt means that the single table simple question and answer scene can now be upgraded to the multi-table joint question and answer scene.

[0055] 4. The application first designs a pattern matching method, which can quickly standardize the user's input of colloquial expressions, abbreviations and other expression methods; secondly, a time conversion method is designed, which converts the time slot into a timestamp that the large model can understand through two steps of time slot extraction and time conversion, and through the combination of the two methods, the flexibility of the user query statement and the accuracy of the SQL statement generation are greatly improved.

[0056] 5. The application rewrites the table name, table annotation, column name, column annotation, and adjusts the column content prone to errors of the large model, adds a column to assist the large model in understanding user problems; finally, for multiple related tables, a wide table management method of table merging is introduced to eliminate the semantic overlap of the fields, and also simplify the length of Prompt, which lays the foundation for the subsequent large model to generate high-quality SQL. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a common model architecture diagram in the background art of the application.

[0058] Figure 2 is a GPT model architecture diagram in the background art of the application.

[0059] Figure 3 is a part of the domain dictionary in step S3 in the embodiment of the application.

[0060] Figure 4 is an example of model fine-tuning data in step S3 in the embodiment of the application.

[0061] Figure 5 is a schematic diagram of a vector matching model in step S5 in the embodiment of the application.

[0062] Figure 6 is a conversion schematic diagram of the vector matching model in step S5 in the embodiment of the application.

[0063] Figure 7 is a component schematic diagram of the Prompt statement in step S6 in the embodiment of the application. DETAILED DESCRIPTION

[0064] The application will be further described in detail below in conjunction with the drawings.

[0065] The embodiment of the application discloses a method for converting natural language into SQL sentences based on a large model.

[0066] The method for converting natural language into SQL sentences based on a large model comprises the following steps:

[0067] S1, data governance is performed on an original data table;

[0068] The original data table is rewritten to generate a rewritten data table, and multiple tables are combined into a wide table.

[0069] The original data table comprises a table name, table annotation, column name and column annotation.

[0070] Specifically, the original data table is "semantically more explicit" transformed, for example, the contents related to natural language processing such as the table name, table annotation, column name and column annotation are rewritten; meanwhile, the experience of data governance is used to combine multiple tables into a wide table.

[0071] On the one hand, the organization form of the original data table is more reasonable, and on the other hand, the accuracy of SQL generation can be further improved.

[0072] Preferably, a corresponding example is given below:

[0073] The following table is taken as an example to rewrite the semantics of the English column name. This is an "institution information table", and the original English column name definition is relatively simple, for example, "institution code" is defined as "id", and "institution name" is defined as "name". Obviously, using "id" and "name" as English column names will lose important information and is not conducive to the semantic understanding of the subsequent large model. The application first rewrites the semantics of the original English column name, and the result is as follows:

[0074] Chinese column name English column name Modified as Institution code id institution_id Institution name name institution_name Whether revoked isredo is_revoked Institution level level institution_level Subordinate institution frommec parent_institution ... ... ...

[0075] Obviously, the semantic information of the rewritten English column name is more explicit.

[0076] The following table is taken as an example to rewrite the column content. This is a "student basic information table", and when the user uses it, he / she can ask "how many domestic and foreign students are there" and "age distribution of foreign male students". When the application performs table data governance, it adjusts the column content that is prone to errors, for example, a new column "is Chinese" is added in this scenario, which is a Boolean type with values "yes / no".

[0077] Chinese English column name Modified as Student ID student_id student_id Name name name Education level educational_level_name educational_level Gender gender_name gender Place of birth birthplace_name place_of_birth Native place native_place_name native_place National region name region_name national_region_name ... ... ... Whether Chinese is_chinese is_chinese

[0078] In addition, the application can also associate multiple tables into a wide table. For example, the related tables such as the "student basic information table" and the "student enrollment information table" can be combined into a wide table, thereby reducing redundant fields and simplifying the token length for subsequent assembly of the Prompt for the large model.

[0079] S2, index all values in the database and select TOP-N related values from the database based on the user question, then calculate the matching degree between the user question and the TOP-N related values based on the longest common substring matching algorithm, and select the related value with the highest matching degree;

[0080] The constructed index is a BM25 index, and the BM25 index considers factors such as term frequency, inverse document frequency, and document length.

[0081] Specifically, an index based on a full-text retrieval service is created for the table content. In order to solve the problem of slow query performance, a "from coarse to fine" matching method is used. The essence of this method is to use the index for a fast but coarse-grained initial search, and then use the LCS algorithm for a fine-grained matching process.

[0082] Specifically, first, Lucene is used to build a BM25 index for all values stored in each database. The BM25 algorithm takes into account factors such as term frequency, inverse document frequency, and document length, and can provide more accurate relevance scores. This indexing method not only improves the accuracy of the search, but also significantly improves the retrieval efficiency of large-scale datasets. When a user question is received, the BM25 index first extracts TOP-N potential related values from the entire database based on the user question, where N is a natural number and N can be preferably 50. Then, the longest common substring matching algorithm is used to calculate the matching degree between the user question and the TOP-N related values, and the related value with the highest matching degree is selected.

[0083] The integration of the BM25 index significantly improves the retrieval speed of the extensive database, and the number of calls to the longest common substring matching algorithm is greatly reduced from potentially millions of times to hundreds of times.

[0084] Preferably, a corresponding example is given below:

[0085] For example, the user inputs the query statement "Does the school have a student named Han Meimei", and the tokenized statement is "school | whether | have | call | Han Meimei | student". The full-text retrieval service can first match the TOP-N related column values in milliseconds, such as "Han Meimei", "Han Mei", "Han Xue Mei", and "Korea". After the longest common substring matching algorithm, the column value "Han Meimei" can be matched with 100% similarity. Then the table name and column name to which "Han Meimei" belongs can be obtained, which lays the foundation for subsequent assembly of the Prompt for the SQL large model.

[0086] S3, paraphrasing the user question;

[0087] Paraphrasing the user question with the help of the domain dictionary, the abbreviations, abbreviations and aliases in the domain involved in the user question are rewritten into professional full name;

[0088] When the user question involves time content, the time content is uniformly converted into UNIX timestamp;

[0089] Specifically, the user question rewriting model is constructed. When facing various user questions, the first thing to consider is how to unify various different ways of asking the same question to a more semantic clear expression. This embodiment adopts the mode matching combined with large model.

[0090] With the help of the question and answer system platform, the domain dictionary can be quickly uploaded, and the dictionary format supports.xlsx. With the help of the domain dictionary, the abbreviations, abbreviations and aliases in the domain involved in the user question are rewritten into professional full name;

[0091] Referring to Figure 3 , the following is a corresponding example given:

[0092] For example, the user input query statement is "query the number of students in the telecommunications college", and the actual stored college name in the table is "electronic and information engineering college"; The SQL large model itself does not know that "telecommunications college" is the abbreviation of "electronic and information engineering college", at this time the query statement can be rewritten with the help of pattern matching dictionary.

[0093] In the aspect of large model fine-tuning, the embodiment upgrades the fine-tuning ability for the time involved scene. For example, the user input statement is "how many doctoral students have graduated in the last 3 months to now", the large model cannot understand the time concepts such as "recent 3 months" and "now", and the embodiment uniformly processes the time into UNIX timestamp. In the step of converting to UNIX timestamp, it is divided into two steps: first, use the large model to extract the time slot, that is, "recent 3 months", "now"; Second, use the large model to generate the Java program code for time conversion.

[0094] The model fine-tuning data please refer to Figure 4 .

[0095] The input statement is the extracted time slot, and the output part is part of Java 8 code, which is embedded into other code to form a complete time conversion code.

[0096] S4, word segmentation matching of user question;

[0097] Based on steps S2 and S3 respectively, the user's question is matched with all values ​​stored in the database;

[0098] The user's query is rewritten and segmented for matching. Step S4 is an application of steps S2 and S3. After rewriting the query in step S2, the user's query can be transformed into a more semantically clear expression suitable for large models. For example, a comprehensive question might be entered as "How many times has Han Meimei from the School of Telecommunications used her campus card in the last 3 months?"

[0099] First, by performing a quick match on the table content in step 202, we can obtain the English table name to which "Han Meimei" belongs, which is "student_information", and the English column name to which it belongs is "name".

[0100] Secondly, in step 203, the time slots are extracted and transformed, and the original question is rewritten as "How many times did Han Meimei from the School of Electronic and Information Engineering use her campus card from 1711900800000 to 1719763200000?", which will obviously improve the accuracy of the generated SQL statement.

[0101] S5. Store the accumulated seed question-and-answer statements into a vector database and build a vector matching model. The vector matching model is a Siamese network, which includes two input channels, left and right, and the two input channels share the same BERT class model. Assign weights to different keywords in the user question, and then match the user question with the accumulated seed question-and-answer statements and select the TOP-S SQL question-and-answer statements.

[0102] The accumulated seed question-and-answer statements include manually compiled statements and statements from the actual question-and-answer phase.

[0103] Specifically, by leveraging the existing similar vector retrieval capabilities of the question-and-answer platform, the accumulated seed question-and-answer statements are stored in a vector database. The vector database can return results quickly in milliseconds, laying the foundation for example few-shots of the Prompt statements in subsequent large models.

[0104] The accumulated seed question-and-answer statements come from two sources: manually compiled statements from the initial stage and statements from the actual question-and-answer stage. It employs a finely tuned vector matching model for high-precision SQL data retrieval. This vector matching model architecture is based on a Siamese network; please refer to the diagram for a schematic. Figure 5 .

[0105] The twin network structure has left and right two input channels, and the two channels share the same BERT model. The loss function can be cosine similarity, Euclidean distance L2, etc. The fine-tuned vectorization model can assign weights to keywords in the SQL scenario, increase the weight of keywords, and reduce the weight of auxiliary words, thereby greatly increasing the semantic similarity between similar and non-similar sentences.

[0106] For example, the initial seed question and the actual question and answer statement are both vectorized by the vectorization model, and then matched with the vectorized statements in the vector database, and the top-S SQL question and answer statements are selected.

[0107] In this embodiment, S is preferably 1.

[0108] Specific reference is made to Figure 6 .

[0109] S6, optimize the DDL statement to build a prompt statement, and call a large model to generate an SQL statement.

[0110] The input of the large model is called a prompt statement. In order to make the large model generate more accurate SQL statements, the prompt statement needs to contain the required key information. However, a longer prompt statement is not necessarily better. A prompt statement that is too long can cause the large model to forget information, which in turn affects the quality of the generated SQL statement. Therefore, generating a high-quality prompt statement is an important step.

[0111] Specifically, S6 includes the following steps:

[0112] S61, the rewritten data table in step S1 is displayed using the adapted DDL statement;

[0113] S62, the TOP-M column values after deduplication of each column content in the adapted DDL statement display in S61 are written into the DDL statement as examples; wherein M in TOP-M is a natural number, and in this embodiment, M is preferably 3.

[0114] S63, the table information and column information matched in step S2 are brought into the prompt statement;

[0115] S64, the top-S SQL question and answer statements matched in step S5 are assembled as few-shot into the prompt statement, and the prompt statement is input into the large model to generate an SQL statement.

[0116] In this embodiment, S is preferably 1.

[0117] Specifically, the composition of the Prompt statement is divided into 5 parts, referring to Figure 7 The 5 parts are as follows:

[0118] 1. Role expression: Here, the large model role and task information are written, for example, "You are an SQL expert, please pay full attention to the information below the table, understand the column name and the corresponding explanation and example values, and generate an SQL statement";

[0119] 2. Table information: Here, the changed DDL statement information is filled in;

[0120] 3. Example: Here, as a few-shot example, the N most similar SQL question and answer pairs are selected from the vector library;

[0121] 4. Matched table information: The table column information here helps the large model correctly select the corresponding table and column when generating an SQL statement;

[0122] 5. Requirements: Here, the output requirements of the large model are written, such as "only output the standard SQL statement".

[0123] In summary, inputting the optimized Prompt statement to the large model can greatly improve the quality of the SQL statement generated by the large model.

[0124] The embodiment of the application also discloses a device for converting natural language to an SQL statement based on a large model, which comprises the following modules:

[0125] A table management module is configured to perform data management on an original data table, rewrite the original data table to generate a rewritten data table, and combine multiple tables into a wide table.

[0126] An index construction module is configured to construct indexes for all values in the database.

[0127] A screening module is configured to select relevant values from the database based on a user question, calculate the matching degree between the user question and the top-N relevant values based on a longest common substring matching algorithm, and select the relevant value with the highest matching degree.

[0128] A rewriting module is configured to rewrite a user question by means of a domain dictionary, rewriting abbreviations, abbreviations and alternative names in the domain involved in the user question into professional full names, and uniformly converting time content into UNIX timestamps when the user question involves time content.

[0129] A word segmentation and matching module is configured to perform word segmentation and matching on a user question, and match the user question with all values stored in the database.

[0130] The vector matching module is used for storing the accumulated seed question and answer sentences into a vector database and building a vector matching model.

[0131] The SQL statement generation module is used for optimizing a DDL statement to build a Prompt statement and calling a large model to generate an SQL statement.

[0132] The SQL statement generation module comprises the following units:

[0133] The DDL statement display unit is used for displaying the rewritten data table generated by the data table management module by using the adapted DDL statement.

[0134] The deduplication unit is used for deduplicating the rewritten data table displayed by the DDL statement display unit.

[0135] The writing unit is used for writing the column values of the top-M of the rewritten data table displayed by the DDL statement as examples into the DDL statement.

[0136] The bringing unit is used for bringing the table information and column information of the related value with the highest matching degree matched by the screening module into the Prompt statement.

[0137] The Prompt statement assembly unit is used for assembling the top-S SQL question and answer sentences selected by the vector matching module as few-shot into the Prompt statement, inputting the Prompt statement into the large model, and generating an SQL statement.

[0138] The electronic device disclosed in the embodiment of the present application comprises a processor and a memory.

[0139] The processor can be a CPU (central processing unit), a general-purpose processor, a DSP (data signal processor), an ASIC (application-specific integrated circuit), an FPGA (field programmable gate array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0140] The bus can include a path for communicating information among the above components. The bus can be a PCI (Peripheral Component Interconnect) Express bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0141] The memory can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0142] The memory is used to store application program codes for implementing the scheme of the present application, and is controlled by the processor to perform. The processor is used to execute the application program codes stored in the memory to realize the content shown in the method for converting natural language to SQL sentence based on a large model disclosed in the above embodiments.

[0143] The electronic device includes, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (e.g., a car navigation terminal), etc., and a stationary terminal such as a digital TV, a desktop computer, etc. It can also be a server, etc.

[0144] The computer readable storage medium of the embodiment of the present application has a computer program stored thereon, when the computer program is run on a computer, the computer can execute the corresponding content in the method for converting natural language to SQL sentence based on a large model disclosed in the above embodiments.

[0145] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, therefore: any equivalent changes made on the structure, shape, principle of the present application shall be covered within the protection scope of the present application.

Claims

1. A method for natural language to SQL sentence conversion based on a large model, characterized in that, The method comprises the following steps: S1, data governance is performed on an original data table; rewriting the original data table to generate a rewritten data table, and merging multiple tables into a wide table; S2, indexes are constructed for all values in the database, and TOP-N related values are selected from the database based on a user question, then a matching degree between the user question and the TOP-N related values is calculated based on a longest common substring matching algorithm, and a related value with the highest matching degree is selected; S3, rewriting the user question; rewriting the user question by means of a domain dictionary, rewriting abbreviations, short forms and alternative terms in a domain involved in the user question into professional full names; when time content is involved in the user question, the time content is uniformly converted into a UNIX timestamp; S4, word segmentation matching is performed on the user question; matching the user question with all values stored in the database based on steps S2 and S3 respectively; S5, storing accumulated seed question and answer sentences in a vector database, and building a vector matching model, the vector matching model being a twin network, the twin network comprising left and right two input channels, and the two input channels sharing a same BERT type model, different keywords in the user question being assigned weights, then the user question is matched with the accumulated seed question and answer sentences, and TOP-S SQL question and answer sentences are selected; S6, optimizing a DDL statement to build a Prompt statement, and calling a large model to generate an SQL statement.

2. The method for converting natural language to SQL sentences based on a large model according to claim 1, characterized in that, The building of the Prompt statement in S6 comprises the following steps: S61, displaying the rewritten data table in S1 by using a modified DDL statement; S62, removing duplication of each column content in the modified DDL statement display in S61, and taking TOP-M column values as examples to write into the DDL statement; S63, bringing table information and column information of the related value with the highest matching degree matched in S2 into the Prompt statement; S64, assembling the TOP-S SQL question and answer sentences matched in S5 as few-shot into the Prompt statement, and inputting the Prompt statement into the large model to generate an SQL statement.

3. The method of claim 1, wherein, The original business table in S1 comprises a table name, a table annotation, a column name and a column annotation.

4. The method for converting natural language to SQL sentences based on a large model according to claim 1, characterized in that, The index in S2 is a BM25 index.

5. The method for converting natural language to SQL sentence based on large model according to claim 4, characterized in that, The BM25 index comprises a word frequency, an inverse document frequency and a document length.

6. The method for converting natural language to SQL sentences based on a large model according to claim 1, characterized in that, The accumulated seed question and answer sentences in S5 comprise artificially sorted sentences and actual question and answer stage sentences.

7. An apparatus for natural language to SQL based on large models, characterized in that, The method comprises the following modules: a table governance module, configured to perform data governance on an original data table, rewrite the original data table to generate a rewritten data table, and merge multiple tables into a wide table; an index construction module, configured to construct indexes for all values in the database; a screening module, configured to select TOP-N related values from the database based on a user question, then calculate a matching degree between the user question and the TOP-N related values based on a longest common substring matching algorithm, and select a related value with the highest matching degree; and a rewriting module, configured to rewrite the user question by means of a domain dictionary, and rewrite abbreviations, short forms and alternative terms in a domain involved in the user question into professional full names. The rewriting module is configured to rewrite the user question; the user question is rewritten by means of a domain dictionary, and an abbreviation, a shorthand and an alias in a domain involved in the user question are rewritten into a professional full name; when time content is involved in the user question, the time content is uniformly converted into a UNIX timestamp; The word segmentation matching module is configured to perform word segmentation matching on the user question; the user question is matched with all values stored in a database; The vector matching module is configured to store the accumulated seed question and answer sentences in a vector database, and build a vector matching model; the vector matching model is a twin network, the twin network includes left and right two input channels, and the two input channels share the same BERT model; different keywords in the user question are assigned weights, and then the user question is matched with the accumulated seed question and answer sentences, and TOP-S SQL question and answer sentences are selected; The SQL statement generation module is configured to optimize a DDL statement to build a Prompt statement, and call a large model to generate an SQL statement.

8. An electronic device, comprising: It comprises: one or more processors; a memory; one or more application programs; one or more of the application programs are stored in the memory and configured to be executed by one or more of the processors, and the one or more application programs are configured to execute the method for converting a natural language into an SQL statement based on a large model according to any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method for converting a natural language into an SQL statement based on a large model according to any one of claims 1 to 6.

Images