SQL (Structured Query Language) generation model and multi-round Text-to-SQL method and system based on SQL generation model

Abstract

The invention relates to the technical field of SQL databases, and provides an SQL generation model and a multi-round Text-to-SQL method and system based on the SQL generation model, and the system comprises a semantic enhancement mode extraction unit, a context extraction unit and an output unit; the semantic enhancement mode extraction unit is used for solving the correlation probability of all tables and columns in the original sequence and the current problem based on the original sequence and the annotation sequence, and screening out the tables and columns of which the correlation probability with the current problem is greater than a preset threshold value to construct a mode item sequence; the original sequence is constructed based on a current question corresponding to a current round, previous questions corresponding to a plurality of rounds before the current round, and pattern item information in an SQL database corresponding to the current question and the previous questions, and the annotation sequence is used for annotating the original sequence; the context extraction unit is used for retrieving an SQL statement corresponding to a previous problem most related to the current problem based on a solving result of the semantic enhancement mode extraction unit; the output unit is used for outputting an SQL statement corresponding to the current question based on the pattern item sequence and the retrieval result of the context extraction unit; according to the model, the method and the system, the SQL statement can be accurately generated under the scene of multiple rounds of dialogues.

Description

Technical Field

[0001] The present invention relates to the technical field of SQL databases, and more specifically, to an SQL generation model and a multi-round Text-to-SQL method and system based on the SQL generation model. Background Art

[0002] With the advent of the big data era, data-driven decision-making plays an increasingly crucial role in all walks of life. Enterprises and organizations not only need to quickly and accurately extract valuable information from massive amounts of data, but also need to ensure that this information can be easily accessed and understood by non-technical personnel. However, in reality, most users do not have the ability to write SQL queries, which greatly limits their ability to directly interact with the database and obtain the information they need. Therefore, developing a technology that can convert natural language into structured query language (SQL) is crucial for improving user experience and efficiency.

[0003] Although a large amount of research has been dedicated to the Text-to-SQL task - that is, the system not only has to understand the user's natural language input, but also generate the correct SQL query statement based on these inputs - existing methods still face many challenges when dealing with complex queries. Especially in the multi-round dialogue scenario, the academic community has paid high attention to improving the accuracy and robustness of the Text-to-SQL system. In this case, the user interacts with the system through a series of consecutive questions, requiring the system to correctly map to the corresponding database operations at each step, and the whole process is coherent and error-free. The new research focus is on how to ensure that during the multi-round dialogue process, the system can continuously and accurately understand and respond to the user's intentions.

[0004] In recent years, methods based on large language models have excelled in the Text-to-SQL task due to their powerful semantic understanding and generation capabilities. Such models typically adopt a sequence-to-sequence decoder architecture and can achieve high conversion accuracy in single-round conversations. However, in multi-round conversation scenarios, even the best-performing generative language models will experience performance degradation due to the error accumulation effect. Specifically, when it comes to processing context information across multiple rounds, the challenges faced by the model become more complex, especially in dynamically linking patterns. This is because databases in different domains have unique schema characteristics, and database naming in practical applications often has ambiguities, making it difficult to accurately link the user's questions with the database schema in multi-round conversations. In addition, context tracking is another important challenge. During the multi-round conversation process, there may be dependencies between consecutive questions, and sometimes these dependencies even span multiple rounds. To generate accurate SQL queries, the system must be able to accurately understand and make full use of the conversation context information from previous rounds. There is still much room for improvement in current technologies in this regard.

[0005] In summary, the existing technologies have the problem of being unable to accurately generate SQL statements in multi-round conversation scenarios. Summary of the Invention

[0006] To overcome the problem of the existing technologies being unable to accurately generate SQL statements in multi-round conversation scenarios, the present invention provides an SQL generation model capable of accurately generating SQL statements, a multi-round Text-to-SQL method and system based on the SQL generation model.

[0007] To solve the above technical problems, the technical solution of the present invention is as follows:

[0008] An SQL generation model, comprising a semantic enhancement pattern extraction unit, a context extraction unit and an output unit;

[0009] The semantic enhancement pattern extraction unit is used to solve the relevant probabilities of all tables and columns in the original sequence with respect to the current question based on the original sequence and the annotation sequence, and filter out the tables and columns with relevant probabilities greater than a preset threshold with respect to the current question to construct a pattern item sequence; the original sequence is constructed based on the current question corresponding to the current round, the previous several rounds corresponding to the past questions, and the schema item information in the SQL database corresponding to the current question and the past questions, and the annotation sequence is used to annotate the original sequence;

[0010] The context extraction unit is used to retrieve the SQL statement corresponding to the past question most relevant to the current question based on the solution result of the semantic enhancement pattern extraction unit;

[0011] The output unit is used to output the SQL statement corresponding to the current question based on the pattern item sequence and the retrieval result of the context extraction unit.

[0012] The present invention also provides a multi-round Text-to-SQL method based on an SQL generation model. By applying the above SQL generation model, the method includes the following steps:

[0013] Construct an original sequence based on the current question corresponding to the current round, the past questions corresponding to several previous rounds of the current round, and the pattern item information in the SQL database corresponding to the current question and the past questions, and generate an annotation sequence for annotating the original sequence based on the SQL database;

[0014] Input the original sequence and the annotation sequence into the SQL generation model. The semantic enhancement pattern extraction unit, based on the original sequence and the annotation sequence, solves the relevant probabilities of all tables and columns in the original sequence with respect to the current question, and filters out the tables and columns with relevant probabilities greater than a preset threshold with respect to the current question to construct a pattern item sequence; the context extraction unit, based on the solution result of the semantic enhancement pattern extraction unit, retrieves the SQL statement corresponding to the past question most relevant to the current question; the output unit outputs the SQL statement corresponding to the current question based on the pattern item sequence and the retrieval result.

[0015] The present invention also provides a multi-round Text-to-SQL system based on an SQL generation model for implementing the above multi-round Text-to-SQL method based on an SQL generation model. The system includes:

[0016] A sequence construction module for constructing an original sequence based on the current question corresponding to the current round, the past questions corresponding to several previous rounds of the current round, and the pattern item information in the SQL database corresponding to the current question and the past questions, and generating an annotation sequence for annotating the original sequence based on the SQL database;

[0017] An SQL statement generation module for using the SQL generation model to solve the relevant probabilities of all tables and columns in the original sequence with respect to the current question based on the original sequence and the annotation sequence, and filtering out the tables and columns with relevant probabilities greater than a preset threshold with respect to the current question to construct a pattern item sequence; using the solved relevant probabilities to retrieve the SQL statement corresponding to the past question most relevant to the current question; and outputting the SQL statement corresponding to the current question based on the pattern item sequence and the retrieval result.

[0018] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0019] The SQL generation model includes a semantic enhancement pattern extraction unit, a context extraction unit, and an output unit. The core of the semantic enhancement pattern extraction unit focuses on solving the problem of dynamic pattern linking. With the help of a special semantic processing mechanism, it can more accurately capture the internal relationship between the user's question and the database pattern, thereby effectively improving the accuracy and stability of pattern linking. The context extraction unit is mainly dedicated to solving the problem of context tracking. It can efficiently identify and integrate the context information in multiple rounds of conversations, providing a more comprehensive and reliable basis for generating accurate SQL queries. These two units cooperate with each other, significantly enhancing the ability of the generation model to convert from natural language to SQL, promising to provide strong technical support for the in-depth development of multi-round Text-to-SQL tasks, strongly promoting the implementation of natural language database interface products in related fields, and further promoting the wide application and in-depth expansion of data-driven decision-making in various industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 FIG. 6 is a first flow diagram of the multi-round Text-to-SQL method based on the SQL generation model proposed in Embodiment 1;

[0021] Figure 2 FIG. 7 is a second flow diagram of the multi-round Text-to-SQL method based on the SQL generation model proposed in Embodiment 2;

[0022] Figure 3 FIG. 8 is a third flow diagram of the multi-round Text-to-SQL method based on the SQL generation model proposed in Embodiment 2;

[0023] Figure 4 FIG. 9 is a schematic diagram of a sequence format proposed in Embodiment 2;

[0024] Figure 5 FIG. 10 is a schematic diagram of the experimental results proposed in Embodiment 2;

[0025] Figure 6 FIG. 11 is a schematic diagram of the actual case analysis proposed in Embodiment 2. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The drawings are only for illustrative purposes and should not be construed as a limitation of this embodiment;

[0027] For better illustration of this embodiment, some components in the drawings are omitted, enlarged, or reduced, and do not represent the dimensions of the actual product;

[0028] For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0029] The technical solutions of the present invention will be further described below with reference to the drawings and embodiments.

[0030] Example 1

[0031] This embodiment proposes an SQL generation model, including a semantic enhancement pattern extraction unit, a context extraction unit, and an output unit;

[0032] The semantic enhancement pattern extraction unit is used to solve the relevant probabilities of all tables and columns in the original sequence and the current problem based on the original sequence and the annotation sequence, and filter out the tables and columns with relevant probabilities greater than a preset threshold to construct a pattern item sequence; the original sequence is constructed based on the current problem corresponding to the current round, the past problems corresponding to several previous rounds of the current round, and the pattern item information in the SQL database corresponding to the current problem and the past problems, and the annotation sequence is used to annotate the original sequence;

[0033] The context extraction unit is used to retrieve the SQL statement corresponding to the past problem most relevant to the current problem based on the solution result of the semantic enhancement pattern extraction unit;

[0034] The output unit is used to output the SQL statement corresponding to the current problem based on the pattern item sequence and the retrieval result of the context extraction unit.

[0035] In the specific implementation process, the SQL generation model includes a semantic enhancement pattern extraction unit, a context extraction unit, and an output unit. The core of the semantic enhancement pattern extraction unit focuses on solving the dynamic pattern linking problem. With the help of a special semantic processing mechanism, it can more accurately capture the internal relationship between the user's question and the database pattern, thus effectively improving the accuracy and stability of pattern linking; the context extraction unit is mainly committed to overcoming the context tracking problem. It can efficiently identify and integrate the context information in multiple rounds of conversations, providing a more comprehensive and reliable basis for generating accurate SQL queries; these two units cooperate with each other, significantly strengthening the ability of the generation model to convert from natural language to SQL, promising to provide strong technical support for the in-depth development of multi-round Text-to-SQL tasks, effectively promoting the implementation of natural language database interface products in related fields, and further promoting the wide application and in-depth expansion of data-driven decision-making in various industries.

[0036] In an optional embodiment, the semantic enhancement pattern extraction unit includes a historical extraction item marking module and a semantic enhancement module;

[0037] The historical extraction item marking module is configured with a RoBERTa model and a pooling sub-module. The pooling sub-module consists of two layers of bidirectional long short-term memory networks and one layer of non-linear fully connected layers connected in sequence. The output of the RoBERTa model is the input of the pooling sub-module;

[0038] An attention gating mechanism is set in the semantic enhancement module. The semantic enhancement module takes the output of the pooling sub-module as input, and the output of the semantic enhancement module is the output of the semantic enhancement pattern extraction unit.

[0039] In an alternative embodiment, the historical extraction item marking module is used to obtain the columns screened out by the semantic enhancement pattern extraction unit in previous rounds, mark these columns in the original sequence with the symbol [SN], and input the marked original sequence and the annotation sequence into the RoBERTa model. The output embedding of each token after tokenization of the marked original sequence by the RoBERTa model is input into the pooling sub-module; the pooling sub-module outputs the embedding representations of each table and its annotation in the original sequence, as well as the embedding representations of each column and its annotation, and the output of the pooling sub-module is the output of the semantic enhancement pattern extraction unit.

[0040] In an alternative embodiment, let the embedding representation of the i-th table in the original sequence be denoted as T i , and the embedding representation of the annotation of the i-th table in the original sequence be denoted as The embedding representation of the k-th column of the i-th table in the original sequence is denoted as C i，k , and the embedding representation of the annotation of the k-th column of the i-th table in the original sequence is denoted as Then the expression of the attention gating mechanism includes:

[0041]

[0042] In the formula, represents the gating vector corresponding to T i , and also represents the probability that the table corresponding to T i is relevant to the current question. σ(·) represents the sigmod function, represents the weight matrix corresponding to T i ; and respectively represent the first and second matrix elements of the weight matrix , represents the corresponding bias term, represents the corresponding bias term. The symbol ; represents the vector concatenation operation, represents the matrix multiplication operation of the weight matrix and the vector in the square brackets, represents the corresponding bias term; represents the result after semantic enhancement of T i . Norm(·) represents the L 2Normalization function, where the symbol · represents element-wise multiplication; represents C i，k The corresponding gating vector, also represents C i，k The probability related to the current problem for the corresponding column, represents C i，k The corresponding weight matrix, and respectively represent the weight matrix The first and second matrix elements of, represents The corresponding bias term, represents The corresponding bias term, represents The corresponding bias term, represents the weight matrix Performs a matrix multiplication operation with the vector within the square brackets; represents C i，k The result after semantic enhancement;

[0043] When is greater than the preset threshold, The corresponding table is regarded as the table with the probability related to the current problem greater than the preset threshold and is filtered out; when is greater than the preset threshold, The corresponding column is regarded as the column with the probability related to the current problem greater than the preset threshold and is filtered out;

[0044] The filtered-out tables and columns are sorted from high to low according to and values, obtaining a sorted sequence, and serializing the foreign key relationships between the column names of the sorted columns, adding the serialized foreign key relationship information to the sorted sequence to form the pattern item sequence.

[0045] In an alternative embodiment, the semantic enhancement pattern extraction unit further includes a full-column intent recognition module, which is used to add a special column corresponding to the wildcard "*" in the SQL query language to each table in the original sequence, and is used to input the original sequence with the special column added into the historical extraction item marking module.

[0046] In an alternative embodiment, when the context extraction unit retrieves the SQL statement corresponding to the past problem most relevant to the current problem based on the solution result of the semantic enhancement pattern extraction unit, it obtains the probability of all tables related to the current problem and the probability of all columns related to the current problem for each problem from the semantic enhancement pattern extraction unit, and forms a probability vector with the probabilities obtained from the semantic enhancement pattern extraction unit; where the past problem Q hThe probabilities of all corresponding tables related to the current problem, and the past problem Q h The probability vector composed of the probabilities of all corresponding columns related to the current problem

[0047] Calculate the comprehensive relevance score between the past problem and the current problem based on the probability vector, and regard the past problem with the highest comprehensive relevance score as the past problem most relevant to the current problem;

[0048] The calculation expression of the comprehensive relevance score includes:

[0049]

[0050]

[0051] h ∈ [1,..., m]

[0052] In the formula, R h represents the comprehensive relevance score between the h-th past problem Q h and the current problem Q m ; represents the similarity between Q h and Q m calculated by the SentenceBERT model; represents the JS divergence corresponding to the past problem Q h , and also represents the JS divergence between and ;

[0053] In an optional embodiment, a target function T is configured in the output unit g . Before generating the SQL statement using the SQL generation model, obtain several multi-round questions of known SQL statements, and the tables and columns in the SQL database corresponding to the multi-round questions to construct several original sequences for training, and generate the annotation sequences of the original sequences for training;

[0054] Input several original sequences for training, the annotation sequences of the original sequences for training, and the known SQL statements corresponding to the original sequences for training into the SQL generation model, and use the target function T g to perform iterative training on the SQL generation model until the number of iterations is greater than or equal to the preset number of iterations, or the function value of the target function T g reaches the minimum, then stop the iteration to obtain the trained SQL generation model;

[0055] When inputting the original sequence and the annotation sequence into the SQL generation model, the original sequence and the annotation sequence are input into the trained SQL generation model, and the output unit of the trained SQL generation model outputs the SQL statement corresponding to the current problem;

[0056]

[0057] In the formula, M * represents all the parameters of the SQL generation model, ε(·) represents the preset sequence format, and Q ≤m represents the multi-round problem sequence obtained by connecting the first round problem to the last round problem of the multi-round problem with the separator & one by one. E(S) represents the pattern item sequence, and SQL base represents the SQL statement corresponding to the past problem most relevant to the current problem. When the current problem is the first round problem of a single multi-round problem, SQL base is an empty sequence, and s m represents the SQL statement generated by the SQL generation model. L(·) represents the cross-entropy loss function. The smaller the function value of L(·), the closer s m is to the known SQL statement, that is, the smaller the function value of L(·), the closer s m is to the true result.

[0058] This embodiment also proposes a multi-round Text-to-SQL method based on the SQL generation model. Figure 1 It is the first process schematic diagram of the multi-round Text-to-SQL method based on the SQL generation model of this embodiment;

[0059] A multi-round Text-to-SQL method based on the SQL generation model proposed in this embodiment includes the following steps:

[0060] S1: Construct an original sequence based on the current problem corresponding to the current round, the past problems corresponding to the previous several rounds of the current round, and the pattern item information in the SQL database corresponding to the current problem and the past problems, and generate an annotation sequence for annotating the original sequence based on the SQL database;

[0061] S2: Input the original sequence and the annotation sequence into the SQL generation model. The semantic enhancement pattern extraction unit, based on the original sequence and the annotation sequence, calculates the relevant probabilities of all tables and columns in the original sequence with respect to the current question, and filters out the tables and columns with relevant probabilities greater than a preset threshold with respect to the current question to construct a pattern item sequence. The context extraction unit, based on the calculation result of the semantic enhancement pattern extraction unit, retrieves the SQL statement corresponding to the past question that is most relevant to the current question. The output unit outputs the SQL statement corresponding to the current question based on the pattern item sequence and the retrieval result.

[0062] As an exemplary illustration, the pattern item information includes a single table name, a single column name, and the foreign key relationship between column names.

[0063] In an alternative embodiment, the expression of the original sequence is as follows:

[0064]

[0065] where X represents the original sequence, Q m represents the question in the m-th round, and also represents the current question corresponding to the current round; the symbol & represents the separator between questions; t N represents the N-th table in the SQL database, and N represents the total number of tables in the SQL database; Nn N represents the total number of columns in the N-th table t N ; represents the Nn-th column of table t N ; the symbol | represents the separator between tables; the symbol : indicates that the columns following this symbol all belong to the table preceding this symbol; N

[0066] When generating an annotation sequence for annotating the original sequence based on the SQL database, several values are randomly sampled from each column in each table in the SQL database, and together with the type and name of the column, they are used as the input to the large language model to prompt the large language model to generate a descriptive annotation.

[0067] The expression of the annotation sequence is as follows:

[0068]

[0069] where represents the annotation sequence, represents the annotation of , and when generating the annotation , several values are randomly sampled from the column , and together with the type and name of the column , they are used as the prompt words to input into the large language model; Denote t N comments, and when generating comments belonging to table t N comments for all columns are used as prompt words to input into the large language model.

[0070] Embodiment 2

[0071] Based on the multi-round Text-to-SQL method based on the SQL generation model proposed in Embodiment 1, the following specific implementation examples are proposed. Figure 2 and Figure 3 are respectively the second and third process schematic diagrams of the multi-round Text-to-SQL method based on the SQL generation model proposed in this embodiment. Figure 3 Both Continent and continent in Figure 3 represent column names, and these column names do not need to distinguish between uppercase and lowercase in actual operations. Continent names represent the annotation of the name of the column with the column name Continent, and Continent identifier represents the annotation of the identifier of the column with the column name Continent. Figure 3 In the original schema item of Figure 3 , Continents corresponds to the table in the original sequence, and continent corresponds to the column in the original sequence. Figure 3 Shows the tables (Continents) and columns (continent, contid) filtered out by the semantic enhancement pattern extraction unit when the question Q m-1 is the current question. When the question Q m is the current question, the column continent in the table Continents in the original sequence is marked with the [SN] identifier, and Gold S represents the label of the schema item.

[0072] The specific implementation examples of the present invention include the following steps:

[0073] (1) Use the semantic enhancement pattern extractor to obtain the most relevant schema item E(S) of Q m ;

[0074] In the database multi-round dialogue scenario, redundant database schema item information will cause great interference to SQL generation. Therefore, the present invention designs a dynamic schema item selector (semantic enhancement pattern extraction unit) to filter redundant table column information. This selector consists of three intertwined sub-elements: historical extraction item marking, semantic enhancement module, and full-column intention recognition, which respectively implement multi-round dynamic schema link encoding, semantic alignment between question entities and schema items, and full-column intention encoding.

[0075] First, define an original sequence X of multi-round question connection pattern item names, where "&" is used to connect multi-round questions, and "|" is used to separate different pattern items:

[0076]

[0077] To enhance the semantic expressiveness of pattern items, this module introduces open-domain semantic knowledge, enhances the semantic information of column names by combining database content and large language models, and further enriches the semantic information of table names using the enhanced column names. Specifically, for each column in each table, some values are randomly sampled, together with the column type and name, as input to prompt the large language model to generate descriptive annotations; then, based on the information of all columns, an annotation for the table is generated. In this way, a pattern annotation sequence is obtained:

[0078]

[0079] (a) Historical extraction item marking

[0080] In a multi-round interaction environment, the columns extracted in the previous round are retrieved from the historical pattern memory, and these columns are marked in the current round's input X using the symbol [SN]. Subsequently, X and are sequentially input into the RoBERTa model. To perform information fusion and classification on pattern items and their corresponding annotations as a complete unit, this module performs a pooling operation on the output embeddings of each token after RoBERTa tokenization of the pattern items. For this purpose, a pooling module consisting of two layers of BiLSTM and a non-linear fully connected layer is used. After pooling, the embedding representation of each table and its annotation is T i , The embedding of each column and its annotation can be represented as represents the size of the hidden layer.

[0081] (b) Semantic enhancement module

[0082] The naming of database pattern items sometimes has ambiguities, which may lead to semantic differences between the user's query intent and the actual data structure. Through parsing by the large language model, it is found that the "continent" column in table Continents and table Countries represents "continent name" and "continent id" respectively. This abbreviated form of column names may cause deviations in understanding, thereby affecting the performance of the pattern item selector. To address this semantic inconsistency problem, this module introduces an attention gating mechanism on the basis of the column enhancement module to aggregate the representations of pattern item names and their related annotations.

[0083]

[0084] Among them, · represents element-wise multiplication. represents the sigmod function, and Norm(·) is the L 2 normalization function by row. and respectively represent the gating vectors of the table and columns. The embeddings of the pattern item names and annotations are weighted averaged and normalized by the probability values of the gating vectors. This module can obtain the enhanced table embedding and column embedding

[0085] After being processed by the semantic enhancement layer, the classification probabilities of all pattern items in the current round can be obtained. Next, according to the predefined threshold s, the pattern items closely related to the question are screened out. This threshold needs to be reasonably set to prevent the loss of important pattern items due to being too high. The screened pattern items are sorted in descending order of their probability values, and the foreign key information is serialized and combined with these pattern items to finally generate the serialized database pattern item representation required by the SQL generation model.

[0086] (c) Full column intention recognition

[0087] In addition, in query interactions, the user's intention often is not always directly expressed as obtaining information about specific columns, but in some cases implies obtaining information about all columns in a specified table. To effectively capture the "full column intention" implicit in the user's query, when designing the pattern item selector, the wildcard "*" in the SQL query language is deliberately regarded as a special column identifier. For example, in the query "Show ids and names of all continents!", the user clearly expresses the hope to obtain information about specific columns (such as "ids" and "names") in the "continents" table. However, when dealing with a more general query like "Which countries do they each have", the user's intention implies the need to query all relevant data. In this case, the pattern item extractor generates the classification probability about "*" for all tables and decides whether and how to insert "*" in the input sequence based on these probability evaluations, so as to guide the SQL generator to more accurately understand and respond to the user's query intention.

[0088] (2) Use the pattern-aware context extractor to retrieve the most relevant past SQL queries SQL for the current request base ;

[0089] As the number of dialogue turns increases, the relationships between turns become more intricate, and there are often cross-turn connections between the current question and historical questions. To this end, a context information extraction module is designed. This module can identify the past SQL queries most relevant to the current request and use this query as a reference for generating the current SQL statement. Since the semantic association modeling between the question and the database schema has been completed in the pattern item extraction stage, during the subsequent information filtering process, the pre-stored pattern item encoding information in the historical pattern memory can be directly utilized, thus avoiding repeated model training.

[0090] During the screening process of context information, this module mainly considers the following two factors: First, it is the semantic relevance between the current question Q m and the historical question Q h (h ∈ 1,..., m - 1). A higher semantic similarity usually means similarity in query intent. For example, if Q m is "How many dog pets are raised by female students?", and the historical question Q h is "How many of those have dogs?". Then there is an obvious semantic association between these two questions. This module uses the SentenceBERT model to quantify this similarity, defined as:

[0091]

[0092] where h ∈ [1,..., m]. When Q h and Q m are semantically similar, their corresponding SQL queries are likely to have a similar structure.

[0093] Secondly, considering that relying solely on the semantic similarity between questions may lead to misjudgments. For example, the questions "Who are the female students?" and "Of those, who has a pet?" are actually consecutive inquiries, but semantically they seem quite different. Therefore, the pattern item probability values provided by the pattern item selector are also introduced to measure the overlap degree of the entities involved in the two questions. Let the normalized pattern item extraction probability vectors obtained from the m-th round and the h-th round of reasoning be and respectively. Then the normalized JS divergence can be used to evaluate the distribution difference measure between these entities, so as to obtain and are two key scoring indicators for retrieving and filtering historical information, and their values are both limited to the interval [0, 1]. Among them represents the maximized metric score, while represents the minimized metric score. To unify the direction of the metrics, is adjusted to the form of so that both metrics can be optimized in the maximized direction. Based on the above definitions, the comprehensive correlation score R h between Q m and Q h can be calculated:

[0094]

[0095] By selecting the historical SQL queries with the highest R h value, this module can use appropriate historical SQL queries as reference inputs during the supervised fine-tuning of the large language model to facilitate the generation of accurate SQL outputs for question Q m , thereby effectively reducing the difference between the input and the output.

[0096] (3) Supervised fine-tuning of the SQL generator;

[0097] By screening the pattern items in the multi-round dataset and filtering out irrelevant historical information, a concise input sequence highly relevant to the target SQL query can be obtained. Based on this, the multi-round dataset is converted into a single-round Text-to-SQL corpus. The input sequence of each entry consists of question Q <m , the extracted pattern item sequence E(S), and SQL base , and the actual SQL query serves as the expected output sequence s m . During the practice of converting multi-round questions into single-round questions, it is found that directly rewriting the questions or other forms of filtering will lead to serious error propagation. Therefore, define question Q ≤m = Q 1 &... & Q m to ensure the semantic integrity of the multi-round question sequence through symbol merging. When processing the first question Q 1 , since there is no previous SQL query as a reference, let SQL base be an empty sequence. Thus, the minimized loss function of a set of interactive samples can be expressed as:

[0098]

[0099] Here, ε(·) defines a sequence format, Figure 4 is a schematic diagram of a sequence format proposed in this embodiment, Figure 4 showing a specific example of the preset sequence format ε(·).

[0100] Specifically, the steps of this application are as follows:

[0101] S1: Input the data of the original pattern items corresponding to the questions in the previous m - 1 rounds and the question in the m-th round into the semantically enhanced pattern extractor to obtain the most relevant pattern item E(S). This pattern item is not only used for model training but also stored in the historical pattern memory. m Specifically, the present invention designs a dynamic pattern item selector. This selector constructs an original sequence X of multi-round question-connected pattern item names, combines the database content and the large language model to enhance the semantic information of column names and table names, and forms a pattern annotation sequence.

[0102] In multi-round interactions, the columns extracted in the previous round are obtained from the historical pattern memory, and these columns are marked with the symbol [SN]. Subsequently, the marked input X and the pattern annotation sequence are input into the RoBERTa model for pooling operations to fuse the information of pattern items and their annotations. To address the naming ambiguity problem, an attention gating mechanism is introduced to aggregate the representations of pattern item names and their related annotations and reduce the impact of semantic inconsistencies. Then, according to a predefined threshold, the pattern items closely related to the question are screened out, sorted by probability, and the serialized database pattern item representation required for the final SQL generation model is generated. In addition, by capturing the user's implicit "full column intention", the wildcard "*" is regarded as a special column identifier to guide the SQL generator to more accurately understand and respond to the user's query intention. This series of steps not only filters redundant information but also improves the accuracy and robustness of SQL generation in a multi-round dialogue environment. In multi-round interactions, the columns extracted in the previous round are obtained from the historical pattern memory, and these columns are marked with the symbol [SN]. Subsequently, the marked input X and the pattern annotation sequence are input into the RoBERTa model for pooling operations to fuse the information of pattern items and their annotations. To address the naming ambiguity problem, an attention gating mechanism is introduced to aggregate the representations of pattern item names and their related annotations and reduce the impact of semantic inconsistencies. Then, according to a predefined threshold, the pattern items closely related to the question are screened out, sorted by probability, and the serialized database pattern item representation required for the final SQL generation model is generated. In addition, by capturing the user's implicit "full column intention", the wildcard "*" is regarded as a special column identifier to guide the SQL generator to more accurately understand and respond to the user's query intention. This series of steps not only filters redundant information but also improves the accuracy and robustness of SQL generation in a multi-round dialogue environment. In multi-round interactions, the columns extracted in the previous round are obtained from the historical pattern memory, and these columns are marked with the symbol [SN]. Subsequently, the marked input X and the pattern annotation sequence are input into the RoBERTa model for pooling operations to fuse the information of pattern items and their annotations. To address the naming ambiguity problem, an attention gating mechanism is introduced to aggregate the representations of pattern item names and their related annotations and reduce the impact of semantic inconsistencies. Then, according to a predefined threshold, the pattern items closely related to the question are screened out, sorted by probability, and the serialized database pattern item representation required for the final SQL generation model is generated. In addition, by capturing the user's implicit "full column intention", the wildcard "*" is regarded as a special column identifier to guide the SQL generator to more accurately understand and respond to the user's query intention. This series of steps not only filters redundant information but also improves the accuracy and robustness of SQL generation in a multi-round dialogue environment.

[0103] S2: Input the question in the m-th round into the pattern-enhanced context extractor, which retrieves the most relevant past SQL query SQL based on the records in the historical pattern memory, the historical questions, and the SQL memory, and uses this query as a reference for generating the current SQL statement. base and uses this query as a reference for generating the current SQL statement.

[0104] In specific operations, first, all historical questions Q h (h ∈ 1,..., m - 1) and the current question Q m are constructed into a question sequence, and then the SentenceBERT model is used to quantify the semantic similarity m between Q h and each Q to evaluate the similarity of query intentions. At the same time, the probability values of the pattern items provided by the pattern item selector are evaluated through the normalized JS divergence to measure the overlap degree of the entities involved in the two questions To unify the index direction, is adjusted to Optimize both indicators in the direction of maximization and calculate the comprehensive relevance score Select the historical SQL query with the highest R h value as the reference input, combine it with the pattern item encoding information pre-stored in the historical mode memory, avoid repeated model training, and finally generate the accurate SQL output for Q m , effectively reducing the difference between the input and the output, improving the accuracy and robustness of SQL generation, and providing a more intelligent and efficient interaction platform for users.

[0105] S3: Utilize the most relevant pattern items E(S), SQL ≤m , Q m for the first m rounds of questions Q base and these three types of information to finally obtain the SQL statement for the m-th round, verify the generated SQL, and store it in the historical question and SQL memory.

[0106] By screening and filtering the pattern items in the multi-round dataset, a concise input sequence that is highly relevant to the target SQL query is obtained. Based on this, the multi-round dataset is converted into a single-round form of the Text-to-SQL corpus. The input sequence of each entry consists of the current question Q m , the extracted pattern item sequence E(S), and the most relevant previous SQL query SQL base , and the actual SQL query serves as the expected output sequence s m .

[0107] In the practical process, directly rewriting the question or other forms of filtering will lead to serious error propagation. Therefore, we define the question Q ≤m = Q 1 &...&Q m , merge all previous questions through the symbol "&" to ensure the semantic integrity of the multi-round question sequence. When processing the first question Q 1 , since there is no previous SQL query as a reference, set SQL base to an empty sequence.

[0108] S4: Construct a loss function based on the labeled SQL and the model output SQL, and update the model parameters to minimize the loss function, where ε(·) is in sequence format.

[0109] To verify the effectiveness of the present invention, EM, EX, and TS score evaluations for single-round and multi-round interactions were conducted on the common datasets SparC and CoSQL. Figure 5 This is the schematic diagram of the experimental results proposed in this embodiment. Figure 5 The SFT in Figure 5The experimental results of this embodiment are shown. Compared with other methods, the present invention has a significant improvement and achieves the best performance on multiple data sets. It also has obvious effect improvements on different large language models (Codellama 7B, SFT Deepseek 7B and SFTMistral 7B). Figure 5 SFT Codellama 7B+Track-SQL represents the addition of the SQL generation model proposed in this application to the original supervised fine-tuning large language model Codellama 7B.

[0110] Figure 5 QM stands for question matching index, that is, the accuracy is calculated based on a single question; IM stands for interaction matching index, that is, the accuracy is calculated based on a single interaction, and multiple questions constitute an interaction; EM Exact Match Accuracy (EM) measures the structural accuracy of the predicted SQL query, that is, whether each of its components is completely consistent with the standard answer (golden SQL) (but not including the actual value part in the query); EX Execution Accuracy (EX) focuses on whether the result after the predicted SQL query is executed is correct, and this indicator is generally considered to be more stringent than EM; TS Test Suite Accuracy (TS) not only examines the accuracy of query execution, but also requires that the query results for each database mode on multiple database instances must be correct. This multi-level evaluation system ensures a comprehensive understanding of model performance.

[0111] Figure 5 In-Context Learning Approach: A Text-to-SQL method based on context learning, which does not involve model training and relies on the generation capabilities of large language models (such as ); Fine-tuned Model represents a Text-to-SQL method based on model supervision fine-tuning, which involves model training.

[0112] Figure 6 This is a schematic diagram of an actual case analysis proposed in this embodiment. Figure 6 The case analysis strongly proves the effectiveness of each module of the present invention.

[0113] Example 3

[0114] This embodiment proposes a multi-round Text-to-SQL system based on an SQL generation model, which is used to implement the multi-round Text-to-SQL method based on an SQL generation model proposed in Embodiment 1.

[0115] A multi-round Text-to-SQL system based on an SQL generation model proposed in this embodiment includes:

[0116] A sequence construction module, configured to construct an original sequence based on the current question corresponding to the current round, the previous several rounds of corresponding past questions, and the schema item information in the SQL database corresponding to the current question and the past questions, and generate an annotation sequence for annotating the original sequence based on the SQL database;

[0117] An SQL statement generation module, configured to use an SQL generation model to solve the relevant probabilities of all tables and columns in the original sequence and the current question based on the original sequence and the annotation sequence, and filter out the tables and columns with relevant probabilities greater than a preset threshold for the current question to construct a schema item sequence; use the solved relevant probabilities to retrieve the SQL statement corresponding to the past question most relevant to the current question; and output the SQL statement corresponding to the current question based on the schema item sequence and the retrieval result.

[0118] This embodiment also proposes a computer device, including a memory and a processor. When the computer-readable instructions stored in the memory are executed by the processor, the processor is caused to execute the steps of the multi-round Text-to-SQL method based on the SQL generation model as described in Embodiment 1.

[0119] It can be understood that the multi-round Text-to-SQL system and the computer device based on the SQL generation model in this embodiment are used to implement the method of Embodiment 1. The optional items in the above Embodiment 1 also apply to this embodiment, so they will not be repeated here.

[0120] Identical or similar reference numerals correspond to identical or similar components;

[0121] The terms used to describe the positional relationship in the drawings are only for illustrative purposes and cannot be construed as a limitation of this embodiment;

[0122] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, rather than a limitation of the embodiments of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all the embodiments here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A SQL generation model, characterized in that: It includes a semantic enhancement pattern extraction unit, a context extraction unit and an output unit; The semantic enhancement pattern extraction unit is used to solve the relevance probability of all tables and columns in the original sequence with the current question based on the original sequence and the annotation sequence, and screen out the tables and columns whose relevance probability with the current question is greater than a preset threshold to construct a pattern item sequence; The original sequence is constructed based on the current question corresponding to the current round, the past questions corresponding to the previous rounds of the current round, and the pattern item information in the SQL database corresponding to the current question and the past questions, and the annotation sequence is used to annotate the original sequence; The context extraction unit is used to retrieve SQL statements corresponding to past questions that are most relevant to the current question based on the solution results of the semantic enhancement pattern extraction unit; The output unit is used to output the SQL statement corresponding to the current question based on the pattern item sequence and the retrieval result of the context extraction unit.

2. The SQL generation model according to claim 1, characterized in that: The semantic enhancement pattern extraction unit includes a history extraction item marking module and a semantic enhancement module; The historical extraction item labeling module is configured with a RoBERTa model and a pooling submodule, wherein the pooling submodule is composed of two layers of bidirectional long short-term memory networks connected in sequence and a layer of nonlinear fully connected layer, and the output of the RoBERTa model is the input of the pooling submodule; An attention gating mechanism is provided in the semantic enhancement module, the semantic enhancement module takes the output of the pooling submodule as input, and the output of the semantic enhancement module is the output of the semantic enhancement pattern extraction unit.

3. The SQL generation model according to claim 2, characterized in that: The historical extraction item labeling module is used to obtain the columns screened out by the semantic enhancement pattern extraction unit in previous rounds, label these columns in the original sequence using the symbol [SN], and input the labeled original sequence and the annotation sequence into the RoBERTa model. The output embedding of each label of the labeled original sequence after word segmentation by the RoBERTa model is input into the pooling submodule; the pooling submodule outputs the embedded representation of each table and its annotations in the original sequence, as well as the embedded representation of each column and its annotations. The output of the pooling submodule is the output of the semantic enhancement pattern extraction unit.

4. The SQL generation model according to claim 2, characterized in that: Let the embedding representation of the i-th table of the original sequence be denoted by T i , the embedded representation of the annotation of the i-th table of the original sequence is recorded as The embedding representation of the kth column of the ith table of the original sequence is denoted by C i,k , the embedding representation of the annotation of the kth column of the ith table of the original sequence is recorded as Then the expression of the attention gating mechanism includes: In the formula, Indicates T i The corresponding gating vector, also represented by T i The corresponding table is the probability related to the current problem, σ(·) represents the sigmoid function, Indicates T i The corresponding weight matrix; and Represent the weight matrix The first and second matrix elements of express The corresponding bias term is, express The corresponding bias term, symbol; represents the vector concatenation operation, Represents the weight matrix Performs matrix multiplication with the vector in square brackets. express The corresponding bias term; Indicates T i The result after semantic enhancement, Norm(·) represents the row-wise L2 normalization function, and the symbol · represents element-wise multiplication; Represents C i,k The corresponding gate vector, also represented by C i,k The probability of the corresponding column being relevant to the current question, Represents C i,k The corresponding weight matrix is, and Represent the weight matrix The first and second matrix elements of express The corresponding bias term, express The corresponding bias term is, express The corresponding bias term is, Represents the weight matrix Perform matrix multiplication with the vector in square brackets; Represents C i,k The result after semantic enhancement; when When it is greater than the preset threshold, The corresponding tables are considered to be those with a probability greater than a preset threshold related to the current problem and are screened out; When it is greater than the preset threshold, The corresponding columns are considered to be those with a probability greater than a preset threshold related to the current question and are filtered out; The tables and columns to be filtered out are based on and The values ​​of are sorted from high to low to obtain a sorted sequence, and the column names of the sorted columns and the foreign key relationships between the column names are serialized, and the serialized foreign key relationship information is added to the sorted sequence to form the pattern item sequence.

5. The SQL generation model according to claim 2, characterized in that: The semantic enhancement pattern extraction unit also includes a full-column intent recognition module, which is used to add a special column to each table of the original sequence, and the special column corresponds to the wildcard "*" in the SQL query language, and is used to input the original sequence with the special column added into the historical extraction item marking module.

6. The SQL generation model according to claim 1, characterized in that: When the context extraction unit retrieves the SQL statements corresponding to the past questions most relevant to the current question based on the solution results of the semantic enhancement pattern extraction unit, it obtains the probabilities of all tables corresponding to each question being relevant to the current question and the probabilities of all columns being relevant to the current question from the semantic enhancement pattern extraction unit, and composes the probabilities obtained from the semantic enhancement pattern extraction unit into a probability vector; wherein, the past question Q h The corresponding probabilities of all tables related to the current question and the past questions Q h The corresponding probabilities of all columns related to the current problem form a probability vector Calculating a comprehensive relevance score between the past question and the current question based on the probability vector, and considering the past question with the highest comprehensive relevance score as the past question most relevant to the current question; The calculation expression of the comprehensive relevance score includes: In the formula, R h represents the hth past question Q h With the current questionQ m The comprehensive correlation score between represents the Q calculated by the SentenceBERT model h With Q m The similarity between Indicates past questions Q h The corresponding JS divergence also means and JS divergence between .

7. The SQL generation model according to any one of claims 1 to 6, characterized in that: The output unit is configured with an objective function T g , before using the SQL generation model to generate SQL statements, obtain a number of multi-round questions of known SQL statements, and construct a number of original sequences for training using tables and columns in an SQL database corresponding to the multi-round questions, and generate an annotation sequence of the original sequence for training; Several original sequences for training, annotation sequences of the original sequences for training, and known SQL statements corresponding to the original sequences for training are input into the SQL generation model, and the objective function T is used to generate the SQL statement. g The SQL generation model is iteratively trained until the number of iterations is greater than or equal to the preset number, or the objective function T g When the function value reaches the minimum, stop the iteration and get the trained SQL generation model; When the original sequence and the annotation sequence are input into the SQL generation model, the original sequence and the annotation sequence are input into the trained SQL generation model, and the output unit of the trained SQL generation model outputs the SQL statement corresponding to the current question; Where M * represents all parameters of the SQL generation model, ε(·) represents the preset sequence format, Q ≤m represents a sequence of multiple rounds of questions obtained by connecting the first round of questions to the last round of questions one by one with the separator &, E(S) represents the pattern item sequence, SQL base Indicates the SQL statement corresponding to the past question that is most relevant to the current question. When the current question is the first round of a single multi-round question, SQL base is an empty sequence, s m represents the SQL statement generated by the SQL generation model, L(·) represents the cross entropy loss function, and the smaller the function value of L(·), the higher the value of s m The closer it is to the known SQL statement, the smaller the function value of L(·) is. m The closer to the real result.

8. A multi-round Text-to-SQL method based on an SQL generation model, applying the SQL generation model according to any one of claims 1 to 7, characterized in that: The following steps are involved: Constructing an original sequence based on a current question corresponding to a current round, past questions corresponding to several rounds before the current round, and pattern item information in an SQL database corresponding to the current question and the past questions, and generating an annotation sequence for annotating the original sequence based on the SQL database; The original sequence and the annotation sequence are input into the SQL generation model, and the semantic enhanced pattern extraction unit solves the relevance probability of all tables and columns in the original sequence with the current question based on the original sequence and the annotation sequence, and selects the tables and columns whose relevance probability with the current question is greater than a preset threshold to construct a pattern item sequence; The context extraction unit retrieves the SQL statement corresponding to the past question most relevant to the current question based on the solution result of the semantic enhancement pattern extraction unit; the output unit outputs the SQL statement corresponding to the current question based on the pattern item sequence and the retrieval result.

9. The multi-round Text-to-SQL method based on SQL generation model according to claim 8, characterized in that: The expression of the original sequence includes: In the formula, X represents the original sequence, Q m Indicates the question of the mth round, and also indicates the current question corresponding to the current round; the symbol & indicates the separator between questions; t N Nn represents the Nth table of the SQL database, and N represents the total number of tables in the SQL database; N Indicates the Nth table t N The total number of columns; Representation table t N Nnth N Column, the symbol | indicates the separator between tables; the symbol : indicates that the columns after this symbol belong to the table before this symbol; When generating an annotation sequence for annotating the original sequence based on the SQL database, a number of values ​​are randomly sampled from each column in each table in the SQL database, together with the type and name of the column, as inputs of the large language model, prompting the large language model to generate descriptive annotations; The expression of the annotation sequence includes: In the formula, represents a sequence of annotations, express , and generate annotations When, from the column Randomly sample several values ​​from The type and name are input into the large language model as prompt words; Indicates t N , and generate annotations When N The annotations of all columns are used as prompt words to input into the large language model.

10. A multi-round Text-to-SQL system based on a SQL generation model, used to implement the multi-round Text-to-SQL method based on a SQL generation model according to claim 8 or 9, characterized in that: include: A sequence construction module, used to construct an original sequence based on the current question corresponding to the current round, the past questions corresponding to the previous rounds of the current round, and the pattern item information in the SQL database corresponding to the current question and the past questions, and generate an annotation sequence for annotating the original sequence based on the SQL database; An SQL statement generation module is used to use an SQL generation model to solve the relevance probabilities of all tables and columns in the original sequence with the current question based on the original sequence and the annotation sequence, and screen out tables and columns whose relevance probabilities with the current question are greater than a preset threshold to construct a pattern item sequence; Using the solved correlation probability, the SQL statement corresponding to the past question that is most relevant to the current question is retrieved; based on the pattern item sequence and the retrieval result, the SQL statement corresponding to the current question is output.

Images