Text-to-SQL (Structured Query Language) statement analysis and optimization method based on large language model

Abstract

The invention discloses a text-to-SQL (Structured Query Language) statement analysis and optimization method based on a large language model. The method comprises the following steps: firstly, analyzing a natural language query or command in a fine-grained manner by using text processing and grammar analysis technologies, and converting the natural language query or command into grammar and semantic units; the process comprises the step of extracting elements such as entities, relationships, conditions, qualifiers and operators from input questions. Then, a large language model technology is used for training a model capable of being used for text-to-SQL statement analysis; and then, a feedback optimization mechanism is adopted to improve the accuracy of SQL statement analysis. And finally, carrying out effect verification on the obtained model on a public data set. Compared with a parser which only pays attention to text-to-SQL statement generation, more attention is paid to comprehensiveness and quality optimization in the text-to-SQL semantic parsing process, and the application performance of natural language processing in the field of databases and the capacity of processing complex grammar structures in the databases are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the cross-technical fields of natural language processing, large language models, database information retrieval, etc., and specifically relates to a text-to-SQL statement parsing and optimization method based on a large language model. Background Art

[0002] Text-to-SQL is an important task in the field of natural language processing (NLP), which aims to convert natural language questions into SQL query statements to query information in the database. This task can be regarded as a translation task between natural language and structured queries, and has important application value, especially in data analysis, business intelligence, and intelligent customer service. Through this technology, non-technical personnel can interact with the database through intuitive natural language, thus saving time to learn and master complex SQL syntax. This has made the Text-to-SQL task widely concerned and applied in enterprises and research fields. The challenge of the Text-to-SQL task is to map natural language questions to SQL queries. This mapping is usually not a simple one-to-one mapping, but complex and diverse. Natural language has rich and diverse expressions, and the same query intent can be proposed in different ways. Therefore, the model needs to have strong semantic understanding and logical reasoning capabilities. In addition, the SQL query itself also has a complex structure, including multiple clauses (such as SELECT, FROM, WHERE, JOIN, etc.) and nested and connected operations. The model needs to be able to generate the correct SQL statement structure. In recent years, with the development of deep learning and natural language processing technology, there are many solutions to the Text-to-SQL task, including rule-based methods, transformation-based methods, machine learning-based methods, and the application of pre-trained models. Among them, the emergence of pre-trained models such as BERT and GPT provides new ideas and performance improvements for solving Text-to-SQL tasks. These models can better understand the complex mapping relationship between natural language and SQL through large-scale pre-training learning, thereby improving the accuracy and efficiency of the task. In the Text-to-SQL semantic parsing task, when generating SQL statements, there are often problems such as grammatical errors, semantic inconsistency, or incomplete logic. These problems may arise from the ambiguity of the input text, insufficient model understanding, or the complexity of a specific field. The training of the error correction model aims to identify and correct these errors and improve the accuracy and usability of the generated SQL statements. By designing corresponding training data and loss functions for error types, the model can learn how to identify parts that do not conform to SQL syntax or the intent of the problem, and provide correction suggestions. The design and training of the error correction model also need to consider the characteristics of the actual application scenario. Different fields or tasks may have different common error patterns, so it is necessary to reflect these specific errors in the training data and adjust the structure and parameters of the model to adapt to different scenarios. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides a text-to-SQL statement parsing and optimization method based on a large language model. The method first captures semantic information by modifying the encoder part in the large language model T5, and obtains the structural information of the database through the graph neural network GNN, and then fuses the information of the two through the gating mechanism. Finally, a model capable of error correction is trained to improve the accuracy and robustness of the results to meet the challenges brought by practical applications.

[0004] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0005] The present invention is a gait recognition method based on adaptive fusion to generate multi-scale time features, comprising the following steps:

[0006] Step 1: Input a natural language text;

[0007] Step 2: remove special characters and redundant spaces from the text obtained in step 1, and standardize English to lowercase letters and Chinese to simplified Chinese to obtain a preprocessed text sequence;

[0008] Step 3: Encode the preprocessed text sequence obtained in step 2 through the large language model to extract the semantic information of the natural language query, including key intent, context relevance, and mapping relationship with the database schema;

[0009] Step 4: Use the graph neural network GNN to model the structural information of the database using the preprocessed text sequence obtained in step 2, and combine the preprocessed text sequence with the metadata of the database, such as table name, column name, data type, primary key and foreign key constraints, to build a heterogeneous graph; the nodes in the graph represent entities such as tables, columns, foreign keys, and edges represent the relationships between them; the graph neural network GNN encodes the database structure through message passing and aggregation operations, captures the complex structural information of tables, columns and their relationships, and fuses this structural information with the semantic information of the query to provide accurate database support for subsequent SQL generation;

[0010] Step 5: Use a fusion strategy based on a gating mechanism for steps 3 and 4 to fuse the information in the graph into the encoder output vector; the graph information is the comprehensive information about the database structure and query semantics obtained after processing by the graph neural network GNN;

[0011] Step 6: The output vector obtained in step 5 is sent to an autoregressive decoder, which is composed of several T5 Transformer layers. The decoder gradually generates various components of the SQL statement according to the vectors output by the encoder and the graph neural network GNN module, including the selected fields, table names, query conditions, etc., until the SQL query statement is completely generated.

[0012] Step 7: Use the SQL statement generated in step 6 as a query and execute it on the given database;

[0013] Step 8: Record the wrong results in step 7 and optimize the results;

[0014] Step 9: Output the results.

[0015] Furthermore, step 3 of encoding the preprocessed text sequence using a large language model includes the following steps:

[0016] Step 31: Combine the natural language question G and a series of words or tokens q1, ..., q |G| With database schema spliced ​​into a sequence; Represented as a collection of columns, Represented as a collection of tables;

[0017] Step 32, the spliced ​​sequence is:

[0018]

[0019] This includes the tags q1, ..., q in the question |G| , database name And the columns of each table *Special column markers in the database;

[0020] Step 33: Use the bidirectional encoder to learn the hidden state h of the input x. The specific formula is as follows:

[0021] h=Encoder(x)

[0022] Step 34: The semantic representation of the hidden state is encoded by the Transformer block, which contains two important parts, the multi-head self-attention network MHA and the fully connected feed-forward network FFN; the hidden state at layer l is represented as Where N is the maximum length of the input;

[0023] Step 35: The MHA part first queries the matrix Key Matrix and Value Matrix Mapped into an attention vector through the self-attention mechanism; the formula is as follows:

[0024]

[0025] Where m is the number of query vectors and n is the number of key or value vectors; MHA performs self-attention on each head. Represents the linear transformation matrix for each attention head; T represents transpose; QK T represents the multiplication of the query matrix Q and the transposed matrix of the key matrix K; d m Indicates the dimension of T5, d k Indicates d m The target dimension mapped to the query vector, d v represents the dimension of the median vector of each attention head, and h represents the number of heads in the multi-head attention mechanism;

[0026] Step 36: In the encoder module of T5, d k =d v =d m / h, MHA calculates the attention output of each head and connects them together to get the result of the multi-head self-attention network;

[0027] Step 37: The semantic hidden state needs to be obtained through another component FFN; the formula is as follows:

[0028]

[0029] Among them, the linear weight matrix is ​​shown as d ff Represents the hidden layer dimension of the feedforward neural network, which is used to expand the feature space; Indicates The input of the layer, b1, b2 represents the bias vector, which is used to provide an offset for the calculation of each layer;

[0030] Step 38: After step 37, perform layer normalization and residual connection to obtain the semantic hidden state; the formula is as follows:

[0031]

[0032] Represents the normalized result.

[0033] Furthermore, the step 4 of modeling the structural information therein by using the graph neural network GNN includes the following steps:

[0034] Step 41: construct a heterogeneous graph through step 31, including three types of nodes and multiple types of edges; the three types of nodes are problem nodes, column nodes and table nodes; the multiple types of edges are pattern relations, pattern link relations and problem relations;

[0035] Step 42: assign an initial embedding vector to each node to represent their semantic information;

[0036] Step 43: Structural encoding of the heterogeneous graph is performed, and the relationship graph attention network RGAT is used to calculate the attention weights according to the different relationships between the nodes, and the embedding representation of each node is updated through message passing;

[0037] Step 44: concatenate the representation vectors of all nodes and input them into the T5 decoder to generate the corresponding SQL query.

[0038] Furthermore, the fusion strategy based on the gating mechanism in step 5 fuses the information in the graph into the encoder output vector and includes the following steps:

[0039] Step 51, fusing the semantic information obtained in step 38 and the database structure information obtained in step 43 through a gating mechanism;

[0040] Step 52: Use a gating mechanism to calculate the weight between the two codes;

[0041] Step 53: Calculate a reset gate for each encoding. The reset gate calculates the importance of information in each dimension by combining the features of semantic encoding and structural encoding, using a trainable weight matrix and bias term; then decides which information should be reset; the formula for calculating the reset gate is as follows:

[0042] r t =σ(W r ·[h sem ,h struct ]+b r )

[0043] Where W r is a trainable weight matrix used to learn how to calculate the reset gate; h sem The hidden state vector representing the semantic encoding; calculated by the pre-trained language model; h struct The hidden state vector representing the structure encoding represents the information of the database structure and is calculated by the graph attention network; b r is the bias term;

[0044] Step 54, calculate the update gate to determine the degree of mixing of new information and old information; the formula is as follows:

[0045] z t =σ(W z·[h sem ,h struct ]+b z )

[0046] Where W z is a trainable weight matrix used to learn how to calculate the update gate; b z is the bias term of the update gate;

[0047] Step 55: Reset gate r t and update gate z t To calculate the mixed code h t , the two codes (semantic and structural) are weighted and added together, where the weights are determined by the reset gate and the update gate; the formula is as follows:

[0048] h t =tanh(W h ·[r t ·h sem , (1-r t )·h struct ]+b h )

[0049] where r t The weight used to control the semantic encoding, (1-r t ) is used to control the weight of the structure encoding; W h 、b h They are respectively the trainable weight matrix and bias term;

[0050] Step 56: Generate a mixed hidden state h t ; Through the gating mechanism, the mixed hidden state h t Represents the fused information; this information combines the query semantic information and database structure information.

[0051] Furthermore, in step 6, the output obtained in step 5 is fed into an autoregressive decoder, including the following steps:

[0052] Step 61 uses the hidden state mixed with semantic and structural information obtained in step 55 as the input of the decoder; the formula is as follows:

[0053] y=Decoder(h mix )

[0054] where h mix Contains semantic information of natural language questions and structural information of database structure; Decoder refers to an autoregressive decoder that processes mixed hidden states and gradually generates SQL query statements;

[0055] Step 62: For each SQL unit, the decoder predicts the next SQL unit based on the historically generated SQL units and the hidden state of the encoder;

[0056] Step 63: Introduce constraint checking to detect and reject erroneous SQL units to improve the quality of generated SQL.

[0057] Further, step 7, the execution of the SQL statement comprises the following steps:

[0058] Step 71: Run the obtained SQL statements in the specified database, and record the SQL statements that are incorrectly run and their corresponding predicted probabilities.

[0059] Furthermore, step 8 of training an error correction model includes the following steps:

[0060] Step 81, recording the SQL executed incorrectly in step 71 and the probability of generating the SQL statement;

[0061] Step 82: assemble the contents obtained in step 81 into a list:

[0062]

[0063] in Indicates an error query;

[0064] Step 83: Target sequence obtained according to step 82:

[0065]

[0066] in, represents the probability of the i-th wrong query, Indicates that given an input sequence x, the generated partial output sequence y1:t-1 and the error query P(y|x) is the conditional probability of generating the t-th element yt of the target sequence under the condition of x; T is the length of the target sequence, that is, the number of components of the complete SQL query that the model needs to generate; t represents the t-th element in the target sequence; P(y|x) represents the probability that the model generates the correct SQL query y under the condition of x.

[0067] Furthermore, in step 9, the error correction model is used to correct the error and then output the result;

[0068] The present invention discloses a text-to-SQL statement parsing and optimization method based on a large language model, which has the following advantages: the present invention first captures semantic information by modifying the encoder part in the large language model T5, and obtains the structural information of the database through the graph neural network GNN, and then fuses the information of the two through a gating mechanism; then, an autoregressive decoder is used to realize efficient parsing from natural language questions to SQL statements; finally, a model capable of error correction is trained to improve the accuracy and robustness of the results, so as to cope with the challenges brought by practical applications;

[0069] Specifically:

[0070] (1) The present invention uses a large language model to better capture the semantic information of the input text;

[0071] (2) The present invention makes full use of the graph neural network (GNN) to obtain the structural information of the database, which can more naturally understand the tables, relationships and architecture of the database;

[0072] (3) The gating mechanism of the present invention ensures the effective fusion of semantic information and database structure information; this fusion not only improves the accuracy of the query, but also helps to avoid unnecessary ambiguity, thereby providing clearer and more consistent results;

[0073] (4) The present invention also includes a well-trained error correction model for improving the accuracy and robustness of the results; this error correction model is capable of detecting and correcting errors caused by unclear or ambiguous inputs. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a flow chart of the Text-to-SQL method of the present invention.

[0075] Figure 2 This is the SQL statement prediction model architecture diagram.

[0076] Figure 3 This is the error correction model architecture diagram. DETAILED DESCRIPTION

[0077] The following will disclose the embodiments of the present invention with drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. That is to say, in some embodiments of the present invention, these practical details are not necessary.

[0078] Considering that Text-to-SQL is a task of converting natural language questions into structured query language (SQL), semantic information can be captured by using large language models such as T5. For example, by modifying the encoder part of T5, the natural language questions raised by users can be understood more accurately, which is expected to be better mapped to SQL queries. As for the diversity and complexity of database structures, the structural information of the database can be obtained by introducing graph neural networks (GNNs). The above semantic information and structural information can be fused by introducing gating mechanisms. Finally, considering the unclear or ambiguous questions that users may ask, the accuracy and robustness of the results can be improved by training an error correction model.

[0079] like Figure 1 , Figure 2 As shown, the present invention is a text-to-SQL statement parsing and optimization method based on a large language model, comprising the following steps:

[0080] Step 1: Enter a natural language question;

[0081] Step 2: Remove special characters and redundant spaces from the text obtained in step 1, and standardize English to lowercase letters and Chinese to simplified Chinese to ensure consistency;

[0082] Step 3: Encode the preprocessed text sequence obtained in step 2 using a large language model (such as T5, CodeBert, etc.);

[0083] Wherein: encoding the preprocessed text sequence using a large language model comprises the following steps:

[0084] Step 31 combines the natural language question G and a series of words or tokens q1, ..., q |G| With database schema spliced ​​into a sequence. Represented as a collection of columns, Represented as a collection of tables.

[0085] The sequence spliced ​​in step 32 is:

[0086]

[0087] This includes the tags q1, ..., q in the question |G| , database name And the columns of each table * indicates a special column marker in the database.

[0088] Step 33 uses a bidirectional encoder to learn the hidden state h of the input x. The specific formula is as follows:

[0089] h=Encoder(x)

[0090] Step 34 The semantic representation of the hidden state is encoded by the Transformer block, which contains two important parts, the multi-head self-attention network MHA and the fully connected feed-forward network FFN. The hidden state at layer l is represented as Where N is the maximum length of the input.

[0091] Step 35 The MHA part first queries the matrix Key Matrix and Value Matrix Mapped into an attention vector through the self-attention mechanism. The formula is as follows:

[0092]

[0093] Where m is the number of query vectors and n is the number of key or value vectors; MHA performs self-attention on each head. Represents the linear transformation matrix for each attention head. T represents transpose. QK T represents the multiplication of the query matrix Q and the transposed matrix of the key matrix K. m Indicates the dimension of T5, d k Indicates d m The target dimension mapped to the query vector, d v represents the dimension of the median vector of each attention head, and h represents the number of heads in the multi-head attention mechanism.

[0094] Step 36 In the encoder module of T5, d k =d v =d m / h, HA calculates the attention output of each head and connects them together to obtain the result of a multi-head self-attention network.

[0095] Step 37 The semantic hidden state also needs to be obtained through another component FFN. The formula is as follows:

[0096]

[0097] Among them, the linear weight matrix is ​​shown as d ff Represents the hidden layer dimension of the feedforward neural network, which is used to expand the feature space. Indicates The input of the layer, b1, b2 represents the bias vector, which is used to provide an offset for the calculation of each layer.

[0098] Step 38 performs layer normalization and residual connection after step 37 to obtain the semantic hidden state. The formula is as follows:

[0099]

[0100] Step 4: Model the structural information of the database through the graph neural network GNN. This step uses the graph neural network GNN to obtain the database structure information, so that the model can more naturally understand the tables, relationships and architecture of the database, and thus better convert the natural language questions raised by users into SQL queries.

[0101] Specifically, extracting the structural confidence of the database includes the following steps:

[0102] Step 41 constructs a heterogeneous graph through step 31), including three types of nodes and multiple types of edges. The three types of nodes are question nodes, column nodes and table nodes, and the multiple types of edges are pattern relations, pattern link relations and question relations.

[0103] Step 42 assigns an initial embedding vector to each node, representing their semantic information.

[0104] Step 43 encodes the structure of the heterogeneous graph, uses the relational graph attention network RGAT, calculates the attention weights according to the different relations between nodes, and updates the embedding representation of each node through message passing.

[0105] Step 44 concatenates the representation vectors of all nodes and inputs them into the T5 decoder to generate corresponding SQL queries.

[0106] Step 5: The semantic information and structural information obtained in step 3 and step 3 are fused into the output vector through the fusion strategy of the gating mechanism.

[0107] Among them, the gating mechanism fusion includes the following steps:

[0108] Step 51 fuses the semantic information obtained in step 38) and the database structure information obtained in step 43) through a gating mechanism.

[0109] Step 52 uses a gating mechanism to calculate the weight between the two codes.

[0110] Step 53 calculates a reset gate for each encoding (semantic and structural) to determine which information should be reset (discarded). The formula for calculating the reset gate is as follows:

[0111] r t =σ(W r ·[h sem ,h struct ]+b r )

[0112] Where W r is a trainable weight matrix used to learn how to calculate the reset gate; hsem The hidden state vector representing the semantic encoding; calculated by the pre-trained language model; h struct The hidden state vector representing the structure encoding represents the information of the database structure and is calculated by the graph attention network; b r is the bias term.

[0113] Step 54 calculates the update gate to determine the degree of mixing of new information and old information. The formula is as follows:

[0114] z t =σ(W z ·[h sem ,h struct ]+b z )

[0115] Where W z is a trainable weight matrix used to learn how to calculate the update gate. z is the bias term of the update gate.

[0116] Step 55: Reset gate r t and update gate z t To calculate the mixed code h t , the two codes (semantic and structural) are weighted and added together, where the weights are determined by the reset gate and the update gate. The formula is as follows:

[0117] h t =tanh(W h ·[r t ·h sem , (1-r t )·h struct ]+b h )

[0118] where r t The weight used to control the semantic encoding, (1-r t ) is used to control the weight of the structure encoding. h 、b h They are respectively the trainable weight matrix and bias term;

[0119] Step 6: Input the hidden state mixed with semantic and structural information obtained in step 5 into the decoder for decoding. Specifically, the steps are as follows:

[0120] Step 61 uses the hidden state mixed with semantic and structural information obtained in step 55) as the input of the decoder. The formula is as follows:

[0121] y=Decoder(h mix )

[0122] where h mixContains semantic information of natural language questions and structural information of database structure.

[0123] In step 62, for each SQL unit, the decoder predicts the next SQL unit based on the historically generated SQL units and the hidden state of the encoder.

[0124] Step 63 introduces constraint checking, detecting and rejecting erroneous SQL units to improve the quality of generated SQL.

[0125] Step 7 and step 71 execute the obtained SQL statements in the specified database, and record the SQL statements that are executed incorrectly and their corresponding predicted probabilities.

[0126] Step 8: Train an error correction model.

[0127] Among them, training the error correction model includes the following steps:

[0128] Step 81) records the SQL statements executed incorrectly in step 71) and the probability of generating the SQL statements.

[0129] Step 82) assembles the contents obtained in step 81) into a list:

[0130]

[0131] in Indicates an error query.

[0132] Step 83) Target sequence obtained according to step 82):

[0133]

[0134] in, represents the probability of the i-th wrong query, Indicates that given an input sequence x, the generated partial output sequence y1:t-1 and the error query The conditional probability of generating the t-th element yt of the target sequence under the condition of x; T is the length of the target sequence, that is, the number of components of the complete SQL query that the model needs to generate; t represents the t-th element in the target sequence; P(y|x) represents the probability of the model generating the correct SQL query y under the condition of x;

[0135] Step 9) The error correction model is used to correct the errors and then output the results.

[0136] The present invention provides a comprehensive solution for Text-to-SQL tasks, combining innovative methods of semantic understanding, database structure understanding, information fusion and error handling, so that the present invention can efficiently, accurately and reliably convert natural language questions into SQL queries.

[0137] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A text-to-SQL statement parsing and optimization method based on a large language model, characterized in that: The following steps are involved: Step 1: Input a natural language text; Step 2: remove special characters and redundant spaces from the natural language text obtained in step 1, and standardize English to lowercase letters and Chinese to simplified Chinese to obtain a preprocessed text sequence; Step 3: Encode the preprocessed text sequence obtained in step 2 through the large language model to extract the semantic information of the natural language query, including key intent, context relevance, and mapping relationship with the database schema; Step 4: Use the graph neural network GNN to model the structural information of the database using the preprocessed text sequence obtained in step 2, which specifically includes the following steps: combine the preprocessed text sequence with the metadata of the database to build a heterogeneous graph; the metadata of the database includes table name, column name, data type, primary key and foreign key constraints; the nodes in the heterogeneous graph represent table, column, and foreign key entities, and the edges in the heterogeneous graph represent the relationship between table, column, and foreign key entities; the graph neural network GNN encodes the database structural information through message passing and aggregation operations, captures the structural information of tables, columns and their relationships, and fuses this structural information with the semantic information of the query; Step 5: Use a fusion strategy based on a gating mechanism to fuse the information in the graph into the encoder output vector for steps 3 and 4; the information in the graph is comprehensive information about the database structure and query semantics obtained after processing by the graph neural network GNN; Step 6: The output vector obtained in step 5 is sent to an autoregressive decoder, which is composed of several T5 Transformer layers. The decoder gradually generates the components of the SQL statement according to the vectors output by the encoder and the graph neural network GNN module, including the selected fields, table names, and query conditions, until the SQL query statement is completely generated. Step 7: Use the SQL statement generated in step 6 as a query and execute it on the given database; Step 8: Record the wrong results in step 7 and optimize the results; Step 9: Output the results.

2. A text-to-SQL statement parsing and optimization method based on a large language model according to claim 1, characterized in that The step 3 uses the large language model as an encoder to encode the preprocessed text, which includes the following steps: Step 3-1: From step 2, we get a natural language question G and a series of words or tags q1,…,q |G| With database schema spliced ​​into a sequence; Represented as a collection of columns, Represented as a collection of tables; Step 3-2: Express the sequence obtained in step 3-1 as: in is the database name, For the columns of each table, * indicates a special column marker in the database; Step 3-3: Use a bidirectional encoder to learn the hidden state h of the input x. The specific formula is as follows: h=Encoder(x) Step 3-4: The semantic representation of the hidden state is encoded by a Transformer block, which contains a multi-head self-attention network MHA and a fully connected feed-forward network FFN. The hidden state at layer l is represented as Where N is the maximum length of the input; Step 3-5: The MHA part in step 3-4 first queries the matrix Key Matrix Value Matrix Mapped into an attention vector through the self-attention mechanism; the formula is as follows: Where m is the number of query vectors and n is the number of key or value vectors; MHA performs self-attention on each attention head. Represents the linear transformation matrix for each attention head; T represents transpose; QK T represents the multiplication of the query matrix Q and the transposed matrix of the key matrix K; d m Indicates the dimension of T5, d k Indicates d m The target dimension mapped to the query vector, d v represents the dimension of the median vector of each attention head, and h represents the number of heads in the multi-head attention mechanism; Step 3-6: In the encoder module of T5, d k =d v =d m / h, MHA calculates the attention output of each head and connects them together to get the result of the multi-head self-attention network; Step 3-7: The semantic hidden state needs to be obtained through another component FFN; the formula is as follows: Among them, the linear weight matrix is ​​shown as d ff Represents the hidden layer dimension of the feedforward neural network, which is used to expand the feature space; Indicates The input of the layer, b1, b2 represents the bias vector, which is used to provide an offset for the calculation of each layer; Step 3-8: After step 3-7, normalization and residual connection are performed to obtain the semantic hidden state; the formula is as follows: Represents the normalized result.

3. The method for parsing text to SQL statements based on a large language model and optimizing the same according to claim 2, characterized in that: The step 4 of modeling the structural information therein by using the graph neural network GNN includes the following steps: Step 4-1: Construct a heterogeneous graph through step 3-1, including three types of nodes and multiple types of edges; the three types of nodes are problem nodes, column nodes and table nodes, and the multiple types of edges are pattern relations, pattern link relations and problem relations; Step 4-2: Assign an initial embedding vector to each node to represent their semantic information; Step 4-3: Structurally encode the heterogeneous graph, use the relational graph attention network RGAT, calculate the attention weights according to the different relationships between nodes, and update the embedding representation of each node through message passing; Step 4-4: Concatenate the representation vectors of all nodes and input them into the T5 decoder to generate the corresponding SQL query.

4. The method for parsing text to SQL statements based on a large language model and optimizing the same according to claim 3, characterized in that: The fusion strategy based on the gating mechanism in step 5 fuses the information in the graph into the encoder output vector and includes the following steps: Step 5-1: Fuse the semantic information obtained in step 3 and the database structure information obtained in step 4 through a gating mechanism; Step 5-2: Use a gating mechanism to calculate the weight between the two codes; Step 5-3, calculate a reset gate for each encoding. The reset gate calculates the importance of information in each dimension by combining the features of semantic encoding and structural encoding, using a trainable weight matrix and bias term; then decides which information should be reset; the formula for calculating the reset gate is as follows: r t =σ(W r ·[h sem ,h struct ]+b r ) Where W r is a trainable weight matrix used to learn how to calculate the reset gate; h sem The hidden state vector representing the semantic encoding; calculated by the pre-trained language model; h struct The hidden state vector representing the structure encoding represents the information of the database structure and is calculated by the graph attention network; b r is the bias term; Step 5-4: Calculate the update gate to determine the degree of mixing of new information and old information; the formula is as follows: z t =σ(W z ·[h sem ,h struct ]+b z ) Where W z is a trainable weight matrix used to learn how to calculate the update gate; b z is the bias term of the update gate; Step 5-5, by resetting gate r t and update gate z t To calculate the mixed code h t , the semantic code and the structural code are weighted and added together, where the weight is determined by the reset gate and the update gate; the formula is as follows: h t =tanh(W h ·[r t ·h sem ,(1-r t )·h struct ]+b h ) where r t The weight used to control the semantic encoding, (1-r t ) is used to control the weight of the structure encoding; W h , b h They are respectively the trainable weight matrix and bias term; Step 5-6: Generate the mixed hidden state h t ; Through the gating mechanism, the mixed hidden state h t Represents the fused information; this information combines the query semantic information and database structure information.

5. The method for parsing text to SQL statements based on a large language model and optimizing the same according to claim 4, characterized in that: In step 6, the output obtained in step 5 is fed into an autoregressive decoder, which includes the following steps: Step 6-1: The hidden state mixed with semantic and structural information obtained in step 5 is used as the input of the decoder; the formula is as follows: y=Decoder(h mix ) where h mix Contains semantic information of natural language questions and structural information of database structure; Decoder refers to an autoregressive decoder that processes mixed hidden states and gradually generates SQL query statements; Step 6-2: For each SQL unit, the decoder predicts the next SQL unit based on the historically generated SQL units and the encoder’s hidden state. Step 6-3: Introduce constraint checking to detect and reject erroneous SQL units to improve the quality of generated SQL.

6. The method for parsing text to SQL statements based on a large language model and optimizing the same according to claim 5, characterized in that: The SQL statement obtained in step 7 is used as a query and executed on a given database, comprising the following steps: running the obtained SQL statement in the specified database, and recording the SQL statement that is incorrectly run and its corresponding predicted probability.

7. The method for parsing text to SQL statements based on a large language model and optimizing the same according to claim 6, characterized in that: Step 8 of optimizing the results according to the error results by training an error correction model includes the following steps: Step 8-1, record the SQL statement executed incorrectly in step 7 and the probability of generating the SQL statement; Step 8-2: Compose the contents obtained in step 8-1 into a list: in represents the i-th error query; Step 8-3: Obtain the target sequence according to step 8-2; the target sequence represents the correct SQL query sequence that is expected to be generated by the error correction model; the formula is as follows: in, represents the probability of the i-th wrong query, Indicates that given an input sequence x, the generated partial output sequence y 1:t-1 and error query Under the condition of t where t is the conditional probability of the target sequence; T is the length of the target sequence, that is, the number of components of the complete SQL query that the model needs to generate; t represents the tth element in the target sequence; P(y|x) represents the probability that the model generates the correct SQL query y under the condition of x.

8. The method for parsing text to SQL statements based on a large language model and optimizing the same according to claim 7, characterized in that: In step 9, the error correction model is used to correct the error and then output the result.

Images