Text-to-sql parsing and optimization method based on large language model

Abstract

The application discloses a text-to-SQL parsing and optimization method based on a large language model. First, text processing and syntax analysis techniques are used to parse natural language queries or commands in a fine-grained manner, converting them into syntax and semantic units. This process includes extracting elements such as entities, relationships, conditions, qualifiers, and operators from the input question. Then, a large language model technique is used to train a model that can be used for text-to-SQL parsing. Next, a feedback optimization mechanism is used to improve the accuracy of SQL statement parsing. Finally, the resulting model is validated on a public dataset. Compared to parsers that only focus on text-to-SQL generation, the application places more emphasis on the comprehensiveness and quality optimization of the text-to-SQL semantic parsing process, effectively improving the performance of natural language processing in the database field and the ability to handle complex syntax structures in databases.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary technical fields of natural language processing, large language models, and database information retrieval. Specifically, it relates to a method for parsing and optimizing text-to-SQL statements based on a large language model. Background Technology

[0002] Text-to-SQL is an important task in the field of Natural Language Processing (NLP), aiming to transform natural language questions into SQL queries to retrieve information from a database. This task can be viewed as a translation between natural language and structured queries, with significant application value, especially in data analysis, business intelligence, and intelligent customer service. Through this technology, non-technical personnel can interact with databases using intuitive natural language, saving time spent learning and mastering complex SQL syntax. This has led to widespread attention and application of Text-to-SQL tasks in both enterprise and research fields. The challenge of Text-to-SQL lies in mapping natural language questions to SQL queries. This mapping is usually not a simple one-to-one correspondence but rather complex and diverse. Natural language has rich and varied expressions, and the same query intent can be expressed in different ways; therefore, the model needs strong semantic understanding and logical reasoning capabilities. Furthermore, SQL queries themselves have complex structures, including multiple clauses (such as SELECT, FROM, WHERE, JOIN, etc.) and nested and join operations, requiring the model to generate correct SQL statement structures. In recent years, with the development of deep learning and natural language processing technologies, various solutions have emerged for Text-to-SQL tasks, including rule-based methods, transformation-based methods, machine learning-based methods, and the application of pre-trained models. Among these, pre-trained models such as BERT and GPT have provided new ideas and performance improvements for solving Text-to-SQL tasks. These models, through large-scale pre-training, can better understand the complex mapping relationship between natural language and SQL, thereby improving the accuracy and efficiency of the task. In Text-to-SQL semantic parsing tasks, generating SQL statements often faces problems such as syntax errors, semantic inconsistencies, or logical incompleteness. These problems may stem from the ambiguity of the input text, insufficient model understanding, or the complexity of a specific domain. The training of error correction models aims to identify and correct these errors, improving the accuracy and usability of the generated SQL statements. By designing corresponding training data and loss functions for different error types, the model can learn how to identify parts that do not conform to SQL syntax or the intent of the question and provide correction suggestions. The design and training of error correction models also need to consider the characteristics of real-world application scenarios. Different domains or tasks may have different common error patterns, so it is necessary to reflect these specific errors in the training data and adjust the structure and parameters of the model to adapt to different scenarios. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a text-to-SQL statement parsing and optimization method based on a large language model. This method first captures semantic information by modifying the encoder part of the large language model T5, and simultaneously obtains database structural information through a graph neural network (GNN). Then, it fuses the two information sets using a gating mechanism. Finally, an error-correcting model is trained to improve the accuracy and robustness of the results, addressing the challenges posed by practical applications.

[0004] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0005] This invention is a gait recognition method based on adaptive fusion to generate multi-scale temporal features, comprising the following steps:

[0006] Step 1: Input a piece of natural language text;

[0007] Step 2: Remove special characters and redundant spaces from the text obtained in Step 1, and standardize the English letters to lowercase and the Chinese letters to simplified Chinese, to obtain a preprocessed text sequence.

[0008] Step 3: Encode the preprocessed text sequence obtained in Step 2 using a large language model to extract semantic information of the natural language query, including key intent, contextual relevance, and mapping relationship with the database schema.

[0009] Step 4: The preprocessed text sequence obtained in Step 2 is used to model the structural information of the database using a Graph Neural Network (GNN). The preprocessed text sequence is combined with the database metadata, such as table names, column names, data types, primary keys, and foreign key constraints, to construct a heterogeneous graph. Nodes in the graph represent entities such as tables, columns, and foreign keys, and edges represent the relationships between them. The GNN encodes the database structure through message passing and aggregation operations, capturing the complex structural information of tables, columns, and their relationships, and integrates this structural information with the semantic information of the query to provide accurate database support for subsequent SQL generation.

[0010] Step 5: Apply a gating mechanism-based fusion strategy to steps 3 and 4 to fuse the information in the graph into the encoder output vector; where the graph information is the comprehensive information about the database structure and query semantics obtained after processing by the graph neural network (GNN).

[0011] Step 6: The output vector obtained in Step 5 is fed into an autoregressive decoder, which consists of several T5 Transformer layers. The decoder generates the various components of the SQL statement step by step based on the vectors output by the encoder and the graph neural network (GNN) module, including the selected fields, table names, query conditions, etc., until the complete SQL query statement is generated.

[0012] Step 7: Execute the SQL statement generated in Step 6 as a query on the given database;

[0013] Step 8: Record the erroneous results from Step 7 and optimize the results;

[0014] Step 9: Output the results.

[0015] Furthermore, step 3, encoding the preprocessed text sequence using a large language model, includes the following steps:

[0016] Step 31: Combine the natural language problem G with a series of words or tags q1, ..., q |G| With database schema Concatenate them into a sequence; where Represented as a set of columns, Represented as a set of tables;

[0017] Step 32, the assembled sequence is:

[0018]

[0019] This includes the labels q1, ..., q in the question. |G| Database name and the columns of each table Special column markers in the database;

[0020] Step 33: Use a bidirectional encoder to learn the hidden state h of the input x. The specific formula is as follows:

[0021] h = Encoder(x)

[0022] Step 34: The semantic representation of the hidden state is encoded using a Transformer block, which contains two important parts: a multi-head self-attention network (MHA) and a fully connected feedforward network (FFN). The hidden state representation at layer l is as follows: Where N is the maximum length of the input;

[0023] Step 35, the MHA part first queries the matrix. Key matrix and value matrix This is mapped to an attention vector using a self-attention mechanism; the formula is as follows:

[0024]

[0025] Where m is the number of query vectors and n is the number of key or value vectors; MHA performs self-attention on each head. Let Q and K represent the linear transformation matrices for each attention head; T represents the transpose; QK T This represents the product of the query matrix Q and the transpose of the key matrix K; d m d represents the dimension of T5. k d m Mapped to the target dimension of the query vector, d v represents the dimension of the value vector in each attention head, and h represents the number of heads in the multi-head attention mechanism;

[0026] Step 36: In the encoder module of T5, d k =d v =d m / h, MHA calculates the attention output of each head and then concatenates them together to obtain the result of the multi-head self-attention network;

[0027] Step 37: The semantic hidden state also needs to be obtained through another component, FFN; the formula is as follows:

[0028]

[0029] The linear weight matrix is ​​shown as follows: d ff This represents the hidden layer dimension of the feedforward neural network, used to expand the feature space; Indicates the first The input to the layer, b1 and b2 represent bias vectors, which are used to provide offsets for the calculation of each layer;

[0030] Step 38: After step 37, perform layer normalization and residual connections to obtain the semantic hidden state; the formula is as follows:

[0031]

[0032] This represents the result after normalization.

[0033] Furthermore, step 4, which models the structural information using a graph neural network (GNN), includes the following steps:

[0034] Step 41: Construct a heterogeneous graph based on Step 31, including three types of nodes and multiple types of edges; the three types of nodes are problem nodes, column nodes, and table nodes; the multiple types of edges are schema relationships, schema link relationships, and problem relationships.

[0035] Step 42: Assign an initial embedding vector to each node to represent its semantic information;

[0036] Step 43: Perform structural encoding on the heterogeneous graph, use the Relationship Graph Attention Network (RGAT) to calculate attention weights based on the different relationships between nodes, and update the embedding representation of each node through message passing;

[0037] Step 44: Concatenate the representation vectors of all nodes and input them into the T5 decoder to generate the corresponding SQL query.

[0038] Furthermore, step 5, the gating mechanism-based fusion strategy, fuses the information in the graph into the encoder output vector, including the following steps:

[0039] Step 51: Merge the semantic information obtained in step 38 and the database structure information obtained in step 43 through a gating mechanism;

[0040] Step 52: Use a gating mechanism to calculate the weight between the two codes;

[0041] Step 53: Calculate a reset gate for each encoding. The reset gate calculates the information importance of each dimension by combining the features of semantic and structural encodings, using a trainable weight matrix and bias terms; then it determines which information should be reset; the formula for calculating the reset gate is as follows:

[0042] r t =σ(W r ·[h sem h struct ]+b r )

[0043] Among them W r It is a trainable weight matrix used to learn how to compute the reset gate; h sem The hidden state vector represents the semantic encoding; it is calculated by a pre-trained language model; h struct The hidden state vector, representing the structure encoding, contains information about the database structure and is calculated by the graph attention network; b r It is a bias term;

[0044] Step 54: Calculate the update gate to determine the degree of mixing between new and old information; the formula is as follows:

[0045] z t =σ(W z·[h sem h struct ]+b z )

[0046] Among them W z It is a trainable weight matrix used to learn how to compute the update gate; b z It updates the bias term of the gate;

[0047] Step 55: Reset the door r t and update gate z t To calculate the hybrid encoding h t The two codes (semantic and structural) are weighted and summed, with the weights determined by the reset and update gates; the formula is as follows:

[0048] h t =tanh(W h ·[r t ·h sem , (1-r t )·h struct ]+b h )

[0049] Where r t Weights used to control semantic encoding, (1-r t The weights (W) are used to control the structural encoding. h b h These are the trainable weight matrix and the bias term, respectively.

[0050] Step 56: Generate the blended hidden state h t Through the gating mechanism, the hybrid hidden state h is obtained. t This represents the merged information; it integrates the semantic information of the query and the database structure information.

[0051] Furthermore, step 6, which involves feeding the output obtained in step 5 into an autoregressive decoder, includes the following steps:

[0052] Step 61 uses the hidden state obtained in step 55, which combines semantic and structural information, as input to the decoder; the formula is as follows:

[0053] y = Decoder(h) mix )

[0054] Where h mix It contains semantic information about natural language problems and structural information about the database structure; the Decoder refers to an autoregressive decoder that processes mixed hidden states and progressively generates SQL query statements;

[0055] Step 62: For each SQL unit, the decoder predicts the next SQL unit based on the previously generated SQL units and the encoder's hidden state;

[0056] Step 63: Introduce constraint checks to detect and reject erroneous SQL units, thereby improving the quality of generated SQL.

[0057] Furthermore, step 7, executing the SQL statement, includes the following steps:

[0058] Step 71: Run the obtained SQL statements in the specified database and record the SQL statements that were executed incorrectly and their corresponding prediction probabilities.

[0059] Furthermore, step 8, training an error correction model, includes the following steps:

[0060] Step 81: Record the SQL statement that was executed incorrectly in step 71 and the probability of generating that SQL statement;

[0061] Step 82: Compile the contents obtained in Step 81 into a list:

[0062]

[0063] in Indicates an incorrect query;

[0064] Step 83: Based on the target sequence obtained in step 82:

[0065]

[0066] in, This represents the probability of the i-th incorrect query. This represents a given input sequence x, a partially generated output sequence y1:t-1, and an error query. Under the condition of x, P(y|x) represents the conditional probability of generating the t-th element yt of the target sequence; T is the length of the target sequence, i.e., the number of components of the complete SQL query that the model needs to generate; t represents the t-th element in the target sequence; P(y|x) represents the probability that the model generates the correct SQL query y under the condition of x.

[0067] Furthermore, in step 9, the error correction model is used to correct errors, and then the results are output;

[0068] This invention discloses a text-to-SQL statement parsing and optimization method based on a large language model, which has the following advantages: First, this invention captures semantic information by modifying the encoder part of the large language model T5, and simultaneously obtains the structural information of the database through a graph neural network (GNN). Then, it fuses the two information through a gating mechanism. Next, it achieves efficient parsing from natural language questions to SQL statements through an autoregressive decoder. Finally, it trains an error-correcting model to improve the accuracy and robustness of the results, addressing the challenges brought by practical applications.

[0069] Specifically:

[0070] (1) This invention utilizes a large language model to better capture the semantic information of the input text;

[0071] (2) This invention makes full use of graph neural networks (GNNs) to obtain the structural information of the database, and can understand the tables, relationships and architecture of the database more naturally;

[0072] (3) The gating mechanism of the present invention ensures the effective integration of semantic information and database structure information; this integration not only improves the accuracy of queries, but also helps to avoid unnecessary ambiguity, thereby providing clearer and more consistent results;

[0073] (4) The present invention also includes a trained error correction model for improving the accuracy and robustness of the results; the error correction model is capable of detecting and correcting errors caused by unclear or ambiguous input. Attached Figure Description

[0074] Figure 1 This is a flowchart of the Text-to-SQL method of the present invention.

[0075] Figure 2 This is a diagram of the SQL statement prediction model architecture.

[0076] Figure 3 This is a diagram of the error correction model architecture. Detailed Implementation

[0077] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.

[0078] Given that Text-to-SQL is a task of translating natural language questions into structured query language (SQL), semantic information can be captured by using large language models such as T5. For example, by modifying the encoder part of T5, it is possible to more accurately understand the natural language questions posed by users, thus potentially mapping them better to SQL queries. For the diversity and complexity of database structures, graph neural networks (GNNs) can be introduced to obtain structural information about the database. The aforementioned semantic and structural information can be fused by introducing gating mechanisms. Finally, considering that users may pose unclear or ambiguous questions, an error correction model can be trained to improve the accuracy and robustness of the results.

[0079] like Figure 1 , Figure 2 As shown, this invention is a text-to-SQL statement parsing and optimization method based on a large language model, comprising the following steps:

[0080] Step 1: Input a natural language question;

[0081] Step 2: Remove special characters and redundant spaces from the text obtained in Step 1, and standardize the English letters to lowercase and the Chinese letters to simplified Chinese to ensure consistency.

[0082] Step 3: Encode the preprocessed text sequence obtained in Step 2 using a large language model (such as T5, CodeBert, etc.);

[0083] The encoding of the preprocessed text sequence using a large language model includes the following steps:

[0084] Step 31: Given a natural language problem G and a series of words or tags q1, ..., q2, we can define the natural language problem G and the set of words or tags q1, ..., q3. |G| With database schema Concatenate them into a sequence. Represented as a set of columns, It is represented as a set of tables.

[0085] The sequence assembled in step 32 is:

[0086]

[0087] This includes the labels q1, ..., q in the question. |G| Database name and the columns of each table * indicates a special column marker in the database.

[0088] Step 33 uses a bidirectional encoder to learn the hidden state h of the input x. The specific formula is as follows:

[0089] h = Encoder(x)

[0090] The semantic representation of the hidden state in step 34 is encoded using a Transformer block, which contains two important parts: a multi-head self-attention network (MHA) and a fully connected feedforward network (FFN). The hidden state representation at layer l is as follows: Where N is the maximum length of the input.

[0091] Step 35, the MHA part, first queries the matrix. Key matrix and value matrix This is mapped to an attention vector using a self-attention mechanism. The formula is as follows:

[0092]

[0093] Where m is the number of query vectors and n is the number of key or value vectors; MHA performs self-attention on each head. Let QK represent the linear transformation matrix for each attention head. T denotes the transpose. T This represents the product of the query matrix Q and the transpose of the key matrix K. m d represents the dimension of T5. k d m Mapped to the target dimension of the query vector, d v represents the dimension of the value vector in each attention head, and h represents the number of heads in the multi-head attention mechanism.

[0094] Step 36: In the encoder module of T5, d k =d v =d m / h, HA calculates the attention output of each head and then concatenates them together to obtain the result of the multi-head self-attention network.

[0095] Step 37: The semantic hidden state also needs to be obtained through another component, FFN. The formula is as follows:

[0096]

[0097] The linear weight matrix is ​​shown as follows: d ff This represents the hidden layer dimension of the feedforward neural network, used to expand the feature space. Indicates the first The inputs to the layer, b1 and b2, represent bias vectors used to provide offsets for the calculation of each layer.

[0098] Step 38, following step 37, performs layer normalization and residual connections to obtain the semantic hidden state. The formula is as follows:

[0099]

[0100] Step 4: Model the database structure using a Graph Neural Network (GNN). This step utilizes a GNN to obtain database structure information, enabling the model to more naturally understand the database tables, relationships, and architecture, thereby better translating user-generated natural language questions into SQL queries.

[0101] Specifically, extracting structural information from a database includes the following steps:

[0102] Step 41 builds a heterogeneous graph from step 31), including three types of nodes and multiple types of edges. The three types of nodes are problem nodes, column nodes, and table nodes, and the multiple types of edges are schema relationships, schema link relationships, and problem relationships.

[0103] Step 42 assigns an initial embedding vector to each node, representing its semantic information.

[0104] Step 43 performs structural encoding on the heterogeneous graph, uses the Relational Graph Attention Network (RGAT), calculates attention weights based on the different relationships between nodes, and updates the embedding representation of each node through message passing.

[0105] Step 44 concatenates the representation vectors of all nodes and inputs them into the T5 decoder to generate the corresponding SQL query.

[0106] Step 5: The semantic and structural information obtained in Step 3 and Step 3 are fused into the output vector through a gating mechanism fusion strategy.

[0107] The gating mechanism integration includes the following steps:

[0108] Step 51 integrates the semantic information obtained in step 38) and the database structure information obtained in step 43) through a gating mechanism.

[0109] Step 52 uses a gating mechanism to calculate the weight between the two codes.

[0110] Step 53 calculates a reset gate for each encoding (semantic and structural) to determine which information should be reset (discarded). The formula for calculating the reset gate is as follows:

[0111] r t =σ(W r ·[h sem h struct ]+b r )

[0112] Among them W r It is a trainable weight matrix used to learn how to compute the reset gate; hsem The hidden state vector represents the semantic encoding; it is calculated by a pre-trained language model; h struct The hidden state vector, representing the structure encoding, contains information about the database structure and is calculated by the graph attention network; b r It is a bias term.

[0113] Step 54 calculates the update gate to determine the degree of mixing between new and old information. The formula is as follows:

[0114] z t =σ(W z ·[h sem h struct ]+b z )

[0115] Among them W z It is a trainable weight matrix used to learn how to compute the update gate. z It is the bias term for updating the gate.

[0116] Step 55 involves resetting the gate r t and update gate z t To calculate the hybrid encoding h t The two codes (semantic and structural) are weighted and summed, with the weights determined by the reset and update gates. The formula is as follows:

[0117] h t =tanh(W h ·[r t ·h sem , (1-r t )·h struct ]+b h )

[0118] Where r t Weights used to control semantic encoding, (1-r t The weights (W) are used to control the weights of the structural encoding. h b h These are the trainable weight matrix and the bias term, respectively.

[0119] Step 6: Input the hidden state obtained in Step 5, which combines semantic and structural information, into the decoder for decoding. This specifically includes the following steps:

[0120] Step 61 uses the hidden state obtained in step 55, which combines semantic and structural information, as input to the decoder. The formula is as follows:

[0121] y = Decoder(h) mix )

[0122] Where h mixIt contains semantic information about natural language issues and structural information about the database structure.

[0123] Step 62: For each SQL unit, the decoder predicts the next SQL unit based on the previously generated SQL units and the encoder's hidden state.

[0124] Step 63 introduces constraint checks to detect and reject erroneous SQL units, thereby improving the quality of the generated SQL.

[0125] Steps 7 and 71 involve running the obtained SQL statements in the specified database and recording the SQL statements that were executed incorrectly and their corresponding prediction probabilities.

[0126] Step 8: Train an error correction model.

[0127] Training the error correction model includes the following steps:

[0128] Step 81) Record the SQL statement that was executed incorrectly in step 71) and the probability of generating that SQL statement.

[0129] Step 82) Compile the contents obtained in step 81) into a list:

[0130]

[0131] in This indicates an incorrect query.

[0132] Step 83) Based on the target sequence obtained in step 82):

[0133]

[0134] in, This represents the probability of the i-th incorrect query. This represents a given input sequence x, a partially generated output sequence y1:t-1, and an error query. Given x, P(y|x) represents the conditional probability of generating the t-th element yt of the target sequence; T is the length of the target sequence, i.e., the number of components of the complete SQL query that the model needs to generate; t represents the t-th element in the target sequence; P(y|x) represents the probability that the model generates the correct SQL query y given x. ；

[0135] Step 9) The error correction model is used to correct errors, and then the results are output.

[0136] This invention provides a comprehensive solution for Text-to-SQL tasks, combining innovative methods of semantic understanding, database structure understanding, information fusion, and error handling, enabling it to efficiently, accurately, and reliably transform natural language problems into SQL queries.

[0137] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for parsing and optimizing text-to-SQL statements based on a large language model, characterized in that, Includes the following steps: Step 1: Input a piece of natural language text; Step 2: Remove special characters and redundant spaces from the natural language text obtained in Step 1, and standardize the English to lowercase letters and the Chinese to simplified Chinese, to obtain a preprocessed text sequence. Step 3: Encode the preprocessed text sequence obtained in Step 2 using a large language model to extract semantic information of the natural language query, including key intent, contextual relevance, and mapping relationship with the database schema. Step 4: Use a Graph Neural Network (GNN) to model the structural information of the database using the preprocessed text sequence obtained in Step 2. This includes the following steps: combining the preprocessed text sequence with the database metadata to construct a heterogeneous graph; the database metadata includes table names, column names, data types, primary keys, and foreign key constraints; the nodes in the heterogeneous graph represent tables, columns, and foreign key entities, and the edges in the heterogeneous graph represent the relationships between tables, columns, and foreign key entities; the GNN encodes the database structural information through message passing and aggregation operations, captures the structural information of tables, columns, and their relationships, and fuses this structural information with the semantic information of the query. Step 5: Apply a gating mechanism-based fusion strategy to steps 3 and 4 to fuse the information in the graph into the encoder output vector; where the information in the graph is comprehensive information about the database structure and query semantics obtained after processing by a graph neural network (GNN). Step 6: The output vector obtained in Step 5 is fed into an autoregressive decoder, which consists of several T5Transformer layers. The decoder generates the various components of the SQL statement step by step based on the vectors output by the encoder and the graph neural network (GNN) module, including the selected fields, table names, and query conditions, until the complete SQL query statement is generated. Step 7: Execute the SQL statement generated in Step 6 as a query on the given database; Step 8: Record the erroneous results from Step 7 and optimize the results; Step 9: Output the results; Step 3, which uses a large language model as an encoder to encode the preprocessed text, includes the following steps: Step 3-1: From Step 2, we obtain a natural language problem G and a series of words or tags q1,…,q |G| With database schema Concatenate them into a sequence; where Represented as a set of columns, Represented as a set of tables; Step 3-2: Represent the sequence obtained in Step 3-1 as follows: in For database name, For columns in each table, * indicates a special column marker in the database; Step 3-3: Use a bidirectional encoder to learn the hidden state h of the input x. The specific formula is as follows: h = Encoder(x) Steps 3-4: The semantic representation of the hidden state is encoded using a Transformer block, which contains a Multi-Head Self-Attention Network (MHA) and a Fully Connected Feedforward Network (FFN). The hidden state representation at layer l is as follows: Where N is the maximum length of the input; The MHA part in steps 3-5 and 3-4 first queries the matrix. Key matrix Value matrix This is mapped to an attention vector using a self-attention mechanism; the formula is as follows: Where m is the number of query vectors and n is the number of key or value vectors; MHA performs self-attention on each attention head. Let Q and K represent the linear transformation matrices for each attention head; T represents the transpose; QK T This represents the product of the query matrix Q and the transpose of the key matrix K; d m d represents the dimension of T5. k d m Mapped to the target dimension of the query vector, d v represents the dimension of the value vector in each attention head, and h represents the number of heads in the multi-head attention mechanism; Steps 3-6: In the encoder module of T5, d k =d v =d m / h, MHA calculates the attention output of each head and then concatenates them together to obtain the result of the multi-head self-attention network; Steps 3-7: The semantic hidden state also needs to be obtained through another component, FFN; the formula is as follows: The linear weight matrix is ​​shown as follows: d ff This represents the hidden layer dimension of the feedforward neural network, used to expand the feature space; b1 and b2 represent the input of the l-th layer, and b1 and b2 represent the bias vectors used to provide offsets for the computation of each layer. Steps 3-8: After step 3-7, normalization and residual connections are performed to obtain the semantic hidden state; the formula is as follows: This represents the result after normalization.

2. The method for parsing and optimizing text-to-SQL statements based on a large language model according to claim 1, characterized in that, Step 4, which involves modeling the structural information using a graph neural network (GNN), includes the following steps: Step 4-1: Construct a heterogeneous graph based on Step 3-1, including three types of nodes and multiple types of edges; the three types of nodes are problem nodes, column nodes, and table nodes, and the multiple types of edges are schema relationships, schema link relationships, and problem relationships. Step 4-2: Assign an initial embedding vector to each node to represent its semantic information; Step 4-3: Perform structural encoding on the heterogeneous graph, use the Relationship Graph Attention Network (RGAT) to calculate attention weights based on the different relationships between nodes, and update the embedding representation of each node through message passing; Step 4-4: Concatenate the representation vectors of all nodes and input them into the T5 decoder to generate the corresponding SQL query.

3. The method for parsing and optimizing text-to-SQL statements based on a large language model according to claim 2, characterized in that: The gating mechanism-based fusion strategy in step 5 fuses the information in the graph into the encoder output vector, including the following steps: Step 5-1: Merge the semantic information obtained in step 3 and the database structure information obtained in step 4 using a gating mechanism; Step 5-2: Use a gating mechanism to calculate the weight between the two codes; Step 5-3: Calculate a reset gate for each encoding. The reset gate combines the features of semantic and structural encodings, and uses a trainable weight matrix and bias terms to calculate the information importance of each dimension; then it determines which information should be reset; the formula for calculating the reset gate is as follows: r t =σ(W r ·[h sem ,h struct ]+b r ) Among them W r It is a trainable weight matrix used to learn how to compute the reset gate; h sem The hidden state vector represents the semantic encoding; it is calculated by a pre-trained language model; h struct The hidden state vector, representing the structure encoding, contains information about the database structure and is calculated by the graph attention network; b r It is a bias term; Step 5-4: Calculate the update gate to determine the degree of mixing between new and old information; the formula is as follows: z t =σ(W z ·[h sem ,h struct ]+b z ) Among them W z It is a trainable weight matrix used to learn how to compute the update gate; b z It updates the bias term of the gate; Step 5-5: Reset the door r t and update gate z t To calculate the hybrid encoding h t The semantic encoding and structural encoding are weighted and summed, with the weights determined by the reset gate and update gate; the formula is as follows: h t =tanh(W h ·[r t ·h sem ,(1-r t )·h struct ]+b h ) Where r t The weights used to control semantic encoding, and (1-rt) used to control structural encoding weights; W h b h These are the trainable weight matrix and the bias term, respectively. Steps 5-6: Generate the mixed hidden state h t Through the gating mechanism, the hybrid hidden state h is obtained. t This represents the merged information; it integrates the semantic information of the query and the database structure information.

4. The method for parsing and optimizing text-to-SQL statements based on a large language model according to claim 3, characterized in that: Step 6, which involves feeding the output obtained in step 5 into an autoregressive decoder, includes the following steps: Step 6-1: Use the hidden state obtained in Step 5, which combines semantic and structural information, as the input to the decoder; the formula is as follows: y=Decoder(h mix ) Where h mix It contains semantic information about natural language problems and structural information about the database structure; the Decoder refers to an autoregressive decoder that processes mixed hidden states and progressively generates SQL query statements; Step 6-2: For each SQL unit, the decoder predicts the next SQL unit based on the previously generated SQL units and the encoder's hidden state. Step 6-3: Introduce constraint checks to detect and reject erroneous SQL units, thereby improving the quality of generated SQL.

5. The method for parsing and optimizing text-to-SQL statements based on a large language model according to claim 4, characterized in that, The SQL statement obtained in step 7 is used as a query and executed on a given database, which includes the following steps: running the obtained SQL statement in the specified database and recording the SQL statement that was executed incorrectly and its corresponding prediction probability.

6. The method for parsing and optimizing text-to-SQL statements based on a large language model according to claim 5, characterized in that, Step 8, optimizing the results based on the errors, is achieved by training an error correction model and includes the following steps: Step 8-1: Record the SQL statement that was executed incorrectly in step 7 and the probability of generating that SQL statement; Step 8-2: Compile the contents obtained in Step 8-1 into a list: in This indicates the i-th incorrect query; Step 8-3: Obtain the target sequence based on Step 8-2; the target sequence represents the correct SQL query sequence that is expected to be generated by the error correction model; the formula is as follows: in, This represents the probability of the i-th incorrect query. This represents a given input sequence x and a partially generated output sequence y. 1:t-1 and error query Under the condition of generating the t-th element y of the target sequence t The conditional probability; T is the length of the target sequence, i.e., the number of components of the complete SQL query that the model needs to generate; t represents the t-th element in the target sequence; P(y|x) represents the probability that the model generates the correct SQL query y given x.

7. The method for parsing and optimizing text-to-SQL statements based on a large language model according to claim 6, characterized in that, In step 9, the error correction model is used to correct errors, and then the results are output.

Images