A multi-layer semantic-aware enhanced NL2SQL model and method

Abstract

The application designs a multi-layer semantic perception enhanced NL2SQL model and method, and belongs to the technical field of natural language processing; in the stage of constructing a semantic enhanced schema link graph, on the basis of the existing method of constructing a schema link graph by similarity, hyperbolic space is organically integrated to construct a schema link graph coupled by hyperbolic space and cosine similarity; the semantic enhanced schema link graph obtained in this stage is transmitted to a semantic relation embedding encoder module, the natural language query is converted into an intermediate semantic vector by the encoder module, in the process, similarity is taken as a further enhanced relation embedding, finally, the processing result of the encoder module is delivered to a decoder module considering pruning of irrelevant tables, and then the final SQL statement is obtained; the application enables many non-expert users to query data in a database by using natural language, saves the learning cost of learning the related knowledge of SQL statements for users, improves the accuracy of the model, and reduces the complexity of the model.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically relating to a multi-layer semantic awareness enhanced NL2SQL model and method. Background Technology

[0002] The NL2SQL (Natural Language to SQL) task refers to parsing user-generated natural language questions using an NL2SQL model to generate corresponding SQL statements. The emergence of NL2SQL allows users to perform database queries without understanding SQL syntax, thus gaining widespread popularity and becoming one of the most popular semantic parsing models. Its popularity stems from its ability to make database queries more convenient and intuitive, providing convenience for many users. The NL2SQL model has a wide range of applications, including but not limited to intelligent assistants and database management systems, providing users with a more intelligent and efficient way to query and process data. It is worth noting that with the continuous development and improvement of NL2SQL technology, its effectiveness and performance in practical applications are constantly improving, bringing a better user experience. In summary, the emergence and development of the NL2SQL task provides users with a more convenient and intuitive way to query databases, making it popular and widely used in various fields.

[0003] NL2SQL is a subtask within the semantic parsing domain of natural language understanding. Its technical framework diagram is as follows: Figure 1 As shown, the NL2SQL model takes a user-generated natural language question and an existing database schema as input. These are semantically encoded to obtain semantic vectors. Simultaneously, schema linking is used to understand which tables and columns in the database the question queries, and the question and database schema are linked. Further, the SQL encoding module encodes the alignment relationship and semantic vectors obtained from schema linking to obtain a relationship vector between the question and the database schema. This relationship vector is a high-dimensional vector representation of the character features, semantic features, and relational features of the natural language question and the database schema. Next, SQL decoding is performed to generate an SQL statement according to SQL syntax rules, and the final SQL statement is returned to the user as output.

[0004] Despite the long history of research in semantic parsing, accurately understanding the semantics of natural language queries remains a challenging problem. Pattern linking, a crucial submodule, connects a word in a natural language query with a table, column, or value in a database schema. Simultaneously, pattern linking identifies the target table and columns the queryer needs and the constraints that constitute the SQL statement. It also incorporates the relevant tables, columns, and values ​​found in the pattern linking process into the generated SQL statement. Therefore, the accuracy of the pattern linking process in discovering the relationship between natural language words and the database schema significantly impacts the accuracy of parsing. Semantic parsing has always been a challenging problem in the field of natural language processing. Despite long-term efforts by researchers in this area, accurately understanding the semantics of natural language queries remains a complex task. With the increasing demands for information retrieval and database queries, the accuracy of semantic parsing has become particularly important. In this context, pattern linking, as a vital component of semantic parsing, plays a crucial role. The role of pattern linking cannot be underestimated. It connects a word in a natural language query with a table, column, or value in a database schema, thereby helping the parsing system better understand the user's query intent. Through schema linking, the system can locate the target table and columns that the user needs to query, and determine the constraints that constitute the SQL statement. This process is crucial for generating accurate SQL queries, as it directly affects the accuracy of the system's understanding and response to user queries. The accuracy of schema linking has a significant impact on the overall accuracy of semantic parsing. If schema linking fails to accurately discover the relationship between natural language words and database schemas, it can lead to the parsing system misunderstanding the user's query intent, thereby affecting the system's response quality. Therefore, improving the accuracy of schema linking is a key step in enhancing semantic parsing capabilities.

[0005] Existing NL2SQL models fail to provide correct parsing results when there are discrepancies between the natural language question and the database schema—that is, when the question words and database schema fields (literal values) are not represented correctly. This reduces the domain generalization ability of such models, primarily due to the limitations of their schema linking capabilities. Methods based on neural parsers consider the relational information of continuous representations in the PLM (Programmable Logic Model), but they can fail due to synonyms and spelling errors. Furthermore, neural parsers ignore grammatical information during processing, which can easily carry errors downstream, leading to incorrect final results. Therefore, existing methods for schema linking primarily employ n-gram fuzzy string matching and similarity-based methods. While n-gram fuzzy string matching considers the relational information of continuous representations in the PLM, its accuracy still needs improvement. Synonyms and spelling errors can easily lead to inappropriate alignment, causing schema linking failures. Similarity-based schema linking methods are affected by static word embeddings.

[0006] Meanwhile, many current decoding methods employ a depth-first search order to generate an abstract syntax tree (AST) of the context-free syntax of the target query results, including APPYRULE, SELECTCOLUMN, and SELECTTABLE. APPYRULE(r) applies rule r to the current derived tree of the query. SELECTCOLUMN(c) and SELECTTABLE(t) select column c and table t from the schema, respectively. However, these methods generate SQL queries containing irrelevant tables. This can lead to unexpected query results, impacting data analysis and decision-making processes. Therefore, more efficient decoding methods are needed to avoid the presence of irrelevant tables. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention designs a multi-layer semantic awareness-enhanced NL2SQL model and method, aiming to solve many problems existing in current NL2SQL models, including insufficient schema linking capabilities, redundant information in generated queries, high computational and search complexity of decoders, and a large number of tables that need to be searched, thereby improving the accuracy of the model and reducing its complexity.

[0008] A multi-layer semantically enhanced NL2SQL model comprises three parts: constructing a semantically enhanced pattern link graph, an encoder for semantic relation embedding, and a decoder that considers pruning irrelevant tables. In the stage of constructing the semantically enhanced pattern link graph, hyperbolic space is organically integrated into the existing method of constructing the pattern link graph based on similarity, resulting in a pattern link graph coupled with hyperbolic space and cosine similarity. The semantically enhanced pattern link graph obtained in this stage is transmitted to the encoder module for semantic relation embedding. The encoder module converts the natural language query into an intermediate semantic vector, incorporating similarity as a further enhanced relation embedding during this process. Finally, the processing result of the encoder module is sent to the decoder module that considers pruning irrelevant tables, thereby obtaining the final SQL statement.

[0009] A multi-layer semantic awareness enhancement method, based on the aforementioned multi-layer semantic awareness enhancement NL2SQL model, specifically includes the following steps:

[0010] Step 1: Define the Natural Language Problem and Database Schema; given an NL problem Q and the corresponding database schema S =<T,C> Therefore, the input is I =<Q,S> The output target is to generate an SQL query Y; specifically, the problem Q = {q1,q2,…,q} |Q|} is a token-level sequence of question |Q|; where the database schema consists of table |T| and columns |C|, where T = {t1, t2, ..., t} |T|}, C = {c1, c2, ..., c |C| Each table ti consists of a name containing multiple words: Table t i Each column in They are all composed of words or a phrase; The last word of table ti, that is, the last word of the table name; This represents the name of the j-th column in table ti, i.e., the column name;

[0011] Step 2: Construct a semantically enhanced pattern link graph:

[0012] Step 2-1: Construct a pattern link graph based on hyperbolic space optimization;

[0013] Given a database schema item S =<T,C> Where T is the corresponding database table, T = {t1, t2, ..., t} |T|}, where C is a column in the corresponding table, C = {c1, c2, ..., c |C| First, concatenate the database tables and columns: S = (T, C) = <s1,s2,…s |T|+|C| Then, the question word is concatenated with S, serialized, and used as input I:

[0014] I = (q1, q2, ..., q |Q| ,s1,s2,…,s |T|+|C| )

[0015] Input I into the PLM to obtain the context representation c of the j-th pattern item. j At the same time, replace the question word q with the [MASK] marker. i The new question word sequence containing the [MASK] marker is re-inputted into the PLM. At this time, the PLM will obtain the result when the question word q is... i When blocked, the j-th pattern item s j New contextual representation The pattern term s is measured by measuring the difference between two context representations. j and the question word q i The correlation between them; where the distance between the two contexts represents c. j , The difference between the two context representations is measured using the Poincaré distance in hyperbolic space. j , The distance between them;

[0016] In order to represent two context vectors c in hyperbolic space j , To compare them, we first need to compare the two vectors c. j , Mapping to hyperbolic space yields the corresponding representation c in hyperbolic space. j ′、 The mapping function is represented as M():FvI→HvI, where M() is expressed as:

[0017]

[0018] μ represents a coefficient used to adjust the degree of processing of the input vector;

[0019] The vector representation c in hyperbolic space is obtained. j ′、 Then, the problem word q is measured by measuring the Poincaré distance between the two vector representations. i and pattern item s j Correlation between them:

[0020]

[0021] This leads to a question word q. i and pattern item s jThe relationship matrix between them is used, but the pattern link graph based on hyperbolic space is affected by low-confidence noise. Therefore, a predefined threshold h is introduced. Items below the predefined threshold are considered noise, i.e., items less than the threshold are set to h(q). i ,s j If h(q) = 0, otherwise it is 1. Therefore, the noisy pattern linking results are initially filtered out, resulting in an adjacency matrix HS, which is represented as the optimized hyperbolic space-based pattern linking graph. The adjacency matrix HS is a two-dimensional matrix, where each element h(q) = 0. i ,s j ) indicates the question word q i and pattern item s j The relationship between h(q); i ,s j If the value is less than the predefined threshold h, it is set to 0 to indicate noise; otherwise, it is set to 1 to indicate a relationship. Therefore, the specific form of the adjacency matrix HS is a matrix composed of 0 and 1, where 0 represents noise and 1 represents a relationship.

[0022] Step 2-2: Construct a pattern link graph based on cosine similarity optimization;

[0023] Cosine similarity is used to measure the similarity of the question word q. i and pattern item s j The correlation between them is first calculated by the question word q. i and pattern item s j The cosine similarity between them yields the similarity matrix S:

[0024]

[0025] Where ReLU is the modified linear unit activation function, and W1 and W2 are two learnable weight parameters; i *W1,s j *W2> represents two vectors q i and s j Inner product operations between them;

[0026] The resulting pattern link graph S based on cosine similarity ij However, there may be a phenomenon where one database schema item corresponds to multiple question words. Therefore, only each database schema item s is retained. j The question word q with the highest similarity score i To improve the accuracy of pattern linking, an optimized pattern linking graph CS based on cosine similarity is obtained; CS is a two-dimensional matrix, where each element CS(i,j) represents a database pattern item s. j and the question word q i Similarity score between them; for each database pattern item s j ​Only the question word q with the highest similarity score is retained. i The other scores are set to 0;

[0027] Steps 2-3: Construct a semantically enhanced pattern link graph;

[0028] Joint modeling is performed on the HS and CS obtained in steps 2-1 and 2-2 to obtain a semantically enhanced pattern link graph; given the optimized hyperbolic space-based adjacency matrix HS and the optimized cosine similarity-based pattern link graph CS, the HS and CS are finally... Addition yields the final, complete pattern link graph. The similarity between the question word and the pattern item in this semantically enhanced pattern link graph is denoted as Q. ji Meanwhile, the integrated pattern link graph is used as the input to the encoder;

[0029] Step 3: Using an encoder, encode the semantically enhanced pattern link graph obtained in Step 2 to obtain an intermediate semantic vector;

[0030] The encoder is based on the Graph Attention Network (GAT) and incorporates a relational attention mechanism, taking into account the relationships between edges. It first processes a graph with relation type R and N nodes... The pattern link graph is taken as input, processed through layers, and output as the transformed feature vector matrix F(E) of all nodes. ji Secondly, attention weights are calculated using the features of the corresponding neighboring nodes. Next, a set of weights for neighboring nodes is calculated using the attention weights of the corresponding relationships. Simultaneously, the features to be passed back after the relationship attention calculation are obtained by multiplying the features of the corresponding neighboring nodes by their weights. Finally, the output of this layer is calculated by combining the attention weights of the nodes and the relationship attention weights with the features of the neighboring nodes. Specifically:

[0031]

[0032] The above formula describes how to calculate the attention weights between nodes using node features and a weight matrix; where α ji x represents the attention weight of node j towards node i. i and x j Represent the feature representations of node i and node j respectively, and the operators. First, the vector is uniformly divided into H parts, and the h-th partition is returned; [F(E ji The returned value is the eigenvector matrix, W. q W k These are trainable parameters;

[0033]

[0034] The above formula describes the process of normalizing and scaling the attention weights; This represents the scaled attention weight for the h-th partition. This represents the normalization of the vector matrix, meaning the unit vectors are normalized by softmax and scaled after applying the original attention weights. The scaling factor representing the feature dimension;

[0035]

[0036] The above formula describes the node encoding update process, where node features are aggregated through weighted summation to obtain a new representation of the node; where, Let F(E) represent the encoded representation of node i at layer l. This is obtained by weighted summation of the feature representations of neighboring nodes and the attention weights; ji The returned value is the eigenvector matrix, W. v These are trainable parameters;

[0037] Building upon the GAT encoder, in order to enhance the impact of the correlation between problem nodes and pattern nodes on graph encoding, relational semantic embedding is introduced during the encoding process to further enhance the influence of relational embedding.

[0038] The encoder simultaneously employs multi-head proportional dot product to calculate attention weights; since learnable relational embeddings are used to control the different influences between edge types, the similarity Q between problem nodes and pattern-oriented nodes calculated based on hyperbolic space and cosine similarity during the pattern linking stage is used. ji As an embedding to further enhance the relationship:

[0039] α′ ji =(x i W q (x) j W k +Q ji [F(E ji )]) T

[0040]

[0041]

[0042] α′ ji The attention weights describe the input vector x. i and x j The correlation score between them; It also represents attention weights, but after a softmax operation, the attention weights are converted into a probability distribution;

[0043] Then, the embedding relationships are calculated through normalization operations to generate... The output of the layer is represented as follows:

[0044]

[0045] Wherein, matrix W z These are trainable parameters. FFN(·) represents a feedforward neural network layer, and LayerNorm(·) represents a normalized layer.

[0046] Step 4: Use a decoder to decode the result obtained in step 3 to obtain the final SQL statement;

[0047] SQL statements are generated using a syntax-based decoder that uses LSTM output decoder operations, specifically: (i) expanding the last generated node into a syntax rule called APPYRULE, or, upon completion of a leaf node; and (ii) selecting columns / tables named SELECTCOLUMN and SELECTTABLE from the schema. The decoder interacts with three types of operations to generate an SQL query, including APPYRULE, SELECTCOLUMN, and SELECTTABLE. APPYRULE(r) applies the generation rule r to the current derivation tree of the query. SELECTCOLUMN(c) and SELECTTABLE(t) select column c and table t from the schema, respectively.

[0048] The decoder for pruning irrelevant tables first predicts a column using a pointer-enhanced network, then predicts the table it belongs to, and finally outputs in SQL form. During column prediction, memory is used to record the selected columns. When the decoder needs to select a column, it first decides whether to select it from memory, and then selects a column from memory or the schema based on that decision. Once a column is selected, it is removed from the schema and recorded in memory. Based on this, irrelevant tables are eliminated using the relationship between columns and tables; that is, if a column only appears in a specific table, the decoder directly selects the corresponding table by predicting this column, without considering other tables, thus choosing the most suitable table.

[0049] The specific operation is as follows: The decoder implements the SELECTCOLUMN operation through a memory-enhanced pointer network, with memory used to record the selected columns. When the decoder wants to select a column, it first decides whether to select from memory, and then selects a column from memory or the pattern based on the decision. Once a column is selected, it is removed from the pattern and recorded in memory. The probability of selecting column c is:

[0050] p(a j =SELECTCOLUMN[c]|a <j ,s,x)=

[0051] p(MEM|a <j ,s,x)p(c|a <j ,s,x,MEM)+p(S|a <j ,s,x)p(c|a <j ,s,x,S)

[0052] The above formula represents the conditional probability of selecting a specific output value "SELECTCOLUMN[c]" at position j; this probability is given by the input x, the state s, and the previous output a. <j In the given case, the probability of selecting "Memory" is multiplied by the probability of selecting the output value "c" given "Memory", and then the probability of selecting "S" is multiplied by the probability of selecting the output value "c" given "S"; where S represents selecting from the pattern and Memory represents selecting from memory.

[0053]

[0054] The above equation defines the probability of selecting "Memory" at position t, given input x, state s, and previous output a. <j As a weight vector and eigenvector q j The sigmoid function of the dot product; this means that the selection probability of "Memory" is obtained by transforming this dot product using the sigmoid function; where q j This represents the context vector obtained by performing attention on a series of hidden states encoded by the encoder input I. These are trainable parameters;

[0055] p(S|a <j ,s,x)=1-p(Memory|a <j ,s,x)

[0056] The above equation defines the probability of choosing "S" at position t, given input x, state s, and previous output a. <j This serves as the complement of the probability of choosing "Memory"; this means that when "Memory" is not chosen, "S" is chosen.

[0057]

[0058] The above formula defines the condition when "Memory" is selected, given an input x, a state s, and a previous output a. <jIn the case of "Memory", the probability of the output value "c" is related to the feature vector q. t and embedding matrix The probability of the output value "c" after selecting "Memory" is proportional to the exponent of the dot product; this means that the probability of the output value "c" is proportional to the exponent of the dot product after selecting "Memory"; where, This indicates embedding columns in memory;

[0059]

[0060] The above formula defines the condition when "S" is selected, given input x, state s, and previous output a. <j In the case of "S", the probability of the output value "c" is related to the eigenvector. and embedding matrix The exponent of the dot product is proportional to the eigenvector; specifically, the eigenvector... It is a feature vector associated with position t, which captures information about the input, state, and previous output, and is used to predict the next output value; embedding matrix It contains an embedding representation of the output value "c"; this embedding representation is seen as an abstract representation of the output value "c", which captures the semantic relationships and similarities between output values; Representing the eigenvector The dot product with the embedding representation of the output value "c"; this dot product is considered as the feature vector. With Embedded Representation A measure of similarity between them; This represents the exponent of the dot product; the exponential function maps the value of the dot product to the non-negative real number domain, while amplifying the difference; ∝ means "proportional to"; this means that the probability of the output value "c" is proportional to... The value of is directly proportional, meaning the probability is related to the magnitude of this exponent; where, Indicates the embedding of columns that have never been selected;

[0061] When SELECTTABLE is involved, the decoder selects a table t from the schema via a pointer network.

[0062]

[0063] The above equation describes the condition given input x, state s, and previous output a. <j In the case of "SELECTTABLE[t]", the probability of selecting "SELECTTABLE[t]" is related to the eigenvector q. j and embedding matrix E t The exponent of the dot product is proportional to the eigenvector q; where the eigenvector q is the exponent of the product. j is a feature vector associated with position j, which captures information from the input, state, and previous output, used to predict the next choice; embedding matrix E tIt contains an embedded representation of the selection "SELECTTABLE[t]"; this embedded representation is regarded as an abstract representation of the selection "SELECTTABLE[t]", which captures the semantic relationships and similarities between selections; Represents the eigenvector q j The dot product with the embedding representation of "SELECTTABLE[t]"; this dot product is considered as the feature vector q j With embedding representation E t A measure of similarity between them; This indicates the exponent of the dot product; the exponential function maps the value of the dot product to the non-negative real number domain, while amplifying the difference; ∝ means "proportional to"; this means that the probability of selecting "SELECTTABLE[t]" is proportional to... The value of is directly proportional to the magnitude of this exponent; therefore, given the input, state, and previous output, the probability of selecting "SELECTTABLE[t]" is related to the eigenvector q. j And the embedded representation of selecting "SELECTTABLE[t]" E t The similarity index is directly proportional to the similarity index;

[0064] Therefore, the decoder first predicts a column, and then predicts the table to which it belongs; to do this, it uses the relationship between columns and tables to prune irrelevant tables; based on the current derivation tree, and the predicted columns and tables, it generates SQL statements in a depth-first traversal order.

[0065] Beneficial technical effects of the present invention:

[0066] The proposed model employs a pipeline approach, simplifying the overall task complexity, reducing the model's learning pressure, and facilitating implementation. The entire invention is divided into two sub-modules: Module 1 constructs the pattern linking graph. This module decomposes the pattern linking task into two simple sub-tasks: one based on hyperbolic space and the other based on cosine similarity. Module 2 is the encoder-decoder model, which uses the results from Module 1 for encoding and decoding to obtain the final SQL statement.

[0067] In summary, this invention utilizes a multi-layer semantically enhanced NL2SQL model to convert natural language queries into corresponding SQL statements. This natural language interface allows many non-expert users to query data in the database using natural language, saving users the learning cost of SQL-related knowledge and the time cost of understanding each database table. This improves the accuracy and reduces the complexity of the model. Attached Figure Description

[0068] Figure 1A technical framework diagram of a natural language to SQL query task in the prior art according to embodiments of the present invention;

[0069] Figure 2 An overall framework diagram of a multi-layer semantic awareness enhanced NL2SQL model according to an embodiment of the present invention;

[0070] Figure 3 The embodiments of the present invention are based on a pattern link diagram in hyperbolic space;

[0071] Figure 4 This invention provides a pattern link diagram based on cosine similarity.

[0072] Figure 5 The encoder-decoder model structure diagram of this invention embodiment. Detailed Implementation

[0073] The present invention will be further described below with reference to the accompanying drawings and embodiments;

[0074] The challenges and pain points of current NL2SQL models or methods in complex contexts are as follows:

[0075] I. Accuracy: Existing technologies can convert natural language into SQL. However, these methods suffer from low accuracy when there is a mismatch between the natural language question words and the database schema. This mismatch may stem from differences in vocabulary selection, syntactic structure, and other aspects, which can affect the accuracy of the natural language to SQL conversion. Therefore, addressing these mismatches and improving accuracy is a challenging problem. A technical framework diagram of existing natural language to SQL query tasks is attached. Figure 1 As shown;

[0076] II. Complexity: Existing technologies generate SQL statements with high computational and search complexity, requiring searches of numerous tables. Improving SQL generation efficiency, pruning irrelevant tables, and removing redundant information are also challenges. Pruning irrelevant tables and removing redundant information is particularly important. In practical applications, existing data models often result in a large number of irrelevant tables and redundant information in the generated SQL statements. This not only increases query complexity but also impacts query performance. Therefore, it is necessary to consider how to prune irrelevant tables and remove redundant information. In short, improving SQL generation efficiency, pruning irrelevant tables, and removing redundant information is a significant challenge. In practical applications, it is necessary to select appropriate methods to optimize the generated SQL statements based on specific query requirements, thereby improving query performance.

[0077] A multi-layer semantically enhanced NL2SQL model comprises three parts: a semantically enhanced schema link graph, an encoder that embeds semantic relationships, and a decoder that considers pruning irrelevant tables; as shown in the appendix. Figure 2 As shown, in the stage of constructing the semantically enhanced pattern link graph, hyperbolic space is organically integrated into the existing method of constructing the pattern link graph based on similarity, and a pattern link graph coupled with hyperbolic space and cosine similarity is constructed. The semantically enhanced pattern link graph obtained in this stage is transmitted to the encoder module for semantic relation embedding. The encoder module converts the natural language query into an intermediate semantic vector. In this process, similarity is used as a further enhanced relation embedding. Finally, the processing result of the encoder module is sent to the decoder module that considers pruning irrelevant tables, and then the final SQL statement is obtained.

[0078] A multi-layer semantic awareness enhancement method, based on the aforementioned multi-layer semantic awareness enhancement NL2SQL model, specifically includes the following steps:

[0079] Step 1: Define the Natural Language Problem and Database Schema Items. The first step in defining the Natural Language Processing (NLP) and database schema items is to clearly define the problem. First, it's necessary to clearly understand the specific content and scope of the NLP problem so that it can be accurately transformed into database schema items. This includes identifying and understanding the keywords, entities, and relationships of the problem. Second, defining the database schema items is also crucial. This involves clearly defining the tables, fields, relationships, and other elements in the database so that they can accurately correspond and match with the NLP problem. During the problem definition process, the consistency and matching degree between the NLP problem and the database schema items must be fully considered to ensure the smooth progress of subsequent NLP processing and database queries. Therefore, the key to Step 1 is the accurate definition of the problem, which will lay a solid foundation for subsequent work. Formally, given an NLP problem Q and a corresponding database schema S = ...<T,C> Therefore, the input is I =<Q,S> The output target is to generate an SQL query Y; specifically, the problem Q = {q1,q2,…,q} |Q|} is a token-level sequence of question |Q|; where the database schema consists of table |T| and columns |C|, where T = {t1, t2, ..., t} |T|}, C = {c1, c2, ..., c |C| Each table ti consists of a name containing multiple words: Table t i Each column in They are all composed of words or a phrase; The last word of table ti, that is, the last word of the table name; This represents the name of the j-th column in table ti, i.e., the column name.

[0080] Step 2: Constructing a Semantically Enhanced Pattern Link Graph: Constructing a semantically enhanced pattern link graph is a crucial step in the model. This step utilizes advanced technologies and algorithms to build a pattern link graph capable of understanding and processing semantic information. This graph helps to better understand the semantic relationships within the text, thereby improving the efficiency and accuracy of text processing and understanding.

[0081] Step 2-1: Construct a pattern link graph based on hyperbolic space optimization;

[0082] Hyperbolic space is used in word vector representation to capture the relationships between words. Traditional word vector representation methods such as Word2Vec and GloVe use Euclidean space to represent the similarity between words, but this representation cannot handle some complex semantic relationships well. For example, the relationship between "man" and "woman" is similar to the relationship between "king" and "queen," but their distances are equal in Euclidean space. Therefore, hyperbolic space is proposed to represent word vectors. In hyperbolic space, the distance between words can better capture their semantic relationships. For example, word vectors represented using hyperbolic space can better reflect the similarity between the relationship between "man" and "woman" as the relationship between "king" and "queen." This representation can improve the performance of natural language processing tasks, such as word semantic similarity calculation and word association analysis.

[0083] Given a database schema item S =<T,C> Where T is the corresponding database table, T = {t1, t2, ..., t} |T|}, where C is a column in the corresponding table, C = {c1, c2, ..., c |C| First, concatenate the database tables and columns: S = (T, C) = <s1,s2,…s |T|+|C| Then, the question word is concatenated with S, serialized, and used as input I:

[0084] I = (q1, q2, ..., q |Q| ,s1,s2,…,s |T|+|C| )

[0085] The core of constructing a pattern link graph using hyperbolic space is determining the relevance between question words and given database pattern items. A pattern link graph optimized using hyperbolic space is shown below. Figure 3 As shown, first, input I is input into the PLM to obtain the context representation c of the j-th pattern item. j At the same time, replace the question word q with the [MASK] marker. iThe new question word sequence containing the [MASK] marker is re-inputted into the PLM. At this time, the PLM will obtain the result when the question word q is... i When blocked, the j-th pattern item s j New contextual representation The pattern term s is measured by measuring the difference between two context representations. j and the question word q i The correlation between them; where the distance between the two contexts represents c. j , The difference between the two context representations is measured using the Poincaré distance in hyperbolic space. j , The distance between them; hyperbolic space is a type of differential manifold, with negative constant curvature at each location, and is commonly described by the Poincaré disk model. Compared to Euclidean (planar) space, it can capture the hierarchical structure of language encoded in context representations very well. Furthermore, vector comparisons can be performed in hyperbolic space with minimal distortion.

[0086] In order to represent two context vectors c in hyperbolic space j , To compare them, we first need to compare the two vectors c. j , Mapping to hyperbolic space yields the corresponding representation c in hyperbolic space. j ′、 The mapping function is represented as M():FvI→HvI, where M() is expressed as:

[0087]

[0088] μ represents a coefficient used to adjust the degree of processing of the input vector;

[0089] The vector representation c in hyperbolic space is obtained. j ′、 Then, the problem word q is measured by measuring the Poincaré distance between the two vector representations. i and pattern item s j Correlation between them:

[0090]

[0091] This leads to a question word q. i and pattern item s j The relationship matrix between them is used, but the pattern link graph based on hyperbolic space is affected by low-confidence noise. Therefore, a predefined threshold h is introduced. Items below the predefined threshold are considered noise, i.e., items less than the threshold are set to h(q). i ,sj If h(q) = 0, otherwise it is 1. Therefore, the noisy pattern linking results are initially filtered out, resulting in an adjacency matrix HS, which is represented as the optimized hyperbolic space-based pattern linking graph. The adjacency matrix HS is a two-dimensional matrix, where each element h(q) = 0. i ,s j ) indicates the question word q i and pattern item s j The relationship between h(q); i ,s j If the value is less than the predefined threshold h, it is set to 0 to indicate noise; otherwise, it is set to 1 to indicate a relationship. Therefore, the specific form of the adjacency matrix HS is a matrix composed of 0 and 1, where 0 represents noise and 1 represents a relationship.

[0092] Step 2-2: Construct a pattern link graph based on cosine similarity optimization;

[0093] To further enhance the construction of the optimization pattern link graph, the goal of this module is to learn the question word q. i and pattern item s j The implicit matrix between them. Cosine similarity is the cosine angle between two n-dimensional vectors in n-dimensional space. It is the dot product of the two vectors divided by their lengths (or sizes). The most common application of cosine similarity is calculating text similarity. By creating two vectors for each word in two texts and calculating the cosine of these two vectors, we can determine the statistical similarity between the two texts. Assuming there are two points, as the distance between them increases, their similarity decreases, and vice versa. The angle result will show the outcome. If the angle between the text vectors is 0, the cosine function is 1, and the two texts are identical. If the angle is any other value, the cosine function will be less than 1. Therefore, calculating the cosine angle between two vectors determines whether they point in the same direction.

[0094] Cosine similarity is used to measure the similarity of the question word q. i and pattern item s j The correlation between them, and the pattern link graph based on cosine similarity optimization, are as follows: Figure 4 As shown. First, by calculating the question word q... i and pattern item s j The cosine similarity between them yields the similarity matrix S:

[0095]

[0096] Where ReLU is the modified linear unit activation function, and W1 and W2 are two learnable weight parameters; i *W1,s j *W2> represents two vectors q i and s​j Inner product operations between them;

[0097] The resulting pattern link graph S based on cosine similarity ij However, there may be a phenomenon where one database schema item corresponds to multiple question words. Therefore, only each database schema item s is retained. j The question word q with the highest similarity score i To improve the accuracy of pattern linking, an optimized pattern linking graph CS based on cosine similarity is obtained; CS is a two-dimensional matrix, where each element CS(i,j) represents a database pattern item s. j and the question word q i Similarity score between them; for each database pattern item s j Only the question word q with the highest similarity score is retained. i The other scores are set to 0;

[0098] Steps 2-3: Construct a semantically enhanced pattern link graph;

[0099] Joint modeling is performed on the HS and CS obtained in steps 2-1 and 2-2 to obtain a semantically enhanced pattern link graph; given the optimized hyperbolic space-based adjacency matrix HS and the optimized cosine similarity-based pattern link graph CS, the HS and CS are finally... Addition yields the final, complete pattern link graph. The similarity between the question word and the pattern item in this semantically enhanced pattern link graph is denoted as Q. ji Meanwhile, the integrated pattern link graph is used as the input to the encoder;

[0100] In the optimization process of the pattern link graph, the adjacency matrix HS based on hyperbolic space and the pattern link graph CS based on cosine similarity were first obtained. These two optimized pattern link graphs represent the complex relationships and similarities between data. To further improve the quality of the pattern link graph, HS and CS were further optimized. By adding elements and integrating their relationships, a final and complete pattern link diagram is obtained. Addition is a commonly used graph theory operation that can merge the relationships between two graphs to obtain a new graph. Addition integrates the relationships between HS and CS, resulting in a more comprehensive and accurate final pattern link graph. This step successfully integrates and refines the information from the two optimized pattern link graphs, yielding a more comprehensive understanding of data relationships. The final, improved pattern link graph will serve as input to the encoder described below. This means that the rich information contained within the pattern link graph will be utilized to train and optimize the encoder model. By using the integrated pattern link graph as input, it is expected that the encoder will better understand the relationships and features between the data, thereby improving its performance and accuracy.

[0101] Step 3: Using an encoder, encode the semantically enhanced pattern link graph obtained in Step 2 to obtain an intermediate semantic vector;

[0102] The GAT encoder can be used to process semantic relationships in natural language problems and encode them into a graph structure. Then, the GAT encoder can utilize graph attention mechanisms to encode the input semantic graph to better capture the relationships and importance between nodes. Specifically, for the NL2SQL task, the GAT encoder can be used as follows: First, construct a semantic graph: treat words or phrases from the natural language problem as nodes and construct a graph structure based on their semantic relationships. Second, use the GAT encoder: input the constructed semantic graph into the GAT encoder, and use the GAT model to encode the nodes in the graph, considering the relationships between nodes and attention weights. Finally, output the encoded results: the GAT encoder will output the encoded representation of each node, which can be used as input for subsequent query generation models. By using the GAT encoder, semantic information in natural language problems can be better captured and converted into a structured representation for database queries. However, this method does not consider the dependencies between nodes.

[0103] The encoder is based on the Graph Attention Network (GAT) and incorporates a relational attention mechanism, taking into account the relationships between edges. It first processes a graph with relation type R and N nodes... The pattern link graph is taken as input, processed through layers, and output as the transformed feature vector matrix F(E) of all nodes. ji Secondly, attention weights are calculated using the features of the corresponding neighboring nodes. Next, a set of weights for neighboring nodes is calculated using the attention weights of the corresponding relationships. Simultaneously, the features to be passed back after the relationship attention calculation are obtained by multiplying the features of the corresponding neighboring nodes by their weights. Finally, the output of this layer is calculated by combining the attention weights of the nodes and the relationship attention weights with the features of the neighboring nodes. Specifically:

[0104]

[0105] The above formula describes how to calculate the attention weights between nodes using node features and a weight matrix; where α ji x represents the attention weight of node j towards node i. i and x j Represent the feature representations of node i and node j respectively, and the operators. First, the vector is uniformly divided into H parts, and the h-th partition is returned; [F(E ji The returned value is the eigenvector matrix, W. q W k These are trainable parameters;

[0106]

[0107] The above formula describes the process of normalizing and scaling the attention weights; This represents the scaled attention weight for the h-th partition. This represents the normalization of the vector matrix, meaning the unit vectors are normalized by softmax and scaled after applying the original attention weights. The scaling factor representing the feature dimension;

[0108]

[0109] The above formula describes the node encoding update process, where node features are aggregated through weighted summation to obtain a new representation of the node; where, Let F(E) represent the encoded representation of node i at layer l. This is obtained by weighted summation of the feature representations of neighboring nodes and the attention weights; ji The returned value is the eigenvector matrix, W. v These are trainable parameters;

[0110] Building upon the GAT encoder, in order to enhance the impact of the correlation between problem nodes and pattern nodes on graph encoding, relational semantic embedding is introduced during the encoding process to further enhance the influence of relational embedding.

[0111] The encoder simultaneously employs multi-head proportional dot product to calculate attention weights; since learnable relational embeddings are used to control the different influences between edge types, the similarity Q between problem nodes and pattern-oriented nodes calculated based on hyperbolic space and cosine similarity during the pattern linking stage is used. ji As an embedding to further enhance the relationship:

[0112] α′ ji =(x i W q (x) j W k+Q ji [F(E ji )]) T

[0113] Formula α′ ji This represents the attention weight from node j to node i, which is based on the node's features and the learned weight matrix W. q W k and W v The calculation yielded a similarity Q between problem nodes and pattern-oriented nodes, which was also introduced. ji This leads to the encoding of graph semantic relations.

[0114]

[0115] formula By applying softmax normalization and according to Scaling attention weights obtained by scaling

[0116]

[0117] formula This represents the updated encoding of node i at layer l', obtained by weighted aggregation of features from neighboring nodes, where the weights are α′. ji It also introduces the similarity Q between problem nodes and database schema item nodes. ji .

[0118] Then, the embedded relationships are calculated using a similar method to generate... The output of the layer is represented as follows:

[0119]

[0120] Wherein, the update code of node i It is through the encoding of its upper layer. This is obtained by processing a weighted sum of data from a feedforward neural network (FFN). Specifically, it involves first processing the encoding of node i in the previous layer. and normalized encoding after residual connection Perform summation. Normalize the sum using LayerNorm. Input the normalized result into a feedforward neural network for further processing, where W... z This is the weight matrix of the network, i.e., the training parameters. The output of the feedforward neural network is used as the update encoding for node i in the next layer. The formula describes the update process of node encodings in the model, including residual connections, normalization, and feedforward neural network processing. These steps help improve the model's ability to extract features and learn representations for graph-structured data.

[0121] Step 4: Use the decoder to decode the result obtained in Step 3 to obtain the final SQL statement; the encoder-decoder model structure diagram is attached. Figure 5 As shown;

[0122] SQL statements are generated using a syntax-based decoder that uses LSTM output decoder operations, specifically: (i) expanding the last generated node into a syntax rule called APPYRULE, or, upon completion of a leaf node; and (ii) selecting columns / tables named SELECTCOLUMN and SELECTTABLE from the schema. The decoder interacts with three types of operations to generate an SQL query, including APPYRULE, SELECTCOLUMN, and SELECTTABLE. APPYRULE(r) applies the generation rule r to the current derivation tree of the query. SELECTCOLUMN(c) and SELECTTABLE(t) select column c and table t from the schema, respectively.

[0123] The decoder for pruning irrelevant tables first predicts a column using a pointer-enhanced network, then predicts the table it belongs to, and finally outputs in SQL form. During column prediction, memory is used to record the selected columns. When the decoder needs to select a column, it first decides whether to select it from memory, and then selects a column from memory or the schema based on that decision. Once a column is selected, it is removed from the schema and recorded in memory. Based on this, irrelevant tables are eliminated using the relationship between columns and tables; that is, if a column only appears in a specific table, the decoder directly selects the corresponding table by predicting this column, without considering other tables, thus choosing the most suitable table.

[0124] The specific operation is as follows: The decoder implements the SELECTCOLUMN operation through a memory-enhanced pointer network, with memory used to record the selected columns. When the decoder wants to select a column, it first decides whether to select from memory, and then selects a column from memory or the pattern based on the decision. Once a column is selected, it is removed from the pattern and recorded in memory. The probability of selecting column c is:

[0125] p(a j =SELECTCOLUMN[c]|a <j ,s,x)=

[0126] p(MEM|a <j ,s,x)p(c|a <j ,s,x,MEM)+p(S|a <j ,s,x)p(c|a <j ,s,x,S)

[0127] The above formula represents the conditional probability of selecting a specific output value "SELECTCOLUMN[c]" at position j; this probability is given by the input x, the state s, and the previous output a. <j In the given case, the probability of selecting "Memory" is multiplied by the probability of selecting the output value "c" given "Memory", and then the probability of selecting "S" is multiplied by the probability of selecting the output value "c" given "S"; where S represents selecting from the pattern and Memory represents selecting from memory.

[0128]

[0129] The above equation defines the probability of selecting "Memory" at position t, given input x, state s, and previous output a. <j As a weight vector and eigenvector q j The sigmoid function of the dot product; this means that the selection probability of "Memory" is obtained by transforming this dot product using the sigmoid function; where q j This represents the context vector obtained by performing attention on a series of hidden states encoded by the encoder input I. These are trainable parameters; during training, the neural network updates the weight vector using the backpropagation algorithm. This minimizes the model's loss function.

[0130] p(S|a <j ,s,x)=1-p(Memory|a <j ,s,x)

[0131] The above equation defines the probability of choosing "S" at position t, given input x, state s, and previous output a. <j This serves as the complement of the probability of choosing "Memory"; this means that when "Memory" is not chosen, "S" is chosen.

[0132]

[0133] The above formula defines the condition when "Memory" is selected, given an input x, a state s, and a previous output a. <j In the case of "Memory", the probability of the output value "c" is related to the feature vector q. t and embedding matrix The probability of the output value "c" after selecting "Memory" is proportional to the exponent of the dot product; this means that the probability of the output value "c" is proportional to the exponent of the dot product after selecting "Memory"; where, This indicates embedding columns in memory;

[0134]

[0135] The above formula defines the condition when "S" is selected, given input x, state s, and previous output a. <j In the case of "S", the probability of the output value "c" is related to the eigenvector. and embedding matrix The exponent of the dot product is proportional to the eigenvector; specifically, the eigenvector... It is a feature vector associated with position t, which captures information about the input, state, and previous output, and is used to predict the next output value; embedding matrix It contains an embedding representation of the output value "c"; this embedding representation is seen as an abstract representation of the output value "c", which captures the semantic relationships and similarities between output values; Representing the eigenvector The dot product with the embedding representation of the output value "c"; this dot product is considered as the feature vector. With Embedded Representation A measure of similarity between them; This represents the exponent of the dot product; the exponential function maps the value of the dot product to the non-negative real number domain, while amplifying the difference; ∝ means "proportional to"; this means that the probability of the output value "c" is proportional to... The value of is directly proportional, meaning the probability is related to the magnitude of this exponent; where, Indicates the embedding of columns that have never been selected;

[0136] and The embedding vectors for these columns are typically obtained by encoding the column information. During training, the model learns how to transform column information into fixed-length embedding vectors. This process can be achieved by inputting column features (such as column name, type, and position) into a neural network, which then learns to obtain the column embedding vectors. These embedding vectors help the model better understand the semantics and context of the columns. This is a trainable parameter vector, typically used as parameters in the attention mechanism. During training, the model learns how to adjust these parameter vectors to better compute the attention weights. These parameter vectors are usually obtained by random initialization and then learning through backpropagation.

[0137] When SELECTTABLE is involved, the decoder selects a table t from the schema via a pointer network.

[0138]

[0139] The above equation describes the condition given input x, state s, and previous output a. <jIn the case of "SELECTTABLE[t]", the probability of selecting "SELECTTABLE[t]" is related to the eigenvector q. j and embedding matrix E t The exponent of the dot product is proportional to the eigenvector q; where the eigenvector q is the exponent of the product. j is a feature vector associated with position j, which captures information from the input, state, and previous output, used to predict the next choice; embedding matrix E t It contains an embedded representation of the selection "SELECTTABLE[t]"; this embedded representation is regarded as an abstract representation of the selection "SELECTTABLE[t]", which captures the semantic relationships and similarities between selections; Represents the eigenvector q j The dot product with the embedding representation of "SELECTTABLE[t]"; this dot product is considered as the feature vector q j With embedding representation E t A measure of similarity between them; ∝ represents the exponent of the dot product; the exponential function maps the value of the dot product to the non-negative real number domain, while amplifying the difference; ∝ means "proportional to"; this means that the probability of selecting "SELECTTABLE[t]" is proportional to ∝ ... The value of is directly proportional to the magnitude of this exponent; therefore, given the input, state, and previous output, the probability of selecting "SELECTTABLE[t]" is related to the eigenvector q. j And the embedded representation of selecting "SELECTTABLE[t]" E t The similarity index is directly proportional to the similarity index;

[0140] Therefore, the decoder first predicts a column, and then predicts the table to which it belongs; to do this, it uses the relationship between columns and tables to prune irrelevant tables; based on the current derivation tree, and the predicted columns and tables, it generates SQL statements in a depth-first traversal order.

[0141] The matching accuracy of the method in this application and other existing methods is compared below, as shown in the table below;

[0142] Table 1. Exact match accuracy (%) in three different datasets;

[0143]

[0144] For a comprehensive comparison, several state-of-the-art baseline models were selected, including: IRNet, which generates a SemQL using a syntax-based neural parsing model and determines the domain knowledge corresponding to the SQL query; RATSQL, a relation-aware self-attention framework applicable to functional representations in NL2SQL tasks; ETA, a weakly supervised method that eliminates and then re-enables potential links from the PLM; LGESQL, which uses line graphs to mine relational information from underlying meta-paths; and RYANSQL, which uses the Sketch method, defining a basic SQLSketch and incorporating SPC (Statement Position Code) to recursively solve subquery problems. 2 SQL injects syntactic information into the parser of the Question-Schema-based graph encoder in NL2SQL, effectively utilizing the syntactic dependency information of NL2SQL questions.

[0145] The model of this invention was compared with some recent state-of-the-art methods, and experimental results show that the method of this invention has significant improvements. The improvement in accuracy is particularly significant, indicating that the model of this invention has a significant performance improvement on the NL2SQL dataset and effectively solves the problem of insufficient accuracy in Text-to-SQL problems.

Claims

1. A multi-layer semantic awareness enhancement method, implemented based on the multi-layer semantic awareness enhancement NL2SQL model, characterized in that, Specifically, the following steps are included: Step 1: Define the natural language problem and the database schema item; Step 2: Construct a semantically enhanced pattern link graph; Step 3: Using an encoder, encode the semantically enhanced pattern link graph obtained in Step 2 to obtain an intermediate semantic vector; Step 4: Use a decoder to decode the result obtained in step 3 to obtain the final SQL statement; Step 2 specifically involves: Step 2-1, constructing a pattern link graph HS based on hyperbolic space optimization; Step 2-2: Construct a pattern link graph CS based on cosine similarity optimization; Steps 2-3: Construct a semantically enhanced pattern link graph; Jointly model the HS and CS obtained in steps 2-1 and 2-2 to obtain a semantically enhanced pattern link graph. Given the optimized hyperbolic space-based adjacency matrix HS and the optimized cosine similarity-based pattern link graph CS, perform Möbius addition on HS and CS to obtain the final, complete pattern link graph. The similarity between the question word and the pattern item in the semantically enhanced pattern link graph is then denoted as . Meanwhile, the integrated pattern link graph is used as the input to the encoder; The multi-layer semantically enhanced NL2SQL model comprises three parts: constructing a semantically enhanced pattern link graph, an encoder for semantic relation embedding, and a decoder that considers pruning irrelevant tables. In the stage of constructing the semantically enhanced pattern link graph, hyperbolic space is organically integrated into the existing method of constructing the pattern link graph based on similarity, resulting in a pattern link graph coupled with hyperbolic space and cosine similarity. The semantically enhanced pattern link graph obtained in this stage is transmitted to the encoder module for semantic relation embedding. The encoder module converts the natural language query into an intermediate semantic vector, incorporating similarity as a further enhanced relation embedding during this process. Finally, the processing result from the encoder module is sent to the decoder module that considers pruning irrelevant tables, thereby obtaining the final SQL statement.

2. The method for multi-layer semantic perception enhancement according to claim 1, characterized in that, Step 1 defines the natural language problem and the database schema item, specifically as follows: Given an NL problem Q and the corresponding database schema S=<T,C> Therefore, the input is I =<Q,S> The output target is to generate an SQL query Y; specifically, the problem That is the problem. A token-level sequence; where the database schema consists of table |T| and column |C|, where , Each table They are all composed of multiple words in their names: [ , ,···, ];surface Each column in They are all composed of words or a phrase; Representation Table The last word, that is, the last word of the table name; Representation Table The name of the j-th column in the table, i.e., the column name.

3. The method for multi-layer semantic perception enhancement according to claim 1, characterized in that, Step 2-1 is as follows: Given a database schema item S=<T,C> Where T is the corresponding database table, C is the column in the corresponding table. First, concatenate the tables and columns in the database. Then the question word is concatenated with S, and the serialized result is used as input I: Input I into the PLM to obtain the context representation of the j-th pattern item. At the same time, replace the question words with the [MASK] tag. The new question word sequence containing the [MASK] marker is re-inputted into the PLM. The PLM will then retrieve the current question word sequence. When blocked, the j-th pattern item New contextual representation ; The pattern item is measured by measuring the difference between two contextual representations. and question words The correlation between them; where the distance between the two contexts represents the relationship. , The difference between the two context representations is measured using the Poincaré distance in hyperbolic space. , The distance between them; To represent two context vectors in hyperbolic space , To compare them, the two vectors must first be... , Mapping to hyperbolic space yields the corresponding representation in hyperbolic space. The mapping function is represented as M():FvI→HvI, where M() is expressed as: Used to adjust the degree of processing of the input vector; To obtain the vector representation in hyperbolic space Then, the problem words are measured by measuring the Poincaré distance between the two vector representations. and pattern items Correlation between them: This leads to a question word. and pattern items The relationship matrix between them is used, but the pattern link graph based on hyperbolic space is affected by low-confidence noise. Therefore, a predefined threshold h is introduced. Entries below the predefined threshold are considered noise, i.e., those less than the threshold are set to noise. Otherwise, it is 1, thus initially filtering out noisy pattern linking results, resulting in an adjacency matrix HS represented as the optimized hyperbolic space-based pattern linking graph; the adjacency matrix HS is a two-dimensional matrix, each element Indicating problem words and pattern items The relationship between them; if If the value is less than the predefined threshold h, it is set to 0 to indicate noise; otherwise, it is set to 1 to indicate a relationship. Therefore, the specific form of the adjacency matrix HS is a matrix composed of 0 and 1, where 0 represents noise and 1 represents a relationship.

4. The method for multi-layer semantic perception enhancement according to claim 1, characterized in that, Step 2-2 specifically involves: Using cosine similarity to measure the similarity of problem words and pattern items The correlation between them is first calculated by question words. and pattern items The cosine similarity between them yields the similarity matrix S: Where ReLU is the modified linear unit activation function. and These are two learnable weight parameters; > represents two vectors and Inner product operations between them; The resulting pattern link graph based on cosine similarity However, there may be a situation where one database schema item corresponds to multiple question words. Therefore, only each database schema item is retained. The question word with the highest similarity score This improves the accuracy of pattern linking, resulting in an optimized pattern linking graph CS based on cosine similarity. CS is a two-dimensional matrix, where each element CS(i,j) represents a database pattern item. and question words Similarity score between them; for each database pattern item Only the question word with the highest similarity score is retained. The scores for the other scores are set to 0.

5. The method for multi-layer semantic perception enhancement according to claim 1, characterized in that, The encoder described in step 3 is based on the Graph Attention Network (GAT), with the addition of a relational attention mechanism to consider the relationships between edges; firstly, it takes a graph with R relation types and N nodes... The pattern link graph is taken as input, processed through layers, and the output is the transformed feature vector matrix of all nodes. Secondly, attention weights are calculated using the features of the corresponding neighboring nodes. Next, a set of weights for neighboring nodes is calculated using the attention weights of the corresponding relationships. Simultaneously, the features to be passed back after the relationship attention calculation are obtained by multiplying the features of the corresponding neighboring nodes by their weights. Finally, the output of this layer is calculated by combining the attention weights of the nodes and the relationship attention weights with the features of the neighboring nodes. Specifically: The above formula describes how to calculate the attention weights between nodes using node features and a weight matrix; where, This represents the attention weight of node j towards node i. and Represent the feature representations of node i and node j respectively, and the operators. First, the vector is uniformly divided into H parts, and the h-th partition is returned; The returned value is the eigenvector matrix. , These are trainable parameters; The above formula describes the process of normalizing and scaling the attention weights; This represents the scaled attention weight for the h-th partition. This represents the normalization of the vector matrix, meaning the unit vectors are normalized by softmax and scaled after applying the original attention weights. The scaling factor representing the feature dimension; The above formula describes the node encoding update process, where node features are aggregated through weighted summation to obtain a new representation of the node; where, The encoding representation of node i at layer l is obtained by weighted summation of the feature representations of neighboring nodes and attention weights. The returned value is the eigenvector matrix. These are trainable parameters; Building upon the GAT encoder, in order to enhance the impact of the correlation between problem nodes and pattern nodes on graph encoding, relational semantic embedding is introduced during the encoding process to further enhance the influence of relational embedding. The encoder simultaneously employs multi-head proportional dot product to calculate attention weights; since learnable relational embeddings are used to control the different influences between edge types, the similarity between problem nodes and pattern-oriented nodes calculated based on hyperbolic space and cosine similarity during the pattern linking stage is used. As an embedding to further enhance the relationship: The attention weights represent the input vector. and The correlation score between them; It also represents attention weights, but after a softmax operation, the attention weights are converted into a probability distribution; Then, the embedded relationships are calculated through normalization operations to generate... The output of the layer is represented as follows: Among them, matrix These are trainable parameters; FFN(·) represents a feedforward neural network layer. This represents a normalization layer.

6. The method for multi-layer semantic perception enhancement according to claim 1, characterized in that, Step 4 specifically involves: SQL statements are generated using a syntax-based decoder that uses LSTM output decoder operations, specifically: (i) expanding the last generated node into a syntax rule called APPYRULE, or when the leaf node is completed, (ii) selecting a column / table named SELECTCOLUMN and SELECTTABLE from the schema. The decoder interacts with three types of operations to generate an SQL query, including APPYRULE, SELECTCOLUMN, and SELECTTABLE; APPYRULE(r) applies the generation rule r to the current derivation tree of the query; SELECTCOLUMN(c) and SELECTTABLE(t) select column c and table t from the schema, respectively; The decoder for pruning irrelevant tables first predicts a column using a pointer-enhanced network, then predicts the table it belongs to, and finally outputs in SQL form. During column prediction, memory is used to record the selected columns. When the decoder needs to select a column, it first decides whether to select it from memory, and then selects a column from either memory or the schema based on that decision. Once a column is selected, it is removed from the schema and recorded in memory. Based on this, irrelevant tables are eliminated using the relationship between columns and tables; that is, if a column only appears in one specific table, the decoder can directly select the corresponding table by predicting this column, without considering other tables, thus choosing the most suitable table. The specific operation is as follows: The decoder implements the SELECTCOLUMN operation through a memory-enhanced pointer network, with memory used to record the selected columns. When the decoder wants to select a column, it first decides whether to select from memory, and then selects a column from memory or the pattern based on the decision. Once a column is selected, it is removed from the pattern and recorded in memory. The probability of selecting column c is: The above formula represents the conditional probability of selecting the specific output value "SELECTCOLUMN[c]" at position j; This probability is given the input x, the state s, and the previous output. In the case of "Memory", the probability of selecting "Memory" is multiplied by the probability of selecting the output value "c" given "Memory", and then the probability of selecting "S" is multiplied by the probability of selecting the output value "c" given "S"; where S represents selecting from the pattern and Memory represents selecting from memory. The above equation defines the probability of selecting "Memory" at position t, given input x, state s, and previous output. As a weight vector and eigenvectors The sigmoid function of the dot product; this means that the selection probability of "Memory" is obtained by transforming this dot product using the sigmoid function; where, This represents the context vector obtained by performing attention on a series of hidden states encoded by the encoder input I. These are trainable parameters; The above equation defines the probability of choosing "S" at position t, given input x, state s, and previous output. This is the complement of the probability of choosing "Memory"; this means that when "Memory" is not chosen, "S" is chosen. The above formula defines the condition when "Memory" is selected, given an input x, a state s, and a previous output. In the case of "Memory", the probability of the output value "c" and the eigenvector and embedding matrix The probability of the output value "c" after selecting "Memory" is proportional to the exponent of the dot product; this means that the probability of the output value "c" is proportional to the exponent of the dot product after selecting "Memory"; where, This indicates embedding columns in memory; The above formula defines the condition when "S" is selected, given input x, state s, and previous output. In the case of "S", the probability of the output value "c" is related to the eigenvector. and embedding matrix The exponent of the dot product is proportional to the eigenvector; specifically, the eigenvector... It is a feature vector associated with position t, which captures information about the input, state, and previous output, and is used to predict the next output value; embedding matrix It contains an embedding representation of the output value "c"; this embedding representation is seen as an abstract representation of the output value "c", which captures the semantic relationships and similarities between output values; Representing the eigenvector The dot product with the embedding representation of the output value "c"; this dot product is considered as the feature vector. With Embedded Representation A measure of similarity between them; This represents the exponent of the dot product; the exponential function maps the value of the dot product to the non-negative real number domain while amplifying the difference. This means "proportional to"; it implies that the probability of the output value "c" is proportional to... The value of is directly proportional, meaning the probability is related to the magnitude of this exponent; where, Indicates the embedding of columns that have never been selected; When it comes to SELECTTABLE, the decoder selects a table t from the schema via a network of pointers. The above equation describes the condition given input x, state s, and previous output. In the case of "SELECTTABLE[t]", the probability of selecting "SELECTTABLE[t]" and the eigenvectors and embedding matrix The exponent of the dot product is proportional to the eigenvector; where the eigenvector is... is a feature vector associated with position j, capturing information from the input, state, and previous output, used to predict the next choice; embedding matrix It contains an embedded representation of the selection "SELECTTABLE[t]"; this embedded representation is regarded as an abstract representation of the selection "SELECTTABLE[t]", which captures the semantic relationships and similarities between selections; Representing the eigenvector The dot product with the embedding representation of "SELECTTABLE[t]"; this dot product is considered as the feature vector. With Embedded Representation A measure of similarity between them; This represents the exponent of the dot product; the exponential function maps the value of the dot product to the non-negative real number domain while amplifying the difference. This means "proportional to"; it implies that the probability of selecting "SELECTTABLE[t]" is proportional to... The value of the index is directly proportional to the probability; that is, the probability is related to the magnitude of this index. Therefore, given the input, state, and previous output, the probability of selecting "SELECTTABLE[t]" is proportional to the value of the eigenvector. And the embedded representation of selecting "SELECTTABLE[t]" The similarity index is directly proportional to the similarity index; Therefore, the decoder first predicts a column, and then predicts the table to which it belongs; to do this, it uses the relationship between columns and tables to prune irrelevant tables; based on the current derivation tree, and the predicted columns and tables, it generates SQL statements in a depth-first traversal order.

Images