Method and framework for decoding sql syntax tree based on structure perception

Abstract

The embodiment of the application provides a decoding method and framework of a SQL syntax tree based on structure perception. The method comprises the following steps: inputting an encoding state of text-to-SQL into a node input module, and using the node input module to construct absolute position encoding of a front node in an abstract syntax tree of text-to-SQL at a current time; inputting the absolute position encoding into a feature encoding module constructed by at least a mask self-attention module fusing relative position relations, to obtain node features with relative position relations between the front node and other nodes which have been expanded in the abstract syntax tree; determining an expansion action probability distribution of the front node based on the node features; and expanding and updating the abstract syntax tree through the expansion action probability distribution and the encoding state, until all the front nodes in the abstract syntax tree are expanded, to obtain a decoded abstract syntax tree. The embodiment of the application restores the complete syntax tree structure through the relative position and absolute position encoding between nodes.

Description

Technical Field

[0001] This invention relates to the field of intelligent voice, and in particular to a decoding method and framework based on structure-aware SQL syntax trees. Background Technology

[0002] Text-to-SQL is the task of converting a natural language question into an executable SQL program, given a database schema. For example, a user's question in an interaction with a smart voice, "Can you check how much money is left in my account?", is a task of converting a natural language question into executable SQL (a simple example is SELECT moneyFROM account, which means retrieving the amount from the account table).

[0003] In existing technologies, deep learning models are typically used to generate SQL programs in an end-to-end manner. These include two main decoding schemes: symbol-based decoding and syntax-based decoding. Symbol-based decoding treats the SQL program as a string sequence, generating it word by word in a manner similar to text generation, with SQL keywords like SELECT and FROM included as words in the target vocabulary. Syntax-based decoding first generates an equivalent abstract syntax tree of the SQL program, and then converts it into SQL through post-processing. Based on this classification, the above models can be further subdivided into:

[0004] 1. BRIDGE: Symbol-based decoding that introduces a copying mechanism during decoding;

[0005] 2. IRNet: The first model to apply syntax decoding in the field of text-to-SQL, and introduced a simplified top-down syntax, SemQL;

[0006] 3. SmBoP: Also a syntax-based decoding scheme, the difference is that it generates equivalent SQL syntax trees in a bottom-up manner;

[0007] 4. Picard: Symbol-based decoding, using generative pre-trained models to improve generalization, and introducing stepwise checks during model inference to standardize and constrain the generated sequences to conform to the syntax definition of SQL.

[0008] In the process of realizing this invention, the inventors discovered at least the following problems in the related technology:

[0009] 1. BRIDGE: Because it directly generates each word of SQL, it lacks a clear understanding of the internal structure of SQL and cannot utilize the syntax knowledge and rules defined in the standard. It is very easy to generate SQL programs that do not conform to the syntax standard, and errors will occur during execution.

[0010] 2. Picard: Although checking and constraining each time step during decoding improves the grammatical regularity of the final output, it requires building a parser for incomplete SQL, and the learning barrier for writing constraint rules at each step is high. At the same time, it also relies on a large generative model, making it unsuitable when resources are limited.

[0011] 3. IRNet: It uses an LSTM model as the decoding network, which is very time-consuming to train. Furthermore, LSTM itself is not constructed for generating structured trees, so it has few structural priors. In addition, IRNet only generates an abstract framework of SQL, not SQL values, so the actual SQL program is not executable.

[0012] 4. SmBoP: Iterates upwards from the subtree to the complete tree. The search space is very large. Since the depth of different leaf nodes is different, the depth of the target SQL syntax tree needs to be balanced first. This results in many actual equivalent trees and the training target is not unique.

[0013] In summary, the above problems are mainly due to inefficient use of resources caused by excessively long training time, and the lack of structural priors for the model's target output SQL program. Summary of the Invention

[0014] In order to at least address the structural and timing issues of decoding networks in existing technologies.

[0015] In a first aspect, embodiments of the present invention provide a decoding method based on a structure-aware SQL syntax tree, comprising:

[0016] The encoding status of the text-to-SQL is input to the node input module. The node input module is used to construct the absolute position encoding of the leading nodes in the abstract syntax tree of the text-to-SQL at the current time. The leading nodes include nodes in the abstract syntax tree that have not been expanded at the current time.

[0017] The absolute position encoding is input into a feature encoding module constructed from at least a mask self-attention module that fuses relative positional relationships, to obtain node features with the relative positional relationships of the leading edge node and other expanded nodes in the abstract syntax tree;

[0018] Based on the node features, the probability distribution of the expansion actions of the leading edge nodes is determined. The abstract syntax tree is then expanded and updated using the probability distribution of the expansion actions and the encoding state until all leading edge nodes in the abstract syntax tree have been expanded, thus obtaining the decoded abstract syntax tree.

[0019] Secondly, embodiments of the present invention provide a decoding framework based on a structure-aware SQL syntax tree, comprising:

[0020] The node input module is used to construct the absolute position encoding of the leading nodes in the abstract syntax tree of the text-to-SQL at the current time according to the encoding status of the received text-to-SQL, wherein the leading nodes include: nodes in the abstract syntax tree that have not been expanded at the current time;

[0021] A feature encoding module, constructed from at least a mask self-attention module that integrates relative positional relationships, is used to determine node features in the abstract syntax tree with the relative positional relationships of the leading edge node and other expanded nodes, based on the absolute positional encoding.

[0022] An action output module is used to determine the extended action probability distribution of the leading edge node based on the node features.

[0023] Thirdly, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the structure-aware SQL syntax tree decoding method according to any embodiment of the present invention.

[0024] Fourthly, embodiments of the present invention provide a storage medium storing a computer program thereon, characterized in that, when the program is executed by a processor, it implements the steps of the decoding method based on a structure-aware SQL syntax tree according to any embodiment of the present invention.

[0025] The beneficial effects of this invention are as follows: The AST structure-aware Transformer decoder of this method encodes the position of each node to be expanded in the syntax tree into the Transformer decoder through both absolute position encoding and relative position encoding. On the one hand, while ensuring compatibility between the AST and the improved Transformer decoder, it greatly shortens the iterative decoding training time; on the other hand, when each node performs attention calculation with historical nodes, it considers the relative positional relationship between the two. With the relative position between any two nodes, and through absolute position encoding—that is, combining its own node type and depth in the tree—the complete syntax tree structure can be recovered losslessly, ensuring the accuracy of subsequent text-to-SQL conversion. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of a decoding method for SQL syntax trees based on structure awareness, provided in an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram of the ASTormer decoder structure of a structure-aware SQL syntax tree decoding method provided in an embodiment of the present invention.

[0029] Figure 3 This is a schematic diagram illustrating three types of actions in a structure-aware SQL syntax tree decoding method provided in an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of experimental results for a structure-aware SQL syntax tree decoding method provided in an embodiment of the present invention;

[0031] Figure 5 This refers to the time consumed in iterative training of a structure-aware SQL syntax tree decoding method provided in an embodiment of the present invention.

[0032] Figure 6 This is a schematic diagram of an ablation experiment of different components in ASTormer, which provides a structure-aware SQL syntax tree decoding method according to an embodiment of the present invention.

[0033] Figure 7 This is a schematic diagram of the maximum relative distance ablation experiment of a structure-aware SQL syntax tree decoding method provided in an embodiment of the present invention;

[0034] Figure 8 This is a schematic diagram of an ablation experiment on the traversal order of an AST-based SQL syntax tree decoding method according to an embodiment of the present invention.

[0035] Figure 9 This is a schematic diagram of a structure-aware SQL syntax tree decoding framework provided in an embodiment of the present invention;

[0036] Figure 10 This is a schematic diagram of an embodiment of an electronic device for decoding SQL syntax trees based on a structure-aware method, as provided in an embodiment of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] like Figure 1 The diagram shows a flowchart of a structure-aware SQL syntax tree decoding method according to an embodiment of the present invention, which includes the following steps:

[0039] S11: Input the encoding status of the text to SQL to the node input module, and use the node input module to construct the absolute position encoding of the leading node in the abstract syntax tree of the text to SQL at the current time, wherein the leading node includes: the node in the abstract syntax tree that has not been expanded at the current time;

[0040] S12: Input the absolute position encoding into a feature encoding module constructed by at least a mask self-attention module that fuses relative positional relationships, to obtain node features with the relative positional relationships of the leading edge node and other expanded nodes in the abstract syntax tree;

[0041] S13: Determine the probability distribution of the expansion actions of the leading edge nodes based on the node features, and expand and update the abstract syntax tree using the probability distribution of the expansion actions and the encoding state until all leading edge nodes in the abstract syntax tree have been expanded, thereby obtaining the decoded abstract syntax tree.

[0042] In this embodiment, considering the symbol-based decoding in the prior art, although parallel training using the Transformer structure can be accelerated, the introduction of structural knowledge is lacking. The Transformer is a general framework designed for sequence generation, and the temporal relationship is introduced only through positional encoding, which is not compatible with the generation of SQL abstract syntax trees.

[0043] This method incorporates knowledge of syntax tree structures during decoding, with its core function being the encoding of the position of each node within the syntax tree. It's important to note that the abstract syntax tree (AST) in this method is dynamically constructed step-by-step in real-time. The AST is a tree representation of the source code. Each node n in the AST is assigned a type attribute n. τThe syllogism indicates its syntactic function. For example, among existing node types, a node of type sql represents the root of a complete SQL program. Nodes can be categorized into non-terminal nodes and terminal nodes based on their type. In this method, in addition to the commonly used SQL types from and select, node types also include tab_id, col_id, and tok_id, which represent indexes of tables, columns, and tokens, respectively. Each node type n τ The embedding is determined by the function Sure.

[0044] For each node, the rule `r` takes the form `p_type := RuleName(c_type1, c_type2, ...)`, where `p_type` is the type of the parent non-terminal node to be expanded, and `(c_type1, ...)` represents the type of the child nodes to be appended (which can be repeated or empty). Only non-terminal symbol types can appear on the left-hand side of a rule. This is directly derived from the embedding function. Retrieve the features of each grammar rule r.

[0045] For step S11, since this method improves the decoder part, the corresponding encoder can use an existing graph encoder. For example, RATSQL (Relation-Aware Transformer) can be used to encode the user input query that needs to be converted into SQL, resulting in X = [Q; T; C], where Q is the encoding matrix of the query, and T and C are the encoding matrices of the table and columns.

[0046] The decoder in this method can be called ASTormer (AST Structure-aware TransformerDecoder), which consists of three parts:

[0047] 1. Node input module;

[0048] 2. Feature encoding module (including L Astromer layers);

[0049] 3. Action output module.

[0050] To facilitate a detailed explanation of each step, a general overview of the three parts of the ASTormer encoder will be provided first. At each decoding time step j, the ASTormer encoder proceeds as follows:

[0051] 1. From the incomplete syntax tree of the previous moment In the set of all unexpanded nodes, select a node n. jAnd construct its input features using the node input module.

[0052] 2. Process node features through the feature encoding module and obtain the decoder state n. j ;

[0053] 3. The action output module is based on the type of the input node. Calculate the distribution of actions P(a) j |n j );

[0054] 4. From P(a) j Choose an action a from the list. j and update the incomplete syntax tree. get

[0055] By iterating through the above four steps until... No leading node can be found, meaning all nodes have already expanded.

[0056] Regarding the node input module, given the selected front node n... j First, the input features need to be constructed at time step j. It should reflect, to some extent, the type and location information of nodes in the AST. Therefore, it is initialized as the sum of four vectors, including:

[0057] 1. The action characteristics of the leading edge node at the previous time step (a) j-1 ), used to inform the decoder how to update the incomplete AST in the neural space.

[0058] 2. Types of leading-edge nodes ( ).

[0059] 3. The action characteristics of the parent node of the leading node ( ).

[0060] 4. The depth of the leading edge node within the abstract syntax tree ( Represents node n j Depth in AST ).

[0061] It's important to note that vectors 2-4 serve as the original APEs (absolute position embeddings) in the Transformer decoder. A LayerNorm layer is applied to the sum of these vectors. The form is:

[0062]

[0063] During the syntax tree construction process, a node n is selected from the set of nodes that have not been expanded. j This node can be called the frontier node, representing a node in the abstract syntax tree that has not been expanded at the current time. Its input features are constructed using the node input module.

[0064] like Figure 2 As shown in the lower part of the diagram, at time 1, the "depth of the leading node in the abstract syntax tree (node ​​depth embedding)" is 0, and the node type embedding is "sql". Since it is the first round, the corresponding "action features of the parent node of the leading node (i.e., parent rule embedding)" and "action features of the leading node in the previous time step (i.e., previous round action embedding)" are both [0,0,…,0]. At this time, these four features are added together to obtain the leading node n at time 1. j The absolute position encoding of (j=0) is then passed through a LayerNorm layer, and the resulting features are used as the input features for the current node. .

[0065] Input features The input is processed by the feature encoding module and the action output module, which are limited to three types of operations (APPLYRULE, SELECTITEM, GENTOKEN). For example, the output action of the input node SQL is determined to be APPLYRULE. (The processing of the feature encoding module and the action output module will be explained in detail in subsequent steps. This part mainly describes the real-time processing of the node input module.)

[0066] At time step 2, we can see that the "depth of the leading node within the abstract syntax tree (node ​​depth embedding)" is 1, the node type embedding is "from", and the parent rule embedding and the previous action embedding are also present, which is the action APPLYRULE of the node's SQL statement. At this point, these four features are summed to form the absolute position encoding for the next time step, used for decoding in subsequent steps.

[0067] Following the steps described above, the absolute position encoding for each time step is constructed. At time step 4, the node depth embedding is 1, the node type embedding is "select", and the parent rule embedding is the "sql" node's APPLYRULE. It's important to note that the previous action embedding was the "SelectTable:list" of the "tab_id" node at time step 3, not the "sql" node's APPLYRULE. These four features are then summed to form the absolute position encoding for the next time step, used for decoding in subsequent steps. By following these steps, the absolute position encoding of the leading nodes within the abstract syntax tree (abstract syntax tree) for text-to-SQL conversion at all time steps can be constructed using the node input module during the real-time dynamic construction of the abstract syntax tree.

[0068] For step S12, the feature encoding module is used to process the node features. Further processing is performed by stacking the previous layers in L layers, resulting in the final decoder hidden state n. j for:

[0069]

[0070]

[0071] in, Z represents the input features of all nodes before time j in the current layer l. j The matrix represents the current node n j and all expanded nodes n< j The relative relationship encoding.

[0072] As one implementation, the feature encoding module includes: a mask self-attention module that fuses relative positional relationships, a multi-head cross-attention module, and a feedforward neural network module, wherein...

[0073] The mask self-attention module that integrates relative positional relationships is used to determine the relative positional relationship between the leading edge node and other expanded nodes in the abstract syntax tree.

[0074] The multi-head cross-attention module and the feedforward neural network module are used to incorporate the relative positional relationship into the attention weights and weighted vectors of the feature encoding module to obtain node features with relative positional relationship.

[0075] The relative positions of the leading edge node and other expanded nodes in the abstract syntax tree are determined by the lowest common ancestor of the leading edge node and other expanded nodes.

[0076] In this embodiment, the ASTormer of this method includes three sub-modules: a mask self-attention module that fuses relative positional relationships, a multi-head cross-attention module, and a feedforward neural network module. The multi-head cross-attention module and the feedforward neural network module are the latter two sub-modules of a traditional Transformer decoder; this method primarily focuses on the mask self-attention module that fuses relative positional relationships. It borrows the concept of the lowest common ancestor in data structures, where the relative positional relationship z between any two nodes in the tree is used. ji They can be defined by their respective distances to their lowest common ancestor.

[0077] Specifically, node n j and n i The relationship between z ji Defined as a tuple:

[0078] Z ji =(dist(n j n lca ), dist(n i n lca ))

[0079] Where, n lca It is node n j and n i The lowest common ancestor node. For example... Figure 2 The self-attention module of the mask that integrates relative positional relationships is exemplified by z. ji Partial examples. z ji = (1,0) represents n j It is node n i The child nodes of z, and z ji = (1,1) represents the sibling relationship of a node. By definition, z ji It is symmetrical, that is, if z ji = (2,3), then the inverse relation must be z. ij = (3,2). However, it should be noted that some relations (such as z) ji =(0,1)) will never be used because during the top-down traversal of the AST, descendant nodes will not be expanded until all ancestor nodes have been completed.

[0080] Given the relative positional relationship between any two nodes, this method can encode the feature z using relative positional encoding. ji Integrating into attention weights and weighted vector The calculation is as follows: d is the vector dimension, and H is the number of heads in the multi-head attention mechanism. yes The softmax normalized version, node features n j The superscript 'l' for the layer number is ignored here:

[0081]

[0082]

[0083] Wherein, q / k / v represent the query, key, and value network parameters when calculating attention, respectively, and the superscript h indicates which head in the multi-head attention.

[0084] In summary, by combining the RPE (relative position embedding) determined in step S12 and the APE (absolute position embeddings) determined in step S11 as node features, the entire AST can be easily reconstructed using all the relative relationships between two nodes and the node types of the endpoints.

[0085] For step S13, the action output module is based on the encoded node features n j Based on node type Calculate the distribution P(a) of the output action. j The process generates a first expansion action probability for words in the statement, a second expansion action probability for selecting the table or column, and a third expansion action probability for the grammatical rules used to expand the frontier node. Specifically, there are three action types:

[0086] 1. Apply a syntax rule (APPLYRULE[r]) to expand the probability of the node;

[0087] 2. The probability of selecting an item (SelectTable or SelectColumn) from memory [T; C] is expressed as: SELECTITEM[i] (ITEM∈{TABLE,COLUMN}) for tab_id and col_id;

[0088] 3. The probability of generating a word (GENTOKEN).

[0089] Regarding the APPLYRULE action, it is determined by the type of the current node to be expanded. Choose one rule r from a defined set of constrained grammar rules. i Specifically, the syntax rule r i The left side must be equal to Apply this action to AST Rule r i All subtypes on the right are appended to the parent node n. j These child nodes are marked as unexpanded. Regarding the SELECTTABLE action, to expand the leading node of type tab_id, it directly selects an entry and appends it to the input node. Regarding the GENTOKEN action, a pointer generator network is used to generate the SQL value for a node of type tok_id. The original token w i It can be generated from a vocabulary list or from an input question. Copy from the middle. The calculation formula is,

[0090]

[0091] P gen (w i ) = softmax i (n j w gen e(w i ) T ),

[0092]

[0093] P(a j =G EN T OKEN [w i ]|a ＜j n j ,X)=g j P gen (w i )+(1-g j )P copy (w i )

[0094] Among them, g j ∈[0,1] is the balance fraction. For token w i Word embedding. Since each SQL value may have multiple tokens, a special " <eos>"Node n will not be expanded before the token is issued" j If the current output action is a j no" <eos>"Then buffer the intermediate results, and the leading edge node selected at time step j+1 is still n." j In issuing " <eos>"Then, node n is expanded by appending a list of all buffer tokens." j .

[0095] Action embedding can be defined (as decoder input for the next time step):

[0096]

[0097] Where r j / w j The rule / token selected at the current time step j, t j / c j This refers to the entries selected in the table / column memory T / C. The parameters of the pointer network are shared among the SELECTTABLE, SELECTCOLUMN, and GENTOKEN operations to avoid over-parameterization.

[0098] After determining the probability distribution of the extended actions, the AST also needs to be extended and updated, such as... Figure 3 As shown, in ApplyRule, the edge node set contains two nodes of type "from" and "select". To select the boundary node from the node set, the order is primarily DFS (depth-first search) followed by L2R (left-to-right). In this way, the boundary node set can be stored as a stack. At each time step j, the top element of the stack is directly selected as the current boundary node n. j In the action a j Applied to n j Next, remove the top element and rearrange n in reverse order. j All child nodes are pushed onto the stack. Similarly, if the AST traversal order is BFS (breadth-first search) and L2R, only a first-in-first-out queue needs to be maintained for the boundary node set.

[0099] The predefined canonical traversal order restricts the degrees of freedom in constructing the target AST. This eliminates the constraint of structure tree generation. During the training of iterative decoding, a traversal or expansion path is randomly sampled on the labeled AST in each iteration, with attention paid to using DFS or BFS as a constraint on the set of leading nodes. This randomness prevents the model from overfitting permutation bias and motivates the decoder to truly understand the relative positions z of different node pairs. ji As for inference, instead of selecting a candidate from the set of boundary nodes at each time step, all distinct options are enumerated as input to the decoder to expand the search space. This relaxation encourages the model itself to discover the optimal traversal path and provides further explanation for neural symbolic processing.

[0100] As can be seen from this implementation, the AST structure-aware Transformer decoder of this method encodes the position of each node to be expanded in the syntax tree into the Transformer decoder through both absolute position encoding and relative position encoding. On the one hand, while ensuring compatibility between the AST and the improved Transformer decoder, it greatly shortens the iterative decoding training time; on the other hand, when each node performs attention calculation with historical nodes, it considers the relative positional relationship between the two. With the relative position between any two nodes, and through absolute position encoding—that is, combining its own node type and depth in the tree—the complete syntax tree structure can be recovered losslessly, ensuring the accuracy of subsequent text-to-SQL conversion.

[0101] Experiments illustrate this method using Datasets Spider, a large-scale, cross-domain text-to-SQL benchmarking tool, employing SparC and CoSQL multi-turn dialogue versions.

[0102] The model used in this method is implemented in PyTorch. The PLM (pretrained language model) and its initial checkpoints were downloaded from the transformer library. For the graph encoder, the currently popular RATSQL was reproduced. The hidden dimension d was 512, and the number of encoder / decoder layers was 8 / 2. The number of heads and the dropout rate were set to 8 and 0.2, respectively. Throughout the experiments, the AdamW optimizer and a linear warm-up scheduler were used. The warm-up ratio for the total training steps was 0.1. The learning rates for different PLM sizes were 4e-4 (small), 2e-4 (basic), or 1e-4 (large), with a fixed weight decay rate of 0.1. The PLM parameters were further optimized with a layer-wise decay of the learning rate (coefficient 0.8). The batch size was 20, and the number of training iterations was 100k. For inference, a beam search of size 5 was used.

[0103] Experimental results are as follows Figure 4 As shown, the following conclusions can be drawn:

[0104] (1) The ASTormer decoder in this method consistently outperforms the LSTM decoder in all three baseline tests on both metrics. For example, against the existing baseline method ELECTRA, ASTormer beats the traditional LSTM decoder by 1.7 points on the Spider metric in EM, and by 2 points or more on SParC and CoSQL. Furthermore, in... Figure 5 The average training time per 100 iterations for the LSTM and ASTormer decoders was calculated. ASTormer is three times faster than the LSTM decoder (and even four times faster on the Spider due to the longer action sequences). Although it introduces some overhead regarding structural features, the overall training time is still comparable to the token-based Transformer due to the shorter output sequences. That is, the average number of AST nodes is less than the average number of tokenized SQL queries.

[0105] (2) In models of the same size, syntax-based decoders outperform token-based methods. This can be explained by the fact that syntax-based methods explicitly inject structural knowledge of the SQL program into the decoder.

[0106] (3) Equipped with a PLM tailored for structured data, performance can be further improved steadily. Therefore, task-adaptive PLMs focused on enhancing discriminative capabilities are more suitable for text-to-SQL.

[0107] Ablation experimental results of structural features are as follows Figure 6 As shown, this method analyzes the contribution of each module in ASTormer, including node type embedding. parent rule embedding Node deep embedding D a (n j ), and the relative positions z between node pairs. ji For any i ≤ j, the overall results show that performance inevitably decreases regardless of which part is omitted. Even when the proposed method replaces node depth embedding with traditional absolute position embedding, performance still degrades. This indicates that position embedding is unsuitable for structured tree generation and may introduce incorrect biases. When all structural features are removed, the proposed ASTormer decoder degenerates into a purely Transformer-based AST decoder. It also exhibits the worst performance, which conversely underscores the importance of integrating structural knowledge into the decoder.

[0108] The impact on the number of relations, Figure 7 The last column calculates the maximum tree depth for each dataset. By changing the size of R, a feasible option is to set R to approximately half the maximum tree depth.

[0109] This method also conducts an ablation study on the traversal order of the AST, and even introduces randomness in the decoding order during training. Figure 8 The method uses a "+random" approach, and simultaneously attempts to use all possible frontier nodes as input to the decoder during inference, thereby expanding the search space during the inference process. According to... Figure 8 The experimental results lead to the conclusion that the proposed ASTormer framework has good compatibility with different decoding orders, meaning that the performance difference between models obtained according to different decoding orders is not significant.

[0110] In summary, this method proposes a structure-aware AST decoder for text-to-SQL. It integrates the absolute and relative positions of each node in the AST into a Transformer decoder. To eliminate unreasonable permutation biases in the AST node expansion order and avoid training the model in canonical order, some randomness is injected into the AST traversal. The proposed model framework outperforms existing baseline methods on three datasets with the same model size.

[0111] like Figure 9 The diagram shows a structural schematic of a structure-aware SQL syntax tree decoding framework provided in an embodiment of the present invention. The system can execute the structure-aware SQL syntax tree decoding method described in any of the above embodiments and is configured in the terminal.

[0112] This embodiment provides a structure-aware SQL syntax tree-based decoding framework 10, which includes a node input module 11, a feature encoding module 12, and an action output module 13.

[0113] The node input module 11 is used to construct the absolute position encoding of the leading node in the abstract syntax tree of the text-to-SQL at the current time according to the received text-to-SQL encoding state. The leading node includes nodes that have not been expanded in the abstract syntax tree at the current time. The layout configuration adjustment program module 12 is used to determine the node features with the relative positional relationship between the leading node and other expanded nodes in the abstract syntax tree according to the absolute position encoding. The screen adaptation program module 13 is used to determine the expansion action probability distribution of the leading node based on the node features.

[0114] Furthermore, the decoding framework also includes an expansion update module, used to expand and update the abstract syntax tree using the expansion action probability distribution and the encoding state, until all front-end nodes in the abstract syntax tree have been expanded, to obtain the decoded abstract syntax tree.

[0115] Furthermore, the feature encoding module includes: a mask self-attention module that fuses relative positional relationships, a multi-head cross-attention module, and a feedforward neural network module, wherein,

[0116] The mask self-attention module that integrates relative positional relationships is used to determine the relative positional relationship between the leading edge node and other expanded nodes in the abstract syntax tree.

[0117] The multi-head cross-attention module and the feedforward neural network module are used to incorporate the relative positional relationship into the attention weights and weighted vectors of the feature encoding module to obtain node features with relative positional relationship.

[0118] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the structure-aware SQL syntax tree decoding method in any of the above method embodiments.

[0119] In one embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, which are configured as follows:

[0120] The encoding status of the text-to-SQL is input to the node input module. The node input module is used to construct the absolute position encoding of the leading nodes in the abstract syntax tree of the text-to-SQL at the current time. The leading nodes include nodes in the abstract syntax tree that have not been expanded at the current time.

[0121] The absolute position encoding is input into a feature encoding module constructed from at least a mask self-attention module that fuses relative positional relationships, to obtain node features with the relative positional relationships of the leading edge node and other expanded nodes in the abstract syntax tree;

[0122] Based on the node features, the probability distribution of the expansion actions of the leading edge nodes is determined. The abstract syntax tree is then expanded and updated using the probability distribution of the expansion actions and the encoding state until all leading edge nodes in the abstract syntax tree have been expanded, thus obtaining the decoded abstract syntax tree.

[0123] As a non-volatile computer-readable storage medium, it can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this invention. One or more program instructions are stored in the non-volatile computer-readable storage medium, and when executed by a processor, they perform the structure-aware SQL syntax tree-based decoding method in any of the above method embodiments.

[0124] Figure 10 This is a schematic diagram of the hardware structure of an electronic device using a structure-aware SQL syntax tree decoding method according to another embodiment of this application, as shown below. Figure 10 As shown, the device includes:

[0125] One or more processors 1010 and memory 1020, Figure 10 Taking a processor 1010 as an example, the device for decoding SQL syntax trees based on structure awareness may further include an input device 1030 and an output device 1040.

[0126] The processor 1010, memory 1020, input device 1030, and output device 1040 can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0127] The memory 1020, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the structure-aware SQL syntax tree decoding method in the embodiments of this application. The processor 1010 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 1020, thereby implementing the structure-aware SQL syntax tree decoding method in the above-described method embodiments.

[0128] The memory 1020 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data, etc. Furthermore, the memory 1020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 1020 may optionally include memory remotely located relative to the processor 1010, and these remote memories can be connected to the mobile device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0129] Input device 1030 can receive input numerical or character information. Output device 1040 may include display devices such as a display screen.

[0130] The one or more modules are stored in the memory 1020, and when executed by the one or more processors 1010, they execute the structure-aware SQL syntax tree decoding method in any of the above method embodiments.

[0131] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0132] Non-volatile computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the device, etc. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely located relative to the processor, and these remote memories may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0133] This invention also provides an electronic device comprising: at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the structure-aware SQL syntax tree decoding method of any embodiment of this invention.

[0134] The electronic devices described in this application exist in various forms, including but not limited to:

[0135] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0136] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as tablet computers.

[0137] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players, handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0138] (4) Other electronic devices with data processing functions.

[0139] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising" or "including" include not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.< / eos> < / eos> < / eos>

Claims

1. A structure-aware SQL syntax tree decoding method, comprising: inputting a text-to-SQL encoding state into a node input module to construct an absolute position encoding of a frontier node in an abstract syntax tree of the text-to-SQL at a current time, wherein the absolute position encoding comprises: an action feature of the frontier node at a previous time, a type of the frontier node, an action feature of a parent node of the frontier node, and a depth of the frontier node in the abstract syntax tree, wherein the frontier node comprises a node in the abstract syntax tree that has not been expanded at the current time; inputting the absolute position encoding into a feature encoding module constructed by at least a mask self-attention module that fuses relative position relationships to obtain a node feature with relative position relationships of the frontier node and other nodes that have been expanded in the abstract syntax tree; determining an expansion action probability distribution of the frontier node based on the node feature, wherein the expansion action probability distribution comprises: a first expansion action probability of generating a word in a sentence, a second expansion action probability of selecting a table or a column, and a third expansion action probability of a grammar rule used to expand the frontier node, and the abstract syntax tree is updated by expansion through the expansion action probability distribution and the encoding state until all frontier nodes in the abstract syntax tree are expanded to obtain a decoded abstract syntax tree.

2. The method of claim 1, wherein, the absolute position encoding of the frontier node in the abstract syntax tree of the text-to-SQL at the current time is constructed by the node input module, comprising: constructing the absolute position encoding of the frontier node by using the depth of the frontier node in the abstract syntax tree, the type of the frontier node, the action feature of the parent node of the frontier node, and the action feature of the frontier node at the previous time.

3. The method of claim 1, wherein, the structure of the feature encoding module comprises: a mask self-attention module that fuses relative position relationships, a multi-head cross-attention module, and a feedforward neural network module, wherein the mask self-attention module that fuses relative position relationships is used to determine the relative position relationships of the frontier node and other nodes that have been expanded in the abstract syntax tree; the multi-head cross-attention module and the feedforward neural network module are used to fuse the relative position relationships into attention weights and weighted vectors of the feature encoding module to obtain the node feature with the relative position relationships.

4. The method of claim 1, wherein, the relative position relationships of the frontier node and other nodes that have been expanded in the abstract syntax tree are determined by a lowest common ancestor of the frontier node and the other nodes that have been expanded.

5. The method of claim 1, wherein, the encoding state of the text-to-SQL is obtained by a graph encoder on a text sentence that needs to be converted to SQL, wherein the encoding state comprises: a sentence, a table, and a column.

6. A structure-aware SQL syntax tree decoding framework, comprising: The node input module is configured to construct an absolute position code of a frontier node in an abstract syntax tree of the text-to-SQL according to a received encoding state of the text-to-SQL at a current time, wherein the absolute position code comprises an action feature of the frontier node at a previous time, a type of the frontier node, an action feature of a parent node of the frontier node, and a depth of the frontier node in the abstract syntax tree, and the frontier node comprises a node that has not been expanded in the abstract syntax tree at the current time. The feature encoding module constructed by at least the mask self-attention module fusing relative position relationships is configured to determine a node feature with a relative position relationship between the frontier node and other nodes that have been expanded in the abstract syntax tree according to the absolute position code. The action output module is configured to determine an expansion action probability distribution of the frontier node based on the node feature, wherein the expansion action probability distribution comprises a first expansion action probability of a word in a generated sentence, a second expansion action probability of a table or a column, and a third expansion action probability of a syntax rule used for expanding the frontier node.

7. The decoding framework of claim 6, wherein, The decoding framework further comprises an expansion updating module configured to perform expansion updating on the abstract syntax tree by using the expansion action probability distribution and the encoding state until all the frontier nodes in the abstract syntax tree are expanded, so as to obtain a decoded abstract syntax tree.

8. The decoding framework of claim 6, wherein, The feature encoding module comprises a mask self-attention module fusing relative position relationships, a multi-head cross-attention module, and a feed-forward neural network module. The mask self-attention module fusing relative position relationships is configured to determine the relative position relationship between the frontier node and other nodes that have been expanded in the abstract syntax tree. The multi-head cross-attention module and the feed-forward neural network module are configured to fuse the relative position relationship into attention weights and weighted vectors of the feature encoding module, so as to obtain the node feature with the relative position relationship.

9. An electronic device comprising: The at least one processor and a memory connected to the at least one processor in communication, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method in any one of claims 1-5.

10. A storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the steps of the method in any one of claims 1-5.

Images