Text conversion encoder, text conversion to SQL query analysis method and system

Abstract

The invention discloses a text conversion to SQL query analysis method and system based on relation selection and dynamic cutting. First, by calculating the correlation between query words and keywords, the correlation between query and keywords is determined, and the most relevant query words are found; then, irrelevant information is cut and sparse processing is performed; finally, the sparse information is used as input, encoded by a text conversion encoder, and input into a grammar-based decoder to generate an abstract syntax tree of SQL, and then converted into executable SQL. By calculating the mutual correlation between query and keywords, the correlation between query and keywords is determined, the most relevant query is found, irrelevant information is cut, and sparse processing is performed, which has good migration and generalization capabilities.

Description

Technical Field

[0001] The present invention belongs to the information technology field of natural language processing, and in particular relates to a text conversion encoder, a text conversion to SQL query analysis method and a system. Background Art

[0002] Interacting with relational databases through natural language can help any backend database user to easily query and analyze large amounts of data, and unlock the power of large data sets for the vast majority of users who are not proficient in query languages. This requires a method that can understand the user's questions and automatically convert their text into SQL queries. In general, building an algorithm for natural language interfaces with relational databases is an important and challenging problem. Among them, large-scale annotated datasets such as WikiSQL and Spider further point out the challenges currently faced by this technology: generalization, complexity (SQL syntax), correctness (alignment relationship between questions and tables), and also promote the research and system development of a series of subsequent algorithms. It requires an efficient algorithm that can understand natural language questions and generate corresponding SQL queries. For an SQL, there is a lot of information that is irrelevant and unnecessary for the encoder, and the large amount of data to be processed causes the problem of low conversion efficiency; therefore, it is necessary to extract the most relevant information and remove or cut unnecessary information. In order to improve the ability to convert text into SQL query analysis. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a text conversion encoder, a text conversion to SQL query analysis method and system, which solves the problem of large amount of redundant data and low conversion efficiency in the prior art text conversion to SQL query method.

[0004] The present invention adopts the following technical solutions to solve the above technical problems:

[0005] The text conversion encoder based on relation selection and dynamic cutting uses natural language query terms and database patterns as input, uses NLP embedding methods to represent natural language questions and their table patterns, and dynamically calculates and cuts irrelevant elements based on relevance based on a relation self-attention layer containing H heads and a position-level fully connected feedforward LSTM network layer obtained in advance through learning, learns relation selectors to reduce ambiguous incorrect links, and calculates the memory pattern link matrix of columns and tables to capture the relationship between the final encoder output and the table, that is, the alignment relationship between the question and the table.

[0006] The correlation calculation method is as follows:

[0007]

[0008] x jis the query word, x i is the keyword in the dataset, L k is the dimension of keyword vector, d is the dimension of query vector, and K represents the number of most relevant query words.

[0009] The method of using the correlation is as follows:

[0010] For N elements, randomly select L K elements, calculate any element x after sampling i L between all remaining sample elements K dot products, then calculate the difference between the maximum value and the average value, and retain the largest K elements as output.

[0011] The working process of the relationship selector is as follows:

[0012] For any two elements, calculate f(r,x i ,x j ), where f consists of two fully connected layers x i ,x j and a tanh nonlinear layer r, which is used to automatically learn and determine whether to keep the element x i ,x j The links between.

[0013] The text conversion to SQL query analysis method based on relation selection and dynamic cutting first determines the correlation between the query and the keyword by calculating the correlation between the query word and the keyword, and finds the most relevant query word; then, cuts irrelevant information and performs sparse processing; finally, the sparse information is used as input, encoded by the text conversion encoder, and input into the grammar-based decoder to generate the abstract syntax tree of SQL, which is then converted into executable SQL.

[0014] According to the maximum cross-correlation formula, cut irrelevant information, perform sparseness, and leave the most relevant query keyword x after cutting. i ′, and is processed as the input of the encoder.

[0015] According to the alignment relationship between the question and the table, SQL is generated through the prior probability formula to form an abstract syntax tree of SQL.

[0016] Text conversion into SQL query analysis system based on relation selection and dynamic cutting, including

[0017] The most relevant information search module calculates the mutual correlation between the query and the keywords, determines the correlation between the query and the keywords, finds the most relevant information, and cuts off irrelevant information for sparse processing;

[0018] The relation-aware self-attention layer acquisition module takes the sparse information as input, uses the embedding method to represent the natural language question and its table pattern, and obtains a relation-aware self-attention layer with H heads through learning;

[0019] A question and table alignment relationship capture module calculates the memory mode link matrix of columns and tables to capture the relationship between the final encoder output and the table, that is, the alignment relationship between the question and the table; the encoder is the text conversion encoder based on relationship selection and dynamic cutting;

[0020] SQL formation module,During the decoding process, the abstract syntax tree of SQL is obtained through a grammar-based decoder.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. By calculating the cross-correlation between the query and the keyword, the correlation between the query and the keyword is determined, the most relevant query is found, irrelevant information is cut, and sparseness is performed, which has good migration and generalization capabilities.

[0023] 2. Through the trained relationship selector, the relevant information is further refined to make the model more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The present invention is a flow chart of the text conversion into SQL query analysis method based on relationship selection and dynamic cutting.

[0025] Figure 2 This is a schematic diagram of the relationship between the code length and the running time in the simulation of the present invention, which shows that the model running time is reduced by 15% compared to the original time. The actual accuracy that can be achieved is slightly lower.

[0026] Figure 3 This is a block diagram of the text conversion into SQL query analysis system based on relationship selection and dynamic cutting of the present invention. DETAILED DESCRIPTION

[0027] The structure and working process of the present invention will be further described below in conjunction with the accompanying drawings.

[0028] The present invention provides a text-to-SQL query analysis method and system based on relationship selection and dynamic clipping for an aggregate information prediction model and a relationship selection mechanism for dependency calculation, thereby improving the text-to-SQL query capability.

[0029] Under the premise of a given relational database (or table), the user's question generates the corresponding SQL query statement through a neural network. For the general prediction model and the relation selection mechanism of dependency calculation, in order to improve the calculation efficiency, the present invention combines relation selection with dynamic pruning, introduces a dynamic pruning strategy to discard common irrelevant items in the aggregate table, and provides a new and efficient relational database text-to-SQL query method and system.

[0030] The text conversion encoder based on relation selection and dynamic cutting uses natural language query terms and database patterns as input, uses NLP embedding methods to represent natural language questions and their table patterns, and dynamically calculates and cuts irrelevant elements based on relevance based on a relation self-attention layer containing H heads and a position-level fully connected feedforward LSTM network layer obtained in advance through learning, learns relation selectors to reduce ambiguous incorrect links, and calculates the memory pattern link matrix of columns and tables to capture the relationship between the final encoder output and the table, that is, the alignment relationship between the question and the table.

[0031] The correlation calculation method is as follows:

[0032]

[0033] x j is the query word, x i is the keyword in the dataset, L k is the dimension of keyword vector, d is the dimension of query vector, and K represents the number of most relevant query words.

[0034] The method of using the correlation is as follows:

[0035] For N elements, randomly select L K elements, calculate any element x after sampling i L between all remaining sample elements K dot products, then calculate the difference between the maximum value and the average value, and retain the largest K elements as output.

[0036] The working process of the relationship selector is as follows:

[0037] For any two elements, calculate f(r,x i ,x j ), where f consists of two fully connected layers x i ,x j and a tanh nonlinear layer r, which is used to automatically learn and determine whether to keep the element x i ,x j The links between.

[0038] The text conversion to SQL query analysis method based on relation selection and dynamic cutting first determines the correlation between the query and the keyword by calculating the correlation between the query word and the keyword, and finds the most relevant query word; then, cuts irrelevant information and performs sparse processing; finally, the sparse information is used as input, encoded by the text conversion encoder, and input into the grammar-based decoder to generate the abstract syntax tree of SQL, which is then converted into executable SQL.

[0039] Specific embodiment 1, as Figure 1 , Figure 2 As shown,

[0040] The text conversion into SQL query analysis method based on relation selection and dynamic cutting includes the following steps:

[0041] Step 1: Calculate the maximum cross-correlation between query and keyword Identify the key relationship x that is most relevant to the query K information:

[0042]

[0043] x j is the query word, x i is the keyword in the dataset, L k is the dimension of keyword vector, d is the dimension of query vector, and K represents the most relevant query word.

[0044] Step 2: Cut irrelevant information and perform sparseness; through formula (1), the most relevant cut query x is retained i ′.

[0045] Step 3: Cut the query x i ′ is used as a new input, and the following NLP embedding method is used to represent the natural language question and its table mode. Through learning, a relational self-attention layer with H heads is obtained. The encoding between query and keyword is implemented by the following function:

[0046]

[0047] W is the weight coefficient, r ij ′ is the key relationship between the i-th and j-th element tables, r ij is the value relation value. f is the tanh activation function of the two-layer fully connected network. is the attenuation weight of the hth head, d is the dimension of the word vector, is the ith output. is the trainable weight matrix.

[0048] Here, the encoder is divided into two sub-layers, a multi-head self-attention mechanism and a position-level fully connected feed-forward LSTM (long short term memory) network.

[0049] Step 4: Calculate the memory mode linkage matrix of columns and tables to capture the relationship between the final encoder output and the table summary;

[0050]

[0051] Softmax is the activation function, L col is a summary of the aligned tables.

[0052] Step 5: Obtain the predicted distribution through the prior probability formulas (7) and (8), generate SQL, and form the abstract syntax tree (AST) of SQL. Here,

[0053]

[0054] m t ,h t =f LSTM ([a t-1 ||z t ||h pt ||a pt ||n ft ],m t-1 ,h t-1 ) (8)

[0055] Y is the representation given by the encoder. LSTM (long short term memory) has two transmission states, m t is the cell state, indicating long-term memory, h t is its hidden state, representing short-term memory, a t-1 is the embedding of the previous action, pt is the step corresponding to the parent AST node expanded to the current node, n ft is the embedding of the current node type, z t It is a contextual representation.

[0056] Two operations are used in this decoding process, one is as follows:

[0057] P(a t =A PPLY R ULE [r]|a ＜t )=softmax r (g(h t ))

[0058] Among them, softmax is the activation function, g represents thet The activation function tanh operation.

[0059] Another type of operation is to use functions:

[0060] SELECT_COLUMN[ν]or SELECT_TABLE[ν]

[0061] get

[0062]

[0063] When completing a leaf node, it selects a column or table ν from the schema. In this way, the abstract syntax tree of SQL is finally formed through function calculation.

[0064] from Figure 2 As can be seen from the figure, the model running time is reduced by 15% compared to the original time, which effectively improves the text conversion speed.

[0065] Text conversion into SQL query analysis system based on relation selection and dynamic cutting, including

[0066] The most relevant information search module calculates the mutual correlation between the query and the keywords, determines the correlation between the query and the keywords, finds the most relevant information, and cuts off irrelevant information for sparse processing;

[0067] The relation-aware self-attention layer acquisition module takes the sparse information as input, uses the embedding method to represent the natural language question and its table pattern, and obtains a relation-aware self-attention layer with H heads through learning;

[0068] A question and table alignment relationship capture module calculates the memory mode link matrix of columns and tables to capture the relationship between the final encoder output and the table, that is, the alignment relationship between the question and the table; the encoder is a text conversion encoder based on relationship selection and dynamic cutting;

[0069] SQL formation module,During the decoding process, the abstract syntax tree of SQL is obtained through a grammar-based decoder.

[0070] Specific embodiment 2, as Figure 3 As shown,

[0071] This embodiment discloses a text conversion to SQL query analysis system based on relation selection and dynamic cutting, including the following modules connected in sequence:

[0072] The most relevant information search module: by calculating the correlation between the query and the keywords, the correlation between the query and the keywords is determined, and the most relevant information is found; irrelevant information is cut and sparse processing is performed; in this embodiment, the most relevant information search module calculates the maximum correlation between the query and the keywords using the above method.

[0073] Relation-aware self-attention layer acquisition module: takes the sparse information as input, uses the embedding method to represent the natural language question and its table pattern, and obtains a relation-aware self-attention layer containing H heads through learning; the inherent weakness of the encoder-decoder model is that the decoder model only obtains information from the last hidden state of the encoder model. For LSTM, there is a high possibility of losing important context in the sequence. In view of this shortcoming of the encoder-decoder, it is necessary to find a method that allows the decoder model to receive more than just the last state, which leads to a mechanism called "attention". Attention mechanisms are not independent models; instead, they supplement the basic encoder-decoder model by providing an interface to specifically pass more context between the two models by passing more hidden states between the models, thus generating a self-attention layer. The relation-aware self-attention layer acquisition module takes the clipped query words as new input, uses the following NLP embedding method to represent the natural language question and its table pattern, and obtains a relation-aware self-attention layer containing H heads through learning. The specific implementation method refers to the third step of the above method.

[0074] Question and table alignment relation capture module: Calculate the memory pattern link matrix of columns and tables to capture the relationship between the final encoder output and the table, that is, the alignment relationship between questions and tables; Question and table alignment relation capture module calculates the pattern link matrix of columns and tables to capture the relationship between the final encoder output and the table summary. Compared with the ordinary self-attention mechanism, the relation-aware self-attention requires two additional tensor inputs, namely the key relation r ij ′ and value relationship r ij , which respectively amplify or reduce the contribution of the proportional dot product attention of each H attention head. The specific working process and working principle of this module are as shown in the fourth step of the above method.

[0075] SQL formation module: during the decoding process, the abstract syntax tree of SQL is formed through operation calculation. The specific application method of this module is as in the fifth step of the above method.

[0076] Those skilled in the art should understand that those skilled in the art can implement variations by combining the prior art and the above embodiments, which will not be described in detail here. Such variations do not affect the essential content of the present invention, and will not be described in detail here.

[0077] The above describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the devices and structures that are not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can use the above-disclosed methods and technical contents to make many possible changes and modifications to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, or modify them into equivalent embodiments of equivalent changes, which does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention still fall within the scope of protection of the technical solutions of the present invention.

Claims

1. A text conversion encoder based on relation selection and dynamic cutting, characterized by: The natural language query terms and database schema are used as input, and the NLP embedding method is used to represent the natural language questions and their table schema. Based on a relational self-attention layer containing H heads and a position-level fully connected feedforward LSTM network layer obtained in advance through learning, irrelevant elements are dynamically calculated and cut based on relevance, and relation selectors are learned to reduce ambiguous incorrect links. The memory mode link matrix of columns and tables is calculated to capture the relationship between the final encoder output and the table, that is, the alignment relationship between the question and the table; the text conversion into SQL query analysis method based on relation selection and dynamic cutting includes the following steps: Step 1: Calculate the maximum cross-correlation between query and keyword Identify the key relationship x that is most relevant to the query K information: x j is the query word, x i is the keyword in the dataset, L k is the dimension of the keyword vector, d is the dimension of the query word vector, and K represents the most relevant query word; Step 2: Cut irrelevant information and perform sparseness; through formula (1), the most relevant cut query x is retained i ′; Step 3: Cut the query x i ′ is used as a new input, and the following NLP embedding method is used to represent the natural language question and its table mode. A relational self-attention layer with H heads is obtained through learning. Specifically, the encoding between the query and the keyword is realized by the following function: W is the weight coefficient, r ij ′ is the key relationship between the i-th and j-th element tables, r ij is the value relationship value, f is the tanh activation function of the two-layer fully connected network, is the attenuation weight of the hth head, d is the dimension of the word vector, is the ith output, is the trainable weight matrix; Here, the encoder is divided into two sub-layers, a multi-head self-attention mechanism and a position-level fully connected feed-forward LSTM network; Step 4: Calculate the memory mode linkage matrix of columns and tables to capture the relationship between the final encoder output and the table summary; Softmax is the activation function, L col is a summary of the aligned tables; Step 5: Obtain the predicted distribution through the prior probability formulas (7) and (8), generate SQL, and form the abstract syntax tree (AST) of SQL. Here, m t ,h t =f LSTM ([a t-1 ||z t ||h pt ||a pt ||n ft ],m t-1 ,h t-1 ) (8) Y is the representation given by the encoder. LSTM has two transmission states, m t is the cell state, indicating long-term memory, h t is its hidden state, representing short-term memory, a t-1 is the embedding of the previous action, pt is the step corresponding to the parent AST node expanded to the current node, n ft is the embedding of the current node type, z t It is a contextual representation.

2. The text conversion encoder based on relationship selection and dynamic cutting according to claim 1, characterized in that: The correlation calculation method is as follows: x j is the query word, x i is the keyword in the dataset, L k is the dimension of keyword vector, d is the dimension of query vector, and K represents the number of most relevant query words.

3. The text conversion encoder based on relationship selection and dynamic cutting according to claim 2, characterized in that: The method of using the correlation is as follows: For N elements, randomly select L K elements, calculate any element x after sampling i L between all remaining sample elements K dot products, then calculate the difference between the maximum value and the average value, and retain the largest K elements as output.

4. The text conversion encoder based on relationship selection and dynamic cutting according to claim 1, characterized in that: The working process of the relationship selector is as follows: For any two elements, calculate f(r,x i ,x j ), where f consists of two fully connected layers x i ,x j and a tanh nonlinear layer r, which is used to automatically learn and determine whether to keep the element x i ,x j The links between.

5. A text conversion into SQL query analysis method based on relation selection and dynamic cutting, characterized by: First, by calculating the correlation between the query word and the keyword, the correlation between the query and the keyword is determined, and the most relevant query word is found; then, irrelevant information is cut and sparse processing is performed; finally, the sparse information is used as input, and the text conversion encoder described in any one of claims 1 to 4 is applied for encoding, and the abstract syntax tree of SQL is generated by the grammar-based decoder, which is then converted into executable SQL.

6. The text conversion to SQL query analysis method based on relation selection and dynamic cutting according to claim 5 is characterized by: According to the maximum cross-correlation formula, cut irrelevant information, perform sparseness, and leave the most relevant query keyword x after cutting. i ′, and is processed as the input of the encoder.

7. The text conversion to SQL query analysis method based on relation selection and dynamic cutting according to claim 5 is characterized by: According to the alignment relationship between the question and the table, SQL is generated through the prior probability formula to form an abstract syntax tree of SQL.

8. A text conversion into SQL query analysis system based on relation selection and dynamic cutting, characterized by: include The most relevant information search module determines the correlation between the query and the keywords by calculating the mutual correlation between the query and the keywords, and finds the most relevant information; And cut irrelevant information and perform sparse processing; The relation-aware self-attention layer acquisition module takes the sparse information as input, uses the embedding method to represent the natural language question and its table pattern, and obtains a relation-aware self-attention layer with H heads through learning; A question and table alignment relationship capture module calculates the memory mode link matrix of columns and tables to capture the relationship between the final encoder output and the table, that is, the alignment relationship between the question and the table; the encoder is a text conversion encoder based on relationship selection and dynamic cutting as described in any one of claims 1 to 4; SQL formation module,During the decoding process, the abstract syntax tree of SQL is obtained through a grammar-based decoder.

Images