Natural language conversion structure query statement semantic parsing method based on deep learning

Abstract

The invention relates to the field of natural language conversion structure query statement semantic analysis, in particular to a deep learning-based natural language conversion structure query statement semantic analysis method, which comprises the following steps of: performing mode connection processing by utilizing a natural language and a database to obtain a semantic association mode connection processing result of an initial natural language; establishing a natural language basic conversion model based on deep learning by using the semantic association mode connection processing result to obtain structural query statement intermediate data; according to the intermediate data of the structure query statement, obtaining a semantic analysis result of converting the natural language into the structure query statement, realizing accurate semantic matching of the natural language and a database mode by an N-Gram and BERT encoder, and dynamically fusing table and column information in combination with an attention mechanism; syntax tree intermediate representation is generated in an end-to-end mode through LSTM, and complex query generation is optimized; and a syntax tree framework evaluation method is introduced, so that the accuracy and interpretability of text-to-SQL (Structured Query Language) are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of semantic parsing of natural language converted structured query statements, and in particular to a method for semantic parsing of natural language converted structured query statements based on deep learning. Background Art

[0002] Currently, there are problems in natural language generation of structured queries, such as ambiguous matching between natural language and database patterns, insufficient accuracy in generating complex cross-domain SQL, traditional methods relying on hard-coded rules, and poor generalization capabilities. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the present invention provides a semantic parsing method for converting natural language into structured query statements based on deep learning. By establishing an encoder-decoder architecture, a method for converting natural language into structured query statements using an intermediate representation of a syntax tree is implemented.

[0004] To achieve the above objectives, the present invention provides a method for semantic parsing of structured query statements from natural language conversion based on deep learning, comprising: Using natural language and database to perform pattern connection processing to obtain semantic association pattern connection processing results of initial natural language; Using the semantic association pattern connection processing results, a natural language basic conversion model is established based on deep learning to obtain structured query statement intermediate data; A semantic parsing result of the natural language conversion structured query statement is obtained according to the intermediate data of the structured query statement.

[0005] Preferably, the method of performing pattern connection processing on the natural language and the database to obtain the semantic association pattern connection processing result of the initial natural language includes: Obtaining an initial natural language sequence segment based on the natural language based on an N-Gram model; Arrange the initial natural language sequence segments in descending order to obtain natural language sequence segments; The natural language sequence fragments are used to perform pattern connection processing with a database to obtain a semantic association pattern connection processing result of the initial natural language.

[0006] Furthermore, the semantic association pattern connection processing result of the initial natural language is obtained by performing pattern connection processing on the natural language sequence segment and the database, including: Performing string matching processing on the natural language sequence fragment and the database column name and database table name to obtain a string matching processing result; Determine whether the string matching processing result is an exact match. If so, mark the natural language sequence segment corresponding to the string matching processing result and the database column name as Question-column-exact-match; otherwise, mark the natural language sequence segment corresponding to the string matching processing result and the database column name as Question-column-partical-match; Using the string matching processing result, based on the BERT encoder, a natural language sequence segment vector value, a database column name vector, a database table name vector, and a string matching processing result type vector of the string matching processing result are obtained; The calculation formula for the column and table vectors with problematic information is as follows: , , , , , in Indicates the first i The vector representation corresponding to the token is: Indicates the first i A vector corresponding to the column names, Indicates the mode i A vector corresponding to each table name;

[0007] The trainable vector matrix is ​​established by using the sum of the number of corresponding relationship types of the string matching processing results as the rows of the matrix and the type vectors of the string matching processing results as the columns of the matrix. ; Using the trainable vector matrix The formula for calculating the input vector with the database column name vector, database table name vector, and string matching processing result type vector is as follows: ; Using the column and table vectors of the problem information and the input vector as the semantic association pattern connection processing results of the initial natural language; Among them, C represents all column names in the database, and T represents all table names in the database. For the i A string matching result type vector, is the vector corresponding to the k-th column name, is the vector corresponding to the j-th table name, Corresponding to A vector of relation types labeled during the schema linking phase.

[0008] Furthermore, the structured query statement intermediate data is obtained by using the semantic association pattern connection processing result to establish a natural language basic conversion model based on deep learning, including: Using the semantic association pattern connection processing results to establish a natural language basic conversion model based on deep learning; The structured query statement intermediate data is obtained according to the natural language basic conversion model.

[0009] Furthermore, establishing a natural language basic conversion model based on deep learning using the semantic association pattern connection processing results includes: A natural language basic conversion model is established based on deep learning using the natural language sequence fragments corresponding to the semantic association pattern connection processing results, the database and the semantic association pattern connection processing results as input targets, and the SQL query statement intermediate representation corresponding to the semantic association pattern connection processing results as output targets.

[0010] Furthermore, the structured query statement intermediate data obtained according to the natural language basic conversion model includes: The calculation formula for the corresponding nodes of the semantic association pattern connection processing result obtained by establishing the intermediate data sequence based on LSTM is as follows: ; Utilizing the natural language basic conversion model to connect the corresponding nodes of the processing results according to the semantic association pattern to obtain the intermediate data of the structured query statement; Among them, Y represents the generated intermediate representation sequence, X and S represent the question sequence and database model respectively, Indicates at time i All actions taken before, Indicates in i The action taken at each time step.

[0011] Furthermore, obtaining a semantic analysis result of the natural language converted structured query statement according to the intermediate data of the structured query statement includes: Using the structured query statement intermediate data to perform decoding processing to obtain a decoding processing result of the structured query statement intermediate data; The decoding processing result of the intermediate data of the structured query statement is used to obtain the conversion processing accuracy to obtain the semantic parsing result of the natural language conversion structured query statement.

[0012] Furthermore, the decoding process of the structured query statement intermediate data to obtain the decoding process result of the structured query statement intermediate data includes: The intermediate data of the structured query statement is used to perform state tracking processing according to the decoder to obtain the internal hidden state, which is calculated as follows: ; in is the unit calculation function of the long short-term memory network (LSTM), is the cell state at time t-1 in LSTM, which contains some historical information of the past moments. is the output state at time t-1, is the input at time t, It represents the vector representation of the intermediate representation generation rule at time t-1, is a fusion representation of all outputs of the encoder stage, Represents each input The corresponding weight, Equivalent to a single-layer neural network, the input is and the output status of the previous position .

[0013] The action probability of the selected column is obtained according to the internal hidden state, and the calculation formula is as follows: ;

[0014] The action probability of the selection table is obtained according to the internal hidden state, and the calculation formula is as follows: ; Using the action probability of the selection column and the action probability of the selection table as the decoding processing result of the intermediate data of the structured query statement; S is the selected column in the database mode, Mem is the selected column in the dynamic memory module, is the output vector of the decoder at time t, is a trainable parameter vector, is the vector corresponding to the column name in the dynamic memory module, is a vector of column names in the database schema that are never selected, A vector of table names corresponding to the database schema.

[0015] Furthermore, obtaining the conversion processing accuracy using the decoding processing result of the intermediate data of the structured query statement to obtain the semantic parsing result of the natural language conversion structured query statement includes: According to the decoding processing result of the intermediate data of the structured query statement, respectively obtain the syntax tree framework structure with leaf nodes removed and the column name and table name structure without the intermediate data of the structured query statement; The calculation formula for the conversion processing accuracy is as follows based on the syntax tree framework structure without leaf nodes and the column name and table name structure without the intermediate data of the structural query statement: ; Obtaining a semantic analysis result of a natural language conversion structure query statement according to the conversion processing accuracy; in, It represents the predicted label or predicted value, y represents the true label or true value, score represents the score of a sample, and acc represents the accuracy.

[0016] Compared with the closest prior art, the present invention has the following beneficial effects: Significantly Improved SQL Statement Generation Accuracy: This paper combines the N-Gram model with the BERT encoder and introduces an attention mechanism to dynamically integrate database schema information, significantly improving the semantic matching accuracy between natural language and database schemas. Experimental results on the Spider dataset show that the generated SQL statements achieve an accuracy of 56.9%, significantly outperforming traditional Seq2Seq models (20.9%), SQLNet models (32.8%), and TypeSQL models (34.5%). The BERT model enables a deep fusion representation of natural language and database schemas, capturing bidirectional contextual information through a masked language model (MLM), effectively addressing the shortcomings of traditional BiLSTM encoders in long-range dependencies and context integration. Experiments show that using BERT as the encoder improves logical accuracy by approximately 10% compared to using BiLSTM. During the decoding phase, a dynamic memory module records selected column names and prioritizes frequently used column names from the memory module, reducing the overhead of repeated matching and improving column name selection accuracy. Separation experiments demonstrate that the introduction of the dynamic memory module further optimizes the model's convergence speed and final accuracy.

[0017] Reduce the labor cost of cross-domain applications: By introducing an intermediate representation as a transitional form between natural language and SQL statements, the intermediate representation adopts an abstract syntax tree structure, which can cover all complex clauses of SQL statements (such as GROUP BY, HAVING, nested queries, etc.), and its generation rules are decoupled from the specific database model. When applied to a new field, it is only necessary to adjust the conversion rules from natural language to intermediate representation or intermediate representation to SQL, without the need to redesign the entire model. In contrast, other existing models rely more on indirect supervision and still need to adjust the generation strategy through reinforcement learning, while the present invention directly generates intermediate representations, reduces the complexity of training, and can adapt to complex scenarios such as multi-table queries and cross-domain databases. For example, in the 138 different fields of the Spider dataset, the model can generate the target SQL statement only through the transition of the intermediate representation, which significantly reduces the workload of manual labeling and rule formulation.

[0018] Pattern chaining technology improves semantic matching accuracy: The pattern chaining method proposed in this paper defines seven relationship types (such as exact match and partial match) to accurately match entities in natural language with column and table names in the database schema. During the encoding phase, attention weights are used to calculate the correlation between the question sequence and the column and table names, generating vector representations of the columns and tables that incorporate contextual information. Experiments have shown that this method significantly improves the recognition rate of column and table names, especially in multi-table query scenarios.

[0019] The dynamic memory module optimizes decoding efficiency: During the decoding phase, the dynamic memory module improves performance through selective memory and search space reduction. Specifically, the selected column names are stored in the dynamic memory module. By dynamically removing matched column names, the search scope of the database pattern is narrowed, accelerating decoding. This module is prioritized in subsequent decoding steps, avoiding duplicate matches and invalid searches. This module is particularly suitable for complex query scenarios involving a large number of column names. Experimental results show that this module improves the model's post-convergence sketch accuracy and logical accuracy compared to a baseline model that does not use the memory module. Most other existing models do not explicitly record selected column names, resulting in a high proportion of redundant queries.

[0020] Adaptability to complex query scenarios: The performance of the present invention on the Spider dataset verifies its ability to support complex query tasks. It can correctly handle query requirements involving multi-table connections and foreign key associations. For example, when generating SQL statements containing JOIN clauses, the accuracy of the present invention is increased to 70.0% in similar scenarios, which is a significant improvement compared to other existing models (such as an improvement of about 5% compared to SQLGen). It also supports nested queries and aggregation operations. By generating rules through the tree structure of the intermediate representation, the model can flexibly handle complex logic such as nested queries and aggregation functions (such as MAX, COUNT), and its accuracy is significantly improved compared to traditional models. The SQLGen model is limited by the answer-driven mechanism of indirect supervision, and its coverage of complex logic is insufficient.

[0021] Compared with traditional rule-based or static word vector methods, the trainable vector matrix of the present invention can handle fuzzy matching problems in a finer granularity, especially with higher accuracy when processing complex queries; the joint optimization of the dynamic memory module and the syntax tree framework structure enables the model to better maintain the integrity of the syntax structure and reduce logical errors when generating complex nested queries; the dual-path probability calculation that introduces internal hidden states in the decoding stage solves the column / table omission problem caused by long dependencies in traditional methods; the proposed dual accuracy evaluation method (score / acc) based on the syntax tree framework structure and leaf-deleted node comparison not only measures the correctness of the final SQL, but also locates the source of errors (such as syntax structure errors or column name matching errors), providing more refined optimization directions for model iteration, and is more practical than the existing technology that only focuses on execution accuracy evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flowchart of the semantic parsing method for natural language conversion structure query sentences based on deep learning provided by the present invention; Figure 2 This is a schematic diagram of the SQL conversion model structure of the semantic parsing method for converting natural language into structured query statements based on deep learning provided by the present invention; Figure 3 Schematic diagram of the loss value change of the model of the natural language conversion structure query statement semantic parsing method based on deep learning provided by the present invention; Figure 4 This is a schematic diagram of the accuracy change of the natural language conversion structure query statement semantic parsing method based on deep learning provided by the present invention. DETAILED DESCRIPTION

[0023] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0025] Example 1

[0026] The present invention provides a semantic parsing method for natural language conversion structure query sentence based on deep learning, such as Figure 1 Shown, including: S1. Using natural language and database to perform pattern connection processing to obtain the semantic association pattern connection processing result of the initial natural language; S2. Using the semantic association pattern connection processing results, a natural language basic conversion model is established based on deep learning to obtain structured query statement intermediate data; S3. Obtain a semantic analysis result of the natural language converted structured query statement according to the intermediate data of the structured query statement.

[0027] S1 specifically includes: S1-1, using the natural language to obtain an initial natural language sequence segment based on the N-Gram model; S1-2, arranging the initial natural language sequence segments in descending order to obtain natural language sequence segments; S1-3. Perform pattern connection processing on the natural language sequence fragment and the database to obtain a semantic association pattern connection processing result of the initial natural language.

[0028] S1-3 specifically includes: S1-3-1. Perform string matching processing on the natural language sequence segment and the database column name and database table name to obtain a string matching processing result; S1-3-2. Determine whether the string matching processing result is an exact match. If so, mark the natural language sequence fragment corresponding to the string matching processing result and the database column name as Question-column-exact-match; otherwise, mark the natural language sequence fragment corresponding to the string matching processing result and the database column name as Question-column-partical-match; S1-3-3. Using the string matching processing result, obtain the natural language sequence segment vector value, database column name vector, database table name vector and string matching processing result type vector of the string matching processing result based on the BERT encoder; S1-3-4. Based on the attention mechanism, the database column name vector and the database table name vector are fused to obtain the column and table vectors with problematic information as follows: , , , , , in Indicates the first i The vector representation corresponding to the token is: Indicates the first i A vector corresponding to the column names, Indicates the modei A vector corresponding to table names.

[0029] S1-3-5. Use the sum of the number of corresponding relationship types of the string matching processing results as the rows of the matrix and the string matching processing result type vectors as the columns of the matrix to establish a trainable vector matrix ; S1-3-6, using the trainable vector matrix The formula for calculating the input vector with the database column name vector, database table name vector, and string matching processing result type vector is as follows: ; S1-3-7, using the column and table vectors of the problem information and the input vector as the semantic association pattern of the initial natural language connection processing result; Among them, C represents all column names in the database, and T represents all table names in the database. For the i A string matching result type vector, is the vector corresponding to the k-th column name, is the vector corresponding to the j-th table name, Corresponding to A vector of relation types labeled during the schema linking phase.

[0030] S2 specifically includes: S2-1. Establishing a natural language basic conversion model based on deep learning using the semantic association pattern connection processing results; S2-2. Obtaining structured query statement intermediate data according to the natural language basic conversion model.

[0031] S2-1 specifically includes: S2-1-1. Use the natural language sequence fragments corresponding to the semantic association pattern connection processing results, the database and the semantic association pattern connection processing results as input targets, and the intermediate representation of the SQL query statement corresponding to the semantic association pattern connection processing results as output targets to establish a natural language basic conversion model based on deep learning.

[0032] S2-2 specifically includes: S2-2-1. The calculation formula for the corresponding node of the semantic association pattern connection processing result obtained by establishing the intermediate data sequence based on LSTM is as follows: ; S2-2-2. Using the natural language basic conversion model to connect the corresponding nodes of the processing results according to the semantic association pattern to obtain the intermediate data of the structured query statement; Among them, Y represents the generated intermediate representation sequence, X and S represent the question sequence and database model respectively, Indicates at time i All actions taken before, Indicates in i The action taken at each time step.

[0033] S3 specifically includes: S3-1, performing decoding processing on the structured query statement intermediate data to obtain a decoding processing result of the structured query statement intermediate data; S3-2. Obtain a conversion processing accuracy rate using the decoding processing result of the intermediate data of the structured query statement to obtain a semantic parsing result of the natural language conversion structured query statement.

[0034] S3-1 specifically includes: S3-1-1. Use the intermediate data of the structured query statement to perform state tracking processing according to the decoder to obtain the internal hidden state. The calculation formula is as follows: ; in is the unit calculation function of the long short-term memory network (LSTM), is the cell state at time t-1 in LSTM, which contains some historical information of the past moments. is the output state at time t-1, is the input at time t, It represents the vector representation of the intermediate representation generation rule at time t-1, is a fusion representation of all outputs of the encoder stage, Represents each input The corresponding weight, Equivalent to a single-layer neural network, the input is and the output status of the previous position , the output is at position Chuhe The matching score at position j.

[0035] S3-1-2. Obtain the action probability of the selected column according to the internal hidden state, and the calculation formula is as follows: ; S3-1-3. Obtain the action probability of the selection table according to the internal hidden state, and the calculation formula is as follows: ; S3-1-4, using the action probability of the selection column and the action probability of the selection table as the decoding processing result of the intermediate data of the structured query statement; S is the selected column in the database mode, Mem is the selected column in the dynamic memory module, is the output vector of the decoder at time t, is a trainable parameter vector, is the vector corresponding to the column name in the dynamic memory module, is a vector of column names in the database schema that are never selected, A vector of table names corresponding to the database schema.

[0036] S3-2 specifically includes: S3-2-1. Obtain, based on the decoding result of the intermediate data of the structured query statement, a syntax tree framework structure with leaf nodes removed and a column name and table name structure without the intermediate data of the structured query statement; S3-2-2. The calculation formula for the conversion processing accuracy is as follows based on the syntax tree framework structure without leaf nodes and the column name and table name structure without the intermediate data of the structural query statement: ; S3-2-3. Obtain semantic analysis results of the natural language conversion structure query statement based on the conversion processing accuracy; in, It represents the predicted label or predicted value, y represents the true label or true value, score represents the score of a sample, and acc represents the accuracy.

[0037] In this embodiment, a semantic parsing method for natural language conversion structure query sentences based on deep learning is used. Figure 2 As shown, the specific implementation includes: (1) Overview of the model architecture: This research model uses the encoder-decoder approach from deep learning as its foundational structure, and employs a higher-dimensional syntax tree as the intermediate representation between natural language and SQL statements. During the encoding phase, pattern connection is used to match and tag semantic entities mentioned in natural language with column and table names in the database. The deep learning model BERT is employed for enhanced data representation. During the decoding phase, a dynamic memory module is employed to record selected column names, improving the recognition rate of database column names. Experimental results demonstrate that compared to other models, the proposed method achieves a significant improvement in the accuracy of SQL statement generation on the Spider dataset, approaching the performance of current state-of-the-art models.

[0038] (2) Coding technical details: This study first uses the N-Gram model to enumerate all sequence segments of length N, where N ranges from 1 to 5. Then, starting from the sequence segment with N=5, string matching is performed with the column names and table names in the database schema in descending order. If the sequence segment and the column name match exactly, the sequence segment and the column name are both marked as Question-column-exact-match. If it is an incomplete match, the sequence segment and the column name are marked as Question-column-partical-match. If the sequence segment matches both the column name and the table name, it is preferentially identified as a match with the column name. To identify and match values ​​in natural language questions, if a sequence segment begins and ends with a single quote, it is identified as a unit value. For unit value matching, ConceptNet is introduced. It is an open, large-scale knowledge graph designed to organize the vast amount of information about entities and concepts in the world into a structured form. Each entity or concept corresponds to a node in the graph, each node has its own attributes, and the relationships between nodes correspond to real-world relationship types. Nodes, attributes, and relationships are all represented in the form of natural language words or phrases. Using this knowledge graph, computers can better search and process questions and understand human intent. The BERT model is also used to jointly encode natural language questions, database schemas, and schema linking results to obtain richer semantic representations. BERT's multi-layered Transformer architecture captures long-range dependencies in natural language, which is crucial for generating complex SQL queries. Specifically, each word in the input sequence is encoded by BERT into a high-dimensional vector representation. For tokens identified in schema linking (such as column and table names), this study further adds and averages the vectors of the relational tag type to enhance their semantic information. For example, if a token is labeled "Question-column-exact-match," its vector representation incorporates the vector of that relation type, more accurately reflecting its association with the database schema during the encoding phase. Furthermore, through the attention mechanism, the model dynamically calculates the correlation weights between different tokens in natural language questions and column and table names in the database schema, generating more targeted vector representations. This process not only improves the accuracy of schema linking but also lays a solid foundation for SQL generation in the subsequent decoding phase.

[0039] The natural language to SQL query conversion process is divided into three stages. The first stage is schema linking, matching the column names and table names in the natural language and database schema, and marking the relationship types of the matched words and column names; the second stage is the inference training of the model. In this embodiment, an end-to-end encoder-decoder is used as the basic structure of the model. The input is the natural language question, the database schema and the result of schema linking, and the output is the intermediate representation of the SQL query statement, such as Figure 2 As shown in Figure 2, in the last stage, the intermediate representation generated by the model is converted into SQL based on rules.

[0040] The input in the figure is the sequence after question segmentation, such as: "[cls] List the qce name, qce bornstate and qce age of the qte heads ordered by qce age [sep] ...[sep] qce name [sep] qte head [sep]", in qce Linking to the mode question-column-exact Abbreviation for relationship, qte for question- table-exact In this embodiment, BERT is used as the encoder. The input natural language question, database pattern and pattern link result are represented by symbols: X={Q,C,T} ,in Q={q 1 ,q 2 ,…,q t },C={c 1 ,c 2 ,..,c m },T ={t 1 ,t 2 ,…t n }， Q is a natural language question, C represents all column names in the database, and T represents all table names in the database. q i ={(x 1 ,…,x t ),τi },q i Represents the token identified in the pattern link part, each q i May consist of multiple words x i composition, τ i Indicates the result of mode linking. Similarly, c i ={(c 1 ,…,c t ),τ i },t i ={(t 1 ,…,t t ),τ i } , where the column name c i and table name t i It may also consist of multiple words. After passing through the BERT encoder, each word in the input sequence will obtain a corresponding vector representation. For tokens identified during the pattern linking process, the final vector representation is taken as the average of the word and relationship tag type vectors. Column names and table names are also averaged by adding them to the relationship tag type vectors. In addition, the attention mechanism is further used to calculate the weights of the tokens in the question Q to obtain the vector representation of the columns and tables containing the question information. The formula is as follows: ; ; in Indicates the first i The vector representation corresponding to the token is: Indicates the first i A vector corresponding to the column names, Indicates the mode i In order to integrate the information of the question sequence, Do the following calculation.

[0041] ; In order to make a more accurate vector representation of the relationship in the pattern link, a trainable vector matrix is ​​constructed , whose rows and columns are the number and type of relationship in the pattern link stage, respectively. The vector length of . Before entering the decoder, Performs vector addition operation.

[0042] ;

[0043] in They are A vector of relation types labeled during the schema linking phase.

[0044] The goal of the decoder is to generate an intermediate representation of the SQL query statement. In this embodiment, LSTM is used to model the sequence generation process of the intermediate representation. The intermediate representation can essentially be regarded as an abstract syntax tree. The relationship between each parent node and child node in the tree corresponds to the generation rule of the intermediate representation. The sequence generation problem of the intermediate representation can be transformed into the problem of finding the type of child node when the parent node is known. The process of solving the child node is divided into three types of behaviors, namely, applying rules (ApplyRelu[r]), selecting columns (SelectColumn[c]), and selecting tables (SelectTable[t]). Among them, ApplyRelu[r] means applying rule r to the parent node to generate a child node; SelectColumn[c] means selecting column c from the database model; SelectTable[t] means selecting the table corresponding to t from the database model. The node generation process is expressed by formula as follows: ; Y represents the generated intermediate representation sequence, X and S represent the question sequence and database schema respectively. Indicates at time i All actions taken before, Indicates in i The action taken per time step is of the type [apply rules, select columns, select tables]. T is the total number of time steps required to generate the intermediate representation. The entire generation process begins at root node Z. Based on the complete grammar set, non-leaf nodes of the tree are generated by applying rules. Leaf nodes are generated by selecting columns or selecting tables. Applying rules generates the basic structure of the abstract syntax tree. The generation principle follows a depth-first, left-to-right approach. When the parent node is known, the intermediate representation rule set is used to find a matching rule for the parent node, and rule R is applied to add child nodes. For example, if the parent node is a Select node, the child nodes can be a single A node or two AA nodes. Nodes C and T in the intermediate representation are considered leaf nodes of the abstract syntax tree. When generating child nodes, if a C or T node is matched, the Select Column (SelectColumn[c]) or Select Table (SelectTable[t]) action is taken.

[0045] (3) Technical details of the decoding stage: LSTM's time series modeling capabilities enable it to effectively handle the problem of sequence generation of intermediate representations. During the decoding process, the actions of each time step are divided into three categories: applying rules, selecting columns, and selecting tables. Among them, "applying rules" is responsible for generating the basic structure of the abstract syntax tree, while "selecting columns" and "selecting tables" are used to fill leaf nodes. To improve the accuracy of selecting column and table names, this study introduced a dynamic memory module to record the selected column names and avoid repeated selection in subsequent steps. In addition, the decoder also fuses the output of the encoder through the attention mechanism to ensure that the generated intermediate representation is highly consistent with the input natural language question and database schema. Experimental results show that this combination of dynamic memory and attention mechanism significantly improves the generation accuracy of complex SQL queries.

[0046] The decoding phase can be divided into two steps: state tracking and action probability calculation. The decoder generates an internal hidden state at each step and then uses the hidden state to calculate the action probability. In the state tracking phase, the generation formula for each state is ; in is the unit calculation function of the long short-term memory network (LSTM), is the cell state at time t-1 in LSTM, which contains some historical information of the past moments. is the output state at time t-1, The input at time t contains two parts of information: one is the action taken at the previous moment, and the other is the natural language sequence and database model, namely: ,

[0047] in Indicates the vector representation of the intermediate representation generation rule at time t-1. In the actual training process, a trainable vector matrix will be constructed To represent the entire set of application rules, each row of the matrix corresponds to the vector representation of a rule. It is a fusion representation of all the outputs of the encoder stage. It is calculated by applying the attention mechanism to assign a certain weight to each output and finally summing them up. The formula is as follows: ; in Represents each input The corresponding weight, Equivalent to a single-layer neural network, the input is and the output status of the previous position , the output is at positioni Chuhe Location j The matching score at . It will be continuously adjusted during the training process. After obtaining the hidden state, the next step is to use the hidden state to calculate the action probability. Consistent with the three action types, there are three ways to calculate the action probability, as follows: ; in is nonlinear function, It's a rule r When calculating the probability of selecting a column, a dynamic memory module is introduced to dynamically store the selected columns. When a column is selected as a leaf node of the abstract syntax tree at time t, the column is stored in the dynamic memory module and then deleted from the database schema. The probability of selecting a column is calculated as follows: ; Where S means selecting columns from the database schema, and Mem means selecting columns from the dynamic memory module. is the output vector of the decoder at time t, is a trainable parameter vector, Represents the vector corresponding to the column name in the dynamic memory module, A vector of column names that are never selected in the database schema. The probability of selecting a table is calculated as follows: ; in A vector representing table names in the database schema.

[0048] When generating table and column names, the decoder first predicts the column name and then the table name. After generating a complete abstract syntax tree, it performs a relationship check on the column and table names in the syntax tree. Based on the column-table dictionary generated in the preprocessing phase, it determines whether the generated column belongs to the generated table. If not, the column is replaced with the table to which the column belongs according to the column-table dictionary.

[0049] (4) Experimental results and analysis: Accuracy is used to measure the results generated by the model. Since the intermediate representation used can essentially be represented by an abstract syntax tree, we divide the accuracy into two forms. One is the prediction accuracy of the syntax tree framework structure without the leaf nodes, that is, removing the column names and table names in the intermediate representation, which is called sketch accuracy. The other is the accuracy including the leaf nodes, that is, the complete intermediate representation, which is called logical accuracy. For example, for the intermediate representation of "Z(3) R(4) Sel(0) N(2) A(0) C(2) T(1) A(0) C(8) T(1) A(0) C(9) T(1) Order(1) A(0) C(9) T(1)", after removing its leaf nodes, it becomes "Z(3) R(4) Sel(0) N(2) A(0)A(0) A(0) Order(1) A(0)". Then the prediction accuracy of these two labels is respectively used. One benefit of dividing the accuracy into sketch accuracy and logical accuracy is that it helps analyze where the mismatch between the predicted label and the true label is most likely to occur, making it easier to locate the problem. The accuracy calculation formula is as follows: ; in It represents the predicted label or predicted value, y represents the true label or true value, score represents the score of a sample, and acc represents the accuracy.

[0050] The deep learning framework PyTorch is used to implement the model architecture. The dimension of the rule vector of the intermediate representation is set to 300, and the node type vector size of the abstract syntax tree is set to 300. The input is natural language, database schema, and the result of schema linking. The output is the intermediate representation. After 50 iterations, the loss value of the model is as follows Figure 3 As shown, the accuracy is Figure 4 shown.

[0051] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0052] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0053] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0054] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A semantic parsing method for natural language conversion structured query statements based on deep learning, characterized by: include: Using natural language and database to perform pattern connection processing to obtain semantic association pattern connection processing results of initial natural language; Using the semantic association pattern connection processing results, a natural language basic conversion model is established based on deep learning to obtain structured query statement intermediate data; A semantic parsing result of the natural language conversion structured query statement is obtained according to the intermediate data of the structured query statement.

2. The method for semantic parsing of natural language structured query statements based on deep learning according to claim 1, characterized in that: The method of performing pattern connection processing on the natural language and the database to obtain the semantic association pattern connection processing result of the initial natural language includes: Obtaining an initial natural language sequence segment based on the natural language based on an N-Gram model; Arrange the initial natural language sequence segments in descending order to obtain natural language sequence segments; The natural language sequence fragments are used to perform pattern connection processing with a database to obtain a semantic association pattern connection processing result of the initial natural language.

3. The method for semantic parsing of natural language structured query statements based on deep learning according to claim 2, characterized in that: The semantic association pattern connection processing result of the initial natural language obtained by performing pattern connection processing on the natural language sequence segment and the database includes: Performing string matching processing on the natural language sequence fragment and the database column name and database table name to obtain a string matching processing result; Determine whether the string matching processing result is an exact match. If so, mark the natural language sequence segment corresponding to the string matching processing result and the database column name as Question-column-exact-match; otherwise, mark the natural language sequence segment corresponding to the string matching processing result and the database column name as Question-column-partical-match; Using the string matching processing result, based on the BERT encoder, a natural language sequence segment vector value, a database column name vector, a database table name vector, and a string matching processing result type vector of the string matching processing result are obtained; The calculation formula for the column and table vectors with problematic information is as follows: ， ， ， ， ， in Indicates the first i The vector representation corresponding to the token is: Indicates the first i A vector corresponding to the column names, Indicates the mode i A vector corresponding to each table name; The trainable vector matrix is ​​established by using the sum of the number of corresponding relationship types of the string matching processing results as the rows of the matrix and the type vectors of the string matching processing results as the columns of the matrix. ; Using the trainable vector matrix The formula for calculating the input vector with the database column name vector, database table name vector, and string matching processing result type vector is as follows: ； Using the column and table vectors of the problem information and the input vector as the semantic association pattern connection processing results of the initial natural language; Among them, C represents all column names in the database, and T represents all table names in the database. For the i A string matching result type vector, is the vector corresponding to the k-th column name, is the vector corresponding to the j-th table name, Corresponding to A vector of relation types labeled during the schema linking phase.

4. The method for semantic parsing of natural language structured query statements based on deep learning according to claim 3 is characterized in that: The structured query statement intermediate data is obtained by using the semantic association pattern connection processing result to establish a natural language basic conversion model based on deep learning, including: Using the semantic association pattern connection processing results to establish a natural language basic conversion model based on deep learning; The structured query statement intermediate data is obtained according to the natural language basic conversion model.

5. The method for semantic parsing of natural language-converted structured query statements based on deep learning according to claim 4, characterized in that: Using the semantic association pattern connection processing results to establish a natural language basic conversion model based on deep learning includes: A natural language basic conversion model is established based on deep learning using the natural language sequence fragments corresponding to the semantic association pattern connection processing results, the database and the semantic association pattern connection processing results as input targets, and the SQL query statement intermediate representation corresponding to the semantic association pattern connection processing results as output targets.

6. The method for semantic parsing of natural language-to-structured query statements based on deep learning according to claim 5, characterized in that: The intermediate data of the structured query statement obtained according to the natural language basic conversion model includes: The calculation formula for the corresponding nodes of the semantic association pattern connection processing result obtained by establishing the intermediate data sequence based on LSTM is as follows: ； Utilizing the natural language basic conversion model to connect the corresponding nodes of the processing results according to the semantic association pattern to obtain the intermediate data of the structured query statement; Among them, Y represents the generated intermediate representation sequence, X and S represent the question sequence and database model respectively, Indicates at time i All actions taken before, Indicates in i The action taken at each time step.

7. The method for semantic parsing of natural language-to-structured query statements based on deep learning according to claim 6, characterized in that: Obtaining a semantic analysis result of a natural language conversion structured query statement according to the structured query statement intermediate data includes: Using the structured query statement intermediate data to perform decoding processing to obtain a decoding processing result of the structured query statement intermediate data; The decoding processing result of the intermediate data of the structured query statement is used to obtain the conversion processing accuracy to obtain the semantic parsing result of the natural language conversion structured query statement.

8. The method for semantic parsing of natural language-to-structured query statements based on deep learning according to claim 7, characterized in that: The decoding process of the structured query statement intermediate data to obtain the decoding process result of the structured query statement intermediate data includes: The intermediate data of the structured query statement is used to perform state tracking processing according to the decoder to obtain the internal hidden state, which is calculated as follows: ； in is the unit calculation function of the long short-term memory network (LSTM), is the cell state at time t-1 in LSTM, which contains some historical information of the past moments. is the output state at time t-1, is the input at time t, It represents the vector representation of the intermediate representation generation rule at time t-1, is a fusion representation of all outputs of the encoder stage, Represents each input The corresponding weight, Equivalent to a single-layer neural network, the input is and the output status of the previous position , the output is at position Chuhe The matching score at position j; The action probability of the selected column is obtained according to the internal hidden state, and the calculation formula is as follows: ； The action probability of the selection table is obtained according to the internal hidden state, and the calculation formula is as follows: ； Using the action probability of the selection column and the action probability of the selection table as the decoding processing result of the intermediate data of the structured query statement; S is the selected column in the database mode, Mem is the selected column in the dynamic memory module, is the output vector of the decoder at time t, is a trainable parameter vector, is the vector corresponding to the column name in the dynamic memory module, is a vector of column names in the database schema that are never selected, A vector of table names corresponding to the database schema.

9. The method for semantic parsing of natural language-converted structured query statements based on deep learning according to claim 8, characterized in that: Obtaining the conversion processing accuracy using the decoding processing result of the intermediate data of the structured query statement to obtain the semantic parsing result of the natural language conversion structured query statement includes: According to the decoding processing result of the intermediate data of the structured query statement, respectively obtain the syntax tree framework structure with leaf nodes removed and the column name and table name structure without the intermediate data of the structured query statement; The calculation formula for the conversion processing accuracy is as follows based on the syntax tree framework structure without leaf nodes and the column name and table name structure without the intermediate data of the structural query statement: ； Obtaining a semantic analysis result of a natural language conversion structure query statement according to the conversion processing accuracy; in, It represents the predicted label or predicted value, y represents the true label or true value, score represents the score of a sample, and acc represents the accuracy.

Images