Method and device for converting natural language text into SQL (Structured Query Language) statement

Abstract

The invention discloses a method and a device for converting a natural language text into an SQL (Structured Query Language) statement, which are used for converting a natural language problem into a structured query language (SQL) and realizing efficient and accurate text-to-SQL conversion. Comprising a relation perception self-attention mechanism, interaction graph construction and relation propagation, a low-rank adaptation fine tuning strategy and a decoding optimization method. By innovatively and deeply fusing a relation perception mechanism and a self-attention module and introducing low rank adaptation (LoRA) and a decoding optimization strategy, high efficiency and accuracy of a text-to-SQL conversion task are realized. The invention also provides a computer system which comprises a processor, a memory and an input / output interface and is used for receiving the text and outputting the converted SQL. According to the model, the accuracy and generalization ability of complex query generation are remarkably improved by fusing relation information of a database mode and a natural language, and the model can be widely applied to an intelligent question-answering system, a data query auxiliary tool and an automatic database management system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing and database interaction, specifically a LLaMA model combined with a relationship perception mechanism and self-attention enhancement for converting natural language text into structured SQL query statements. This technology can be applied to intelligent assistants, data management systems, automated query generation systems, conversational database interfaces, and other scenarios to improve human-computer interaction efficiency and query accuracy, especially in handling complex multi-table associations, nested conditions, and dynamic query requirements. BACKGROUND

[0002] With the advancement of semantic web technology, numerous knowledge bases have emerged to meet the growing demand for structured knowledge storage. In knowledge base question-answering systems, the task of converting natural language questions into formal query languages is particularly important. Existing methods for converting natural language text into SQL generally face the following challenges: semantic understanding and structure conversion mismatch: natural language text contains ambiguous words, omitted expressions, and other ambiguities, while SQL statements require strict syntax and structure, making it difficult for traditional methods to accurately capture the mapping relationship between semantics and structure; complex relationship capture deficiency: multi-table connections, nested clauses, dynamic conditions, and other complex relationships in text are difficult for existing models to effectively analyze and model; attention mechanism limitations: fixed or static attention cannot dynamically adapt to the association characteristics of different queries, leading to missing or incorrect associations of key information.

[0003] Existing technical solutions such as BERT-based encoder-decoder models and rule template matching methods have some effect, but have high error rates when handling complex queries and limited generalization ability. Therefore, there is an urgent need for a technical solution that can dynamically perceive the semantic and structural association between text and SQL and improve query generation accuracy through enhanced attention mechanisms. SUMMARY

[0004] The purpose of the present application is to provide a method and device for converting natural language text into SQL statements, which introduces a relationship perception module and dynamic self-attention mechanism to achieve high-precision conversion of natural language text to SQL statements.

[0005] To achieve the above invention purposes, the technical solutions adopted by the present application include the following specific steps:

[0006] S1, obtain a labeled dataset D = {(T1, Q1), (T2, Q2),..., (Tn, Qn)} containing natural language query text and corresponding SQL statements, covering single-table queries, multi-table joins, nested clauses, aggregation functions, and other scenarios, where T represents the original text and Q represents the parsed SQL. n n

[0007] ​​S2, multi-modal feature fusion: text feature enhancement: deeply fuse text semantics and SQL structure features, use bidirectional GRU to capture the context dependency of text, output enhanced vectors and preprocess text T by word segmentation, stop word removal, part-of-speech tagging, etc., and use a pre-trained language model to generate d1-dimensional semantic vectors T ′ The conversion formula is:

[0008]

[0009] Among them, WordEmbedding(T) is the word segmentation of text T, SQL structure enhancement: sql uses graph neural network to model the node relationship of SQL syntax tree, including operator, table name, field, etc., Q extracts table name, field, operator, condition expression, aggregation function, etc., and converts it into a structured feature vector Q ′ , and constructs a SQL syntax tree, where Q ′ The conversion formula is:

[0010]

[0011] Fusion operation: form joint representation vector W by concatenation and linear transformation:

[0012] W = W1·T ′ +W2·Q ′ +b

[0013] Where W1, W2 are learnable weight matrices, and b is the bias term.

[0014] S3, build a relationship-aware self-attention layer to dynamically calculate the association weight between text and SQL, the input of self-attention is a series of n embeddings Where Then it calculates its output z as follows, || represents the connection operation:

[0015]

[0016] Where H is the number of heads, and is a learnable weight. And is two different relationship embeddings used to represent the relationship r between the i-th and j-th tokens.

[0017] S4, the model of the invention inherits the transformer decoder architecture of LLaMA, which is composed of N layers of decoder. The self-attention module in the decoder is replaced by a relationship-aware self-attention, and two additional relationship embedding lookup tables R k and R vWe convert the sequence input into an interaction graph by introducing various types of relations and adapting them to the subword level. In the forward process, the relation-aware self-attention module reads the relationship between each token through the interaction graph and retrieves the corresponding relationship embedding from the lookup table R k and R v The calculation method is as follows:

[0018]

[0019] where t i is the table name, c ij is the jth column name of the ith table, and the input of the model is the question Q and the database schema S. v∈V appearing after the column c 11 is the database content in the column with n-gram matching in the token in the question. The separator “|” represents the boundary between Q, S and different tables. Within each table, we use “:” to separate the table name and its columns, and use “,” as the separator between each column. To fine-tune the LRASA model, the model inherits all the parameters from LLaMA and randomly initializes the additional relationship embeddings introduced by the relation-aware self-attention. Interaction graph construction and relationship propagation: Using relation-aware self-attention, various types of relationships are represented as a triple, thereby constructing an interaction graph containing semantic relationships, with input sequence tokens as vertices and relationships as directed edges, expressed as:

[0020] <H,r,T>

[0021] where H, T are the head and tail items in the triple, and r represents the relationship. Relationship propagation is achieved through a cross-modal propagation mechanism, which propagates schema-level relationships (such as foreign key constraints) to the subword level, ensuring that each token can perceive global relationship information. Loss function calculation: A combined loss is used to balance semantic and structural accuracy, ensuring the accuracy of the generated SQL function, and the loss function is calculated as follows:

[0022] L = λ1L_cel + λ2L_struct + λ3L_flux

[0023] where the cross-entropy loss L_cel measures the symbol-level difference between the predicted sequence and the true SQL, the structural similarity loss L_struct evaluates the structural difference between the generated SQL and the true SQL based on the edit distance or tree structure similarity, and the flux regularization loss L_flux encourages attention flow to focus on key association areas and avoid excessive dispersion.

[0024] S5, SQL query generation and optimization, input the text T to be converted, after preprocessing and model inference, output the predicted SQL sequence M, and use the syntax checking module to check the legality of M based on the database schema, including whether the table name / field exists, and whether the operator combination is correct. Rewrite the SQL with low performance, such as merging consecutive conditions, adjusting the order of clauses, and finally output the standardized SQL query statement to ensure execution efficiency and accuracy.

[0025] The application also provides a computer readable storage medium, which stores a computer program, when the program is executed by a processor, the processor executes a method for converting natural language text into SQL statements.

[0026] The application also provides a computer system, comprising:

[0027] The processor is configured to execute instructions stored in the storage medium to implement a method for converting natural language text into SQL statements.

[0028] The memory is configured to store the text and intermediate results generated during the computation of a method for converting natural language text into SQL statements.

[0029] The input / output interface is configured to receive specified text or related information and output converted SQL.

[0030] A method and device for converting natural language text into SQL statements, characterized in that the device is configured for data acquisition and preprocessing, multi-modal feature fusion, relationship-aware self-attention, LLaMA model enhancement and training, SQL query generation and optimization.

[0031] The method and device for converting natural language text into SQL statements have the following effects:

[0032] 1. Multi-modal feature fusion: combine text semantics and SQL structure features to construct a joint semantic space.

[0033] 2. Relationship-aware self-attention layer: dynamically capture the complex association between text and SQL, and strengthen key relationship features.

[0034] 3. Combined loss function: consider cross-entropy loss and structural similarity loss to optimize model training effect.

[0035] 4. Syntax checking and optimization: based on the database schema, the generated SQL is checked for legality and performance optimization. This technology significantly improves the accuracy of complex query generation, reduces semantic ambiguity, and enhances the model's adaptability to different database schemas. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a flowchart of the method as a whole;

[0037] Figure 2 is a vectorization diagram for parsing. DETAILED DESCRIPTION

[0038] The specific embodiments of the present application are described below to facilitate understanding of the present application by those skilled in the art, but it should be clear that the present application is not limited to the scope of the specific embodiments, and that all applications utilizing the concept of the present application are within the scope of protection as long as various changes are obvious to those skilled in the art within the spirit and scope of the present application as defined and determined by the appended claims.

[0039] The technical solutions of the present application are described in detail below through a multi-table association query specific embodiment. The input text is: "Query the average R&D investment of technology enterprises in Shanghai Pudong New Area in the first quarter of 2025, and sort by total R&D investment."

[0040] S1: Data acquisition and preprocessing

[0041] Text preprocessing:

[0042] Tokenization and vectorization: After tokenizing the text, generate semantic vectors T through a pre-trained model. Assume that the tokenization result is [query, 2025 first quarter, Shanghai Pudong New Area, technology enterprises, average R&D investment, by, total R&D investment, sort].

[0043] Word vector dimension: After encoding, the dimension of each word vector is d1=512, so the dimension of the text T is

[0044] SQL parsing and vectorization:

[0045] Parse SQL structure: extract table name (enterprise table, region table), field (R&D investment), operator (SELECT, JOIN, WHERE, ORDER BY), condition (2023 second quarter, Shanghai Pudong New Area, technology enterprises).

[0046] Use graph neural network (GNN) to encode the SQL syntax tree, and assume that the generated structure vector Q has a dimension of d2=256, i.e. S2: Multi-modal feature fusion

[0047] Text feature enhancement:

[0048] Use bidirectional GRU (Bidirectional GRU) to capture context dependence, output enhanced vector T ′ . Assume that the GRU hidden layer dimension is d ′1 = 2d1=1024, then

[0049] SQL structure enhancement:

[0050] Apply GNN to the SQL syntax tree to get the enhanced structure vector Q', assuming the dimension d ′ 2 = 2d2=512, i.e.

[0051] Fusion operation:

[0052] Linear transformation: use weight matrices W1 and W2, assuming bias Then the fusion result W = W1·T ′ +W2·Q ′ +b.

[0053] S3: Construct relationship-aware self-attention

[0054] Construct relationship-aware self-attention, where is a learnable weight. And do normalization processing and connection operation, finally get z i .

[0055] S4: LLaMA model enhancement and training

[0056] Input z i to the LLaMA model, output the probability distribution of SQL symbols where each element corresponds to the probability of a SQL symbol (such as SELECT, FROM, table name, field name, etc.).

[0057] Loss function calculation: cross-entropy loss L_CEL: measure the symbol-level difference between predicted symbols and real SQL; structure similarity loss L_Struct: use edit distance to evaluate the structural difference between generated SQL and real SQL; combined loss L = λ1L_cel + λ2L_struct + λ3L_flux (where λ1 = 0.8, λ2 = 0.2, λ3 = 0.1).

[0058] S5: SQL query generation and optimization

[0059] Generate SQL sequence: The decoder outputs the symbol sequence with the highest probability to generate the initial SQL. Syntax checking and optimization, check if the table name / field exists (e.g. "R&D expenses" should be "R&D input"), adjust field names, optimize time range expressions, convert to continuous date intervals, and finally generate a legal and efficient SQL. Generate SQL: "SELECT AVG(R&D expenses) AS average, SUM(R&D expenses) AS total FROM enterprise data table JOIN region information table ON enterprise data table.region code = region information table.code WHERE region name = 'Pudong' AND enterprise type = 'technology enterprise' AND time range BETWEEN '2025-01-01' AND '2025-03-30' ORDER BY total DESC".

Claims

1. A method of converting natural language text into SQL statements, characterized by, Comprise the following steps: S1: data acquisition and pretreatment, obtain the annotated data set containing natural language query text and corresponding SQL statement; S2: multi-modal feature fusion, deep fusion of text semantics and SQL structure features; S3: relationship-aware self-attention module, build relationship-aware self-attention layer, dynamically calculate the association weight between text and SQL; S4: LLaMA model enhancement and training; S5: SQL query generation and optimization.

2. The method of claim 1, wherein, In step S1, an annotated dataset D = {(T1,Q1),(T2,Q2),...,(T...} containing natural language query text and corresponding SQL statements is obtained. n Q n This covers scenarios such as single-table queries, multi-table joins, nested clauses, and aggregate functions, where T represents the original text and Q represents the parsed SQL.

3. The method of claim 1, wherein, In step S2, multi-modal feature fusion is performed: 3.1 Text feature enhancement: deep fusion of text semantics and SQL structure features, using bidirectional GRU to capture the context dependency of text, output enhanced vector and preprocess the text T by word segmentation, stop word removal, part-of-speech tagging, etc., and use the pre-trained language model to generate d1-dimensional semantic vector T', the conversion formula is: Where WordEmbedding(T) is the word segmentation of text T; 3.2 SQL structure enhancement: sql uses graph neural network to model the node relationship of SQL syntax tree, including operator, table name, field, etc., Q extracts table name, field, operator, conditional expression, aggregation function, etc., and converts it into a structured feature vector Q', and builds a SQL syntax tree, the conversion formula of Q' is: 3.3 Fusion operation: joint representation vector W is formed by concatenation and linear transformation: W = W1·T' + W2·Q' + b Where W1, W2 are learnable weight matrices, and b is the bias term.

4. The method of claim 1, wherein, In step S3, a relation-aware self-attention layer is constructed to dynamically calculate the association weight between the text and the SQL, and the input of the self-attention is a series of n embeddings wherein It then computes its output z as follows, with || denoting the concatenation operation: where H is the number of heads, and are learnable weights; and are two different relational embeddings for representing the relationship r between the ith and jth tokens.

5. The method of claim 1, wherein, In step S4, the LLaMA model in this invention inherits the transformer decoder architecture of traditional LLaMA and consists of N layers of decoders; the self-attention module in the decoder is replaced with relation-aware self-attention, and two additional relation embedding lookup tables R are introduced. k and R v , where R k For the mapping relationship, R v The sequence input is transformed into an interaction graph by introducing various types of relationships, such as contextual relationships and key-value relationships, and adapting them to the word level. During the forward pass, the relationship-aware self-attention module reads the relationships between each token through the interaction graph and retrieves them from the lookup table R. k and R v The formula for retrieving the corresponding relation embedding is as follows: where t i is the table name, c ij is the jth column name of the ith table, the input of the model is the question Q, the database schema S; v∈V appearing after the column c 11 is the database content in the column that has n-gram match with the token in the question; the separator “|” indicates the boundary between Q, S and different tables; within each table, “:” is used to separate the table name and its columns; between each column, “,” is used as the separator; in order to fine-tune the LLaMA model of the present application, the model inherits all the parameters from LLaMA and randomly initializes the additional relationship embedding introduced by the relationship-aware self-attention; 5.1 Interaction graph construction and relationship propagation: use relationship-aware self-attention to represent various types of relationships as a triple, thereby constructing an interaction graph containing semantic relationships, using input sequence tokens as vertices and relationships as directed edges, expressed as: <H, r, T> Where H, T are the head and tail items in the triple, and r represents the relationship. Relationship propagation is achieved through cross-modal propagation mechanism to pass mode-level relationships to sub-word level, ensuring that each token can perceive global relationship information. 5.2 Loss function calculation: use combined loss to balance semantic and structural accuracy, and design to ensure the accuracy of the generated sql function, the loss function is calculated as: L = λ1L_cel + λ2L_struct + λ3L_flux Where the cross-entropy loss L_cel measures the symbol-level difference between the predicted sequence and the true SQL, the structure similarity loss L_struct is based on the edit distance or tree structure similarity to evaluate the structural difference between the generated SQL and the true SQL, and the traffic regularization loss L_flux encourages attention flow to focus on key associated areas and avoid excessive dispersion.

6. The method of claim 1, wherein, In step S5, SQL query generation and optimization, input the text T to be converted, after preprocessing and model inference, output the predicted SQL sequence M, and use the syntax checking module to check the legality of M based on the database schema, including whether the table name / field exists and the operator combination is correct; rewrite the SQL with low performance, including merging consecutive conditions and adjusting clause order, and finally output the standardized SQL query statement.

7. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to perform the method of any one of claims 1-6 for converting natural language text into SQL statements.

8. A computer system comprising: a processor configured to execute instructions stored in a storage medium to implement the method of any one of claims 1-6 for converting natural language text into SQL statements; a memory configured to store text and intermediate results generated during the computation of the method for converting natural language text into SQL statements; an input / output interface configured to receive specified text or related information and output converted SQL.

9. An apparatus for converting natural language text into SQL statements, the apparatus comprising: The device is configured to: data acquisition and preprocessing, acquire an annotated dataset containing natural language query text and corresponding SQL statements; multi-modal feature fusion, deeply fuse text semantics and SQL structure features; relationship perception self-attention module, build a relationship perception self-attention layer to dynamically calculate the association weight between the text and the SQL; LLaMA model enhancement and training; SQL query generation and optimization.

Images