Text-to-SQL (Structured Query Language) generation method and device based on enhanced atlas embedding technology

Abstract

The invention discloses a text-to-SQL (Structured Query Language) generation method and device based on an enhanced atlas embedding technology, and the method comprises the following steps: entity extraction: extracting entities such as tables, fields and condition values from a natural language query, and obtaining a database mode and business constraints at the same time; constructing a multi-relation heterogeneous knowledge graph: constructing four types of nodes including tables, fields, values and constraints, establishing semantic edges and injecting attributes; enhanced atlas embedding: obtaining a sparse embedded vector through pre-training word embedding, KG-Adapter, a self-attention mechanism and L2 regularization-pruning; and cross-modal alignment and SQL generation: aligning an embedded vector with a text vector, and inputting a fine-tuned large language model (LLM) to generate a compliant SQL. According to the method, the database structure information can be effectively utilized to analyze the natural language query, the accuracy and efficiency of SQL generation are improved, and meanwhile, reliable intelligent support is provided for complex database query.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of natural language processing (NLP) and intelligent database query, specifically relating to a text-to-SQL generation method and apparatus based on enhanced graph embedding technology. Background Technology

[0002] Text-to-SQL technology is a core bridge connecting natural language and databases. Its goal is to allow non-technical users to directly query databases using natural language without needing to master SQL syntax.

[0003] In recent years, methods such as KG-SQL and GraPPa have simplified the data query process from complex SQL to a more intuitive natural language format by jointly learning database structures and natural language representations. This innovation facilitates user-friendly data querying and analysis, democratizes access to database systems, thereby improving data processing efficiency and expanding its application areas.

[0004] The current mainstream traditional text-to-SQL methods have the following shortcomings: (1) They rely heavily on sequence-to-sequence models, utilizing only text information and ignoring the database table structure and the relationships between fields. In addition, although some models introduce database schema information, they mainly rely on surface features such as table names and field names, failing to deeply explore the implicit semantic relationships and business logic constraints between tables. (2) The modeling information on structured features such as foreign key relationships and value range constraints is incomplete. The graph structure information is not fully utilized, resulting in insufficient semantic understanding of the database by the model and ignoring the semantic connotation of foreign key constraints.

[0005] Therefore, there is an urgent need for a text-to-SQL method that can deeply integrate database structure semantics with LLM, reduce illusions, and adapt to complex scenarios. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a text-to-SQL generation method and apparatus based on enhanced graph embedding technology. Through a four-stage process of entity extraction, graph modeling, enhanced embedding, and cross-modal generation, the database semantics are transformed into a structured graph and embedded into a vector space. After being aligned with the text vector, the graph guides the generation of compliant SQL using an LLM (Limited Language Management) system.

[0007] In a first aspect, this invention proposes a text-to-SQL generation method based on enhanced graph embedding technology, which includes the following steps:

[0008] S1. Entity extraction: Through text preprocessing, named entity recognition, relationship extraction, and normalization, the table names, fields, conditional values, and functions in the natural language query are extracted as a subset of structured entities on the natural language side; meanwhile, database schema information and business constraints are obtained to form a subset of structured entities on the database side; the subset of structured entities on the natural language side and the subset of structured entities on the database side are merged to obtain a complete set of structured entities;

[0009] S2. Construction of a multi-relational heterogeneous knowledge graph: Map the entity set to table nodes, field nodes, value nodes, and constraint nodes, construct semantic edges of 'table-field', 'field-field', 'field–value', 'field-function', and 'field-constraint', and inject data types, value range, and business rule attributes into the nodes / edges to obtain a multi-relational heterogeneous knowledge graph;

[0010] S3. Enhanced graph embedding: Map the graph nodes to initial vectors through pre-trained word embeddings, update the node representations through the KG-Adapter layer and the graph neural network, enhance the features by combining the self-attention mechanism and the multi-layer perceptron, and then obtain the sparsified graph embedding vectors through L2 regularization and pruning; and

[0011] S4. Cross-modal alignment and SQL generation: Align the sparsified graph embedding vectors with the text vectors of the natural language query to form a cross-modal joint representation, and input it into the supervised fine-tuned LLM to generate SQL statements that conform to the SQL syntax specification and database business constraints.

[0012] Preferably, the text preprocessing in step S1 includes word segmentation,词性标注 (this should be 'pos tagging' in English), and stop word removal operations, which are preprocessing for directionally retaining key information in database queries to ensure that the extracted entity subset directly matches the database query requirements. Specifically, word segmentation splits the query text in natural language grammar into semantic units; pos tagging retains key verbs, nouns, and numerical词性 (this should be 'pos' in English); stop word removal eliminates words without database semantic associations including 'of', 'in', 'and'.

[0013] Preferably, the named entity recognition in step S1 is performed for entity types related to database queries, and the entity types include time entities, numerical entities, person entities, place entities, and function entities. After recognition, a mapping pair in the format of {<entity keyword>:<database mapping object>} is output.

[0014] Preferably, the database schema information in step S1 includes table structures, field data types, primary and foreign key association relationships, and index information in the database; the business constraints include field value range constraints, table association constraints, and business logic constraints.

[0015] Preferably, the data type injected into the nodes in step S2 is specifically as follows: table nodes are injected with the database identifier, field nodes are injected with the corresponding data type and the 'whether it is not null' and 'whether it is unique' attributes, value nodes are injected with the data category, and constraint nodes are injected with the constraint type.

[0016] Preferably, the value range in step S2 is obtained through normalization, specifically: for numerical value nodes, min-max normalization is used to map the value range to the [0,1] interval; for time value nodes, they are converted to timestamps and then normalized to ensure the consistency of the value range.

[0017] Preferably, the pre-trained word embedding in step S3 is implemented using a pre-trained model of a large language model (LLM), including BERT series models, Qwen series models, or GPT series models, to ensure that the initial vector of the node is consistent with the dimension of the text vector of the LLM.

[0018] Preferably, the KG-Adapter layer in step S3 integrates the graph neural network output using an instant tuning method. Specifically, the node structural features generated by the graph neural network are used as Prompt embeddings, concatenated with the initial vectors of the pre-trained word embeddings, and then input into the KG-Adapter layer. The fusion of structural features and semantic features is achieved through 1 to 2 layers of linear transformation.

[0019] Preferably, the L2 regularization in step S3 is achieved by adding a sum of squared embedding vectors to the loss function, and the expression of the loss function is: Loss = Loss base +L reg , Among them, Loss base Basic task loss, L reg This is an added term in the loss function, where λ is the regularization coefficient, and v i This represents the i-th embedding vector.

[0020] Preferably, the pruning process in step S3 employs a threshold filtering mechanism, specifically: a preset vector dimension weight threshold is established; the absolute value of the weight of each dimension of each embedded vector is calculated; dimensions with absolute weight values ​​less than the threshold are removed, while key semantic dimensions are retained, thereby reducing the dimension of the sparsified embedded vectors; the preset threshold is γ = 1e-4, with a value range of 0.01 to 0.05; if |v i If |<γ=1e-4, then the dimension is considered to have little information content, and the dimension is removed.

[0021] Preferably, the training samples for supervised fine-tuning in step S4 include two types of labeled data: the first type is natural language query-graph answer node ID pairs, used to optimize the semantic association ability of LLM with graph nodes; the second type is natural language query-standard SQL statement pairs, used to optimize the SQL syntax generation ability of LLM; and the fine-tuning process adopts the cross-entropy loss function, the number of iterations is 50 to 100 rounds, and the learning rate is 1e-5 to 5e-5.

[0022] In a second aspect, embodiments of the present invention provide a text-to-SQL generation apparatus based on enhanced graph embedding technology, used to implement the text-to-SQL generation method as described in any one aspect, including:

[0023] The entity extraction module is used to extract table names, fields, condition values, and functions from natural language queries into natural language-side structured entity subsets through text preprocessing, named entity recognition, relation extraction, and standardization; at the same time, it obtains database schema information and business constraints to form database-side structured entity subsets; and merges the natural language-side structured entity subsets and the database-side structured entity subsets to obtain a complete set of structured entities.

[0024] The knowledge graph construction module is used to map the entity set into table nodes, field nodes, value nodes, and constraint nodes, construct semantic edges of 'table-field', 'field-field', 'field-value', 'field-function', and 'field-constraint', and inject data type, value range, and business rule attributes into the nodes / edges to obtain a multi-relationship heterogeneous knowledge graph;

[0025] An enhanced graph embedding module is used to map the graph nodes to initial vectors through pre-trained word embeddings, update the node representations through a KG-Adapter layer and a graph neural network, enhance features by combining self-attention mechanism and multilayer perceptron, and then obtain sparse graph embedding vectors through L2 regularization and pruning; and

[0026] The model fine-tuning module is used to align the sparse graph embedding vector with the text vector of the natural language query to form a cross-modal joint representation. The input is a supervised fine-tuned LLM to generate an SQL statement that conforms to the SQL syntax specification and database business constraints.

[0027] Thirdly, embodiments of the present invention provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect.

[0028] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0029] Compared with the prior art, the beneficial results of the present invention are as follows:

[0030] This invention combines database structure information with natural language queries. It models tables, fields, values, and constraint nodes using a multi-relational heterogeneous knowledge graph, introducing various semantic relationships such as field-value, field-function, and constraint relationships. Through graph embedding technology and text vector alignment, it achieves accurate question answering in small-scale domains, effectively reducing model overhead. Subsequently, combined with methods such as KG-Adapter, it enables efficient embedding of graph information. Furthermore, L2 regularization and pruning techniques are used to sparsify the embedded vectors, reducing storage and computational overhead while preserving key semantic information. This improves the performance of small question-answering systems in specialized domains and enhances the accuracy and reliability of complex database queries and SQL generation. Attached Figure Description

[0031] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many anticipated advantages of the embodiments will be readily recognized as they become better understood through reference to the following detailed description. Elements in the drawings are not necessarily to scale. The same reference numerals refer to corresponding similar parts.

[0032] Figure 1 This is a flowchart illustrating the text-to-SQL generation method based on enhanced graph embedding technology, according to an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram illustrating the process of constructing the SQL generation model according to an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of the graph embedding generation according to an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram illustrating natural language text entity extraction in a specific embodiment of the present invention.

[0036] Figure 5 This is a schematic diagram of a database entity knowledge graph in a specific embodiment of the present invention;

[0037] Figure 6 This is a schematic diagram illustrating the generation of dialogue results from text-to-SQL conversion using enhanced graph embedding, as shown in a specific embodiment of the present invention.

[0038] Figure 7 This is a schematic diagram of the architecture of a text-to-SQL generation device based on enhanced graph embedding technology according to an embodiment of the present invention;

[0039] Figure 8 This is a schematic diagram of the structure of a computer device suitable for implementing electronic devices according to embodiments of the present invention. Detailed Implementation

[0040] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0041] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0042] This invention utilizes a neural network model to achieve semantic alignment between natural language and database structure, generating grammatically correct SQL statements, aiming to improve the accuracy and efficiency of natural language queries.

[0043] To address the shortcomings of existing technologies, the embodiments of this invention propose the following improvements: (1) Enhanced graph embedding encoding. Database tables and field relationships are modeled as knowledge graphs, and structural and semantic information is encoded into a vector space through enhanced graph embedding technology, achieving efficient representation of table structures and potential relationships between fields. (2) Fusion of textual intent and graph embedding encoding. Textual query intent and enhanced graph embedding features are combined and input into a deep learning model, enabling the generation of highly accurate SQL in complex queries, multi-table relationships, and semantically ambiguous scenarios.

[0044] Firstly, Figure 1 An embodiment of the present invention discloses a text-to-SQL generation method based on enhanced graph embedding technology, such as... Figure 1 As shown, the method includes the following steps:

[0045] S1. Entity Extraction: Through text preprocessing, named entity recognition, relation extraction, and standardization, table names, fields, condition values, and functions in natural language queries are extracted into natural language-side structured entity subsets; at the same time, database schema information and business constraints are obtained to form database-side structured entity subsets; the natural language-side structured entity subsets and database-side structured entity subsets are merged to obtain a complete set of structured entities.

[0046] Specifically, the text preprocessing in this step includes word segmentation,词性标注, stop word removal, which is preprocessing for directionally retaining key information for database queries, ensuring that the extracted entity subset directly matches the database query requirements, specifically including: word segmentation splits the query text into semantic units according to natural language grammar;词性标注retains key verbs, nouns, and numerical词性; stop word removal eliminates words without database semantic associations such as '的', '在', '和'.

[0047] Named entity recognition is performed for entity types related to database queries. The entity types include time entities, numerical entities, person entities, place name entities, and function entities. After recognition, a mapping pair in the format of {<entity keyword>: <database mapping object>} is output.

[0048] Database schema information includes table structures, field data types, primary and foreign key association relationships, and index information in the database; the business constraints include field value range constraints, table association constraints, and business logic constraints.

[0049] S2. Construction of a multi-relational heterogeneous knowledge graph: Map the entity set to table nodes, field nodes, value nodes, and constraint nodes, construct semantic edges of 'table - field', 'field - field', 'field – value', 'field - function', 'field - constraint', and inject data types, value range, and business rule attributes into the nodes / edges to obtain a multi-relational heterogeneous knowledge graph;

[0050] Specifically, injecting data types into nodes in this step is specifically as follows: Inject the database identifier to which the table node belongs, inject the corresponding data type and attributes of '是否非空' and '是否唯一' to the field node, inject the data category to the value node, and inject the constraint type to the constraint node.

[0051] The value range is obtained through normalization. Specifically: For numerical value nodes, use min-max normalization to map the value range to the [0,1] interval; for time value nodes, convert to a timestamp and then perform normalization to ensure the consistency of the value range.

[0052] S3. Enhanced graph embedding: Map the graph nodes to initial vectors through pre-trained word embeddings, update the node representations through the KG-Adapter layer and graph neural network, enhance the features by combining self-attention mechanism and multi-layer perceptron, and then obtain a sparsified graph embedding vector through L2 regularization and pruning processing; and

[0053] Specifically, the pre-trained word embedding is implemented using the pre-trained model of a large language model LLM. The pre-trained model includes BERT series models, Qwen series models, or GPT series models, ensuring that the initial vector of the node is consistent with the text vector dimension of the LLM.

[0054] It should be noted that the "词性标注" in the original text seems to be a Chinese term that needs to be replaced with the correct English expression. You can provide the accurate English term for a more accurate translation.L2 regularization is achieved by adding a sum of squared embedding vectors to the loss function, which is expressed as: Loss = Loss base +L reg , Among them, Loss base Basic task loss, L reg This is an added term in the loss function, where λ is the regularization coefficient, and v i This represents the i-th embedding vector.

[0055] The pruning process employs a threshold filtering mechanism, specifically: a preset vector dimension weight threshold is established; the absolute value of the weights for each dimension of each embedded vector is calculated; dimensions with absolute weights less than the threshold are removed, while key semantic dimensions are retained, thus reducing the dimension of the sparsified embedded vectors; the preset threshold is γ = 1e-4, with a value range of 0.01 to 0.05; if |v i If |<γ=1e-4, then the dimension is considered to have little information content, and the dimension is removed.

[0056] S4. Cross-modal alignment and SQL generation: Align the sparse graph embedding vector with the text vector of the natural language query to form a cross-modal joint representation. Input the supervised fine-tuned LLM to generate SQL statements that conform to SQL syntax specifications and database business constraints.

[0057] Specifically, the training samples for supervised fine-tuning contain two types of labeled data: the first type is natural language query-graph answer node ID pairs, which are used to optimize the semantic association ability of LLM with graph nodes; the second type is natural language query-standard SQL statement pairs, which are used to optimize the SQL syntax generation ability of LLM; and the fine-tuning process uses the cross-entropy loss function, with 50 to 100 iterations and a learning rate of 1e-5 to 5e-5.

[0058] The text-to-SQL conversion method based on enhanced knowledge graph embedding proposed in this invention has the core technology of achieving high-precision conversion from natural language to structured query language through the synergistic effect of three key stages: deep knowledge graph modeling, multimodal semantic alignment, and SQL generation.

[0059] The first step is entity extraction. Through text preprocessing, named entity recognition, relation extraction and standardization, table names, fields, condition values ​​and functions in natural language problems are extracted into a set of structured entities. At the same time, database schema information and business constraints are obtained to form a complete set of database entities.

[0060] The second step is knowledge graph modeling, which involves turning entity sets and database schema information into nodes, constructing nodes such as tables, fields, values, and constraints, and connecting them through semantic relationship edges to form a multi-relationship heterogeneous graph. At the same time, attribute information is added to nodes and edges to enhance representation capabilities.

[0061] The third step is graph embedding. Knowledge graph nodes are mapped into vectors through pre-trained word embeddings, and the node representations are updated using KG-Adapter and graph neural networks. Then, feature enhancement is performed by combining self-attention mechanism and multilayer perceptron. Sparse and efficient embedding representations are achieved through L2 regularization and pruning. Finally, the embeddings are aligned with text vectors to form a cross-modal joint representation, providing factual constraints and semantic guidance for LLM, reducing fictitious information in generated content, and improving the accuracy and reliability of question answering and reasoning.

[0062] Finally, in the reasoning mechanism, while ensuring the legality of SQL syntax, the optimal mapping from query intent to database operations is achieved. This technical approach, by introducing enhanced graph embedding, effectively establishes a high-order semantic connection between natural language and database structure, significantly improving the accuracy and interpretability of SQL generation.

[0063] The detailed steps of this method are as follows: Figure 2 As shown. The specific process of this method includes the following steps:

[0064] Step 1: Entity Extraction

[0065] (1) First, entity extraction is performed for the training problem. From the natural language problem, through text preprocessing, named entity recognition, relation extraction and standardization, table names, field names, condition values ​​and functions are extracted and organized into a structured entity list. Entity extraction, also known as named entity recognition, is a key technology in natural language processing. Its purpose is to identify and extract entities with specific semantics from unstructured text and classify them into predefined categories.

[0066] Text preprocessing includes word segmentation, part-of-speech tagging, and stop word removal, retaining key verbs, nouns, and numerical values. Subsequently, entities potentially related to the database are identified, such as time, numerical values, and names / places. The final result is an entity set {<keyword>:<field>}. For example, if a user inputs "find the total order amount of Mr. Li in 2021", the entity extraction result will be a candidate set of table names. <users>The time is 2021, the user is Li, and thus the entity collection (table, field, value, function, condition) is obtained.

[0067] (2) Obtain database entities

[0068] First, extract tables, fields, primary and foreign keys, indexes, types, and constraints. Based on the entity set and database schema information, construct a database knowledge graph.

[0069] For example, the Users table contains fields<user_id> int <username>VARCHAR(100). Then, business rules are established, such as "orders must be associated with a user" and "amount ≥ 0", etc. Finally, the schema and constraint set are obtained.

[0070] Step 2: Knowledge Graph Modeling

[0071] (1) Constructing graph nodes

[0072] Based on the entity set (tables, fields, values, functions, conditions) and database schema and constraint information obtained in Step 1, different types of objects are first represented as nodes: Table Node: corresponds to each table in the database, such as Users and Orders; Column Node: corresponds to specific fields in the table, such as Users.user_id, Users.username, and Orders.amount; Value Node: corresponds to conditional values ​​in natural language questions, such as the time value "2021", the user "Li", and the value "100"; Constraint Node: corresponds to primary keys, foreign keys, NOT NULL, business rules, etc.

[0073] (2) Constructing graph edges

[0074] Establish semantic relationships between nodes to form directed or undirected knowledge graphs: Table-field relationships (contains): e.g., Users → username; Field-field relationships (foreign key / primary / foreign key): e.g., Users.user_id Orders.user_id; Field-value relationship (hasValue): e.g., username Li, order_date 2021; Field-Function Relationship (applyTo): e.g., amount SUM; Field-Constraint Relationship (satisfy): e.g., Orders.amount Amount ≥ 0.

[0075] (3) Injecting attribute information

[0076] Attach attributes to nodes and edges to enhance subsequent embedding modeling: Table / field attributes: data type (int / varchar / date), index, uniqueness, whether enumerated; Value attributes: data type (time, numeric, string), range, normalization result.

[0077] (4) Forming a database knowledge graph

[0078] A multi-relation heterogeneous graph is constructed using the nodes, edges, and attributes described above, and its representation is as follows:

[0079] G=(V,E,Attr) (1)

[0080] Where V is the set of nodes, E is the set of edges, and Attr is the additional attribute feature. This forms a graph representation of the database structure.

[0081] Step 3: Graph Embedding

[0082] Large Language Models (LLMs) are deep learning models trained on massive amounts of text data, capable of understanding and generating human language. Knowledge graphs, on the other hand, represent entities and their relationships in a structured way, providing LLMs with explicit factual information and logical constraints, thereby improving the model's accuracy and interpretability in tasks such as reasoning, question answering, and decision-making. Furthermore, the efficient computing power and optimized pre-training structure of LLMs reduce energy consumption during training and inference, enhancing the model's ability to process large-scale knowledge graphs.

[0083] (1) First, construct the graph embedding. See details for this step. Figure 3 We propose a knowledge graph-based embedding method to guide large language models (LLMs), thereby reducing the occurrence of factual inaccuracies or fictional content (i.e., AI illusions) in the text they generate.

[0084] The graph structure consists of nodes and edges. First, each node is mapped to a vector through the pre-trained word embedding of a large language model (LLM) to obtain its semantic information embedding, as shown in formula (2).

[0085]

[0086] Where Emb(n) represents the original graph embedding of the database table. Word embeddings are generated for each node. Word embedding is a method that maps words in natural language to a continuous vector space.

[0087] These hidden vectors are summed to obtain multi-word level representations, and the vector dimensions are compressed to the same size as the pre-trained encodings using a multilayer perceptron (MLP).

[0088]

[0089] In equation (3), Multi-word level representation for LLM word embeddings. Subsequently, graph embedding representation. The input is fed into the KG-Adapter layer, where KG represents the node-centric updates that will be performed via a graph neural network (GNN). KG-Adapter is a parameter-efficient ensemble method based on on-the-fly tuning. It introduces a novel adapter structure to the decoder LLM, encoding CKG from the perspectives of nodes and relation centers for joint inference with LLM.

[0090]

[0091] In equation (4), h ' n This is the vector representation output by the KG-Adapter layer. Subsequently, to determine which node information in the graph is most important to the current node, the SA model, or Self-Attention, is used. SA helps the model fuse information from different nodes in the graph, generating context-sensitive semantic representations. See formula (5) for details.

[0092] H = SA(h) n ′) (5)

[0093] A multilayer perceptron (FFN) typically consists of two linear mapping layers with an activation function (such as ReLU or GELU) in between to enhance the features of each node. The output of equation (5) is used as input to the FFN, as detailed in equation (6), so that each node representation can capture more complex features.

[0094]

[0095] To make the model more lightweight, L2 regularization is incorporated during training, which involves adding a term to the loss function:

[0096]

[0097] In equation (7), v i Let represent the i-th embedding vector. The preset threshold γ = 1e-4, if |v i If |<γ=1e-4, then that dimension is considered to have low information content and can be pruned. Regularization makes the embedding vector exhibit sparse properties, thereby reducing storage and computational overhead while preserving semantic information. Pruning, as a typical method, reduces redundancy by removing dimensions with low information content or weights close to zero from the vector, achieving efficient compression of the embedding. Finally, we obtain... This involves learning joint embedding representations. After obtaining the initial knowledge graph embeddings, they are aligned with the text vectors of the question to form a cross-modal joint representation.

[0098] Step 4: Fine-tuning the large language model

[0099] To make knowledge graph embeddings more suitable for question-answering tasks while preserving their semantic and structural information, this method employs supervised fine-tuning. During fine-tuning, question-answering data is first prepared, with each sample including a natural language question and its corresponding answer node ID. Then, the question is converted into a vector using a text encoder, and mapped to the same vector space as the KG node embedding. The probability of each node is predicted using similarity calculation. Finally, the model is trained using cross-entropy loss to optimize the embedding, ensuring it retains the original structural and semantic information while effectively supporting question-answering tasks.

[0100] In one specific embodiment, the steps are as follows:

[0101] Step 1: Entity Extraction

[0102] (1) Natural Language Problem: Text Entity Extraction

[0103] First, after the user inputs the natural language question "Find the total order amount of Mr. Li in 2021", the question undergoes text preprocessing, including word segmentation, part-of-speech tagging, and basic cleaning, breaking the sentence down into the smallest processable units. Then, entity recognition technology is used to extract key entities, such as time (2021), person (Mr. Li), attribute (order amount), and operation (total), and these are organized into a structured entity list. Next, these entities are mapped to corresponding nodes in a knowledge graph; for example, time corresponds to "node_time", user to "node_user", attribute to "node_attribute", and operation to "node_operation". The extraction process is detailed below. Figure 4 .

[0104] (2) Database table field entity extraction

[0105] During the database table entity extraction process, the table structure is first parsed, and each table is identified as an entity node (such as "Users" and "Orders"). Then, each field in the table is identified as an attribute node (such as user_id, username, order_date, etc.). At the same time, the type and constraint information of the fields (such as primary key, foreign key, uniqueness, etc.) are extracted.

[0106] Step 2: Knowledge Graph Modeling

[0107] In the process of converting database tables into a graph structure, each table is first treated as a graph node; for example, "Users" and "Orders" are created as table nodes. Then, the fields in the tables are mapped to attribute nodes, such as user_id, username, and email in the "Users" table, and order_id, user_id, and order_date in the "Orders" table. Next, foreign key constraints are used to establish relationships between the tables; for example, "Orders.user_id" points to "Users.user_id," forming an edge link from the "Orders" node to the "Users" node in the knowledge graph. Ultimately, this transformation converts the structured information of the original relational database into a graph structure. The graph structure is shown below. Figure 5 As shown.

[0108] Step 3: Fine-tune the large language model

[0109] The fine-tuned LLM takes a union representation as input and generates SQL through a conditional generation mechanism. The model can automatically identify query types (such as SELECT, JOIN, GROUP BY, etc.) and filtering conditions, and generate SQL statements that conform to the syntax rules.

[0110] Step 5: Enhance the graph embedding to generate dialogue results via text-to-SQL conversion.

[0111] When a user inputs the query "{Find the total order amount of Mr. Li in 2021}", the model performs syntax and logic checks on the generated SQL, including field existence validation, type matching, and constraint checks, and then responds to the user, such as... Figure 6 As shown.

[0112] This invention combines database structure information with natural language queries. It models tables, fields, values, and constraint nodes using a multi-relational heterogeneous knowledge graph, introducing various semantic relationships such as field-value, field-function, and constraint relationships. Through graph embedding technology and text vector alignment, it achieves accurate question answering in small-scale domains, effectively reducing model overhead. Subsequently, combined with methods such as KG-Adapter, it enables efficient embedding of graph information. Furthermore, L2 regularization and pruning techniques are used to sparsify the embedded vectors, reducing storage and computational overhead while preserving key semantic information. This improves the performance of small question-answering systems in specialized domains and enhances the accuracy and reliability of complex database queries and SQL generation.

[0113] Further reference Figure 7 As an implementation of the methods shown in the above figures, this application provides an embodiment of a text-to-SQL generation device based on enhanced graph embedding technology. This device embodiment is similar to... Figure 1 and Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0114] Secondly, embodiments of the present invention also disclose a text-to-SQL generation device based on enhanced graph embedding technology, used to implement the text-to-SQL generation method as described in any one of the first aspects, such as... Figure 7 As shown, it includes: entity extraction module 71, knowledge graph construction module 72, enhanced graph embedding module 73, and model fine-tuning module 74.

[0115] In a specific embodiment, the entity extraction module 71 is used to extract table names, fields, condition values, and functions from natural language queries into natural language-side structured entity subsets through text preprocessing, named entity recognition, relation extraction, and standardization; simultaneously, it acquires database schema information and business constraints to form a database-side structured entity subset; and merges the natural language-side structured entity subsets and the database-side structured entity subsets to obtain a complete structured entity set. The knowledge graph construction module 72 is used to map the entity set into table nodes, field nodes, value nodes, and constraint nodes, construct 'table-field', 'field-field', 'field-value', 'field-function', and 'field-constraint' semantic edges, and inject data types, value ranges, and business rule attributes into the nodes / edges to obtain a multi-relation heterogeneous knowledge graph.

[0116] The enhanced graph embedding module 73 is used to map the graph nodes into initial vectors through pre-trained word embeddings, update the node representations through a KG-Adapter layer and a graph neural network, enhance features by combining a self-attention mechanism and a multilayer perceptron, and then obtain sparse graph embedding vectors through L2 regularization and pruning. The model fine-tuning module 74 is used to align the sparse graph embedding vectors with the text vectors of natural language queries to form a cross-modal joint representation, and input the supervised fine-tuned LLM to generate SQL statements that conform to SQL syntax specifications and database business constraints.

[0117] The functions and methods of the above modules correspond to each other, and will not be repeated here.

[0118] The beneficial effects of this invention are as follows:

[0119] 1. Improve SQL generation accuracy: By covering full-dimensional semantics through multi-relational heterogeneous graphs, fact constraints are provided for LLM, and the accuracy of multi-table join queries is improved to 85.2% (compared to 72.0% of the existing LGESQL), and the hallucination rate is reduced to 2.7% (compared to 18.4% of GPT-4).

[0120] 2. Optimized computational efficiency: L2 regularization and pruning reduce the embedding vector dimension by 30%-50%, and the average SQL generation time is 0.4s (compared to 0.6s for HES-SQL), making it suitable for small question-answering systems;

[0121] 3. Enhanced scenario adaptability: Supports multi-table joins, nested subqueries, and aggregate function queries, which can be directly applied to professional fields such as financial credit reports (e.g., "query customers with loan overdue amounts ≥ 100,000 in 2023") and manufacturing yield analysis (e.g., "query batches with a yield rate < 95% in March 2024 for production line A").

[0122] 4. Improve interpretability: Output SQL statement descriptions (field sources, relational logic), allowing users to trace the basis of their queries and lowering the barrier to entry.

[0123] The following is for reference. Figure 8 It illustrates an electronic device suitable for implementing embodiments of the present invention (e.g., Figure 1 The diagram shows the structure of a computer device 800 (a server or terminal device). Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0124] like Figure 8 As shown, the computer device 800 includes a central processing unit (CPU) 801 and a graphics processing unit (GPU) 802, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 803 or programs loaded from storage section 809 into random access memory (RAM) 804. The RAM 804 also stores various programs and data required for the operation of the device 800. The CPU 801, GPU 802, ROM 803, and RAM 804 are interconnected via bus 805. An input / output (I / O) interface 806 is also connected to bus 805.

[0125] The following components are connected to I / O interface 806: an input section 807 including a keyboard, mouse, etc.; an output section 808 including an LCD, speakers, etc.; a storage section 809 including a hard disk, etc.; and a communication section 810 including a network interface card, such as a LAN card or modem. The communication section 810 performs communication processing via a network such as the Internet. A drive 811 may also be connected to I / O interface 806 as needed. A removable medium 812, such as a hard disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 811 as needed so that computer programs read from it can be installed into storage section 809 as needed.

[0126] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 810, and / or installed from removable medium 812. When the computer program is executed by central processing unit (CPU) 801 and graphics processing unit (GPU) 802, the functions defined in the methods of this invention are performed.

[0127] It should be noted that the computer-readable medium described in this invention can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus, or any combination thereof. More specific examples of a computer-readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution apparatus, device, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than a computer-readable medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution device, apparatus, or apparatus. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0128] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using dedicated hardware-based devices that perform the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0130] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be located in a processor.

[0131] In another aspect, the present invention also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods and steps described in the first aspect of the embodiments of the present invention.

[0132] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the specific combination of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this invention.< / username> < / users>

Claims

1. A text-to-SQL generation method based on enhanced graph embedding technology, characterized in that, The method includes the following steps: S1. Entity extraction: Through text preprocessing, named entity recognition, relation extraction, and normalization, the table names, fields, conditional values, and functions in the natural language query are extracted as a subset of structured entities on the natural language side; meanwhile, database schema information and business constraints are obtained to form a subset of structured entities on the database side; the subset of structured entities on the natural language side and the subset of structured entities on the database side are merged to obtain a complete set of structured entities. S2. Construction of a multi-relational heterogeneous knowledge graph: The entity set is mapped to table nodes, field nodes, value nodes, and constraint nodes, semantic edges of 'table-field', 'field-field', 'field–value', 'field-function', and 'field-constraint' are constructed, and data types, value range, and business rule attributes are injected into the nodes / edges to obtain a multi-relational heterogeneous knowledge graph. S3. Enhanced graph embedding: The graph nodes are mapped to initial vectors through pre-trained word embeddings, the node representations are updated through the KG-Adapter layer and graph neural network, the features are enhanced by combining the self-attention mechanism and multi-layer perceptron, and then the sparse graph embedding vectors are obtained through L2 regularization and pruning processing. And S4. Cross-modal alignment and SQL generation: The sparse graph embedding vectors are aligned with the text vectors of the natural language query to form a cross-modal joint representation, and the supervised fine-tuned LLM is input to generate SQL statements that conform to the SQL syntax specification and database business constraints.

2. The text-to-SQL generation method according to claim 1, characterized in that, The text preprocessing in step S1 includes word segmentation,词性标注, and stop word removal operations, which are preprocessing for定向保留数据库查询关键信息 to ensure that the extracted entity subset directly matches the database query requirements. Specifically, it includes: Word segmentation splits the query text in natural language grammar into semantic units;词性标注 retains key verbs, nouns, and numerical词性; stop word removal剔除词汇 without database semantic associations including '的', '在', and '和'.

3. The text-to-SQL generation method according to claim 1, characterized in that, The named entity recognition in step S1 is performed for entity types related to database queries, and the entity types include time entities, numerical entities, person entities, place entities, and function entities. After recognition, a mapping pair in the format of {<entity keyword>:<database mapping object>} is output.

4. The text-to-SQL generation method according to claim 1, characterized in that, The database schema information in step S1 includes table structures, field data types, primary and foreign key association relationships, and index information in the database; the business constraints include field value range constraints, table association constraints, and business logic constraints.

5. The text-to-SQL generation method according to claim 1, characterized in that, In step S2, injecting data types into nodes specifically means: injecting the database identifier to which the table node belongs, injecting the corresponding data type and attributes of '是否非空' and '是否唯一' to the field node, injecting the data category to the value node, and injecting the constraint type to the constraint node.

6. The text-to-SQL generation method according to claim 1, characterized in that, The value range in step S2 is obtained through normalization processing. Specifically, for numerical value nodes, min-max normalization is used to map the value range to the [0,1] interval; for time value nodes, they are converted to timestamps and then normalized to ensure the consistency of the value range. It should be noted that there is an unclear part "词性标注" in the original text which is left in Chinese as it seems to be a misspelling or an unclear term. If this is a known specific term in Chinese that needs to be translated accurately, more context or clarification is required. Also, "定向保留数据库查询关键信息" and "剔除词汇" seem a bit unclear in the context and might need further refinement in the original text for more accurate translation.

7. The text-to-SQL generation method according to claim 1, characterized in that, The pre-trained word embedding in step S3 is implemented using a pre-trained model of a large language model (LLM). The pre-trained model includes BERT series models, Qwen series models, or GPT series models, ensuring that the initial vector of the node is consistent with the dimension of the text vector of the LLM.

8. The text-to-SQL generation method according to claim 1, characterized in that, In step S3, the KG-Adapter layer integrates the graph neural network output using an instant tuning method. Specifically, the node structural features generated by the graph neural network are used as the Prompt embedding, concatenated with the initial vector of the pre-trained word embedding, and then input into the KG-Adapter layer. The fusion of structural features and semantic features is achieved through 1 to 2 layers of linear transformation.

9. The text-to-SQL generation method according to claim 1, characterized in that, The L2 regularization described in step S3 is achieved by adding a sum of squared embedding vectors to the loss function, and the expression for the loss function is: Loss = Loss base +L reg , Among them, Loss base Basic task loss, L reg This is an added term in the loss function, where λ is the regularization coefficient, and v i Let represent the i-th embedding vector.

10. The text-to-SQL generation method according to claim 1, characterized in that, The pruning process in step S3 employs a threshold filtering mechanism, specifically: a preset vector dimension weight threshold is established; the absolute value of the weights for each dimension of each embedded vector is calculated; dimensions with absolute weights less than the threshold are removed, while key semantic dimensions are retained, thus reducing the dimension of the sparsified embedded vectors; the preset threshold is γ = 1e-4, with a value range of 0.01 to 0.05; if |v i If |<γ=1e-4, then the dimension is considered to have little information content, and the dimension is removed.

11. The text-to-SQL generation method according to claim 1, characterized in that, The training samples for supervised fine-tuning in step S4 contain two types of labeled data: the first type is natural language query-graph answer node ID pairs, which are used to optimize the semantic association ability of LLM with graph nodes; the second type is natural language query-standard SQL statement pairs, which are used to optimize the SQL syntax generation ability of LLM; and the fine-tuning process adopts the cross-entropy loss function, the number of iterations is 50 to 100 rounds, and the learning rate is 1e-5 to 5e-5.

12. A text-to-SQL generation device based on enhanced graph embedding technology, characterized in that, A method for implementing the text-to-SQL generation method as described in any one of claims 1-11, comprising: The entity extraction module is used to extract table names, fields, condition values, and functions from natural language queries into natural language-side structured entity subsets through text preprocessing, named entity recognition, relation extraction, and standardization; at the same time, it obtains database schema information and business constraints to form database-side structured entity subsets; and merges the natural language-side structured entity subsets and the database-side structured entity subsets to obtain a complete set of structured entities. The knowledge graph construction module is used to map the entity set into table nodes, field nodes, value nodes, and constraint nodes, construct semantic edges of 'table-field', 'field-field', 'field-value', 'field-function', and 'field-constraint', and inject data type, value range, and business rule attributes into the nodes / edges to obtain a multi-relationship heterogeneous knowledge graph; An enhanced graph embedding module is used to map the graph nodes to initial vectors through pre-trained word embeddings, update the node representations through a KG-Adapter layer and a graph neural network, enhance features by combining self-attention mechanism and multilayer perceptron, and then obtain sparse graph embedding vectors through L2 regularization and pruning; and The model fine-tuning module is used to align the sparse graph embedding vector with the text vector of the natural language query to form a cross-modal joint representation. The input is a supervised fine-tuned LLM to generate an SQL statement that conforms to the SQL syntax specification and database business constraints.

13. An electronic device, comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 11.

Images