NL2SQL method and system based on reinforcement learning

Abstract

The invention relates to the technical field of artificial intelligence and databases, in particular to an NL2SQL method and system based on reinforcement learning, and the method comprises the following steps: natural language input preprocessing, semantic analysis and abstract representation construction, mode linking and candidate set determination, SQL generation, query execution and feedback optimization, and multi-round interaction processing. The method has the beneficial effects that complex scenes such as nested sub-query, multi-table JOIN and aggregation function combination are supported, and the SQL generation accuracy is improved by more than 30% compared with that of a traditional method (test data is from a spider data set); a mode link mechanism based on reinforcement learning can quickly adapt to a new database mode, and in a cross-domain test (such as switching from an e-commerce database to a medical database), the accuracy rate decrease amplitude is less than 15%.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and database technology, and specifically to a NL2SQL method and system based on reinforcement learning. Background Art

[0002] Natural language to SQL (NL2SQL) conversion is a key technology for enabling non-technical users to interact with databases. It is widely used in scenarios such as intelligent customer service and business data analysis. Traditional NL2SQL methods are primarily based on rule matching, template generation, or statistical learning models (such as recurrent neural networks (RNNs) and Transformers), and have the following limitations:

[0003] (1) Insufficient semantic understanding: Difficulty processing complex sentence structures (such as nested queries, aggregate functions, and multi-table joins) and domain-specific terminology, leading to logical errors in the generated SQL statements;

[0004] (2) Insufficient generalization capability: Relying on a large amount of labeled data, performance degrades significantly when the database schema changes (such as table or column name changes) or new query scenarios;

[0005] (3) Lack of dynamic interaction capabilities: It is unable to adjust the generation strategy based on query result feedback, and it is difficult to handle context dependency issues in multi-round conversations.

[0006] The existing technology has the following defects:

[0007] (1) Rule and template method: It requires manual definition of a large number of rules, has poor scalability, and cannot cope with flexible natural language expressions;

[0008] (2) Statistical learning methods: Based on supervised learning, they rely on strongly labeled data and do not explicitly model the database schema structure, resulting in inaccurate schema linking.

[0009] (3) Lack of feedback mechanism: Traditional methods lack closed-loop feedback on execution results after generating SQL, and are unable to correct implicit errors in semantic parsing.

[0010] As enterprise databases become more complex (e.g., multi-table joins and hierarchical queries), users are placing higher demands on the accuracy and interactivity of natural language queries. An NL2SQL approach with dynamic semantic parsing, adaptive schema linking, and feedback optimization capabilities is urgently needed to meet the needs of automated data queries in complex scenarios. Summary of the Invention

[0011] The purpose of the present invention is to provide a NL2SQL method and system based on reinforcement learning to solve the problems raised in the above background technology.

[0012] To achieve the above objectives, the present invention provides the following technical solution: a NL2SQL method based on reinforcement learning, comprising the following steps:

[0013] Natural language input preprocessing: Receive natural language queries input by users, perform word segmentation, part-of-speech tagging, and named entity recognition on them to extract key semantic components from the query;

[0014] Semantic parsing and abstract representation construction steps: Use a deep learning model to convert the preprocessed natural language query into an abstract semantic tree, which includes operation nodes, entity nodes, function nodes, and logic nodes;

[0015] Schema linking and candidate set determination steps: Represent the database schema as a graph structure G(Schema) = (T, C, E), where T is the table set, C is the column set, and E is the foreign key relationship edge between tables. Establish a mapping relationship between abstract semantic tree nodes and database tables and columns to generate a set of candidate SQL fragments;

[0016] SQL generation step: With the help of a reinforcement learning agent, based on the preset state space, action space, and reward function, the optimal fragments are selected from the set of candidate SQL fragments and combined to generate a complete SQL statement;

[0017] Query execution and feedback optimization step: Execute the generated SQL statement in the database, calculate the reward value based on the execution result, and use the reward value to optimize the parameters of the reinforcement learning model;

[0018] Multi-round interaction processing steps: Maintain the conversation history stack, record the semantic tree of each query round, the generated SQL and execution results, and extract context entities; perform differential parsing on new queries, processing only the parts that differ from historical conversations, and combine the historical semantic vectors with the current query vector through the attention mechanism to generate context-aware semantic representations to support SQL generation for subsequent queries.

[0019] Preferably, in the semantic parsing and abstract representation construction steps, a pre-trained language model is used in combination with a bidirectional long short-term memory network LSTM for semantic encoding: the pre-trained language model is used to perform contextual semantic encoding on natural language queries to obtain hidden state vectors at the character / word level; the hidden state vectors are processed through a bidirectional LSTM network to capture long-distance dependencies and output sentence-level semantic vectors and phrase-level semantic vectors; a syntactic tree of the query is generated based on dependency syntactic analysis, and the syntactic tree is converted into an abstract semantic tree in combination with the domain knowledge graph.

[0020] Preferably, in the schema linking and candidate set determination steps, the database schema graph structure G (Schema) = (T, C, E) is constructed and used as follows: each table node in the table set T contains the table name and description information; each column node in the column set C contains the column name, data type and the table ID to which it belongs; for the entity nodes in the abstract semantic tree, the corresponding tables and columns are searched in the database schema graph through string matching, type constraint matching and relational reasoning to generate a candidate mapping set; based on the similarity of the candidate mappings, the Top-K candidate schema objects are selected for each semantic entity node to form the final candidate SQL fragment set.

[0021] Preferably, in the SQL generation step, the state space, action space and reward function of the reinforcement learning agent are defined as follows: the state space includes the semantic tree fragment that has not been fully parsed, the generated SQL fragment, the candidate mapping set and the current access status of the database schema; the action space includes selecting candidate mappings to link semantic entities to the schema, adding SQL operators, and confirming the current SQL fragment and entering the next stage; the reward function comprehensively considers correctness, efficiency, and semantic matching factors, specifically including: giving positive rewards if the generated SQL is executed successfully and the results meet expectations; giving execution efficiency rewards based on the SQL execution time; giving semantic matching rewards based on the similarity of candidate mappings; and giving penalties for duplicate linked tables / columns, grammatical errors and invalid actions.

[0022] Preferably, the query execution and feedback optimization step includes the following operations: integrating the database connection pool, performing syntax verification and security filtering on the generated SQL statements, and then executing them in the database; if the SQL execution fails, parsing the error type and returning a fine-grained penalty signal; if the execution is successful, verifying the correctness of the results by comparing the key constraints in the user query with the query result statistics and using active learning technology to request the user to mark low-confidence results; storing the valid query-result pairs in the real-time interaction in the incremental training set, regularly triggering model fine-tuning, and using hot update technology to update the parameters of the reinforcement learning model without interrupting the service.

[0023] A system for a reinforcement learning-based NL2SQL method, comprising:

[0024] Natural language input preprocessing module: Receives natural language queries input by users, performs word segmentation, part-of-speech tagging, and named entity recognition on them, and extracts key semantic components from the queries;

[0025] Semantic parsing and abstract representation building module: Using deep learning models, the pre-processed natural language query is converted into an abstract semantic tree, which includes operation nodes, entity nodes, function nodes, and logic nodes.

[0026] Schema linking and candidate set determination module: The database schema is represented as a graph structure G(Schema) = (T, C, E), where T is the table set, C is the column set, and E is the foreign key relationship edge between tables. The mapping relationship between abstract semantic tree nodes and database tables and columns is established to generate a set of candidate SQL fragments.

[0027] SQL generation module: With the help of reinforcement learning agents, based on the preset state space, action space, and reward function, it selects the best fragments from the candidate SQL fragment set and combines them to generate a complete SQL statement;

[0028] Query execution and feedback optimization module: executes the generated SQL statements in the database, calculates the reward value based on the execution results, and uses the reward value to optimize the parameters of the reinforcement learning model;

[0029] Multi-round interaction processing module: Maintains the conversation history stack, records the semantic tree of each query round, the generated SQL and execution results, and extracts contextual entities; performs differential parsing on new queries, processing only the parts that differ from historical conversations, and combines the historical semantic vectors with the current query vector through the attention mechanism to generate context-aware semantic representations to support SQL generation for subsequent queries.

[0030] Preferably, in the semantic parsing and abstract representation construction module, a pre-trained language model is combined with a bidirectional long short-term memory network LSTM for semantic encoding: the pre-trained language model is used to perform contextual semantic encoding on natural language queries to obtain hidden state vectors at the character / word level; the hidden state vectors are processed through a bidirectional LSTM network to capture long-distance dependencies and output sentence-level semantic vectors and phrase-level semantic vectors; a syntactic tree of the query is generated based on dependency syntactic analysis, and the syntactic tree is converted into an abstract semantic tree in combination with the domain knowledge graph.

[0031] Preferably, in the schema linking and candidate set determination module, the database schema graph structure G (Schema) = (T, C, E) is constructed and applied as follows: each table node in the table set T contains the table name and description information; each column node in the column set C contains the column name, data type and the table ID to which it belongs; for the entity nodes in the abstract semantic tree, the corresponding tables and columns are searched in the database schema graph through string matching, type constraint matching and relational reasoning to generate a candidate mapping set; based on the similarity of the candidate mappings, the Top-K candidate schema objects are selected for each semantic entity node to form the final candidate SQL fragment set.

[0032] Preferably, in the SQL generation module, the state space, action space and reward function of the reinforcement learning agent are defined as follows: the state space includes the semantic tree fragment that has not been fully parsed, the generated SQL fragment, the candidate mapping set and the current access status of the database schema; the action space includes selecting candidate mappings to link semantic entities to the schema, adding SQL operators, and confirming the current SQL fragment and entering the next stage; the reward function comprehensively considers correctness, efficiency, and semantic matching factors, specifically including: giving positive rewards if the generated SQL is executed successfully and the results meet expectations; giving execution efficiency rewards based on the SQL execution time; giving semantic matching rewards based on the similarity of candidate mappings; and giving penalties for duplicate linked tables / columns, grammatical errors and invalid actions.

[0033] Preferably, the query execution and feedback optimization module includes the following operations: integrating the database connection pool, performing syntax verification and security filtering on the generated SQL statements, and then executing them in the database; if the SQL execution fails, parsing the error type and returning a fine-grained penalty signal; if the execution is successful, verifying the correctness of the results by comparing the key constraints in the user query with the query result statistics and using active learning technology to request the user to mark low-confidence results; storing the valid query-result pairs in the real-time interaction in the incremental training set, regularly triggering model fine-tuning, and using hot update technology to update the parameters of the reinforcement learning model without interrupting the service.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The reinforcement learning-based NL2SQL method and system proposed in this paper support complex scenarios such as nested subqueries, multi-table JOINs, and aggregate function combinations, and the SQL generation accuracy is more than 30% higher than that of traditional methods (test data comes from the Spider dataset); the reinforcement learning-based pattern linking mechanism can quickly adapt to new database patterns. In cross-domain tests (such as switching from an e-commerce database to a medical database), the accuracy drop is less than 15% (traditional methods drop by about 40%); it reduces dependence on labeled data and, through simulated data augmentation technology, maintains stable performance while reducing the number of labeled samples by 50%; the execution result feedback mechanism can automatically correct semantic parsing errors. After 5 rounds of iterative training, the error rate is reduced by 60%; in multi-round dialogue scenarios, context utilization is increased to 85%, and user interaction efficiency is improved by 50%; it supports mainstream relational databases and provides standardized API interfaces, which can be quickly integrated into existing data query systems; the online learning mechanism supports continuous model evolution and adapts to dynamic changes in the enterprise database schema. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0037] In order to clearly and completely describe the objectives and technical solutions of the present invention and make the advantages more clearly understood, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, not all of them, and are only used to explain the embodiments of the present invention, not to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] For example 1, please refer to Figure 1 The present invention provides a technical solution: a NL2SQL method based on reinforcement learning, comprising the following steps:

[0039] Natural language input preprocessing step: Receive the natural language query input by the user, perform word segmentation, part-of-speech tagging, and named entity recognition operations on it, and extract the key semantic components in the query.

[0040] Semantic parsing and abstract representation construction steps: Use a deep learning model to convert the preprocessed natural language query into an abstract semantic tree, which covers operation nodes, entity nodes, function nodes and logic nodes; use a pre-trained language model combined with a bidirectional long short-term memory network LSTM for semantic encoding: Use the pre-trained language model to perform contextual semantic encoding on the natural language query to obtain a hidden state vector at the character / word level; process the hidden state vector through a bidirectional LSTM network to capture long-distance dependencies and output sentence-level semantic vectors and phrase-level semantic vectors; generate a syntactic tree for the query based on dependency syntactic analysis, and convert the syntactic tree into an abstract semantic tree in combination with the domain knowledge graph.

[0041] Steps for determining pattern links and candidate sets: Represent the database schema as a graph structure G(Schema) = (T, C, E), where T is the table set, C is the column set, and E is the foreign key relationship edge between tables. Establish a mapping relationship between abstract semantic tree nodes and database tables and columns to generate a set of candidate SQL fragments. The construction and application of the database schema graph structure G(Schema) = (T, C, E) are as follows: each table node in the table set T contains the table name and description information; each column node in the column set C contains the column name, data type and the table ID to which it belongs. For entity nodes in the abstract semantic tree, find the corresponding tables and columns in the database schema graph through string matching, type constraint matching and relational reasoning to generate a set of candidate mappings. Based on the similarity of the candidate mappings, select the Top-K candidate schema objects for each semantic entity node to form the final set of candidate SQL fragments.

[0042] SQL generation steps: With the help of a reinforcement learning agent, based on the preset state space, action space and reward function, the optimal fragments are selected from the set of candidate SQL fragments for combination to generate a complete SQL statement; the state space, action space and reward function of the reinforcement learning agent are defined as follows: the state space contains the semantic tree fragments that have not been fully parsed, the generated SQL fragments, the candidate mapping set and the current access status of the database schema; the action space includes selecting candidate mappings to link semantic entities to the schema, adding SQL operators, and confirming the current SQL fragment and entering the next stage; the reward function comprehensively considers correctness, efficiency, and semantic matching factors, specifically including: if the generated SQL is executed successfully and the results are as expected, a positive reward is given; an execution efficiency reward is given based on the SQL execution time; a semantic matching reward is given based on the similarity of the candidate mappings; and penalties are imposed for duplicate linked tables / columns, syntactic errors and invalid actions.

[0043] Query execution and feedback optimization steps: Execute the generated SQL statement in the database, calculate the reward value based on the execution result, and use the reward value to optimize the parameters of the reinforcement learning model; including the following operations: integrate the database connection pool, perform syntax verification and security filtering on the generated SQL statement, and then execute it in the database; if the SQL execution fails, parse the error type and return a fine-grained penalty signal; if the execution is successful, verify the correctness of the result by comparing the key constraints in the user query with the query result statistics and using active learning technology to request the user to mark low-confidence results; store the valid query-result pairs in real-time interaction in the incremental training set, regularly trigger model fine-tuning, and use hot update technology to update the parameters of the reinforcement learning model without interrupting service.

[0044] Multi-round interaction processing steps: Maintain the conversation history stack, record the semantic tree of each query round, the generated SQL and execution results, and extract context entities; perform differential parsing on new queries, processing only the parts that differ from historical conversations, and combine the historical semantic vectors with the current query vector through the attention mechanism to generate context-aware semantic representations to support SQL generation for subsequent queries.

[0045] In the second embodiment, based on the first embodiment, a system for the NL2SQL method based on reinforcement learning is proposed, including:

[0046] Natural language input preprocessing module: Receives natural language queries input by users, performs word segmentation, part-of-speech tagging, and named entity recognition operations on them, and extracts key semantic components in the queries.

[0047] Semantic parsing and abstract representation construction module: Use a deep learning model to convert the preprocessed natural language query into an abstract semantic tree, which covers operation nodes, entity nodes, function nodes and logic nodes; use a pre-trained language model combined with a bidirectional long short-term memory network LSTM for semantic encoding: Use the pre-trained language model to perform contextual semantic encoding on the natural language query to obtain a hidden state vector at the character / word level; process the hidden state vector through a bidirectional LSTM network to capture long-distance dependencies and output sentence-level semantic vectors and phrase-level semantic vectors; generate a syntactic tree for the query based on dependency syntactic analysis, and convert the syntactic tree into an abstract semantic tree in combination with the domain knowledge graph.

[0048] Schema linking and candidate set determination module: The database schema is represented as a graph structure G(Schema) = (T, C, E), where T is the table set, C is the column set, and E is the foreign key relationship edge between tables. A mapping relationship is established between abstract semantic tree nodes and database tables and columns to generate a set of candidate SQL fragments. The database schema graph structure G(Schema) = (T, C, E) is constructed and applied as follows: each table node in the table set T contains the table name and description information; each column node in the column set C contains the column name, data type, and the ID of the table to which it belongs. For entity nodes in the abstract semantic tree, the corresponding tables and columns are searched in the database schema graph through string matching, type constraint matching, and relational reasoning to generate a set of candidate mappings. Based on the similarity of the candidate mappings, the Top-K candidate schema objects are selected for each semantic entity node to form the final set of candidate SQL fragments.

[0049] SQL generation module: With the help of reinforcement learning agents, based on the preset state space, action space and reward function, the optimal fragments are selected from the candidate SQL fragment set for combination to generate complete SQL statements; the state space, action space and reward function of the reinforcement learning agent are defined as follows: the state space contains the semantic tree fragments that have not been fully parsed, the generated SQL fragments, the candidate mapping set and the current access status of the database schema; the action space includes selecting candidate mappings to link semantic entities to the schema, adding SQL operators, and confirming the current SQL fragment and entering the next stage; the reward function comprehensively considers correctness, efficiency, and semantic matching factors, specifically including: if the generated SQL is executed successfully and the results meet expectations, a positive reward is given; an execution efficiency reward is given based on the SQL execution time; a semantic matching reward is given based on the similarity of the candidate mappings; and penalties are imposed for duplicate linked tables / columns, syntactic errors and invalid actions.

[0050] Query execution and feedback optimization module: Executes the generated SQL statements in the database, calculates the reward value based on the execution results, and uses the reward value to optimize the parameters of the reinforcement learning model; includes the following operations: integrates the database connection pool, performs syntax verification and security filtering on the generated SQL statements, and then executes them in the database; if the SQL execution fails, parses the error type and returns a fine-grained penalty signal; if the execution is successful, verifies the correctness of the result by comparing the key constraints in the user query with the query result statistics and using active learning technology to request the user to mark low-confidence results; stores valid query-result pairs in real-time interaction in the incremental training set, regularly triggers model fine-tuning, and uses hot update technology to update the parameters of the reinforcement learning model without interrupting service.

[0051] Multi-round interaction processing module: Maintains the conversation history stack, records the semantic tree of each query round, the generated SQL and execution results, and extracts contextual entities; performs differential parsing on new queries, processing only the parts that differ from historical conversations, and combines the historical semantic vectors with the current query vector through the attention mechanism to generate context-aware semantic representations to support SQL generation for subsequent queries.

[0052] Example 3, based on Example 2, proposes:

[0053] 1. Semantic parsing module design

[0054] Goal: Convert natural language queries into structured semantic representations and support complex sentence parsing.

[0055] Technical implementation:

[0056] Multi-level semantic encoding

[0057] (1) Use a pre-trained language model (such as BERT) to perform contextual semantic encoding on the query text and generate character / word level hidden state vectors h1,h2,...,hn.

[0058] (2) Capture long-distance dependencies through a bidirectional LSTM network and output a sentence-level semantic vector s sent and phrase-level semantic vector s phrase .

[0059] Semantic tree construction

[0060] (1) Generate a syntactic tree of the query based on dependency syntactic analysis (such as the LSTM-CRF model) and annotate components such as the subject, predicate, and object.

[0061] (2) Combined with the domain knowledge graph (such as the database operation keyword set), the syntax tree is converted into an abstract syntax tree (AST). The node types include:

[0062] Operation nodes: SELECT, FROM, WHERE, GROUP BY, HAVING, ORDER BY, etc.

[0063] Entity node: table name, column name, and constant value involved in the query;

[0064] Function nodes: aggregate functions (SUM, COUNT, AVG), date functions, string functions, etc.

[0065] Logical nodes: Boolean operators such as AND, OR, and NOT.

[0066] Example

[0067] Enter the query: "Query the names and sales volume of employees in each department with sales exceeding 1 million in 2023"

[0068] Key nodes of the semantic tree:

[0069] SELECT: employee name, sales volume

[0070] FROM: employee table, department table, sales table

[0071] WHERE: Sales Time = 2023 AND Sales > 1 million AND Employee.Department ID = Department.Department ID AND Sales.Employee ID = Employee.Employee ID

[0072] GROUP BY: department table.department name

[0073] 2. Mode link module design

[0074] Goal: Establish a mapping relationship between semantic tree nodes and database schema to resolve table name / column name ambiguity.

[0075] Technical implementation:

[0076] Database schema representation

[0077] The database schema is modeled as a graph structure G(Schema) = (T, C, E), where:

[0078] T: table collection, each table node contains table name and description information;

[0079] C: Column set, each column node contains the column name, data type, and table ID;

[0080] E: Foreign key relationship edge between tables, labeling the associated fields (such as employee table.department ID-department table.department ID).

[0081] Semantic-Pattern Matching Algorithm

[0082] String matching: Perform fuzzy matching on entity nodes in the semantic tree (such as "sales"), find columns in the database whose names contain the keyword (such as "sales", "sales amount"), and calculate the edit distance similarity sim str .

[0083] Type constraint matching: Filters out candidate columns that do not match based on their data types (such as INT, DATE, and VARCHAR). For example, the "time" in the query condition "time > 2023-01-01" must be mapped to a DATE type column.

[0084] Relational reasoning: Inferring multi-table association paths through foreign key relationship diagrams. For example, if the semantic tree contains "employee" and "department" entities, and the two tables are related by "department ID," the JOIN condition "employee table.department ID = department table.department ID" is generated.

[0085] Candidate set generation strategy

[0086] Generate Top-K candidate schema objects (tables / columns) for each semantic entity node to form a candidate mapping set:

[0087] M={(e i ,c j ,sim j )|e i ∈Semantic Entity,c j ∈ candidate mode,sim j ∈([0,1]}

[0088] 3. Reinforcement Learning Decision Module Design

[0089] Objective: To optimize SQL generation strategies through reinforcement learning and achieve dynamic selection and combination of candidate fragments.

[0090] Technical implementation:

[0091] State space definition:

[0092] Status t Include:

[0093] The current semantic tree fragment St (the part that has not been parsed);

[0094] Generated SQL fragment Qt;

[0095] Candidate mapping set Mt;

[0096] The current access status of the database schema (G) (such as whether the table or column has been referenced).

[0097] Action space definition:

[0098] Action at include:

[0099] Select candidate mapping (e i ,e j ), the semantic entity e i Link to mode c j ;

[0100] Add SQL operators (such as JOIN, WHERE, GROUP BY);

[0101] Confirm the current SQL fragment and proceed to the next stage.

[0102] Reward function design

[0103] Correctness reward: If the generated SQL is executed successfully and the result is as expected, the reward is +R correct ; Otherwise, penalty -R error .

[0104] Efficiency Reward: Based on the SQL execution time (such as the cost estimate in the execution plan), rewards strategies with high execution efficiency + R efficiency .

[0105] Semantic matching reward: based on the similarity of candidate mappings j , reward high confidence mapping +sim j ×R match .

[0106] Penalty item: Penalty for invalid actions such as duplicate link tables / columns, syntax errors, etc. -R penalty .

[0107] Algorithm selection

[0108] The proximal policy optimization (PPO) algorithm is adopted as the reinforcement learning framework, combining the value function network V(s) and the policy network π(a|s) for end-to-end training.

[0109] The training data includes:

[0110] Manually annotated NL-SQL pairs (for pre-training semantic parsing models);

[0111] Automatically generated simulated queries (generated by random combinations of database patterns to expand training scenarios);

[0112] User query logs in real-time interaction (online learning mode).

[0113] 4. Query execution and feedback module design

[0114] Objective: Establish a closed loop of “generation-execution-feedback” to optimize model parameters.

[0115] Technical implementation:

[0116] SQL execution engine

[0117] Integrated database connection pool, supporting mainstream relational databases such as MySQL and PostgreSQL;

[0118] Perform syntax verification and security filtering (such as preventing SQL injection) on the generated SQL to ensure execution security.

[0119] Feedback signal processing

[0120] If SQL execution fails (such as syntax error or table does not exist), the error type is parsed and a fine-grained penalty signal (such as syntax error penalty or schema link error penalty) is returned.

[0121] If the execution is successful, verify the correctness of the result by:

[0122] Compare the key constraints in the user query (such as "more than 1 million") with the query result statistics;

[0123] Using active learning technology, high-confidence results are automatically confirmed, and low-confidence results require user annotation.

[0124] Online learning mechanism

[0125] Store valid query-result pairs in real-time interactions into incremental training sets, and periodically trigger model fine-tuning.

[0126] Adopt hot update technology to update reinforcement learning strategy parameters without interrupting service.

[0127] 5. Multi-round interaction module design

[0128] Goal: To handle context dependencies in multi-round conversations and avoid repetitive parsing of the same semantics.

[0129] Technical implementation:

[0130] Conversation state tracking

[0131] Maintain the conversation history stack, recording the semantic tree of each query round, the generated SQL and the execution results;

[0132] Extract contextual entities (such as "Department A" mentioned in the previous round) and automatically link to corresponding schema objects in subsequent queries.

[0133] Semantic incremental parsing

[0134] Perform differential parsing on new queries, processing only the parts that differ from historical conversations;

[0135] The historical semantic vectors are combined with the current query vector through the attention mechanism to generate context-aware semantic representations.

[0136] Example

[0137] First round of query: "Query the list of employees in the sales department"

[0138] Generate SQL: SELECT employee name FROM employee table WHERE department ID IN (SELECT department ID FROM department table WHERE department name = 'sales department')

[0139] Second round of query: "Calculate the annual sales of these employees"

[0140] The context entity "these employees" is mapped to the "employee name" column in the previous query results, generating the SQL: SELECT employee name, SUM (sales) AS annual sales FROM sales table WHERE employee ID IN (SELECT employee ID FROM employee table WHERE department ID IN (SELECT department ID FROM department table WHERE department name = 'sales department')) GROUP BY employee name

[0141] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A reinforcement learning-based NL2SQL method, characterized by: The following steps are involved: Natural language input preprocessing: Receive natural language queries input by users, perform word segmentation, part-of-speech tagging, and named entity recognition on them to extract key semantic components from the query; Semantic parsing and abstract representation construction steps: Use a deep learning model to convert the preprocessed natural language query into an abstract semantic tree, which includes operation nodes, entity nodes, function nodes, and logic nodes; Schema linking and candidate set determination steps: Represent the database schema as a graph structure G(Schema) = (T, C, E), where T is the table set, C is the column set, and E is the foreign key relationship edge between tables. Establish a mapping relationship between abstract semantic tree nodes and database tables and columns to generate a set of candidate SQL fragments; SQL generation step: With the help of a reinforcement learning agent, based on the preset state space, action space, and reward function, the optimal fragments are selected from the set of candidate SQL fragments and combined to generate a complete SQL statement; Query execution and feedback optimization step: Execute the generated SQL statement in the database, calculate the reward value based on the execution result, and use the reward value to optimize the parameters of the reinforcement learning model; Multi-round interaction processing steps: Maintain the conversation history stack, record the semantic tree of each query round, the generated SQL and execution results, and extract context entities; perform differential parsing on new queries, processing only the parts that differ from historical conversations, and combine the historical semantic vectors with the current query vector through the attention mechanism to generate context-aware semantic representations to support SQL generation for subsequent queries.

2. The NL2SQL method based on reinforcement learning according to claim 1, characterized in that: In the semantic parsing and abstract representation construction steps, a pre-trained language model combined with a bidirectional long short-term memory network (LSTM) is used for semantic encoding: the pre-trained language model is used to perform contextual semantic encoding on natural language queries to obtain hidden state vectors at the character / word level; the hidden state vectors are processed through a bidirectional LSTM network to capture long-distance dependencies and output sentence-level semantic vectors and phrase-level semantic vectors; a syntactic tree for the query is generated based on dependency syntactic analysis, and the syntactic tree is converted into an abstract semantic tree in combination with the domain knowledge graph.

3. The NL2SQL method based on reinforcement learning according to claim 2, characterized in that: In the schema linking and candidate set determination steps, the database schema graph structure G(Schema) = (T, C, E) is constructed and applied as follows: each table node in the table set T contains the table name and description information; each column node in the column set C contains the column name, data type and the table ID to which it belongs; for entity nodes in the abstract semantic tree, the corresponding tables and columns are searched in the database schema graph through string matching, type constraint matching and relational reasoning to generate a candidate mapping set; based on the similarity of the candidate mappings, the top-K candidate schema objects are selected for each semantic entity node to form the final candidate SQL fragment set.

4. The NL2SQL method based on reinforcement learning according to claim 3, characterized in that: In the SQL generation step, the state space, action space, and reward function of the reinforcement learning agent are defined as follows: the state space contains the currently unparsed semantic tree fragment, the generated SQL fragment, the candidate mapping set, and the current access status of the database schema graph; the action space includes selecting a candidate mapping to link the semantic entity to the schema, adding SQL operators, and confirming the current SQL fragment and entering the next stage; The reward function comprehensively considers factors such as correctness, efficiency, and semantic matching. Specifically, it provides positive rewards if the generated SQL is successfully executed and the results meet expectations; rewards for execution efficiency based on the SQL execution time; rewards for semantic matching based on the similarity of candidate mappings; and penalties for duplicate linked tables / columns, syntax errors, and invalid actions.

5. The NL2SQL method based on reinforcement learning according to claim 4, characterized in that: The query execution and feedback optimization step includes the following operations: integrating with the database connection pool, performing syntax verification and security filtering on the generated SQL statements, and then executing them in the database; If SQL execution fails, parse the error type and return a fine-grained penalty signal; If the execution is successful, the correctness of the result is verified by comparing the key constraints in the user query with the query result statistics and using active learning technology to request the user to mark low-confidence results; the valid query-result pairs in the real-time interaction are stored in the incremental training set, and the model fine-tuning is triggered regularly. The hot update technology is used to update the parameters of the reinforcement learning model without interrupting the service.

6. A system for the reinforcement learning-based NL2SQL method according to claim 5, characterized in that: include: Natural language input preprocessing module: Receives natural language queries input by users, performs word segmentation, part-of-speech tagging, and named entity recognition on them, and extracts key semantic components from the queries; Semantic parsing and abstract representation building module: Using deep learning models, the pre-processed natural language query is converted into an abstract semantic tree, which includes operation nodes, entity nodes, function nodes, and logic nodes. Schema linking and candidate set determination module: The database schema is represented as a graph structure G(Schema) = (T, C, E), where T is the table set, C is the column set, and E is the foreign key relationship edge between tables. The mapping relationship between abstract semantic tree nodes and database tables and columns is established to generate a set of candidate SQL fragments. SQL generation module: With the help of reinforcement learning agents, based on the preset state space, action space, and reward function, it selects the best fragments from the candidate SQL fragment set and combines them to generate a complete SQL statement; Query execution and feedback optimization module: executes the generated SQL statements in the database, calculates the reward value based on the execution results, and uses the reward value to optimize the parameters of the reinforcement learning model; Multi-round interaction processing module: Maintains the conversation history stack, records the semantic tree of each query round, the generated SQL and execution results, and extracts contextual entities; performs differential parsing on new queries, processing only the parts that differ from historical conversations, and combines the historical semantic vectors with the current query vector through the attention mechanism to generate context-aware semantic representations to support SQL generation for subsequent queries.

7. A system according to claim 6, characterized in that: In the semantic parsing and abstract representation construction module, a pre-trained language model combined with a bidirectional long short-term memory network (LSTM) is used for semantic encoding: the pre-trained language model is used to perform contextual semantic encoding on natural language queries to obtain hidden state vectors at the character / word level; the hidden state vectors are processed through a bidirectional LSTM network to capture long-distance dependencies and output sentence-level semantic vectors and phrase-level semantic vectors; a syntactic tree for the query is generated based on dependency syntactic analysis, and the syntactic tree is converted into an abstract semantic tree in combination with the domain knowledge graph.

8. A system according to claim 6, characterized in that: In the schema linking and candidate set determination module, the database schema graph structure G(Schema) = (T, C, E) is constructed and applied as follows: each table node in the table set T contains the table name and description information; each column node in the column set C contains the column name, data type and the table ID to which it belongs; for entity nodes in the abstract semantic tree, the corresponding tables and columns are searched in the database schema graph through string matching, type constraint matching and relational reasoning to generate a candidate mapping set; based on the similarity of the candidate mappings, the top-K candidate schema objects are selected for each semantic entity node to form the final candidate SQL fragment set.

9. A system according to claim 6, characterized in that: In the SQL generation module, the state space, action space, and reward function of the reinforcement learning agent are defined as follows: the state space includes the currently unparsed semantic tree fragment, the generated SQL fragment, the candidate mapping set, and the current access status of the database schema graph; the action space includes selecting a candidate mapping to link the semantic entity to the schema, adding SQL operators, and confirming the current SQL fragment and entering the next stage; The reward function comprehensively considers factors such as correctness, efficiency, and semantic matching. Specifically, it provides positive rewards if the generated SQL is successfully executed and the results meet expectations; rewards for execution efficiency based on the SQL execution time; rewards for semantic matching based on the similarity of candidate mappings; and penalties for duplicate linked tables / columns, syntax errors, and invalid actions.

10. A system according to claim 6, characterized in that: The query execution and feedback optimization module includes the following operations: integrating with the database connection pool, performing syntax verification and security filtering on the generated SQL statements, and then executing them in the database; If SQL execution fails, parse the error type and return a fine-grained penalty signal; If the execution is successful, the correctness of the result is verified by comparing the key constraints in the user query with the query result statistics and using active learning technology to request the user to mark low-confidence results; the valid query-result pairs in the real-time interaction are stored in the incremental training set, and the model fine-tuning is triggered regularly. The hot update technology is used to update the parameters of the reinforcement learning model without interrupting the service.

Images