An NL2SQL method based on LLM collaboration under a structured inference space

Abstract

The application belongs to the field of database intelligent interaction, and relates to an NL2SQL method based on LLM cooperation under a structured reasoning space, comprising the following steps: constructing a structured reasoning space based on a natural language query and a target database mode; obtaining a skeleton candidate set; screening the skeleton candidates to obtain an orthogonal skeleton candidate set; taking the orthogonal skeleton candidate set as a starting point, layer by layer progressing along a granularity dimension and verifying to obtain a verification result; when the verification result is passed, calculating a fusion reward of a candidate state; taking the fusion reward as a guide, performing an operation on the candidate state through tree search to promote to a next granularity level; when the verification result is not passed, performing structured backtracking to locate to an effective ancestor state; when the candidate state progresses to a final granularity level and passes a consistency verification, terminating the search and outputting a structured query language statement; and achieving closed-loop cooperation of the three of search-verification-structure, which has reliability in a complex NL2SQL query scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent database interaction, and specifically discloses an NL2SQL method based on LLM collaboration in a structured reasoning space. Background Technology

[0002] The Natural Language to Structured Query Language (Text-to-SQL, NL2SQL) task aims to automatically convert users' natural language questions into Structured Query Language (SQL) statements that can be executed on relational databases. This technology is one of the core technologies for intelligent database interaction and is widely used in business intelligence, data analysis, intelligent customer service, and other fields, enabling non-specialist users to conveniently access and query databases using natural language. With the continuous breakthroughs in the capabilities of Large Language Models (LLMs) in natural language understanding and code generation, LLM-based NL2SQL methods have become the mainstream technical approach in this field.

[0003] Existing LLM-based NL2SQL systems typically employ a multi-stage inference optimization architecture, comprising four main stages: preprocessing, SQL generation, post-processing, and output. The preprocessing stage aims to optimize the input quality of the LLM, primarily including schema linking, semantic value retrieval, and question rewriting. Schema linking refers to identifying tables and columns relevant to the user's question from the database schema to reduce the schema size that the LLM needs to process. Semantic value retrieval involves retrieving specific data values ​​related to question keywords from the database and injecting them as contextual hints to bridge the semantic gap between natural language expressions and actual database stored values.

[0004] The SQL generation phase is the core reasoning stage of the system. Existing methods typically employ one or more of the following strategies: instruction following, which guides the LLM to perform structured reasoning through carefully designed prompt templates, such as thought chain reasoning; context learning, which provides a few examples in the prompts to guide the LLM to understand the task pattern; intermediate representation, which decomposes the complex SQL generation task into simpler subtasks, such as generating the SQL skeleton first and then filling in specific pattern elements, or decomposing the complex query into multiple subproblems, solving them separately, and then combining them; and tree search, which models SQL generation as a tree search problem and explores the optimal reasoning path in the action space or structure space using algorithms such as Monte Carlo tree search.

[0005] The post-processing stage optimizes the quality of the SQL generated by LLM, mainly including SQL revision, candidate selection, and result verification. SQL revision refers to correcting SQL with syntax or logical errors; candidate selection refers to selecting the optimal result from multiple candidate SQLs through mechanisms such as voting on execution results and self-consistency; result verification refers to verifying the correctness by executing the SQL and checking the reasonableness of the results.

[0006] As the complexity of NL2SQL tasks continues to increase, such as those involving multi-table joins, multi-level nested subqueries, and aggregate functions, the aforementioned multi-stage architecture still faces many challenges in achieving high accuracy and high reliability, and exhibits the following structural defects at the system level: The search space is disconnected from structural guidance, resulting in a high proportion of invalid explorations. In mainstream tree-search NL2SQL methods, the search process typically operates on a flat action space, meaning all SQL construction actions (such as selecting tables, adding conditions, and generating clauses) are at the same level of abstraction. The search tree lacks guidance from high-level structural information during expansion. When faced with complex problems requiring multi-table joins or nested subqueries, the search process is prone to deviating from the correct direction at the structural level, leading to a significant number of search iterations being wasted on invalid paths. Another approach employs a sequential strategy of predicting the skeleton first and then filling in the details. After skeleton prediction, it is used as a static template input into the SQL generation module. This strategy works well when the skeleton is correct, but when the skeleton prediction is flawed, subsequent SQL generation and revision cannot fundamentally correct structural errors, leading to the cumulative amplification of structural errors.

[0007] The verification feedback is disconnected from the search process, resulting in delayed error detection. In the existing architecture, the verification mechanism is usually treated as an independent module in the post-processing stage, only intervening after the SQL is fully generated. This post-verification model has two inherent drawbacks: First, error detection is severely delayed. If key tables or join conditions are missed during the schema linking stage, no matter how precise the subsequent SQL generation and revision stages are, the final SQL will inevitably be incorrect, and verification can only detect the error after generation is complete. Second, verification results cannot be fed back to guide the search. The error types found by verification are only used for local repair of the current SQL and are not fed back into the search strategy to prevent the same error from recurring in other paths.

[0008] The search reward signal source is singular, and its ability to distinguish complex errors is insufficient. Existing search methods typically rely solely on the self-consistency of SQL execution results in their reward functions. This involves multiple high-temperature samplings of the same candidate SQL and comparing the consistency of the execution results across these samplings. This reward function has two inherent flaws: first, when multiple erroneous SQL statements produce the same erroneous results, the self-consistency reward will incorrectly award high scores, misleading the search direction; second, the self-consistency reward cannot distinguish between different types of errors—syntax errors, schema linking errors, join logic errors, and aggregate function errors all have the same reward value, preventing the search process from specifically optimizing weak points.

[0009] Multiple candidate generation lacks diversity guarantees and suffers from information silos between strategies. Existing methods improve candidate SQL coverage by generating multiple strategies in parallel. However, multiple generators share the same LLM framework and input context, which may lead to similar systematic errors when input biases exist, degenerating into monomodal redundancy and failing to truly improve coverage. Furthermore, each generator runs independently without information exchange, and the partially correct information held by each cannot be integrated and complemented during the generation process. Existing skeleton-based methods also suffer from diversity issues, with multiple skeleton candidates coming from multiple generation iterations of the same LLM, exhibiting similarity biases and lacking explicit guarantees of orthogonality between skeletons.

[0010] In view of this, and in response to the shortcomings of existing NL2SQL inference optimization architectures such as disconnected structure guidance, delayed verification feedback, single reward signal, and insufficient diversity guarantee, this invention provides an NL2SQL method based on LLM collaboration in a structured inference space. Through unified SRS three-dimensional space modeling, pre-embedding of multi-granularity verification checkpoints, structured backtracking, three-source fusion reward function, and skeleton distance metric to ensure diversity adaptive search, a closed-loop collaboration of search, verification, and structure is achieved, which can ensure reliability in complex NL2SQL query scenarios. Summary of the Invention

[0011] This invention provides an NL2SQL method based on LLM collaboration within a structured reasoning space, addressing the technical problem of how to achieve closed-loop collaboration among search, verification, and structure to ensure reliability in complex NL2SQL query scenarios. The specific solution is as follows: A method for NL2SQL based on LLM collaboration within a structured reasoning space includes: Step 1: Obtaining the user's natural language query and target database pattern, and constructing a structured reasoning space based on the natural language query and the target database pattern; the structured reasoning space is a three-dimensional space composed of granularity dimension, verification dimension, and operation dimension; the state in the structured reasoning space is jointly represented by the values ​​of the granularity dimension, the verification dimension, and the operation dimension; the granularity dimension describes the abstraction level of the current reasoning, the verification dimension describes the verification level that the current state has passed, and the operation dimension describes the type of search operation currently being performed; Step 2: Generating skeleton candidates in parallel using multiple skeleton generation strategies to obtain a skeleton candidate set; the skeleton is a representation of the query structure in the structured reasoning space; Step 3: Calculating the skeleton distance between skeletons in the skeleton candidate set, retaining skeleton candidates whose skeleton distance is greater than or equal to a preset diversity threshold, and obtaining... Step 4: Starting from the orthogonal skeleton candidate set, proceed layer by layer along the granularity dimension. At each step to the next granularity level, perform verification on the candidate state corresponding to the current granularity level to obtain the verification result. The granularity dimension includes multiple granularity levels from coarse to fine. Step 5: When the verification result is passed, calculate the fusion reward of the candidate state. The fusion reward is obtained by fusing the execution consistency reward, structure verification reward, and data layer reward. Step 6: Guided by the fusion reward, perform generation or refinement operations on the candidate state through tree search, advance to the next granularity level, and return to Step 4. Step 7: When the verification result is failed, perform structured backtracking to relocate the search to the valid ancestor state in the structured reasoning space and return to Step 4. Step 8: When the candidate state advances to the final granularity level and passes the execution consistency verification, terminate the search and output the structured query language statement.

[0012] Furthermore, the multiple granularity levels include sentence level, clause level, expression level, and lexical level; the sentence level is used to represent the classification of complete query intent, including the top-level operation type of the query, the set of root tables involved, and the set of top-level clauses; the clause level is used to represent the structured query language clause structure, including clause nesting information and the inter-table connection graph; the expression level is used to represent the content filled in the expression within the clause; the lexical level is used to represent specific structured query language lexicals, which include column names, constants, and functions.

[0013] Furthermore, the types of verification include a syntax verification layer, a semantic link verification layer, a value matching verification layer, and an execution consistency verification layer. The syntax verification layer is used to verify the structural validity of the structured query language. The structural validity includes successful abstract syntax tree parsing, compatibility between operators inside the clause and column types, and consistency between grouping fields and aggregate items in the select list. The semantic link verification layer is used to verify whether the referenced tables and columns exist in the target database schema and whether foreign key relationships are valid. The value matching verification layer is used to verify whether the values ​​of constants in the conditions fall within the data value range of the corresponding columns. The execution consistency verification layer is used to verify whether the results after simulated execution are consistent with the expected statistics. The execution-related verification corresponding to the current granularity level is one or more of the syntax verification layer, the semantic link verification layer, and the value matching verification layer. The execution consistency verification is performed by the execution consistency verification layer.

[0014] Furthermore, the operation dimensions include generation, refinement, verification, pruning, backtracking, and termination operations. The generation operation generates candidate structured query language fragments at the current granularity level. The refinement operation refines the current granularity level to the next granularity level. The verification operation performs verification at the next level of the verification dimension on the current candidate state. The pruning operation removes branches that fail a certain verification level. The backtracking operation returns to the previous valid granularity level for re-searching. The termination operation terminates the search and outputs a structured query language statement at the finest granularity level after passing consistency verification. Whether each of the various search operations is valid in the current state is constrained by the current value of the granularity dimension and the current value of the verification dimension.

[0015] Furthermore, the various skeleton generation strategies include a top-down strategy, a bottom-up strategy, and a hybrid strategy; the top-down strategy refers to building the skeleton layer by layer downwards from the top layer of the structured reasoning space; the bottom-up strategy refers to building the skeleton upwards from the relevant tables and columns in the target database schema; and the hybrid strategy refers to using both the top-down and bottom-up strategies to build the skeleton.

[0016] Furthermore, the skeleton distance is: ; in, For the i-th skeleton candidate With the j-th skeleton candidate The skeletal distance between them; Edit distance for the skeleton structure; These are the skeleton weighting coefficients; To verify the required Hamming distance.

[0017] Furthermore, the number of generated skeleton candidates is dynamically adjusted based on the query complexity index of the natural language query, where a higher query complexity index is directly proportional to the number of skeleton candidates; the query complexity index is: ; in, This is a metric for query complexity. The number of entities in the problem; It refers to the number of judgment conditions involved in the problem; This is the estimated query depth; This refers to the size of the database schema.

[0018] Furthermore, the fusion reward is: ; in, For integration rewards; To implement consistency rewards; Rewards for structural verification; Rewards for the data layer; , and These are the weight coefficients for execution consistency, structural verification, and data layer, respectively.

[0019] Furthermore, the execution of structured backtracking includes: determining the verification dimension that failed verification, the granularity level when verification failed, the current search operation, and the failure type; searching along the verification dimension towards the ancestor direction in the structured inference space to obtain the nearest valid ancestor state; the valid ancestor state is a state where verification passed and the verification dimension is compatible with the verification dimension that failed verification; calculating the backtracking target offset based on the failure type, the granularity level when verification failed, and the valid ancestor state; encoding the backtracking target offset as a backtracking pointer and injecting it into search control parameters, and positioning the search to the valid ancestor state as the target starting point for the next round of search.

[0020] Furthermore, the nearest effective ancestor state is determined by an effective neighborhood function, which is: ; in, For effective neighborhood functions; To verify the verification dimensions that failed; Δ is the tolerance parameter; For any candidate validation dimension in the set of validation dimensions; To verify the set of dimensions, To verify the distance between dimensions; This is a valid decision function.

[0021] The present invention has the following advantages and beneficial effects: (1) Structured constraints guide the significant compression of the search space and reduce the proportion of ineffective exploration: Existing tree search methods search on a flat action space, where all SQL construction actions are at the same level of abstraction, lacking guidance from high-level structural information. This patent's SRS three-dimensional space divides the search process from coarse to fine into four levels: G_sentence, G_clause, G_expression, and G_token, using the G dimension (granularity dimension). In the skeleton hypothesis generation stage, the system first generates a coarse-grained skeleton, which constrains the action space to a subset compatible only with the top-level structure. Subsequently, after filtering through multi-granularity verification checkpoints, the medium-granularity skeleton further constrains nested structures and connection relationships. Finally, the fine-grained skeleton refines the executable action space into placeholder-filling actions. Through this layer-by-layer refinement of structural constraints, the volume of the search space can be effectively explored.

[0022] (2) Pre-process verification blocks error accumulation, reducing single-path error rate: Existing methods typically intervene as a post-processing stage after the SQL is fully generated, resulting in a significant delay in error detection. This patent addresses this by implementing multi-granularity verification checkpoints, placing verification after each level of skeleton generation. This pre-verification mechanism allows structural errors to be captured and corrected early in the search process, rather than being patched only after errors have accumulated, fundamentally changing the passive pattern of "error occurrence → continuous propagation → eventual discovery → local correction".

[0023] (3) The three-source fusion reward suppresses consistent but erroneous inflated rewards, guiding the search direction to be more accurate: Existing search-based methods rely solely on the self-consistency of SQL execution results for their reward functions. When multiple erroneous SQL statements produce the same incorrect results, the self-consistency reward is incorrectly given a high score, severely misleading the search direction. In the three-source fusion reward function of this patent, the structural verification reward is a weighted sum of the pass rates of four verification checkpoints. When multiple erroneous SQL statements produce the same incorrect execution results, they may be syntactically correct but logically incomplete or data-infeasible. In this case, the score will be significantly reduced, forming a quantifiable distinction between the score of correct SQL statements and effectively suppressing inflated rewards. Furthermore, the different distribution patterns of different error types across the three sub-items allow MCTS to determine the cause of failure based on the type of the lower-order sub-item, and to strategically increase the exploration weight of the corresponding type of action in subsequent rollouts, rather than blindly retrying.

[0024] (4) Explicit skeleton distance metric ensures candidate orthogonality and enhances the synergistic effect between strategies: Existing methods, whether employing a homomodal generation strategy using multiple samplings of the same LLM or a multi-strategy approach with multiple generators running in parallel, suffer from insufficient diversity. This patent explicitly controls the degree of difference between candidate skeletons through a skeleton distance metric. By setting a diversity threshold, the system retains only candidate skeletons whose distances are greater than the threshold. Orthogonalized candidate skeletons cover different structural regions in subsequent action searches, rather than fine-tuning details within the same structural pattern. Furthermore, candidate skeletons generated by multiple strategies are uniformly represented as standardized state nodes in the SRS, and their verification results and search history are managed uniformly within the same framework. This allows the outputs of different strategies to share verification feedback, achieving constraint-level information complementarity and coverage greater than a simple union of strategies running independently. Attached Figure Description

[0025] Figure 1 This is an exemplary flowchart of an NL2SQL method based on LLM collaboration in a structured reasoning space, provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0027] Figure 1 This is an exemplary flowchart of an NL2SQL method based on LLM collaboration in a structured reasoning space, provided by the present invention. Figure 1 As shown, this embodiment provides an NL2SQL method based on LLM collaboration within a structured reasoning space. Taking natural language queries and target database patterns as input, the overall process revolves around the constructed structured reasoning space (SRS): The system first performs multi-strategy skeleton generation and orthogonality filtering. Then, in the progressive process of granularity dimension (G), multi-granularity verification checkpoints (V) are embedded beforehand. A verification-driven structured backtracking mechanism is used to achieve accurate recovery of failed states. Simultaneously, Monte Carlo tree search continuously optimizes the exploration direction under the guidance of the fusion reward function. Finally, when a candidate state passes consistency verification, the search terminates and an executable SQL statement is output. The method specifically includes the following steps: Step 1: Obtain the user's natural language query and target database pattern, and construct a structured reasoning space based on the natural language query and target database pattern.

[0028] Natural language queries refer to queries entered by users in natural language, such as "count the number of orders for each customer after January 1, 2024". The target database schema refers to the structural description of the relational database to be queried, including the tables, columns, data types, and foreign key relationships between tables. The structured inference space refers to modeling the solution of the NL2SQL task as a three-dimensional abstract space, denoted as: SRS = G × V × A; In this space, SRS represents a three-dimensional abstract space; G represents the granularity dimension; V represents the verification dimension; and A represents the operational dimension. The NL2SQL solution process in this space is characterized by alternating exploration and verification, gradually approaching the correct SQL statement.

[0029] Granularity dimension refers to the dimension that characterizes the current level of abstraction in the reasoning. Different values ​​correspond to different structural levels, from coarse to fine. Granularity dimension is: ; in, At the sentence level, it is used to categorize complete query intents (such as aggregate queries, filter queries, and multi-table queries). , The {SELECT,Aggregation,Comparison,SetOp} represents the top-level operation type of the query; This represents the set of root tables that may be involved. {SELECT, WHERE, GROUP BY, ORDER BY} represents the set of top-level clauses. At the clause level, specifically the SQL clause structure, such as FROM table set, WHERE filter condition, GROUP BY grouping field, and ORDER BY sorting rule, etc. NestInfo represents nested clause information; JoinGraph represents the graph of connections between tables. At the expression level, this can specifically refer to filling in expressions within clauses, such as column name-operator-constant triples in WHERE, or join conditions in JOIN. It is at the token level, specifically it can be SQL tokens, such as column name 'orders.order_date', constant '2024-01-01' and function 'MAX(...)', etc.

[0030] The verification dimension refers to the dimension that describes the level of verification the current state has passed. Different values ​​correspond to different verification levels with increasing credibility. The verification dimension is: ; in, For syntax validation, specifically the validity of the SQL structure, such as successful AST parsing, compatibility of operators within clauses with column types, and consistency of GROUP BY and SELECT aggregations. Semantic link verification can specifically include whether the referenced tables and columns exist in the database schema and whether foreign key relationships are valid. For value matching validation, such as whether the value of a constant in the WHERE condition is within the data value range of the corresponding column; To perform consistency verification, such as checking whether the results after simulation are consistent with the expected statistics, the verification dimensions are sequential; only after passing the verification of one dimension will the next dimension be moved on to.

[0031] The operation dimension describes the type of search operation being performed; different values ​​correspond to different search actions. The operation dimension is: ; in, To generate, for example, candidate SQL fragments at a certain granularity level to advance the search; To refine, specifically, the current granularity is refined to the next granularity level, such as... Advance the G dimension; To verify this, specifically, we can perform the next level of verification in dimension V on the current candidate, thus advancing the V dimension. This is pruning, that is, removing branches that do not pass through a certain layer V, thus shrinking the search space; This is for backtracking, that is, returning to the previous valid G layer and searching again (triggered by failure in the V dimension). To terminate, specifically, it can be in Particle size and passing The search is then terminated, and the final SQL is output.

[0032] In the structured reasoning space, the state is represented by values ​​in the granularity dimension, the verification dimension, and the operation dimension. That is, any point in the structured reasoning space can be represented as a triple (g, v, a), representing a structured reasoning state. Here, g is the granularity dimension value, indicating the current level of abstraction and determining the level at which reasoning can be performed; v is the verification dimension value, indicating which level of verification the candidate has passed, determining the credibility of the current state; and a is the operation dimension value, indicating which operation is being performed, determining the next step. For example, when a mismatch is found in the referenced table or column during an expression-level search, the process immediately backtracks to the clause level and re-links the tables and columns. This correction is a state transition within the space, rather than a backflow between stages in the existing architecture.

[0033] Step 2: Generate skeleton candidates in parallel using multiple skeleton generation strategies to obtain a skeleton candidate set.

[0034] A skeleton is a representation of the query structure within a structured inference space. For example, it might be an SQL framework that only specifies the top-level operation type, the tables involved, and the clauses, but hasn't yet included specific column names and constants. In some embodiments, various skeleton generation strategies include top-down, bottom-up, and hybrid strategies. A top-down strategy builds the skeleton layer by layer downwards from the top level of the structured inference space, extracting the skeleton from the top-level intent of the query. A bottom-up strategy builds the skeleton upwards from relevant tables and columns in the target database schema. Relevant tables and columns refer to those identified in the target database schema that are associated with the natural language query. A hybrid strategy uses both top-down and bottom-up strategies to build the skeleton. These three strategies operate independently, and the results are then combined to obtain a candidate skeleton set.

[0035] To address the lack of diversity in existing skeleton-based methods, each strategy's generated skeleton candidates are represented as standardized state nodes in a structured inference space. Their verification results and search history are managed uniformly within the same framework, enabling skeleton candidates generated by different strategies to share verification feedback and search experience, achieving constraint-level information complementarity. Its coverage is greater than that of a simple union of strategies operating independently.

[0036] Furthermore, the number of generated skeleton candidates can be dynamically adjusted based on the query complexity index of the natural language query. A higher query complexity index results in a greater number of generated skeleton candidates; the two are directly proportional. The query complexity index, denoted as C_q, is a quantitative indicator used to measure the difficulty of solving a natural language query, and its calculation formula is: ; in, This is a metric for query complexity. The number of entities in the problem; The number of decision conditions involved in the problem; This is the estimated query depth; This refers to the database schema size. In some embodiments, a low complexity threshold is set. With high complexity threshold :when At that time, 3 to 5 skeleton candidates are generated; when and At that time, 5 to 10 skeleton candidates are generated; when 10 to 15 skeleton candidates are generated at a time.

[0037] Step 3: Calculate the skeleton distance between skeletons in the skeleton candidate set, retain skeleton candidates whose skeleton distance is greater than or equal to the preset diversity threshold, and obtain the orthogonal skeleton candidate set.

[0038] Skeleton distance is a metric used to measure the degree of orthogonality between two skeletons. The preset diversity threshold is a pre-defined lower limit for skeleton distance used to filter candidate skeletons, denoted as [insert threshold here]. Only skeleton candidates whose skeleton distance is greater than or equal to the threshold are retained.

[0039] In some embodiments, the formula for calculating the skeleton distance is: ; in, For the i-th skeleton candidate With the j-th skeleton candidate The skeletal distance between them; For the graph edit distance of the skeleton structure, and Respective skeleton candidates and The skeleton structure is represented to measure the structural differences between two skeletons; To verify the required Hamming distance, and Respective skeleton candidates and The validation requirement is represented by λ, which measures the difference in validation requirements between the two skeletons; λ is the skeleton weighting coefficient that weighs the difference in skeleton structure against the difference in validation requirements.

[0040] By using the skeleton distance metric and diversity threshold screening, the orthogonality of candidate skeleton structures is guaranteed from the source, so that different candidate skeletons cover different structural regions in subsequent action searches, rather than fine-tuning details in the same structural pattern, thus avoiding multiple candidates redundantly exploring the same erroneous region.

[0041] Step 4: Starting with the orthogonal skeleton candidate set, proceed layer by layer along the granularity dimension. At each step, perform the verification corresponding to the current granularity level on the candidate state of the current granularity level to obtain the verification result. The granularity dimension includes multiple granularity levels from coarse to fine.

[0042] "Coarse-to-fine" refers to defining the granularity based on structural inclusion relationships. A unit at a higher granularity level expands into multiple units at lower granularity levels, with higher granularity levels being coarser and lower granularity levels being finer. Candidate state refers to the reasoning state of the SQL structure currently being constructed during the search process. Validation result refers to the judgment result obtained after performing corresponding validations on the candidate states at the current granularity level.

[0043] In some embodiments, the granularity dimension includes four levels of granularity from coarse to fine: sentence level, clause level, expression level, and lexical level. The sentence level is used to represent the classification of the complete query intent. The complete query intent refers to the overall query purpose expressed by the natural language query. The sentence level includes the top-level operation type of the query, the set of root tables involved, and the set of top-level clauses. The top-level operation type refers to the highest-level operation category of the query. The set of root tables involved refers to the set of root tables that the query may involve. The set of top-level clauses refers to the set of clauses contained in the top-level query. The clause level is used to represent the structured query language clause structure, including clause nesting information and the inter-table connection graph. Clause nesting information refers to the nesting relationships between clauses. The inter-table connection graph is a graphical representation of the connection relationships between the tables involved in the query. The expression level is used to represent the filler content of expressions within clauses. The filler content of expressions within clauses refers to the constituent content of specific expressions within clauses. The lexical level is used to represent specific structured query language lexicals, which include column names, constants, and functions. The verification corresponding to the current granularity level is taken from the verification dimension.

[0044] In some embodiments, the validation types include a syntax validation layer, a semantic link validation layer, a value matching validation layer, and an execution consistency validation layer. The syntax validation layer verifies the structural legality of the Structured Query Language (SQL). Structural legality refers to the legality of the SQL statement at the structural level. Successful abstract syntax tree parsing means that the SQL statement can be successfully parsed into an abstract syntax tree. Clause internal operator compatibility with column types means that the operators used within the clause match the data types of the columns on either side. Grouping fields consistent with aggregate items in the select list means that the grouping fields specified by GROUP BY are consistent with the aggregate items in the SELECT select list, and there are no illegal non-aggregate columns. The semantic link validation layer verifies whether the referenced tables and columns exist in the target database schema and whether foreign key relationships are valid. The referenced tables and columns refer to the tables and columns referenced in the SQL statement. Foreign key relationships refer to the associations established between related tables through foreign keys. The value matching validation layer verifies whether the values ​​of constants in the conditions fall within the data value range of the corresponding columns. Conditions refer to the filtering conditions in the SQL statement, such as the conditions in the WHERE clause. The data value range of the corresponding column refers to the actual range of values ​​that the target column compared to the constant can take in the database. The consistency verification layer is used to verify whether the results of the simulated execution are consistent with the expected statistics. Simulated execution refers to the process of trial-executing the generated SQL statements. Expected statistics refer to the statistical characteristics of the results estimated in advance based on the query intent.

[0045] In step 4, the verification corresponding to the current granularity level is performed through one or more of the following: syntax verification layer, semantic link verification layer, and value matching verification layer. This means that the aforementioned verification layers are embedded beforehand in the progressive process from sentence level to clause level to expression level. In step 8, the consistency verification is performed by the consistency verification layer, serving as the final termination condition. Thus, the verification process shifts from post-hoc judgment to in-process, multi-layered, and continuously trackable signals that guide the search.

[0046] In some embodiments, the syntax validation layer performs validation through a deterministic rule engine; the semantic linking validation layer performs validation through a lightweight model; and the value matching validation layer performs validation based on candidate data values ​​retrieved from the target database. A deterministic rule engine is an engine that determines the legality of SQL structures according to pre-defined deterministic rules. A lightweight model is a model with a smaller parameter size and lower inference overhead compared to the large language model used to generate SQL. Candidate data values ​​are the actual data values ​​retrieved from the corresponding columns of the target database and used to match conditional constants.

[0047] The operation dimension is constrained by both the G and V dimensions, meaning not every operation is valid in every state. Any point (g, v, a) in the SRS represents a structured reasoning state: g ∈ G: the current reasoning level, determining the level of abstraction at which reasoning can be performed; v ∈ V: the current candidate has passed through a certain level, determining the credibility of the current state; a ∈ A: which operation is currently being executed, determining the next step. For example, when in… A schema link mismatch was found during the layer search. (Not approved), immediately backtrack to The layers are then re-linked in the schema. This correction is a state transition within the space, not a reflow between stages.

[0048] SRS exhibits monotonicity in the granularity dimension, for any... exist and exist There exists a monotonic mapping. Make Each state s can be uniquely mapped to The set of states φ(s) in the subspace.

[0049] Step 5: When the verification result is passed, calculate the fusion reward of the candidate state; the fusion reward is obtained by fusing the execution consistency reward, the structural verification reward and the data layer reward.

[0050] The fusion reward guides the exploration direction of the tree search. The execution consistency reward is the reward obtained based on the consistency of the SQL statement execution results. The structure validation reward is the reward obtained based on the validation results of each validation layer. The data layer reward is the reward obtained based on the semantic similarity of data values ​​in the SQL.

[0051] In some embodiments, the formula for calculating the fusion reward is: ; in, For integration rewards; To implement the consistency reward, its value is 1 if the generated SQL statement executes successfully, and 0 otherwise; The reward for structural verification is calculated as follows: , The result of the skeleton s being validated at the kth layer (1 for passing, 0 for failing). The weights corresponding to the k-th layer of validation are assigned, so that failure in any of the four layers of validation will directly reduce the weights. ; The data layer reward is calculated as follows: , The number of semantic values ​​in the query. For the k-th semantic value in the query, The table contains candidate data values ​​retrieved from the table. sim(·) is the semantic similarity function, which is calculated using cosine similarity in some embodiments. α, β, and γ are the weight coefficients for the execution consistency reward, structure verification reward, and data layer reward, respectively.

[0052] Therefore, when multiple erroneous SQL statements produce the same erroneous execution results, although their scores are artificially high in terms of execution consistency, their scores in terms of structural verification rewards and data layer rewards are significantly reduced, thus forming a quantifiable distinguishing interval with the scores of correct SQL statements, effectively suppressing artificially high rewards.

[0053] Step 6: Guided by the fusion reward, perform generation or refinement operations on candidate states through tree search, advancing to the next granularity level and returning to Step 4. Generation operation refers to generating candidate structured query language fragments at the current granularity level. Candidate structured query language fragments are candidate fragments that constitute part of an SQL statement generated at the current granularity level. Refinement operation refers to refining the current granularity level to the next granularity level.

[0054] In some embodiments, the tree search is a Monte Carlo tree search. The reward-guided approach includes increasing the exploration weight of operations of the corresponding error type in subsequent searches based on the error type corresponding to the lower-valued sub-item in the reward fusion. Since different error types (e.g., syntax errors, schema linking errors, join logic errors, and aggregation function errors) exhibit different distribution patterns across the three sub-items of execution consistency reward, structure verification reward, and data layer reward, Monte Carlo tree search can determine the cause of failure based on the type of the lower-value sub-item and directionally increase the exploration weight of the corresponding action type in subsequent rollouts, rather than blindly retrying.

[0055] In addition to the generation and refinement operations mentioned above, the operation dimension in some embodiments also includes verification, pruning, backtracking, and termination operations. That is, the operation dimension includes six search operations: generation, refinement, verification, pruning, backtracking, and termination. The verification operation is used to perform verification at the next level of the verification dimension on the current candidate state. The pruning operation is used to remove branches that fail a certain verification level to shrink the search space. The backtracking operation is used to return to the previous effective granularity level for re-searching, triggered by a verification dimension failure. The previous effective granularity level refers to the granularity level that has passed verification before the current granularity level. The termination operation is used to terminate the search and output a structured query language statement after passing consistency verification at the finest granularity level. In some embodiments, the finest granularity level is the term level. Whether each of the above six search operations is valid in the current state is constrained by the current value of the granularity dimension and the current value of the verification dimension; that is, not every operation is valid in every state. Whether an operation is legal in the current state refers to whether it is allowed to be executed in the current inference state (g,v,a).

[0056] Step 7: When the verification result is unsuccessful, perform structured backtracking, relocate the search to the valid ancestor state in the structured reasoning space, and return to step 4.

[0057] Structured backtracking refers to a mechanism that, when verification fails at a point in the structured inference space, precisely locates and relocates the search to a reliable state within the structured inference space. This differs from existing methods that generate the data first, then verify, and only perform local patching after failure. Structured backtracking directly maps verification failures in the verification dimension to backtracking operations in the operation dimension, forming a closed-loop coupling between verification and search. A valid ancestor state refers to a state obtained by searching along the verification dimension towards its ancestor in the structured inference space, a state that has passed verification and is compatible with the verification dimensions that failed verification.

[0058] In some embodiments, performing a structured backtracking includes the following steps: Failure Diagnosis: This involves identifying the validation dimension that failed, the granularity level at which the validation failed, the current search operation, and the failure type. The validation dimension that failed refers to the validation layer at which the validation failed, denoted as... ( / / / (One of them). The granularity level at which verification fails refers to the granularity level at which the verification fails, denoted as... The current search operation refers to the search operation being executed when verification fails, denoted as... Failure type refers to the error type that verification failed, denoted as... .

[0059] Nearest effective ancestor search: Search along the verification dimension in the structured reasoning space towards the ancestor to obtain the nearest effective ancestor state. , , ),,in Is with Compatible verification dimension. Ancestor-direction search refers to searching along the verification dimension in directions that have passed verification. In some embodiments, the nearest valid ancestor state is determined by a valid neighborhood function, which is: ; in, For the effective neighborhood function, it returns a set of candidate verification dimensions that meet the conditions; To verify the verification dimensions that failed; This is a tolerance parameter; for example, the default value is 2. Let V be any candidate validation dimension in the validation dimension set V; V is the validation dimension set. To verify the distance between dimensions, the distance between elements of different dimensions in the verification dimension set V is 1; Verify(·) is the validity judgment function, used to determine whether a candidate verification dimension passes the verification. Since Δ defaults to 2 and the distance between verification dimensions is 1, the backtracking cost is only O(Δ×|V|) verification times.

[0060] Generate a backtrack pointer: based on the failure type Granularity level and effective ancestor state when verification fails The backtracking target offset is calculated, encoded as a backtracking pointer, and injected into the search control parameters. The search is then positioned at the valid ancestor state as the starting point for the next round of search. The backtracking target offset refers to the offset from the failed state to the valid ancestor state in both the granularity and verification dimensions, denoted as [missing information]. ,in and These represent the granularity level of the valid ancestor state and the verification dimension, respectively. The backtracking pointer is used to indicate the backtracking target, denoted as... Search control parameters refer to the parameters used to control the search behavior. The next search round refers to the search round that restarts after backtracking. The target starting point refers to the initial state of the next search round. This completes the closed loop of failure location, ancestor search, and backtracking target generation.

[0061] Step 8: When the candidate state progresses to the final granularity level and passes the consistency verification, terminate the search and output the Structured Query Language statement.

[0062] The final granularity level refers to the finest granularity level in the granularity dimension. In some embodiments, this is the lexical level. Passing consistency verification means that the candidate state passes the verification of the consistency verification layer. The structured query language statement refers to the SQL statement corresponding to the candidate state that can be executed on the target database.

[0063] Example 1 The following example further illustrates the above process. Suppose the natural language query is "to count the number of orders for each customer after January 1, 2024". The target database schema contains a customers table (containing columns such as customer_id and customer_name) and an orders table (containing columns such as order_id, customer_id, and order_date). The two tables are linked by a foreign key relationship through customer_id.

[0064] First, the system constructs a structured reasoning space and generates multiple skeleton candidates in parallel using a top-down strategy, a bottom-up strategy, and a hybrid strategy. The top-down strategy identifies the top-level operation type of the query as aggregation and the top-level clause set as containing SELECT, WHERE, and GROUP BY. The bottom-up strategy constructs a skeleton from the customers and orders tables upwards, including the connections between them. The hybrid strategy combines the results of the two strategies. Then, the system calculates the skeleton distance between each skeleton candidate and retains those with a skeleton distance greater than or equal to a preset diversity threshold, resulting in an orthogonal skeleton candidate set. This avoids multiple highly similar skeletons crowding out search resources.

[0065] Next, starting with the orthogonal skeleton candidate set, the system progressively advances along the granularity dimension from the sentence level to the clause level and then to the expression level. After determining the aggregation query intent at the sentence level, it progresses to the clause level, where semantic link verification is performed on the candidate states to check whether the customers table, orders table, and their foreign key relationships are valid. If they pass, the fusion reward is calculated, and a Monte Carlo tree search is used to perform a refinement operation, progressing to the expression level to populate expressions such as order_date>'2024-01-01' in WHERE and customer_id in GROUPBY, and performs value matching verification on the constant '2024-01-01'.

[0066] Assuming that a referenced column is found to be absent from the target database schema at the expression level (i.e., semantic link validation fails), the system performs structured backtracking: failure diagnosis determines that the validation dimension that failed is the semantic link validation layer, the granularity level of the validation failure is the expression level, and the failure type is schema linking error; then, it searches along the validation dimension towards the ancestor, obtaining the nearest valid ancestor state (i.e., the clause-level state that previously passed semantic link validation) through the valid neighborhood function; then, it calculates the backtracking target offset and encodes it as a backtracking pointer, injecting it into the search control parameters, repositioning the search to this valid ancestor state, and returning to continue the progression. Finally, when the candidate state progresses to the lexical level, specific column names and constants are filled in, and consistency validation is passed, the system terminates the search and outputs an SQL statement that can be executed on the target database.

[0067] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An NL2SQL method based on LLM collaboration in a structured reasoning space, characterized in that, include: Step 1: Obtain the user's natural language query and target database pattern, and construct a structured reasoning space based on the natural language query and the target database pattern; The structured reasoning space is a three-dimensional space consisting of granularity dimension, verification dimension, and operation dimension; The state in the structured reasoning space is represented by the values ​​of the granularity dimension, the verification dimension, and the operation dimension; the granularity dimension describes the abstraction level of the current reasoning, the verification dimension describes the verification level that the current state has passed, and the operation dimension describes the type of search operation currently being performed. Step 2: Generate skeleton candidates in parallel using multiple skeleton generation strategies to obtain a skeleton candidate set; The skeleton is a representation of the query structure in the structured reasoning space; Step 3: Calculate the skeleton distance between skeletons in the skeleton candidate set, and retain skeleton candidates whose skeleton distance is greater than or equal to a preset diversity threshold to obtain an orthogonal skeleton candidate set; Step 4: Starting from the orthogonal skeleton candidate set, proceed layer by layer along the granularity dimension. At each step, perform verification on the candidate state of the current granularity level to obtain the verification result. The granularity dimension includes multiple granularity levels from coarse to fine. Step 5: When the verification result is passed, calculate the fusion reward of the candidate state; the fusion reward is obtained by fusing the execution consistency reward, the structure verification reward, and the data layer reward; Step 6: Guided by the fusion reward, perform generation or refinement operations on the candidate states through tree search, advance to the next granularity level, and return to step 4; Step 7: When the verification result is unsuccessful, perform structured backtracking to relocate the search to the valid ancestor state in the structured reasoning space, and return to step 4; Step 8: When the candidate state progresses to the final granularity level and passes the consistency verification, the search is terminated and a structured query language statement is output.

2. The NL2SQL method based on LLM collaboration in a structured reasoning space according to claim 1, characterized in that, The multiple granularity levels include sentence level, clause level, expression level, and word level; The sentence level is used to represent the classification of the complete query intent, including the top-level operation type of the query, the set of root tables involved, and the set of top-level clauses; The clause level is used to represent the clause structure of the Structured Query Language, including clause nesting information and inter-table connection graphs; The expression level is used to represent the fill content of the expression within the clause; The terminology level is used to represent specific structured query language terms, which include column names, constants, and functions.

3. The NL2SQL method based on LLM collaboration in a structured reasoning space according to claim 1, characterized in that, The types of verification include syntax verification layer, semantic linking verification layer, value matching verification layer, and execution consistency verification layer; The syntax validation layer is used to verify the structural validity of the structured query language; the structural validity includes successful parsing of the abstract syntax tree, compatibility of operators inside the clause with column types, and consistency between the grouping field and the aggregate items in the select list; The semantic link verification layer is used to verify whether the referenced tables and columns exist in the target database schema and whether the foreign key relationship is valid; The value matching verification layer is used to verify whether the value of the constant in the condition falls within the data value range of the corresponding column; The consistency verification layer is used to verify whether the results after the simulation execution are consistent with the expected statistics. The execution verification corresponding to the current granularity level is one or more of the syntax verification layer, the semantic link verification layer, and the value matching verification layer; the execution consistency verification is performed by the execution consistency verification layer.

4. The NL2SQL method based on LLM collaboration in a structured reasoning space according to claim 1, characterized in that, The operation dimensions include generation operation, refinement operation, verification operation, pruning operation, backtracking operation, and termination operation; The generation operation is used to generate candidate structured query language fragments at the current granularity level; The refinement operation is used to refine the current granularity level to the next granularity level; The verification operation is used to perform the next level of verification in the verification dimension on the current candidate state; The pruning operation is used to remove branches that have not passed a certain verification level; The backtracking operation is used to return to the previous effective granularity level and search again; The termination operation is used to terminate the search and output a structured query language statement at the finest level after performing a consistency verification. Whether each of the various search operations is legal in the current state is constrained by the current value of the granularity dimension and the current value of the verification dimension.

5. The NL2SQL method based on LLM collaboration in a structured reasoning space according to claim 1, characterized in that, The various skeleton generation strategies include a top-down strategy, a bottom-up strategy, and a hybrid strategy; The top-down strategy refers to building the skeleton layer by layer from the top layer of the structured reasoning space downwards; The bottom-up strategy refers to building a skeleton upwards from the relevant tables and columns in the target database schema; The hybrid strategy refers to using both top-down and bottom-up strategies to construct the framework.

6. The NL2SQL method based on LLM collaboration in a structured reasoning space according to claim 1, characterized in that, The skeleton distance is: ； in, For the i-th skeleton candidate With the j-th skeleton candidate The skeletal distance between them; Edit distance for the skeleton structure; These are the skeleton weighting coefficients; To verify the required Hamming distance.

7. The NL2SQL method based on LLM collaboration in a structured reasoning space according to claim 1, characterized in that, The number of generated skeleton candidates is dynamically adjusted based on the query complexity index of the natural language query. A higher query complexity index is directly proportional to the number of skeleton candidates. The query complexity index is: ； in, This is a metric for query complexity. The number of entities in the problem; It refers to the number of judgment conditions involved in the problem; This is the estimated query depth; This refers to the size of the database schema.

8. The NL2SQL method based on LLM collaboration in a structured reasoning space according to claim 1, characterized in that, The fusion reward is: ； in, For integration rewards; To implement consistency rewards; Rewards for structural verification; Rewards for the data layer; , and These are the weight coefficients for execution consistency, structural verification, and data layer, respectively.

9. The NL2SQL method based on LLM collaboration in a structured reasoning space according to claim 1, characterized in that, The execution of structured backtracking includes: Determine the verification dimension that failed, the granularity level at which the verification failed, the current search operation, and the failure type; In the structured reasoning space, search along the verification dimension towards the ancestor direction to obtain the nearest valid ancestor state; the valid ancestor state is a state that has passed verification and whose verification dimension is compatible with the verification dimension that failed verification. Calculate the backtracking target offset based on the failure type, the granularity level when verification fails, and the effective ancestor state; The backtracking target offset is encoded as a backtracking pointer and injected into the search control parameters, and the search is positioned at the effective ancestor state as the target starting point for the next round of search.

10. The NL2SQL method based on LLM collaboration in a structured reasoning space according to claim 9, characterized in that, The nearest effective ancestor state is determined by an effective neighborhood function, which is: ； in, For effective neighborhood functions; To verify the verification dimensions that failed; Δ is the tolerance parameter; For any candidate validation dimension in the set of validation dimensions; To verify the set of dimensions, To verify the distance between dimensions; This is a valid decision function.

Images