NL2SQL ambiguity elimination method based on large language model and rule

Abstract

The invention discloses an NL2SQL ambiguity elimination method based on a large language model and rules in the technical field of natural language processing and database query. The NL2SQL ambiguity elimination method based on the large language model and the rule comprises the following steps: generating structured ambiguity mapping through the combination of the large language model and the rule, and covering a sentence pattern structure and pattern matching ambiguity; the ambiguity is clarified in the form of selection questions through user interaction and rule processing of a user interaction interface, and selection mapping is generated; and rewriting the natural language query and database mode based on the selection mapping, and outputting a final unambiguous problem and an unambiguous mode. According to the NL2SQL ambiguity elimination method based on the large language model and the rule, the accuracy and user friendliness of an analysis result are improved through a structured disambiguation framework, and clear SQL statements can be generated by inputting clear natural language query into an analyzer.

Description

Technical Field

[0001] The present invention relates to the technical fields of natural language processing and database query, and specifically, to an NL2SQL ambiguity elimination method based on large language models and rules. Background Art

[0002] A database is a carrier for organizing and storing data in a computer, and among them, relational databases are the most widely used databases at present. Database query languages are the only way to access and operate databases, such as Structured Query Language (SQL), etc. The technology of parsing natural language into SQL (NL2SQL, Text-to-SQL) greatly reduces the learning cost and usage requirements of users for SQL language. The NL2SQL parser takes a natural language query statement and a database schema (i.e., table names and column names) as inputs, outputs an equivalent SQL query, and finally the query result can be obtained by executing through a database engine. The technology of parsing natural language into SQL (NL2SQL) can help those users who are not good at database operations to efficiently query the required data through natural language. Ambiguity is an inherent obstacle commonly existing in natural language processing, which is defined as a phenomenon where the meaning of language words is not clear and there are two or more ways of understanding. In the NL2SQL task, ambiguity comes from the flexible daily language usage habits of users when expressing queries, such as abbreviations, omissions, synonyms, etc.; at the same time, it also comes from the influence of the complex row and column structures in the database, making users unable to pay attention to all the data in the database when asking questions. When the description of a query can match multiple records stored in the database, the query requirements can be met by any one of these columns, thus generating multiple reasonable SQL queries. Therefore, there is an urgent need for a technology to clarify the user's intention before the parser works to ensure the accuracy of the parsing result.

[0003] Existing NL2SQL parsers show good capabilities when dealing with clear queries, but they generally generate an optimal SQL based on probability and cannot solve the ambiguity problem faced in the parsing process. Ambiguous queries will introduce multiple reasonable interpretations, corresponding to multiple reasonable SQLs, thus making the output of the parser unstable and often deviating from the true intention of the user. Existing NL2SQL parsers can convert natural language queries into SQL statements, but there are the following significant defects when dealing with ambiguity problems:

[0004] 1. Insufficient ambiguity recognition: Existing technologies mostly generate a single optimal SQL based on probability and cannot detect ambiguous queries, resulting in the output deviating from the true intention of the user.

[0005] 2. Lack of a structured disambiguation mechanism: Existing methods cannot perform fine-grained classification and expression on syntactic structure ambiguities (such as scope ambiguity, attachment ambiguity) and pattern matching ambiguities (such as table ambiguity, column ambiguity).

[0006] 3. Heavy user interaction burden: Some studies clarify ambiguities by generating multiple candidate SQLs or through complex dialogue interactions, but it is difficult for ordinary users to make efficient selections or provide feedback.

[0007] For example, the literature reports that Huang et al. (Huang Z, Damalapati P K, Wu E. Data Ambiguity Strikes Back: How Documentation Improves GPT's Text-to-SQL[J]. arXiv preprint arXiv:2310.18742, 2023.) assist large language models in disambiguation by manually writing clarification documents, but it relies on manual intervention and cannot be automated; the LogicalBeam model reported by Bhaskar et al. (Bhaskar A, Tomar T, Sathe A, et al. Benchmarking and Improving Text-to-SQL Generation under Ambiguity[J]. arXiv preprint arXiv:2310.13659, 2023) generates multiple candidate SQLs, but users need to select them by themselves, and its applicability is limited. Summary of the Invention

[0008] The purpose of the present invention is to provide an NL2SQL ambiguity elimination method based on large language models and rules, to solve the technical problems of multiple reasonable SQL interpretations caused by natural language query ambiguities in the NL2SQL task and the following significant defects when dealing with ambiguity problems, and to improve the accuracy and user-friendliness of the parsing results through a structured disambiguation framework.

[0009] To achieve the above purpose, the technical solution of the present invention is as follows:

[0010] An NL2SQL ambiguity elimination method based on large language models and rules, comprising the following steps:

[0011] S1. Generate a structured ambiguity mapping by combining the large language model LLM with rules, covering syntactic structure and pattern matching ambiguities;

[0012] S2. Through the user interface, clarify ambiguities in the form of multiple-choice questions through user interaction and rule processing, and generate a selection mapping;

[0013] S3. Rewrite the natural language query and the database schema based on the selective mapping, and output the final unambiguous question and the unambiguous schema.

[0014] Furthermore, the S1 includes the following steps:

[0015] S11. For the syntactic ambiguity candidates, input the designed large language model (LLM) prompts and the user's natural language question into the large language model LLM, so that the large language model outputs the syntactic ambiguity mapping according to the preset instructions and few-shot examples;

[0016] S12. For the schema matching ambiguity candidates, obtain the reliable schema matching ambiguity mapping through the schema matching ambiguity mapping process.

[0017] Furthermore, the S12 includes the following steps:

[0018] S121. Enhance the column names according to the database to obtain the enhanced document for each column name;

[0019] S122. Use vector retrieval to retrieve the top-k column names and their enhanced information most relevant to the question, and integrate them into the designed prompts together with the question, so that the large language model generates the preliminary schema matching ambiguity mapping;

[0020] S123. Use the formulated rules to verify and correct the preliminarily generated mapping, so as to reduce the impact brought by the large model hallucination;

[0021] S124. Determine whether there is ambiguity in the current ambiguity mapping through the ambiguity detection rules, and finally obtain the reliable schema matching ambiguity mapping.

[0022] Furthermore, the S2 includes the following steps:

[0023] S21. The candidate ambiguity mapping can be regarded as a tree-like hierarchical structure, and thus a multiple-choice interface for user interaction is constructed according to the meaning of each layer;

[0024] S22. The user can complete the clarification of the ambiguity by ticking the given options, and the system obtains the selective mapping according to the user's feedback results.

[0025] Furthermore, the S21 includes the following steps:

[0026] S211. The ambiguity mapping is visualized as a tree, where each leaf node represents an interpretation. By combining the interpretations of each ambiguity dimension, the clarification result can be obtained, formalized as "selective mapping". The selective mapping is a mapping related to the ambiguity mapping A consistent JSON structure, but each ambiguous context h i corresponds to only one interpretation p i , and the selection mapping ψ of each ambiguous dimension i is represented in JSON syntax as:

[0027] ψ i ={h i :p i};

[0028] S212. An interactive user interface UI is designed to obtain user feedback, and multiple-choice questions are designed to clarify ambiguities.

[0029] Furthermore, the S22 includes the following steps:

[0030] S221. When the user uses the interactive user interface UI, one option, multiple options, or no option can be selected through the check box before the option, thereby forming a selection set

[0031] S222. Based on the user's selection, a selection feedback SF rule is developed to obtain the selection mapping ψ.

[0032] Furthermore, the S3 includes the following steps:

[0033] S31. The Rewriting function takes the question, pattern, and selection mapping as inputs, and through question rewriting and pattern rewriting, and through rule processing, outputs the final unambiguous question and unambiguous pattern;

[0034] S32. For question rewriting, a question rewriting QR rule is formulated for the ambiguity type to make the syntactic structure of the sentence and the context description of entity reference clear.

[0035] S33. For pattern rewriting, by formulating a pattern rewriting SR rule, the interference of the ambiguous column in the pattern matching ambiguity is eliminated.

[0036] Adopting the above technical solutions, the present invention has the following advantages:

[0037] The present invention provides an NL2SQL disambiguation method based on large language models and rules, which is a pioneering parser-independent systematic disambiguation framework that can be combined with any existing NL2SQL parser as a plugin to help all NL2SQL parsers further improve their performance. It has strong versatility and proposes a unified method for structured expression of ambiguity, namely "ambiguity mapping", which is a structure based on JSON syntax for the structured representation of common ambiguities involved in NL2SQL, and formulates expression specifications for common ambiguities in 7 types of queries. It proposes three modules: ambiguity candidate, clarification selection, and query rewriting to complete the disambiguation operation, improving the accuracy and user-friendliness of the parsing results. It has been successfully verified for syntactic structure ambiguity and pattern matching ambiguity on multiple parsers. By inputting a clear natural language query to the parser, a clear SQL statement can be generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a framework diagram of the NL2SQL disambiguation method based on large language models and rules of the present invention;

[0039] Figure 2 is a schematic diagram of syntactic structure ambiguity candidates of the present invention;

[0040] Figure 3 is a schematic diagram of pattern matching ambiguity candidates of the present invention;

[0041] Figure 4 is a schematic diagram of ambiguity clarification selection of the present invention;

[0042] Figure 5 is a schematic diagram of an SQL query equivalent to the NL2SQL task of an embodiment of the present invention;

[0043] Figure 6 is a schematic diagram of a case study of the prediction results generated by combining an embodiment of the present invention with a parser. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The technical solution of the present invention will be specifically described below in conjunction with the accompanying drawings of the specification. It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device.

[0046] Task description: Given a natural language question Q=(q1,q2,...,q |Q| ) and a database schema where S contains tables columns and primary keys The goal of the NL2SQL parser is to generate the corresponding SQL query y, which can be expressed as y = Parser(Q,S). When there is ambiguity in the parser's input, the disambiguation task is defined to convert the ambiguous Q and S into non-ambiguous inputs before the parser works and which can be expressed as Finally, a non-ambiguous parsing result can be obtained

[0047] To complete this task, the present invention proposes an NL2SQL disambiguation method based on large language models and rules. As a plug-and-play module, it can be integrated with any state-of-the-art parser and incorporate user interaction into the disambiguation process, thereby helping existing parsers mitigate the adverse effects of ambiguity. Specifically, a unified method for structuring the expression of ambiguity (i.e., "ambiguity mapping") is proposed, which formulates expression specifications for the common ambiguities in 7 types of queries; then three modules, namely ambiguity candidate, clarification selection, and query rewriting, are proposed to complete the disambiguation operation. Briefly speaking, 1) The ambiguity candidate module uses a large language model (LLM) combined with few-shot learning and chain-of-thought techniques to guide the model to reason and generate, and then uses the rules we formulated to normalize and constrain the output to obtain the "ambiguity mapping" (i.e., a special mapping structure designed by the present invention for expressing the correspondence between ambiguous descriptions and multiple interpretations); 2) The clarification selection module clarifies the results generated by the candidate through an interactive user interface in the form of multiple-choice questions, so as to obtain the "selection mapping" (an ambiguity mapping in a special scenario where the description and the interpretation are one-to-one) according to the formulated rules; 3) The query rewriting module applies rules based on the selection results to rephrase the original query into a clear query. At this time, when the clear natural language query is input to the parser, a clear SQL statement can be generated. The framework diagram of the NL2SQL disambiguation method based on large language models and rules is specifically as Figure 1 shown, including the following steps:

[0048] S1. Generate a structured ambiguity mapping by combining a large language model with rules, covering syntactic structure and pattern matching ambiguities;

[0049] Structured expression of ambiguity: Assume that the ambiguity occurs in the set of description fragments H={h1,h2,...,h |H|} of the query problem Q, where h=(q j ,q j+1 ,...,q j+n), and the corresponding explanations for each context exist in the set . Ambiguity can be represented by a mapping function such that where represents multiple interpretations related to the ambiguous context h i . This mapping function is called the "ambiguity mapping" and is used to achieve a unified representation of various ambiguities. It is implemented through a special JavaScript Object Notation (JSON) structure. In this structure, the function mapping in mathematics is represented by the key-value pair structure, and the set is represented by the list structure. The ambiguity mapping is defined by the key-value relationship between the ambiguous context H and the corresponding set of explanations . Specifically, given the i-th dimension of the ambiguity mapping can be expressed in JSON syntax as:

[0050]

[0051] The overall ambiguity mapping can be represented as the union of multiple ambiguity dimensions:

[0052]

[0053] According to the specific type of ambiguity, the expression form of the interpretation p i may vary. The expression structure of p i is designed for syntactic structure ambiguity and pattern matching ambiguity, and relevant examples are shown in Table 1.

[0054] Syntactic structure ambiguity mapping: Syntactic structure ambiguity stems from the contradiction between sentence structure and semantics. A sentence can be parsed into different syntactic trees, resulting in different semantics. Therefore, based on the characteristics of each syntactic tree, we extract the key parts from the sentence to obtain p i , and specific templates are developed according to the characteristics of the ambiguity type.

[0055] For scope ambiguity, in the ambiguity mapping, h i here means the quantifiers that limit the scope (i.e., "each", "every", and "all"), and p i is defined as the different scopes modified by this quantifier. A template is designed to represent p i in the presence of wide and narrow scopes. Given an entity E modified by a scope-limiting quantifier, the template for describing the scope ambiguity interpretation is defined as:

[0056] ① "common to all E", indicating the wide scope

[0057] ② “for each E individually” indicates a narrow scope

[0058] For attachment ambiguity, in the ambiguity mapping, h i At this time, the meaning is the attached description, p i is defined as the different objects of attachment. We have designed templates for the cases of complete attachment and partial attachment respectively to represent p in the case where ambiguity is introduced by conjunctions (i.e., “and” and “or”) i . Given entities E1 and E2, the attached description A, and the conjunction “[conj]”, considering whether the entity in the sentence is attached before or after the description, that is, the two cases of “E1 + [conj] + E2 + A” / “A + E1 + [conj] + E2”, the template defining the interpretation of attachment ambiguity is:

[0059] ① “E2 + A + [conj] + E1 + A” / “A + E2 + [conj] + A + E1” indicates complete attachment

[0060] ② “E2 + A + [conj] + E1” / “E2 + [conj] + A + E1” indicates partial attachment

[0061] Pattern matching ambiguity mapping: Pattern matching ambiguity is not only related to the query description but also associated with some table names and column names in the database. In this case, the interpretation p i corresponding to the ambiguity context h i is a combination of database schemas. The pattern subset s = <T, C> is defined as a subset of the table and its respective columns, where is represented in JSON syntax as:

[0062]

[0063] Take the description representing the pattern reference as the ambiguity context h i , and represent the interpretation as p i = s i , where s i = <T i , C i >. Table 1 shows an example of pattern matching ambiguity mapping.

[0064] Table 1: Examples of the expressions of ambiguity mapping under different types of ambiguity

[0065]

[0066]

[0067] Through ambiguity mapping, ambiguity has clear and structured annotations, thus recording various interpretations of ambiguity in multiple dimensions. Additionally, in subsequent prediction tasks, a ^ is added to the elements in the ambiguity mapping to represent the estimated value, and the original symbol represents the true value.

[0068] S1 includes the following steps specifically as Figure 2 shown:

[0069] S11. For syntactic structure ambiguity candidates, input the designed large language model prompt words and the user's natural language question into the large language model, so that the large language model outputs the syntactic structure ambiguity mapping according to the preset instructions and few-shot examples;

[0070] Task description: This task aims to detect and represent ambiguity from the original input, that is, to find a mapping function which can be expressed as

[0071] This module obtains the detected ambiguity mapping where and respectively represent the syntactic structure ambiguity mapping and the pattern matching ambiguity mapping we estimated.

[0072] Syntactic structure ambiguity candidates: Use a large language model (LLM) to detect syntactic structure ambiguity by generating a syntactic structure ambiguity mapping. As Figure 2 shown, a prompt word is designed for the generation of syntactic structure ambiguity candidates, which includes instruction I Q , few-shot learning example set D Q and question Q. Specifically, I Q guides the LLM to perform the tasks of detecting scope ambiguity and attachment ambiguity and provides all possible interpretations. D Q provides three examples of scope ambiguity and attachment ambiguity respectively, which follow the designed corresponding mapping templates. At this time, given the input Q, the LLM can output the syntactic structure ambiguity mapping, formalized as:

[0073]

[0074] Among them, few-shot examples can stimulate the LLM's in-context learning ability and guide it to imitate the output of the examples. Therefore, given the unique syntactic features of syntactic structure ambiguity, few-shot learning can easily capture them and generate output in a specified format. Additionally, to avoid unnecessary LLM calls, we developed a filtering rule, that is, syntactic structure ambiguity is detected only when scope keywords (i.e., "each", "every", and "all") or attachment keywords (i.e., "and" and "or") appear in the question, which is consistent with the detection scope involved in their templates.

[0075] Despite significant efforts to enhance the generation capabilities of LLMs, hallucinations still exist in LLMs, leading to undesired outputs. Therefore, query correction (QC) rules are used to constrain the output. Specifically, assume is an operation that deletes element x from , and Template(p) is a predicate that returns true if the interpretation p conforms to the template of the syntactic structure ambiguity mapping. The QC rule can be described as:

[0076] QC Rule 1.

[0077] Explanation: To avoid nested ambiguity problems, each ambiguity mapping only considers syntactic structure ambiguity in one dimension.

[0078] QC Rule 2.

[0079] Explanation: The generated interpretations must follow the template rules, and their representations cannot exceed the scope defined by the template.

[0080] S12. For pattern-matching ambiguity candidates, through the pattern-matching ambiguity mapping process, reliable pattern-matching ambiguity mappings are obtained. The LLM and rules are used to generate pattern-matching ambiguity candidates for the alignment of unstructured Q and structured S. Specifically, as Figure 3 shown:

[0081] S12 includes the following steps:

[0082] S121. Enhance the column names according to the database to obtain the enhanced document for each column name; use the LLM to generate pattern-matching ambiguity mappings, and assist in the construction of prompt words through column enhancement and retrieval. First, we use the LLM to assign an explanatory document to each column name or directly use the document provided by the database to enrich the semantics of the column name and its position information in the database, thereby obtaining the column enhancement set E.

[0083] S122. Use vector retrieval to retrieve the top-k column names and their enhanced information most relevant to the question, and integrate them with the question into the designed prompt words to enable the large language model to generate preliminary pattern-matching ambiguity mappings; exclude columns irrelevant to Q through vector retrieval and sort the remaining columns according to their relevance to the query question to obtain the subset S r . Then, an instruction I M and a sample demonstration D M, to clearly specify the tasks and output formats of the LLM. Additionally, we introduce Chain-of-Thought (CoT) to enhance the reasoning of the LLM. Finally, given the input Q and S, the pattern matching ambiguity mapping is generated as follows:

[0084]

[0085] where the instruction I M is specifically implemented as follows.

[0086]

[0087] S123. Use the established rules to verify and correct the initially generated mapping, thereby reducing the impact of large model hallucinations; use the structural constraints of the database to reduce the hallucination phenomenon of the LLM, which is achieved through a set of matching mapping correction (MC) rules to optimize For ease of rule expression, we added the definitions of three operations and a predicate on : Let be an operation that adds the interpretation p to the interpretation set ; Let be an operation that modifies the element x to y according to ; Let Ref(Q, h) be a predicate that returns the subsequence in Q that references h. The MC rules can be expressed as follows:

[0088] MC Rule 1.

[0089] Explanation: The table name must actually exist in the database.

[0090] MC Rule 2.

[0091] Explanation: The column name must actually exist in the database.

[0092] MC Rule 3.

[0093] Explanation: The primary key can only be included in the ambiguity mapping if it is explicitly required to be provided in the query Q.

[0094] MC Rule 4.

[0095] Explanation: The ambiguity must correspond to two or more interpretations.

[0096] MC Rule 5.

[0097] Explanation: The context must be a subsequence of Q.

[0098] S124. Determine whether there is ambiguity in the current ambiguous mapping through the ambiguity detection rule, and finally obtain a reliable pattern matching ambiguous mapping. The previous steps effectively detected potential pattern matching ambiguities, but did not consider the ambiguity requirements. Therefore, based on we formulated rules to identify ambiguous ambiguities. As Figure 3 shown, we comprehensively consider from the perspectives of database records and semantics to distinguish whether the pattern subsets in are replacements of each other with the same recorded values or are used to describe different aspects of h (i.e., ambiguous ambiguities). First, make a judgment based on the overall similarity of the database content where and then, when necessary, use the instruction I V to perform semantic detection using the LLM. For the convenience of rule representation, we add that Vagueness(s i , s j ) is a predicate that returns true when there is ambiguity between the pattern subsets s i and s j ; we add that Sim(x, y) is a predicate that returns the similarity between x and y. The ambiguity detection (VD) rules are described as follows:

[0099] VD - Rule 1.

[0100] Explanation: If it is determined that there is ambiguity, then combine the pattern subsets into a new subset and add it to the ambiguous mapping.

[0101] VD - Rule 2.

[0102] Explanation: Used to describe if the number of columns in the two interpretations is different, then it must be a description from different perspectives, that is, ambiguity.

[0103] VD - Rule 3.

[0104] Explanation: Two columns store similar values, which does not meet the definition of ambiguous ambiguity.

[0105] VD - Rule 4.

[0106] Explanation: Realize the judgment of the ambiguity of through the judgment of the semantics by the large language model.

[0107] Among them, the prompt words of the instruction I V are specifically implemented as follows.

[0108]

[0109]

[0110] S2. Through user interaction via the user interface and rule processing, clarify the ambiguity in the form of a multiple-choice question and generate a selection mapping;

[0111] Task description: This task aims to clarify the ambiguity into a unique interpretation based on the user's feedback. Given the ambiguity detection result can be clarified into an unambiguous mapping ψ: H → P such that where each h i has a unique interpretation p i ∈ P. The task is represented as

[0112] where S2 includes the following steps as specifically Figure 4 shown:

[0113] S21. The candidate ambiguity mapping can be regarded as a tree-like hierarchical structure, and thus a multiple-choice question interface for user interaction is constructed according to the meaning of each layer;

[0114] where S21 includes the following steps:

[0115] S211. This module generates a selection mapping based on the detected ambiguity mapping and the user's selection of the multiple-choice clarification question, as specifically Figure 4 shown. The ambiguity mapping can be visualized as a tree, where each leaf node represents an interpretation. Therefore, by combining the interpretations of each ambiguity dimension, a clarification result can be obtained, formalized as a "selection mapping". The selection mapping is a JSON structure consistent with the ambiguity mapping but requires that each ambiguity context h i corresponds to only one interpretation p i , and the selection mapping ψ i of each ambiguity dimension is represented in JSON syntax as:

[0116] ψ i = {h i : p i}

[0117] S212. To complete this task, we designed an interactive user interface (UI) to obtain user feedback. Specifically, we designed multiple-choice questions to clarify ambiguities. The interactive interface includes: the ambiguity question Q, the clarification question (CQ) for the i-th ambiguity dimension, whose template is "What does h i refer to?", and Opt(p i ), which provides options, where Opt(p) is a predicate that returns the clarification option for the interpretation p. For syntactic ambiguity, Opt(p) = p; for pattern matching ambiguity, Opt(p) = Opt(s) = E s , where E s ∈ E represents the column-enhanced document for the pattern s.

[0118] S22. The user can complete the clarification of the ambiguity by ticking the given options, and the system obtains the selection mapping based on the user's feedback results.

[0119] S22 includes the following specific steps:

[0120] S221. When the user uses it, through the checkboxes in front of the options, one option, multiple options, or no option can be selected, thus forming a selection set

[0121] S222. Based on the user's selection, a selection feedback (SF) rule is developed to obtain the selection mapping ψ.

[0122] SF rule 1.

[0123] Explanation: If a single option is selected, the interpretation corresponding to that option is added to the selection mapping.

[0124] SF rule 2.

[0125] Explanation: If multiple options are selected, it indicates that h i requires multiple columns to describe, which means the appearance of ambiguity.

[0126] SF rule 3.

[0127] Explanation: If no option is selected, the selection mapping is an empty set.

[0128] S3. Based on the selection mapping, the natural language query and the database schema are rewritten to output the final unambiguous question and the unambiguous schema. Task description: This task aims to reformulate the original input to make it clear and unambiguous. Given the ambiguous input Q and S, and the clarification result ψ, a clear input can be generated and can be expressed as

[0129] S3 includes the following steps:

[0130] The Rewriting function takes the question, pattern, and selection mapping as inputs. Through question rewriting and pattern rewriting, and by rule processing, it outputs the final unambiguous question and unambiguous pattern;

[0131] For question rewriting, question rewriting QR rules are formulated for ambiguity types, making the syntactic structure of the sentence and the context description of entity reference clear. The purpose of question rewriting is to make the syntactic structure of the sentence and the context description of entity reference clear, thereby enhancing the accuracy of the parser in aligning natural language with patterns. To keep the rewritten question fluent and unambiguous, we have formulated two question rewriting (QR) rules for ambiguity types and shown examples in Table 2.

[0132] For the convenience of rule expression, we define a predicate and two operations for rewriting. Let Seq(s) be a predicate for generating the sequence of s. Specifically, for each table t in s i and column convert it to a string and connect them using the logical connective "and". Given a subsequence of and a new sequence two operations on Q are defined: Insert(Q, h, u) is defined as inserting u after h in Q, such that Replace(Q, h, u) is defined as replacing h with u, such that The QR rules can be expressed as follows:

[0133] QR rule 1.

[0134] QR rule 2.

[0135] Table 2 Examples of question rewriting

[0136]

[0137]

[0138] For pattern rewriting, by formulating pattern rewriting SR rules, the interference of ambiguous columns in pattern matching ambiguity is eliminated.

[0139] The purpose of pattern rewriting is to eliminate the interference of ambiguous columns in pattern matching ambiguity, which is achieved through the following pattern rewriting (SR) rules:

[0140] SR rules.

[0141] Finally, this module takes Q, S, and ψ as inputs and generates clear questions for the parser and clear patterns The disambiguation task is completed.

[0142] In a specific embodiment, an example of the NL2SQL task is shown below as Figure 5 follows. The NL2SQL task takes a natural language query and the schema of a database (i.e., table names and column names) as inputs and outputs an equivalent SQL query. Finally, the query result can be obtained by executing it through an RDB engine.

[0143] A case study of using the prediction results generated by combining the present invention with a parser is shown as Figure 6 follows: When the parser aligns natural language with the schema, since the parser usually selects the column name that is most semantically relevant as the optimal choice, schools.School and schools.County are used in SQL construction. However, through the disambiguation of the present invention, it is found that the actually required column names are satscores.sname and satscores.cname.

[0144] Finally, it should be noted that although the present invention has been described with reference to the current specific embodiments, those of ordinary skill in the art in this technical field should recognize that the above embodiments are only used to illustrate the present invention and are not used as limitations on the present invention. Various equivalent changes or substitutions can be made without departing from the inventive concept of the present invention. Therefore, as long as the changes and variations of the above embodiments are within the scope of the spirit of the present invention, they will fall within the scope of the claims of the present invention.

Claims

1. An NL2SQL ambiguity elimination method based on large language models and rules, characterized in that, It includes the following steps: S1. Generate a structured ambiguity mapping by combining the large language model LLM with rules, covering syntactic structure and pattern matching ambiguities; S2. Through the user interface, clarify ambiguities in the form of multiple-choice questions through user interaction and rule processing to generate a selection mapping; S3. Rewrite the natural language query and database schema based on the selection mapping, and output the final unambiguous question and unambiguous schema.

2. The NL2SQL ambiguity elimination method based on large language models and rules according to claim 1, characterized in that The S1 includes the following steps: S11. For syntactic structure ambiguity candidates, input the designed LLM prompts and the user's natural language question into the large language model LLM, so that the large language model LLM outputs a syntactic structure ambiguity mapping according to the preset instructions and few-shot examples; S12. For pattern matching ambiguity candidates, obtain a reliable pattern matching ambiguity mapping through the pattern matching ambiguity mapping process.

3. The NL2SQL ambiguity elimination method based on large language models and rules according to claim 2, wherein, The S12 includes the following steps: S121. Enhance the column names according to the database to obtain an enhanced document for each column name; S122. Use vector retrieval to retrieve the top-k column names and their enhanced information most relevant to the question, and integrate them into the designed prompts together with the question, so that the large language model generates a preliminary pattern matching ambiguity mapping; S123. Use the formulated rules to verify and correct the preliminarily generated mapping, so as to reduce the impact of large model hallucinations; S124. Determine whether there is ambiguity in the current ambiguity mapping through the ambiguity detection rule, and finally obtain a reliable pattern matching ambiguity mapping.

4. The NL2SQL ambiguity elimination method based on large language models and rules according to claim 3, characterized in that, The S2 includes the following steps: S21. The candidate ambiguity mapping can be regarded as a tree-like hierarchical structure, and thus a multiple-choice question interface for user interaction is constructed according to the meaning of each layer; S22. The user can complete the clarification of the ambiguity by ticking the given options, and the system obtains the selection mapping according to the user's feedback results.

5. A method for eliminating NL2SQL ambiguity based on large language models and rules according to claim 4, characterized in that, The S21 includes the following steps: S211. Ambiguous mapping is visualized as a tree, where each leaf node represents an interpretation. By combining the interpretations of each ambiguous dimension, a clarification result can be obtained, formalized as a "selection mapping". The selection mapping is a JSON structure that is consistent with the ambiguous mapping but requires that each ambiguous context h i corresponds to only one interpretation p i , and the selection mapping ψ of each ambiguous dimension i is expressed in JSON syntax as: ψ i = {h i : p i}; S212. Design an interactive user interface UI to obtain user feedback, and design multiple-choice questions to clarify ambiguities.

6. A method for eliminating NL2SQL ambiguity based on large language models and rules according to claim 5, characterized in that, The S22 includes the following steps: S221. When the user uses the interactive user interface UI, one option, multiple options, or no option can be selected through the check box before the option, thereby forming a selection set S222. Based on the user's selection, develop a selection feedback SF rule to obtain the selection mapping ψ.

7. A method for eliminating NL2SQL ambiguity based on large language models and rules according to claim 6, characterized in that The S3 includes the following steps: S31. The Rewriting function takes the question, schema, and selection mapping as inputs, and through question rewriting and schema rewriting, and through rule processing, outputs the final unambiguous question and unambiguous schema; S32. For question rewriting, formulate a question rewriting QR rule for the ambiguity type to make the syntactic structure of the sentence and the context description of entity reference clear. S33. For schema rewriting, eliminate the interference of ambiguous columns in the pattern matching ambiguity by formulating a schema rewriting SR rule.

Images