Enterprise number asking system and method based on combination of large language model and NL2SQL

Abstract

The invention relates to the technical field of natural language processing, in particular to an enterprise number asking system and method based on combination of a large language model and an NL2SQL, and the method comprises the following steps: filling a prompt project template with a natural language query request and a structured metadata context, and generating a standard prompt; the standard prompt is input into the large language model, deep semantic analysis is carried out, and an SQL query draft is generated; self-evaluation is carried out on the questions, and when the questions are recognized, clarified questions are generated and returned to the user; receiving feedback of a user, and iteratively optimizing the SQL query draft based on the feedback; performing grammar verification and security verification on the final SQL query draft to generate an executable SQL statement; sending the executable SQL statement to a target enterprise database for execution; and the information is visually displayed to a user. The problem that the SQL generated in the prior art deviates from the real intention of a user and cannot meet the requirements of enterprise-level applications for accuracy and reliability can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to an enterprise question system and method based on a large language model combined with NL2SQL. BACKGROUND

[0002] With the deepening of enterprise digital transformation, data-driven decision-making has become the core of enterprise operation. Massive data is stored in various database systems. How to enable non-technical business personnel to efficiently and accurately obtain and utilize these data has become the key to improving the competitiveness of enterprises. Natural language to query language (NL2SQL) technology has emerged as the times require, aiming to reduce the technical threshold of data query through a natural language interface. In recent years, breakthroughs in large language models (LLM) have been made, and due to their strong semantic understanding and generation capabilities, they have been widely introduced into the NL2SQL task. By understanding user intent and generating corresponding SQL statements, the intelligentization and popularization of enterprise data query have been greatly promoted.

[0003] Currently, NL2SQL systems based on large language models generally follow the following process: receiving a user's natural language query; using a pre-trained LLM model to understand the query intent and generate a preliminary SQL statement; after syntax and security checks, executing the query and returning the results. Some advanced systems also introduce an interactive mechanism. When the initial query has ambiguities or ambiguities, the system will generate clarification questions to guide the user to supplement information, thereby optimizing the SQL generation effect.

[0004] However, the existing interactive clarification method has a deep and rare technical defect. Existing technologies usually treat each round of user interaction as an independent, discrete event, and the system lacks the ability to continuously track and maintain consistency of the overall context of multi-round dialogue. This leads to the phenomena of "cognitive decay" and "ambiguity proliferation" in complex multi-round clarification dialogue: the system forgets or ignores semantic information agreed upon in previous interactions in subsequent interactions, and the clarification questions proposed are isolated from each other and even contradictory; at the same time, solving a high-level ambiguity often triggers a series of new, more detailed ambiguities, forming a chain reaction. This problem makes the clarification loop unable to converge effectively, not only significantly reducing query efficiency, but also seriously consuming user patience and possibly leading to a fundamental deviation of the final query result from the user's true intent, which cannot meet the stringent requirements of enterprise-level applications for accuracy and reliability. SUMMARY

[0005] One of the purposes of the present application is to provide an enterprise query method based on a large language model combined with NL2SQL, which solves the problem that the existing interactive NL2SQL system based on a large language model (LLM) lacks the ability to continuously track and maintain consistency of the overall dialogue context in multiple rounds of clarification dialogue, resulting in the phenomena of "cognitive decay" and "ambiguity proliferation", making the clarification process inefficient and difficult to converge, and ultimately leading to the generated SQL deviating from the user's real intention, and failing to meet the accuracy and reliability requirements of enterprise-level applications.

[0006] In order to achieve the above purpose, an enterprise query method based on a large language model combined with NL2SQL is provided, comprising the following steps:

[0007] In response to a natural language query request input by a user, the natural language query request is received and stored;

[0008] From the schema repository of the enterprise database, the database table name, field name, field annotation and inter-table association related to the natural language query request are extracted, and a structured metadata context is constructed;

[0009] The natural language query request and the structured metadata context are filled into a predefined prompt engineering template to generate a standard prompt for a large language model, which contains task instructions, context constraints and examples;

[0010] The standard prompt is input into a trained large language model, and the large language model performs deep semantic analysis on the natural language query, comprehensively understands the query intent, filtering conditions, aggregation logic and association, and generates a preliminary SQL query draft;

[0011] The large language model performs self-evaluation on the SQL query draft it generates, and when it identifies that there is ambiguity, logical conflict or insufficient information in the natural language query request, one or more structured clarification questions are generated and returned to the user;

[0012] The user's selection or supplementary feedback on the clarification question is received, and the SQL query draft is iteratively optimized based on the feedback;

[0013] The final determined SQL query draft is subjected to syntax verification, and a preset security check module is used to check whether it contains potential dangerous operations; if the check passes, a final executable SQL statement is generated;

[0014] The executable SQL statement is sent to the target enterprise database for execution, and the returned data result is received; the data result is presented to the user in the form of a preset visual chart or structured table.

[0015] The second object of the present application is to provide an enterprise question answering system based on a large language model combined with NL2SQL, comprising the following steps:

[0016] A request receiving module is configured to receive and store a natural language query request input by a user;

[0017] A data extraction module is configured to extract database table names, field names, field annotations and inter-table association relationships potentially related to the natural language query request from a schema repository of an enterprise database, and construct a structured metadata context;

[0018] A standard prompt generation module is configured to fill the natural language query request and the structured metadata context into a pre-defined prompt engineering template, and generate a standard prompt for a large language model, which contains task instructions, context constraints and examples;

[0019] A primary query draft generation module is configured to input the standard prompt into a trained large language model, and perform deep semantic analysis on the natural language query by the large language model to comprehensively understand the query intent, filtering conditions, aggregation logic and association relationships, and generate a preliminary SQL query draft;

[0020] A self-evaluation and iterative optimization module is configured to enable the large language model to perform self-evaluation on the SQL query draft generated by it, and generate one or more structured clarification questions and return them to the user when ambiguities, logical conflicts or insufficient information are identified in the natural language query request;

[0021] The user's selection of the clarification questions or supplementary feedback is received, and the SQL query draft is iteratively optimized based on the feedback;

[0022] A syntax verification and security check module is configured to perform syntax verification on the finally determined SQL query draft, and check whether it contains potential dangerous operations through a pre-set security check module; if the check is passed, a final executable SQL statement is generated;

[0023] A SQL statement execution and visualization module is configured to send the executable SQL statement to a target enterprise database for execution, and receive the returned data results; the data results are presented to the user in the form of a pre-set visual chart or structured table.

[0024] Principles and advantages:

[0025] 1. Effectively eliminates cognitive decay in multi-round clarification dialogues, ensuring the coherence and consistency of system questioning, avoiding repeated questioning or logical conflicts caused by forgetting prior consensus, and significantly improving user experience and trust. By introducing and dynamically maintaining a session-specific clarification state tree, this data structure continuously and structurally records all semantic consensus reached from the original query to each round of user feedback in a tree-like form, providing a complete and consistent dialogue context for each new clarification question generation.

[0026] 2. It can effectively curb the chain reaction and infinite proliferation of ambiguity, decompose complex and deep semantic ambiguity into controllable and manageable sequential clarification steps, and guide the dialogue to quickly converge to the user's clear intent.

[0027] By using a clarification state tree to structurally model and manage ambiguities, high-level ambiguities are decomposed into specific nodes to be clarified. Through a context-aware clarification mechanism based on this state tree, these nodes are resolved in an orderly and sequential manner, ensuring that the introduction of each new problem is based on the resolution of all previous problems, thereby systematically narrowing the scope of ambiguity.

[0028] 3. This solution can intelligently determine whether the clarification loop is complete or if an anomaly has occurred, thereby terminating the interaction process promptly and automatically. This avoids unnecessary lengthy dialogues and seeks final confirmation when intent drifts, ensuring query efficiency and result accuracy. It employs an independent intent... Figure One The consistency verification module, based on rules or a lightweight model, analyzes the completeness (whether all core nodes have been confirmed), consistency (whether there are logical conflicts between nodes), and user feedback history of the clarification state tree, and makes decisions on whether to continue clarification, generate the final SQL, or trigger the exception confirmation process. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating an enterprise query method based on a large language model combined with NL2SQL, according to an embodiment of the present invention.

[0030] Figure 2 A flowchart for interactive clarification and SQL iterative optimization;

[0031] Figure 3 This is a schematic diagram of the state tree structure for clarification.

[0032] Figure 4 Flowchart for SQL syntax and security verification;

[0033] Figure 5 Visualize the decision-making process for the results. Detailed Implementation

[0034] The specific embodiments are further described in detail below:

[0035] Embodiments

[0036] An enterprise question answering method based on a large language model combined with NL2SQL, substantially as shown in the accompanying drawings, Figure 1 comprises the following steps:

[0037] S1, in response to a natural language query request input by a user, receiving and storing the natural language query request;

[0038] S2, extracting from the schema repository of the enterprise database the database table name, field name, field annotation and inter-table association relationship potentially related to the natural language query request, and constructing a structured metadata context;

[0039] S3, filling the natural language query request and the structured metadata context into a pre-defined prompt engineering template to generate a standard prompt for a large language model, which contains task instructions, context constraints and examples;

[0040] S4, inputting the standard prompt into a trained large language model, and comprehensively understanding the query intent, filtering conditions, aggregation logic and association relationship by the large language model through deep semantic analysis of the natural language query to generate a preliminary SQL query draft;

[0041] The training of the large language model specifically comprises the following steps:

[0042] A dataset composed of triples is taken as input, where represents the i-th natural language query, which is a query request proposed by a user, represents the database metadata context corresponding to the query, which contains table structure, field information and its constraints, represents the standard SQL statement matched with the natural language request. The training output of the model is a parameter set Θ, which is obtained through iterative optimization, so that the model can realize the correct mapping from the natural language input and the metadata context to the SQL statement. The training process adopts a supervised fine-tuning paradigm, and the target is to minimize the negative log-likelihood loss function The loss function is defined as

[0043]

[0044] where L represents the length of the target SQL sequence , represents the t-th token in the SQL sequence, ​represents the sequence of all tokens before the t-th token is generated. This formula characterizes the conditional probability of the model generating the SQL target sequence at each position given the natural language query and metadata context conditions. The goal of training is to maximize the conditional probability of the entire target SQL sequence, ensuring that the model can accurately generate a complete SQL statement token by token. The training optimization process is implemented through a gradient descent algorithm. Each iteration calculates the gradient of the loss function with respect to the parameters Θ, and adjusts the parameters through an update rule to reduce the loss value, so that the model converges to a better solution after multiple iterations. Training data D train derived from the enterprise's historical query logs, through data cleaning and manual annotation to establish an aligned mapping of natural language queries and SQL statements, and combined with metadata context to construct complete triples. In addition, to improve the robustness of the model in the face of uncertain semantics, a data augmentation strategy is introduced during data construction, including ambiguity rewriting of the original query, so that the model learns how to handle field ambiguity, table ambiguity, value ambiguity and connection ambiguity during training, ultimately ensuring that it can work with the clarification state tree T state and related decision-making links to generate SQL statements that meet the user's real intent.

[0045] S5, the large language model performs self-evaluation on the SQL query draft it generates, and when it identifies ambiguities, logical conflicts or insufficient information in the natural language query request, generates one or more structured clarification questions and returns them to the user;

[0046] receives the user's selection or supplementary feedback on the clarification question, and iteratively optimizes the SQL query draft based on this feedback;

[0047] As Figure 2 shown, this step includes interactive clarification of clarification questions and user intent confirmation, specifically including the following steps:

[0048] S51, first input the preliminary SQL query draft and the structured metadata context C meta to initialize the clarification state tree T state = (N, E), where N is a set of nodes for storing clarification units and confirmation information, as shown in Figure 3 ; E is a set of edges that define the parent-child relationship between nodes and record the two inputs as core attributes in the root node; thus establishing a unified context environment for subsequent ambiguity detection and user interaction.

[0049] S52, parse through a large language model, layer by layer analyzing its semantic structure and logical expression, and combining C metatable names, field names, field annotations and inter-table association relations in C k}; each element Na i in the ambiguity point set A represents a semantic unit with uncertainty or ambiguity in the query draft, including the aggregation manner of fields, the value range of filtering conditions or the selection of table association paths;

[0050] For each ambiguity point a i , a new node n state is created in the clarification state tree T i , and connected to the parent node through an edge set E, while the type and candidate value information of the ambiguity point are attached to the node, forming a dynamically expandable clarification tree structure;

[0051] The generation process of the ambiguity point set A includes the following steps:

[0052] S521, taking the preliminary SQL query draft Q and the structured metadata context C meta as inputs, first input them into the ambiguity detection function f detect , and the function outputs a tuple set, whose calculation formula is In this set, the element structure is a triple A:

[0053] A={(type1,context1,candidates1),...,(type k ,context k ,candidates k )}

[0054] Wherein, each triple corresponds to an semantic ambiguity point in the query draft; the parameter type1 represents the type of the i-th ambiguity point, whose value comes from a predefined type set, and the basic categories of the set include field ambiguity, table ambiguity, value ambiguity and connection ambiguity, which correspond to the syntax units most likely to have ambiguous interpretations in the query. The parameter context1 represents the context information of the ambiguity point in C , which is specifically an SQL fragment or a related semantic position description, aiming to help the model and the user to determine the logical position and context of the ambiguity point; the parameter candidates1 represents the candidate solution set of the ambiguity point, whose content comes from the pattern information retrieval in C meta or the result of directly accessing the database content, which is used to give a variety of possible alternatives, thereby providing a selection space for subsequent clarification interaction. The execution principle of the function f detect is to analyze Q Abstract Syntax Tree parsing is performed to decompose and analyze the structure of the query statement at the syntax level, and whether the syntax unit is ambiguous is determined by layer-by-layer matching with the pattern information in C meta When it is detected that a certain syntax unit corresponds to multiple possible semantic interpretations, it is classified as a new ambiguity point, and a corresponding triple A is constructed and finally added to the ambiguity point set A.

[0055] S53, after the completion of the clarification tree structure construction, according to the clarification tree T state The current form and the metadata information of C meta Generate a specific clarification question set P clarify Present the clarification question in a structured form to the user to narrow the ambiguity space and make the query intent more clear. The generation of the clarification question specifically includes the following steps:

[0056] S531, with the current clarification state tree T state And the ambiguity point set A as input, through the question generation function f generate Convert it into a natural language question string, and its calculation formula is:

[0057] P clarify =f generate (T state ,A)

[0058] In this process, the question generation function first prioritizes the ambiguity points in the set A, and selects the ambiguity point a j With the highest priority; The determination of the priority is jointly determined by the type of the ambiguity point type j And the importance of the ambiguity point in the query logic, where type j Indicates the type of the jth ambiguity point, which comes from the predefined categories of field ambiguity, table ambiguity, value ambiguity or connection ambiguity; The importance of the query logic is related to whether the ambiguity point directly affects the filtering condition, the aggregation logic or the table association path. After selecting the ambiguity point a j , the function selects the corresponding template According to the type type j Of the ambiguity point Each template Contains a structured natural language question framework, and reserves slots for filling in context and candidate value information. Then the function injects the context parameter context j And the candidate solution set candidates j Into the template, context j Indicates the context fragment of the ambiguity point in the preliminary SQL query draft , which is used to prompt the user that the question involves the specific location of the query, and candidatesj represents the candidate solution list obtained by structured metadata context C meta or database content retrieval, for providing the user with selectable answers;

[0059] The final generated P clarify will be presented through the user interface and as an input request in the interactive clarification process, the user's feedback result R user will be directly updated to the related node of the clarification state tree T state .

[0060] S54, when the user returns the feedback R user , the feedback information is written to the corresponding node n i , and the state of the node is updated to clarified, thereby gradually reducing the number of unresolved nodes in the tree. According to the continuous input of user feedback, the clarification state tree T state is dynamically and iteratively updated, and gradually converges to a clear query intent at the semantic level; based on the updated clarification state tree, a new SQL query draft is generated which continues the logical framework of the initial draft in structure, and integrates the user-confirmed constraint conditions and disambiguation information in semantics, thereby forming a SQL query version that better meets the user's real intent, and entering the subsequent iteration optimization and verification link. The integration of the user's feedback result R user and the update of the clarification state tree includes the following steps:

[0061] S541, taking the feedback information I input provided by the user and the current ambiguity point a j to be clarified as input, generating an updated clarification state tree The mathematical expression is

[0062]

[0063] Where I input represents the feedback information submitted by the user in the interactive interface, which can be an item in the candidate set or free text input. In this process, the state update function f update first parses the input feedback information I input When I input belongs to an element in the candidate list candidates j , mark the element as the confirmed value v confirmed , indicating that the user has made a selection in the predefined options; when I input is free text input by the user, the text is directly taken as a new confirmed value v confirmedto support the user to provide more accurate intent description outside the system candidates. Subsequently, the function locates the ambiguity point a state in the clarification state tree T j corresponding node n j , writes the confirmation value v confirmed into the attribute field of the node, and updates the state of the node from "to be clarified" to "confirmed", making the node complete the transition from semantic ambiguity to semantic clarity. The user feedback result is the general term of the user's selection of the system-generated clarification question or supplementary feedback, and the user feedback information is the specific content submitted by the user in the interactive interface, which constitutes the user feedback result, that is, the user feedback information is the specific carrier form of the user feedback result.

[0064] At the same time, the function also records the original feedback information I input and the operation timestamp in the node to ensure that each semantic update in the entire interaction process has complete historical records.

[0065] S55, optimizing the SQL query draft to obtain a new SQL query draft The steps are as follows:

[0066] With the updated clarification state tree and the original natural language query request Q nl as input, the SQL regeneration function f regenerate outputs the optimized SQL query draft, which is mathematically

[0067]

[0068] In this process, the function first traverses to extract all the node information of the confirmed state from the clarification state tree T

[0069] C constraints ={constraint1,...,constraint m}

[0070] The element constraint1 in the set C is a key-value pair structure, where the key represents a semantic unit in the query draft, and the value represents the explicit semantic mapping confirmed by the user, thereby ensuring that the original ambiguity point is uniquely interpreted. Subsequently, the function modifies the original prompt engineering template and injects the constraint set C constraints into the required field to obtain the enhanced prompt Prompt enhancedThis prompt adds semantic constraints of user confirmation to the task instructions and metadata context, thereby making the generation process more deterministic. Next, the function will... enhanced The input is fed into a large language model, which then parses the original natural language query request Q. nl The constraint set C must be strictly followed. constraints During the semantic understanding and SQL generation process, redundant ambiguities or logical deviations are avoided, thereby outputting an optimized SQL query draft.

[0071] S56. In the step of iteratively optimizing the SQL query draft based on this feedback, set the intention... Figure One The consistency verification and iterative loop termination steps specifically include the following steps:

[0072] The input is the clarification state tree T. state User feedback history H user The output is a termination decision signal D. terminate and a final SQL statement Or an abnormal signal S exception Its mathematical form is defined as

[0073]

[0074] In this process, the decision function f decide First, count the number of confirmed nodes in the clarification state tree to the total number of nodes, and then calculate the confirmation ratio r. confirmed :

[0075]

[0076] Where, |N confirmed | represents the number of confirmed nodes, |N total | represents the total number of nodes. This ratio reflects the completeness of the user's intent clarification. The function then checks T. state The system checks for logical conflicts, specifically whether the confirmation values ​​of two confirmed nodes are mutually exclusive in business logic. If such a contradiction is detected, it indicates that the query semantics are no longer consistent. Next, the system analyzes the user feedback history (H). user This includes the user's input sequence and interaction signals during the clarification process, which is used to determine whether the user is showing a clear intention to terminate or strong frustration. Based on the above analysis, if the calculation result satisfies r... confirmed If the value is 1 and no logical conflict is found, the system assumes that the user's intent is completely determined, and then sets D. terminate =True, and call the SQL regeneration function f regenerate Generate the final SQL statement Ensure the output is consistent with the user's confirmation semantics. If a logical conflict is detected or strong negative emotions are identified in H user , it is considered that continuing interaction is no longer feasible, at which point D terminate =True is set and an exception signal S exception is output, prompting the need for human intervention to restore correctness.

[0077] If the proportion of confirmations has not yet reached complete coverage and there are no conflicts or strong negative signals, D terminate =False is set, clarifying that the loop continues to run and enters the next round of interaction.

[0078] S6, perform syntax verification on the final determined SQL query draft, and check whether it contains potential dangerous operations through a pre-set security verification module; if the check passes, generate the final executable SQL statement;

[0079] As shown in Figure 4 , the syntax verification of the SQL and the security verification of the security verification module specifically include the following steps:

[0080] Input the SQL statement to be verified , where represents the candidate query statement generated in the previous step and needs to be executed. The processing process of this step is implemented by defining a verification function f check , and the function expression is

[0081]

[0082] where R check represents the Boolean value of the verification result, and when the value is True, it indicates that the verification is successful, and when it is False, it indicates that the verification fails; represents the executable SQL statement after verification; E error represents the error information generated when the verification fails, including syntax error information and security error information The first processing stage of the verification function is syntax analysis, which uses a standard SQL parser to perform syntax rule checking on the input . If the parsing fails, R check =False is returned, and the corresponding syntax error is output. At this time, no executable statement is generated.

[0083] If the syntax analysis is successful, it enters the security verification stage, which is checked by a pre-defined dangerous operation mode blacklist B danger ={pattern1, pattern2, …, pattern v}, where B dangerrepresents the set of all patterns of statements that are determined to be potentially attack or violation, pattern j represents the jth dangerous pattern, usually composed of a specific sequence of operators or a combination of keywords, such as operations involving database deletion, system table modification, or privilege escalation.

[0084] If the input matches any one pattern pattern danger in the set B j , it is determined that the security check fails, and the output result is R check = False, and the security error information is returned. Similarly, no executable statement is generated.

[0085] Only when both the syntax analysis and the security verification pass, the value of the check result R check is True, and the original input is output as the final executable statement without any error information. In this way, the entire processing process realizes the mapping from the input candidate SQL statement to the output check result and executable statement or error information, ensuring that the SQL statement in the execution phase meets both the syntax correctness and the security constraints.

[0086] S7, sends the executable SQL statement to the target enterprise database for execution, and receives the returned data result; and presents the data result to the user in the form of a preset visual chart or structured table.

[0087] As Figure 5 shown, the specific steps of presenting the data result to the user in the form of a preset visual chart or structured table include:

[0088] Taking the original data result R data returned by the database and the original natural language query Q nl as inputs, a visual mapping function f visualize is used to generate an output result.

[0089] (V chart |T table )=f visualize (R data ,Q nl )

[0090] wherein R data represents the structured result set returned by executing the SQL query in the database via the NL2SQL model, which includes data fields and corresponding values; V chart represents the generated visual chart result, which is used to visually present the query information; and T tableThis indicates structured table output, used to ensure readability of results when charts cannot be generated.

[0091] f visualize It is an input-to-output mapping function, whose internal logic includes three stages: data structure parsing, semantic intent recognition, and chart type selection. This function first parses R... data The structural properties, including the number of dimensions D. num Measure quantity M num and data type collections Dimension count refers to the number of non-numeric fields in the returned data, metric count refers to the number of numeric fields, and data type set records whether each field is numeric, time-based, or categorical.

[0092] Then the function is parsed Q. nl Keyword set The keyword set is used to infer the user's query intent. When the data includes "trend," the intentional tendency variable φ is labeled as a time series analysis; when it includes "comparison," φ is labeled as a multidimensional comparison analysis; and when it includes "proportion," φ is labeled as a proportion decomposition analysis. This is based on data structure characteristics. The result of combining with semantic intent φ is that the function is derived from a predefined set of visualization rules. The optimal chart type, ChartType, is selected from a set of rules. This set includes bar charts, line charts, pie charts, scatter plots, and heatmaps, each defined in the rule set with matching conditions for dimensions and measures. After determining ChartType, the function will then apply the necessary parameters to R. data Convert to structured data that conforms to the chart input format. Then call the visualization library to generate the final chart V. chart When the rule match fails or When no chart input requirements are met, the function outputs a table format T. table To ensure the complete presentation of the results.

[0093] The implementation case includes the following steps:

[0094] Step 1: The medical insurance reviewer enters a natural language query request on the front-end interface: "Query the outpatient expenses details of orthopedics for insured person Li Moumou in the past six months".

[0095] Step 2: After receiving the request, the system automatically extracts data tables and fields related to concepts such as "insured person", "outpatient expenses" and "orthopedics" from the medical insurance database metadata repository, constructs a metadata context, identifies the core tables including "insured person information table", "outpatient expense details table" and "medical item classification table", and clarifies the relationship between each table.

[0096] Step 3, the system fills the user query request and the constructed metadata context into a predefined prompt template containing instructions and examples, generating a structured standard prompt.

[0097] Step 4, the system inputs the standard prompt into a specially trained large language model. The large language model deeply analyzes the query intent and identifies the core elements as "insured person's name = Li Someone", "time range = past half a year", "expense type = outpatient", and "department category = orthopedics". It then generates a preliminary SQL query draft. Since the "orthopedics category" may have different codes and names in different hospital information systems, the `WHERE` condition in the draft contains a fuzzy matching-based filter.

[0098] Step 5, the large language model self-evaluates the generated SQL draft and determines that the "orthopedics category" condition is ambiguous and lacks information. The system initializes a clarification state tree and records the current ambiguity point as a node to be clarified in the tree. Then, the system returns a structured clarification question to the user: "Please select the specific definition of 'orthopedics category' from the following options: 1. Diagnosis code (ICD-10) starting with M and S and related codes; 2. Charge item name containing keywords such as 'orthopedics' or 'fracture'; 3. Execution department is 'orthopedics' or 'orthopedic surgery'".

[0099] The medical insurance reviewer selects the most accurate option based on their review purpose, for example, selecting "1. Diagnosis code (ICD-10) starting with M and S and related codes".

[0100] The system receives user feedback and updates the status and value of the corresponding node in the clarification state tree based on the user's selected option as the confirmed consensus.

[0101] Based on the updated clarification state tree containing confirmed information, the system regenerates the prompt and requests the large language model to optimize the SQL draft. This time, the model generates an accurate SQL query statement that uses the diagnosis code library for association and filtering.

[0102] Step 6, the system calls an independent verification module to evaluate the final determined SQL statement. The module confirms that all ambiguity points in the clarification state tree have been resolved, and the SQL logic is complete and consistent, with no signs of circular questioning or user intent drift, determining that the clarification is complete and permitting execution.

[0103] Step 7, the system performs strict syntax verification on the SQL statement permitted to execute. The security verification module confirms that the statement is only a read-only "SELECT" operation and does not contain any full table scanning or dangerous instructions, and the verification is passed.

[0104] Step 8, the system sends the final executable SQL statement to the medical insurance core production database for execution.

[0105] Step 9, after the database returns the query results, the system presents the outpatient expense details of the insured person Li in a structured table form to the medical insurance auditors, the table contains time, hospital, diagnosis code, item name, amount and other key fields, completing this query.

[0106] The present scheme effectively solves the problem that the existing interactive NL2SQL system based on a large language model (LLM) lacks the ability to continuously track and maintain consistency of the overall dialogue context in multiple rounds of clarification dialogue, resulting in "cognitive decay" and "ambiguity proliferation" phenomena, making the clarification process inefficient and difficult to converge, and ultimately leading to the generated SQL deviating from the user's true intention, failing to meet the accuracy and reliability requirements of enterprise-level applications.

[0107] An enterprise question answering system based on a large language model combined with NL2SQL, comprising a server and a user end, the server comprising the following modules:

[0108] Request receiving module: for responding to the user's input natural language query request, receiving and storing the natural language query request;

[0109] Data extraction module: for extracting database table names, field names, field annotations and inter-table association relationships potentially related to the natural language query request from the schema repository of the enterprise database, and constructing a structured metadata context;

[0110] Standard prompt generation module: for filling the natural language query request and the structured metadata context into a pre-defined prompt engineering template to generate a standard prompt for a large language model, which contains task instructions, context constraints and examples;

[0111] Primary query draft generation module: for inputting the standard prompt into a trained large language model to perform deep semantic analysis of the natural language query, comprehensively understand the query intent, filtering conditions, aggregation logic and association relationships, and generate a preliminary SQL query draft;

[0112] The training of the large language model specifically includes the following steps:

[0113] A dataset composed of triples as input, where represents the i-th natural language query, which is the query request proposed by the user, represents the database metadata context corresponding to the query, including table structure, field information and its constraints, This represents the standard SQL statement that matches the natural language request. The model's training output is a set of parameters Θ, which are obtained through iterative optimization, enabling the model to achieve the correct mapping from natural language input and metadata context to the SQL statement. The training process employs a supervised fine-tuning paradigm, aiming to minimize the negative log-likelihood loss function. The loss function is defined as

[0114]

[0115] Where L represents the target SQL sequence Length, This represents the t-th token in the SQL sequence. This represents the sequence of all tokens up to the generation of the t-th token. This formula characterizes the conditional probability of the model generating the SQL target sequence at each position, given the natural language query and metadata context. The training objective is to maximize the conditional probability of the entire target SQL sequence, thus ensuring that the model can accurately generate complete SQL statements token by token. The training and optimization process is implemented using the gradient descent algorithm. In each iteration, the gradient of the loss function with respect to the parameter Θ is calculated, and the parameter is adjusted through update rules to reduce the loss value, allowing the model to converge to a better solution after multiple iterations. Training data D train Derived from historical query logs of enterprises, data cleaning and manual annotation are used to establish an aligned mapping between natural language queries and SQL statements, and complete triples are constructed by combining metadata context. Furthermore, to improve the model's robustness to uncertain semantics, data augmentation strategies are introduced during the data construction phase, including ambiguous rewriting of the original query. This allows the model to learn how to handle field ambiguity, table ambiguity, value ambiguity, and join ambiguity during training, ultimately ensuring its ability to align with the clarified state tree T in actual inference. state It works closely with relevant decision-making processes to generate SQL statements that match the user's true intent.

[0116] The self-evaluation and iterative optimization module enables the large language model to self-evaluate the SQL query drafts it generates. When it identifies ambiguity, logical conflict, or insufficient information in the natural language query request, it generates one or more structured clarification questions and returns them to the user.

[0117] Receive user selections or supplementary feedback on the clarification questions, and iteratively optimize the SQL query draft based on this feedback;

[0118] This module includes interactive clarification of questions and confirmation of user intent, specifically including the following steps:

[0119] First, draft the initial SQL query. The structured metadata context C meta As input, initialize the clarification state tree T state =(N, E), where N is a set of nodes for storing clarification units and confirmation information, and E is a set of edges for defining the parent-child relationship between nodes and recording the two inputs as core attributes in the root node; thus, a unified context environment is established for subsequent ambiguity detection and user interaction.

[0120] Through large language models, the semantics and logical expressions of are analyzed layer by layer, and combined with the table names, field names, field annotations, and inter-table association relationships in C meta , all potential ambiguity point sets A = {a1, a2,..., a k} are identified; each element a i in the ambiguity point set A represents a semantic unit with uncertainty or ambiguity in the query draft, including the aggregation method of fields, the value range of filtering conditions, or the selection of table association paths;

[0121] For each ambiguity point a i , a new node n state is created in the clarification state tree T i , and connected to the parent node through the edge set E, while the type and candidate value information of the ambiguity point are attached to the node, forming a dynamically expandable clarification tree structure;

[0122] The generation process of the ambiguity point set A includes the following steps:

[0123] Taking the preliminary SQL query draft and the structured metadata context C meta as input, first input into the ambiguity detection function f detect , and the function output is a tuple set, whose calculation formula is In this set, the element structure is a triple A:

[0124] A = {(type1, context1, candidates1),..., (type k , context k , candidates k )}

[0125] Wherein, each triple corresponds to a semantic ambiguity point in the query draft; the parameter type1 represents the type of the i-th ambiguity point, whose value comes from the pre-defined type set, and the basic categories of the set include field ambiguity, table ambiguity, value ambiguity, and connection ambiguity, which correspond to the most ambiguous syntax units in the query. The parameter context1 represents the ambiguity point in the query draft, and the value is the corresponding context in the structured metadata context C context information in C, which is a SQL snippet or relevant semantic position description, aims to help the model and user to explicitly determine the logical position and context of the ambiguity point; parameter candidates1 represents the candidate solution set of the ambiguity point, whose content comes from the result of pattern information retrieval or direct access to database content in C meta , which is used to give a variety of possible alternative options, thereby providing a selection space for subsequent clarification interactions. Function f detect is executed according to the following principle: the abstract syntax tree of C is parsed, and the structure of the query statement is decomposed and analyzed at the syntax level, and the syntax unit is matched with the pattern information in C meta layer by layer to determine whether the syntax unit is ambiguous; when it is detected that a syntax unit corresponds to multiple possible semantic interpretations, it is classified as a new ambiguity point, and a corresponding triple A is constructed, and finally added to the ambiguity point set A.

[0126] After the clarification tree structure is constructed, according to the current form of clarification tree T state and the metadata information of C meta , a specific clarification question set P clarify is generated, which presents the clarification question to the user in a structured form, used to narrow the ambiguity space and make the query intent more clear. The generation of the clarification question specifically includes the following steps:

[0127] Taking the current clarification state tree T state and the ambiguity point set A as input, the question generation function f generate is converted into a natural language question string, and its calculation formula is:

[0128] P clarify =f generate (T state ,A)

[0129] In this process, the question generation function first prioritizes the ambiguity points in the set A, and selects the ambiguity point a j with the highest priority; the determination of the priority is jointly determined by the type type j of the ambiguity point and the importance of the ambiguity point in the query logic, wherein type j represents the type of the jth ambiguity point, which is derived from the predefined categories of field ambiguity, table ambiguity, value ambiguity or connection ambiguity; the importance of the query logic is related to whether the ambiguity point directly affects the filtering condition, the aggregation logic or the table association path. After selecting the ambiguity point a j , the function selects the corresponding template j in the question template library according to its type type Each template Both contain structured natural language question framework and reserve slots for filling in context and candidate value information. The function will inject the context parameter context j of the ambiguity point into the template j , and the candidate solution set candidates j . The template will be filled with the context j representing the context snippet of the ambiguity point in the preliminary SQL query draft to prompt the user that the question is related to the specific location of the query, and the candidates meta represent the candidate solution list obtained by structured metadata context C clarify or database content retrieval, to provide the user with optional answers;

[0130] The final P user will be presented through the user interface and used as input request in the interactive clarification process, and the user's feedback result R state will be directly updated to the relevant node of the clarification state tree T user .

[0131] When the user returns the feedback R i , the feedback information is written to the corresponding node n state , and the state of the node is updated to clarified, thereby gradually reducing the number of unresolved nodes in the tree. According to the continuous input of user feedback, the clarification state tree T user is dynamically and iteratively updated, and gradually converges to a clear query intent at the semantic level; based on the updated clarification state tree, a new SQL query draft is generated, which continues the logical framework of the initial draft in structure, and integrates the user-confirmed constraint conditions and disambiguation information in semantics, thereby forming a SQL query version that better meets the user's real intent, and entering the subsequent iteration optimization and verification link. The integration of the user's feedback result R input and the update of the clarification state tree specifically includes the following steps:

[0132] With the user-provided feedback information I j and the current ambiguity point a input to be clarified as input, an updated clarification state tree is generated, which is mathematically expressed as

[0133]

[0134] where I update represents the feedback information submitted by the user in the interactive interface, which can be an item in the candidate set or free text input. In this process, the state update function f inputWhen I input is one of the elements in the candidate list candidates j , the element is directly marked as the confirmation value v confirmed , indicating that the user makes a selection among predefined options; when I input is free text input by the user, the text is directly taken as the new confirmation value v confirmed to support the user to provide more accurate intent description beyond the system candidates. Subsequently, the function locates the node n state corresponding to the ambiguity point a j in the clarification state tree T j , writes the confirmation value v confirmed into the attribute field of the node, and updates the state of the node from “to be clarified” to “confirmed”, making the node complete the transition from semantic ambiguity to semantic explicitness.

[0135] Meanwhile, the function also records the original feedback information I input and the operation timestamp in the node to ensure that there is a complete historical record of each semantic update in the entire interaction process.

[0136] The steps of optimizing the SQL query draft to obtain a new SQL query draft are as follows:

[0137] Taking the updated clarification state tree and the original natural language query request Q nl as inputs, the SQL regeneration function f regenerate outputs the optimized SQL query draft, which is mathematically expressed as

[0138]

[0139] In this process, the function first traverses , which represents the clarification state tree that has integrated user feedback and completed updates, contains all confirmed nodes and their attributes, and extracts all node information of the confirmed state from it. Each confirmed node contains the semantic value confirmed by the user, and the function converts these information into a constraint set

[0140] C constraints ={constraint1,...,constraint m}

[0141] The element constraint1 in the set is a key-value pair structure, where the key represents a semantic unit in the query draft, and the value represents the explicit semantic mapping confirmed by the user, thereby ensuring that the original ambiguity points are uniquely interpreted. Subsequently, the function modifies the original prompt engineering template, and the constraint set C constraintsInjection is a required field, and an enhanced prompt is received. enhanced This prompt adds semantic constraints of user confirmation to the task instructions and metadata context, thereby making the generation process more deterministic. Next, the function will... enhanced The input is fed into a large language model, which then parses the original natural language query request Q. nl The constraint set C must be strictly followed. constraints During the semantic understanding and SQL generation process, redundant ambiguities or logical deviations are avoided, thereby outputting an optimized SQL query draft.

[0142] In the step of iteratively optimizing the SQL query draft based on this feedback, the intention is set... Figure One The consistency verification and iterative loop termination steps specifically include the following steps:

[0143] The input is the clarification state tree T. state User feedback history H user The output is a termination decision signal D. terminate and a final SQL statement Or an abnormal signal S exception Its mathematical form is defined as

[0144]

[0145] In this process, the decision function f decide First, count the number of confirmed nodes in the clarification state tree to the total number of nodes, and then calculate the confirmation ratio r. confirmed :

[0146]

[0147] Where, |N confirmed | represents the number of confirmed nodes, |N total | represents the total number of nodes. This ratio reflects the completeness of the user's intent clarification. The function then checks T. state The system checks for logical conflicts, specifically whether the confirmation values ​​of two confirmed nodes are mutually exclusive in business logic. If such a contradiction is detected, it indicates that the query semantics are no longer consistent. Next, the system analyzes the user feedback history (H). user This includes the user's input sequence and interaction signals during the clarification process, which is used to determine whether the user is showing a clear intention to terminate or strong frustration. Based on the above analysis, if the calculation result satisfies r... confirmed If the value is 1 and no logical conflict is found, the system assumes that the user's intent is completely determined, and then sets D. terminate =True, and call the SQL regeneration function f regenerateGenerate the final SQL statement Ensure the output is semantically consistent with the user's confirmation. If a logical conflict is detected or in H... user If strong negative emotions are detected, it is determined that continuing the interaction is no longer feasible, and D is set. terminate =True and output an exception signal S exception The message indicates that manual intervention is required to restore correctness.

[0148] If the confirmation ratio has not yet reached full coverage and there are no conflicts or strong negative signals, then set D. terminate =False, the clarification loop continues to run, and enters the next round of interaction.

[0149] Syntax validation and security verification module: This module performs syntax validation on the final SQL query draft and checks it against a preset security verification module to ensure it does not contain any potentially dangerous operations. If the check passes, the final executable SQL statement is generated.

[0150] The security verification of the SQL syntax validation and security verification module specifically includes the following steps:

[0151] Enter the SQL statement to be validated in This represents the candidate query statements generated in the previous step and that need to be executed. The processing in this step is accomplished by defining a validation function f. check The implementation, the function expression is

[0152]

[0153] Where R check A boolean value representing the verification result. A value of True indicates successful verification, and a value of False indicates verification failure. Indicates an SQL statement that has been validated and can be executed; E error This indicates the error message generated when validation fails, including syntax error messages. and security error messages The first processing stage of the validation function is syntax analysis, which uses a standard SQL parser to analyze the input. Perform a syntax rule check; if parsing fails, return R. check =False, and output the corresponding syntax error. No executable statements are generated at this time.

[0154] If the syntax analysis is successful, the process proceeds to the security verification phase, which checks against a predefined blacklist of dangerous operation modes (B). danger ={pattern1,pattern2,…,pattern v} An inspection was conducted, among which Bdanger represents the set of all patterns of statements that are determined to be potentially attack or violation, pattern j represents the jth dangerous pattern, usually composed of a specific sequence of operators or a combination of keywords, such as operations involving database deletion, system table modification, or privilege escalation.

[0155] If the input matches any one pattern pattern danger in the set B j , it is determined that the security check fails, and the output result R check is False, and the security error information is returned. Similarly, no executable statement is generated.

[0156] Only when both syntax analysis and security verification pass, the value of the check result R check is True, at which time the original input is output as the final executable statement without any error information. In this way, the entire processing process realizes the mapping from the input candidate SQL statement to the output check result and executable statement or error information, ensuring that the SQL statement in the execution phase meets both the syntax correctness and the security constraints.

[0157] SQL statement execution and visualization module: for sending the executable SQL statement to the target enterprise database for execution, and receiving the returned data result; presenting the data result to the user in the form of a preset visual chart or structured table.

[0158] The specific steps of presenting the data result to the user in the form of a preset visual chart or structured table include:

[0159] Taking the original data result R data returned by the database and the original natural language query Q nl as inputs, an output result is generated by a visualization mapping function f visualize .

[0160] (V chart |T table )=f visualize (R data ,Q nl )

[0161] wherein R data represents the structured result set returned after executing the SQL query in the database via the NL2SQL model, the content of which includes data fields and corresponding values; V chart represents the generated visual chart result, which is used to visually present the query information; Ttable represents structured table output, which is used to ensure the readability of results when the chart cannot be generated.

[0162] f visualize is the mapping function from input to output, whose internal logic includes three stages: data structure parsing, semantic intent recognition and chart type selection. The function first parses the structure attributes of R data , which includes the number of dimensions D num , the number of metrics M num and the data type set The number of dimensions refers to the number of non-numeric fields in the returned data, the number of metrics refers to the number of numeric fields, and the data type set records whether each field belongs to a numeric, time or category type.

[0163] Then the function parses the keyword set of Q nl The keyword set is used to infer the intent tendency of the user query. When "trend" is contained in Q , the intent tendency variable φ is marked as time series analysis; when "comparison" is contained, φ is marked as multi-dimensional contrast analysis; when "proportion" is contained, φ is marked as proportion decomposition analysis. Based on the combination results of data structure characteristics and semantic intent φ, the function selects the optimal chart type ChartType from the pre-defined visualization rule set , which contains column chart, line chart, pie chart, scatter chart and heat map type, and each type is defined in the rule set with matching conditions of dimensions and metrics. After determining ChartType, the function converts R data into structured data that meets the chart input format, and then calls the visualization library to generate the final chart V chart . When the rule matching fails or does not meet any chart input requirements, the function outputs the table form T table to ensure the complete presentation of the results.

[0164] ​​The above-mentioned are only embodiments of the present application, and the common knowledge of the specific structure and characteristics in the scheme is not described too much, the ordinary skilled in the art knows all the ordinary technical knowledge in the field of the present application before the application date or the priority date, can know all the prior art in the field, and has the ability to apply the conventional experimental means before the date, the ordinary skilled in the art can improve and implement the present scheme under the inspiration given by the present application, and some typical known structure or known method should not become the obstacle for the ordinary skilled in the art to implement the present application. It should be pointed out that for those skilled in the art, without departing from the structure of the present application, a number of modifications and improvements can be made, which should also be regarded as the protection scope of the present application, which will not affect the effect and practicality of the patent. The protection scope of the present application should be subject to the content of its claims, and the specific implementation mode and the like in the specification can be used to explain the content of the claims.

Claims

1. An enterprise question answering method based on a large language model combined with NL2SQL, characterized in that, The method comprises the following steps: receiving and storing a natural language query request input by a user; extracting database table names, field names, field annotations and inter-table association relationships potentially related to the natural language query request from a schema repository of an enterprise database, and constructing a structured metadata context; filling the natural language query request and the structured metadata context into a predefined prompt engineering template to generate a standard prompt for a large language model, which contains task instructions, context constraints and examples; inputting the standard prompt into a trained large language model, and performing deep semantic analysis on the natural language query by the large language model to comprehensively understand the query intent, filtering conditions, aggregation logic and association relationships, and generating a preliminary SQL query draft; making the large language model self-evaluate the SQL query draft generated by it, and generating one or more structured clarification questions and returning them to the user when ambiguities, logical conflicts or insufficient information are identified in the natural language query request; receiving user selection or supplementary feedback on the clarification questions, and iteratively optimizing the SQL query draft based on the feedback; performing syntax verification on the finally determined SQL query draft, and checking whether it contains potential dangerous operations through a preset security verification module; if the check passes, generating a final executable SQL statement; sending the executable SQL statement to a target enterprise database for execution, and receiving the returned data results; and presenting the data results to the user in the form of a preset visual chart or structured table.

2. The enterprise question answering method based on a large language model combined with NL2SQL according to claim 1, characterized in that: The receiving user selection or supplementary feedback on the clarification questions and iteratively optimizing the SQL query draft based on the feedback comprises the following steps: First, a preliminary SQL query draft is created with the structured metadata context C meta As input, the clarification state tree T is initialized state = (N, E), where N is the node set for storing clarification units and confirmation information; E is the edge set for defining the parent-child relationship between nodes, and recording the two inputs as core attributes in the root node; By analyzing the large language model , layer by layer analyzing its semantic structure and logical expression, and combining the table name, field name, field comment and inter-table association relationship in C meta , all potential ambiguity point sets A = {a1, a2,..., a k} are identified; Each element a i in the ambiguity point set A represents a semantic unit with uncertainty or ambiguity in the query draft, including the aggregation method of the field, the value range of the filtering condition or the selection of the table association path; For each ambiguity point a i In the clarification state tree T state A new node n i Is created and connected to the parent node through the edge set E, while the type and candidate value information of the ambiguity point is attached to the node, forming a dynamically scalable clarification tree structure; After the construction of the clarification tree is completed, the specific clarification question set P state is generated according to the current form of the clarification tree T meta and the metadata information of C clarify , and the clarification questions are presented to the user in a structured form. When the user returns the feedback R user , the feedback information is written into the corresponding node n i , and the state of the node is updated to clarified; according to the continuous input of the user feedback, the clarification state tree T state is dynamically and iteratively updated, and gradually converges to an explicit query intention at the semantic level; based on the updated clarification state tree, a new SQL query draft is generated 3. The enterprise question answering method based on a large language model combined with NL2SQL according to claim 2, characterized in that: The generation process of the ambiguity point set A comprises the following steps: with a preliminary SQL query draft and structured metadata context C meta For the input, first pass the ambiguity detection function f detect The function outputs a set of tuples, whose formula is In this set, the element structure is triple A: A = {(type1, context1, candidates1),..., (type k , context k ,candidates k )} where each triple corresponds to a semantic ambiguity point in the query draft; parameter typei represents the type of the ith ambiguity point, whose value is derived from a predefined type set, the basic categories of which include field ambiguity, table ambiguity, value ambiguity and join ambiguity; parameter contexti represents the context information of the ambiguity point in , which is specifically a SQL fragment or a related semantic position description; parameter candidatesi represents the candidate solution set of the ambiguity point, whose content is derived from the result of searching the schema information in C meta or directly accessing the database content, which is used to give a variety of possible alternative options; the execution principle of function f detect is to perform abstract syntax tree parsing on , to decompose and analyze the structure of the query statement at the syntax level, and to judge whether a syntax unit has ambiguity by layer-by-layer matching with the schema information in C meta ; when detecting that a certain syntax unit corresponds to multiple possible semantic interpretations, it is classified as a new ambiguity point, and the corresponding triple A is constructed, and finally added to the ambiguity point set A.

4. The enterprise question answering method based on a large language model combined with NL2SQL according to claim 3, characterized in that: The generation of the clarification question comprises the following steps: With the current clarification state tree T state With the ambiguity point set A as input, the question generation function f generate Transformed into a natural language question string, the calculation formula is: P clarify = f generate (T state , A) In this process, the problem generation function f generate First, the ambiguity points in set A are prioritized, and the ambiguity point a j with the highest priority is selected j The priority is determined by the type of ambiguity point type j and the importance of the ambiguity point in the query logic, where type j represents the type of the jth ambiguity point, derived from the predefined categories of field ambiguity, table ambiguity, value ambiguity, or join ambiguity; after selecting the ambiguity point a j , the corresponding template is selected from the problem template library according to its type type Each template contains a structured natural language question framework and reserves slots for filling in context and candidate value information; The context parameter context of the ambiguity point is then injected into the template j with the set of candidate solutions candidates j into the template, context j represents the context snippet in the preliminary SQL query draft for hinting the user that the question refers to a specific location of the query, candidates j represents the list of candidate solutions obtained by structured metadata context C meta or database content retrieval, for providing the user with selectable answers; The final generated P clarify The feedback result R of the user will be presented through the user interface and as an input request in the interactive clarification process user Will be updated directly to the clarification state tree T state On the relevant nodes of the clarification state tree T.

5. The enterprise question answering method based on a large language model combined with NL2SQL according to claim 4, characterized in that: the feedback result R of the user user The integration of the set of rules with the update of the clarification state tree specifically comprises the following steps: with the user-provided feedback information I input and the current ambiguity point a to be clarified j as input, generating an updated clarification state tree The calculation formula is: where I input represents the feedback information submitted by the user in the interactive interface, either as one of the candidate set or as free text input; in this process, the state update function f update is applied to the state tree T state to update the state of the ambiguity point a j and its corresponding node n j . First, the input feedback information I input is parsed, and when I input belongs to one of the elements in the candidate list candidates j , the element is directly marked as the confirmation value v confirmed , indicating that the user has made a selection among the predefined options; when I input is free text input by the user, the text is directly taken as the new confirmation value v confirmed ; subsequently, the function locates the node n j corresponding to the ambiguity point a j in the clarification state tree T state , writes the confirmation value v confirmed into the attribute field of the node, and updates the state of the node from "to be clarified" to "confirmed", making the node complete the transition from semantic ambiguity to semantic clarity; at the same time, the original feedback information I input and the operation timestamp are recorded in the node. By this process, the clarified state tree T state The structured dynamic evolution is realized, and the update result is optimizing the SQL query draft to obtain a new SQL query draft The steps are as follows: with the updated clarified state tree and the original natural language query request Q nl for input, through the SQL regeneration function f regenerate outputs the optimized SQL query draft, in mathematical form as In this process, the function first traverses The clarified state tree representing the integrated user feedback and completed updates, containing all confirmed nodes and their attributes, from which the node information of all confirmed states is extracted, each confirmed node containing the user-confirmed semantic value, the function converts this information into a set of constraints C constraints = {constraint1,..., constraint m} The element constraint1 in the set is a key-value pair structure, the key represents a semantic unit in the query draft, and the value represents an explicit semantic mapping confirmed by the user; the subsequent function modifies the original prompt engineering template to add the constraint set C constraints Injection is a required field, get enhanced prompt Prompt enhanced , which adds user-confirmed semantic constraints on the basis of task instructions and metadata context; then, the function inputs Prompt enhanced to the large language model, and the large language model follows the constraint set C constraints when parsing the original natural language query request Q nl , and outputs an optimized SQL query draft 6. The enterprise question answering method based on a large language model combined with NL2SQL according to claim 5, characterized in that: In the step of iteratively optimizing the SQL query draft based on the feedback, an intent consistency verification and iteration loop termination step is set, which comprises the following steps: Input is a clear state tree T state with user feedback history H user Output is a termination decision signal D terminate and a final SQL statement or an exception signal S exception whose formula is In this process, the decision function f decide First, the number of confirmed nodes and the total number of nodes in the clarification state tree are counted, and the confirmation ratio r confirmed : where |N confirmed | represents the number of confirmed nodes, |N total | represents the total number of nodes; then the function checks T state whether there is a logical conflict, i.e. whether there are two confirmed nodes whose confirmation values are mutually exclusive in business logic, if such a contradiction is detected, it means that the query semantics has failed to be self-consistent; then the function analyzes the user feedback history H user , which contains the input sequence and interaction signals of the user in the clarification interaction process, through this information, it is judged whether the user shows a clear termination intention or a strong frustration emotion; based on the above analysis, if the calculation result satisfies r confirmed = 1 and no logical conflict is found, it is determined that the user's intention has been completely determined, at this time, D terminate = True is set, and the SQL regeneration function f regenerate is called to generate the final SQL statement If a logical conflict is detected or a strong negative emotion is identified in H user , it is considered that further interaction is no longer feasible, at this time, D terminate = True is set and an exception signal S exception is output, prompting manual intervention to restore correctness; If the confirmation ratio has not yet reached full coverage and there are no conflicts or strong negative signals, then set D. terminate =False, the clarification loop continues to run, and enters the next round of interaction.

7. The enterprise question answering method based on a large language model combined with NL2SQL according to claim 1, characterized in that: The training of the large language model comprises the following steps: with a dataset consisting of triples as input, where denotes the ith natural language query, which is the query request proposed by the user, denotes the database metadata context corresponding to the query, containing table structure, field information and its constraints, denotes the standard SQL statement matched with the natural language request; the training output of the model is a set of parameters Θ, which are obtained through iterative optimization, so that the correct mapping of the model from the natural language input and the metadata context to the SQL statement is achieved; the training process adopts the supervised fine-tuning paradigm, and the goal is to minimize the negative log-likelihood loss function The loss function is defined as wherein L denotes the length of the target SQL sequence , denotes the tth token in the SQL sequence, denotes the sequence of all tokens before the tth token. Data augmentation strategies are introduced in the data construction phase, including ambiguity rewriting of the original query to ensure that it can be consistent with the clarification state tree T state and the related decision-making link closely cooperate to generate SQL statements that meet the user's real intention.

8. The enterprise question answering method based on a large language model combined with NL2SQL according to claim 1, characterized in that: The syntax verification and security verification of the SQL comprise the following steps: Input the SQL statement to be checked wherein represents the candidate query statement generated by the previous step and needs to be executed; by defining a checking function f check Implementation, function expression is where R check is a Boolean value indicating the result of the check, True if the check succeeds, False if the check fails; E is an SQL statement that can be executed after verification; error E is an SQL statement that can be executed after verification; E is an SQL statement that can be executed after verification; The first processing stage of the check function is syntax analysis, using a standard SQL parser to check the input for syntax rules, and returning R check = False and outputting the corresponding syntax error if the parsing fails; In this case, no executable statement is generated; If the syntax analysis is successful, it enters the security check phase, which checks the query against a predefined blacklist B of dangerous operation patterns danger = {pattern1, pattern2, …, pattern v} where B danger represents the set of all statement patterns that are judged as potentially dangerous or illegal, pattern j represents the jth dangerous pattern, usually consisting of a specific sequence of operators or a combination of keywords, including operations such as database deletion, system table modification, or privilege escalation; If the input is Matching set B danger any one of the patterns j If the condition is not met, the security check is deemed to have failed, and the output result is R. check =False, and return a security error message. Similarly, no executable statements are generated; The verification result R is only valid if both the syntax analysis and security verification stages pass. check The value is True, at which point... Output as is as the final executable statement No error message is generated.

9. The enterprise question answering method based on a large language model combined with NL2SQL according to claim 1, characterized in that: The specific steps of presenting the data results to the user in the form of a preset visual chart or structured table comprise: The raw data result R returned by the database data with the original natural language query Q nl for input, through the visualization mapping function f visualize generates the output result; the calculation formula is as follows: (V chart |T table )=f visualize (R data ,Q nl ) wherein R data represents the structured result set returned after executing the SQL query in the database via the NL2SQL model, the content of which includes data fields and corresponding values; V chart represents the generated visualization chart result, which is used to intuitively present the query information; T table represents the structured table output, which is used to ensure the readability of the result when the chart cannot be generated; f visualize is a mapping function from input to output, whose internal logic includes three stages of data structure parsing, semantic intent recognition and chart type selection; the function first parses the structure attributes of R data , which include the dimension number D num , the measure number M num and the data type set The dimension number refers to the number of non-numeric fields in the returned data, the measure number refers to the number of numeric fields, and the data type set records whether each field belongs to a numeric type, a time type or a category type. Subsequently, the function parses Q nl into a set of keywords where the set of keywords is used to infer the intent tendency of the user query, when "trend" is contained in φ, the intent tendency variable φ is labeled as time series analysis; when "comparison" is contained in φ, φ is labeled as multi-dimensional contrast analysis; when "proportion" is contained in φ, φ is labeled as proportion decomposition analysis; based on the characteristics of the data structure combined with the semantic intent φ, the function selects the optimal chart type ChartType from a predefined set of visualization rules , which contains the bar chart, line chart, pie chart, scatter plot and heat map types, each of which is defined in the rule set with matching conditions of dimensions and measures; after determining ChartType, the function converts R data into structured data conforming to the chart input format , and then calls the visualization library to generate the final chart V chart ; when the rule matching fails or does not meet any chart input requirements, the function outputs the table form T table to ensure the complete presentation of the results.

10. An enterprise question answering system based on a large language model combined with NL2SQL, characterized in that: The enterprise question number method based on a large language model combined with NL2SQL uses the method of any one of claims 1-9.

Images