Interactive number asking agent system based on large language model

Abstract

The invention discloses an interactive question intelligent agent system based on a large language model, and relates to the technical field of dialogue interaction systems. Aiming at the technical defects of the traditional BI tool, the technical scheme is as follows: a heterogeneous data management engine realizes unified access and secure access of cross-source data; the NLP semantic recognition engine converts a natural language into structured semantics through multiple steps, drives the hybrid SQL generation engine and provides visual parameters; the generation engine realizes precise generation from semantics to SQL through a two-stage architecture based on metadata and authority rules; and the visual management console is combined with multi-party configuration and parameters to visually build a number-asking agent. The method is used for realizing intelligent conversion from a natural language to a structured query language (SQL) and a data service interface, and automatically generating an interactive data visualization result.

Description

Technical Field

[0001] The present invention relates to the technical field of dialogue interaction systems, and more specifically, to an interactive question-and-number intelligent agent system based on a large language model. Background Art

[0002] With the rapid development of information technology and the advent of the big data era, enterprises and organizations have accumulated a large amount of data. This data contains rich information and has inestimable value for decision-making support, business optimization, trend analysis, etc. Traditional business intelligence (BI) tools have long dominated the enterprise data insight scenario, but their architectures and interaction modes are no longer able to meet the needs of modern agile analysis. How to effectively extract useful information from the vast amount of data and present it in an intuitive and easy-to-understand way has become an important challenge.

[0003] Traditional BI tools have the following problems: 1. The technical threshold is too high and users are highly dependent. Traditional data query methods usually require users to have a certain technical background and be proficient in SQL syntax, data modeling (such as star / snowflake models), and ETL processes. It is difficult for non-technical personnel to use, and on average, each business department needs to be equipped with 1.2 full-time data analysts to handle data extraction requirements (source: Forrester Enterprise Digital Human Cost White Paper).

[0004] 2. The query results are fixed and cannot quickly respond to demand changes. Traditional BI tools usually rely on predefined data models and fixed report templates. This means that before conducting data analysis, users need to first determine information such as analysis dimensions, metrics, and data sets, and generate reports or dashboards based on these pre-set parameters. Once the report or dashboard is generated, the displayed result data is relatively fixed and it is difficult to reflect the latest changes in the data source in real time. If it is necessary to adjust the analysis angle or add new data dimensions, it often requires reconfiguring the entire data model, which is not only time-consuming but also requires a high level of user skills.

[0005] 3. The interaction method is single. Most traditional BI tools use a drag-and-drop interface for users to select data fields and set filtering conditions. Although this method is relatively intuitive for users familiar with the tool operation, it lacks flexibility and cannot support more user-friendly interaction modes (such as natural language queries). In addition, such tools usually only provide basic functions such as clicking, filtering, and sorting. For users who want to deeply explore the story behind the data or hope to interact with the data in a more flexible way, this interaction method seems inadequate.

[0006] 4. The data visualization and query processes are disjointed. Users need to manually export SQL results to CSV / XLSX and then import them into visualization tools for secondary processing, with an average time consumption exceeding 15 minutes. The matching rate between automated chart recommendations and data features is less than 20%, seriously dragging down the analysis efficiency.

[0007] 5. There is a lack of real-time and unstructured data processing. Relying on the T+1 data update mode, it is unable to process streaming data (such as real-time logs of IoT sensors). At the same time, it only supports structured data, and the analysis coverage rate for unstructured data such as text and images is less than 5%.

[0008] Therefore, a system that can understand natural language query requests, automatically convert them into appropriate query statements, and execute query operations is particularly important. Summary of the Invention

[0009] In view of the problems of high technical thresholds of traditional BI tools, inability to quickly respond to demand changes, single interaction mode, and disjointed data visualization and query processes, the present invention provides an interactive data query intelligent agent system based on a large language model. This system can achieve intelligent conversion from natural language to structured query language (SQL) and data service interfaces through multi-round conversations, and automatically generate interactive data visualization results.

[0010] The technical solution adopted by the interactive data query intelligent agent system based on a large language model of the present invention to solve the above technical problems is as follows: An interactive data query intelligent agent system based on a large language model, which includes: A heterogeneous data management engine, responsible for realizing unified access, integration, and secure and efficient access across databases, file systems, API interfaces, and real-time stream data sources, and providing standardized data sources and metadata support for subsequent modules; An NLP semantic recognition engine, supporting functions such as intent recognition, entity extraction, context understanding, and domain adaptation. Through steps such as dimension index recognition, data table matching, context tracking, entity disambiguation, and security verification, it converts the user's natural language input into a structured semantic representation, drives the query logic of the hybrid SQL generation engine, and provides semantic parameters for visual decision-making; A hybrid SQL generation engine, adopting a two-stage architecture that combines a deep learning model and a rule engine. By constructing a training data set, designing a context-aware mechanism, performing column-level lineage analysis and error correction, optimizing performance, and strengthening security verification steps, based on the metadata and permission rules of the heterogeneous data management engine, it realizes the accurate generation of SQL statements from the semantic representation parsed by the NLP semantic recognition engine; The Visual Management Console is responsible for parameterizing the configuration of the large model, training data sets, and data source information of the heterogeneous data management engine by solidifying the data query call process. Combining the NLP semantic recognition results with the SQL generation parameters, it realizes the visual and rapid construction of the data query intelligent agent.

[0011] Optionally, the heterogeneous data management engine involved includes a heterogeneous protocol adaptation connector, a metadata management module, a blockchain data security control module, and a virtualized federated query engine, where: The heterogeneous protocol adaptation connector works in coordination through three major components: a data operation controller, a data type converter, and a metadata controller. It encapsulates the protocol differences of the underlying data sources, provides a unified upper-layer data access interface, and realizes the standardized access and efficient operation of relational databases, NoSQL, file systems, APIs, and real-time stream data; The metadata management module is based on metadata management technology and uses automated collection, ontology model supplementation, graph database storage, and semantic disambiguation technology to construct a unified metadata knowledge graph across data sources, realizing metadata information integration, lineage tracing, and semantic disambiguation; The blockchain data security control module is built based on blockchain technology. Combining the hierarchical and classification information of metadata, it realizes fine-grained permission control of databases, tables, and fields, ensuring the security of the data usage process; The virtualized federated query engine constructs a unified data view based on data virtualization technology. By docking with the blockchain data security control module, it shields the physical differences of heterogeneous data sources and provides a unified SQL / interface access method for different users, realizing the secure and efficient query of cross-source data.

[0012] Further optionally, the data operation controller involved is responsible for processing the operation instructions received by the unified interface and converting them into underlying protocol requests adapted to each data source; The data type converter is responsible for solving the data type incompatibility problem between different data sources and realizing two-way conversion; The metadata controller is responsible for managing the structure information of the data source and supports the automatic discovery and synchronization of metadata.

[0013] Further optionally, the metadata management module involved specifically performs the following operations to realize metadata information integration, lineage tracing, and semantic disambiguation: Use an automated scanning tool to scan each data source and collect basic metadata information such as table structure, field type, and primary and foreign key relationships; Based on the ontology model, deeply supplement the collected metadata; Construct the supplemented and improved metadata into a unified knowledge graph and store it using a graph database. In the graph database, the metadata objects of data tables and fields are represented by nodes, and the relationships between data tables, between data tables and fields, and between fields are represented by edges, thus clearly showing the data lineage relationship. Use the semantic disambiguation module to automatically identify fields with the same name but different meanings and prompt the user to confirm in combination with the context.

[0014] Optionally, the involved NLP semantic recognition engine supports intent recognition, entity extraction, context understanding, and domain adaptation functions, where: the user query type is judged through the intent recognition function, metrics, dimensions, and filtering conditions are extracted through the entity extraction function, the integration and correction of historical conditions in multi-turn conversations are solved through context understanding, and industry-specific terminology parsing is performed on data from different industries through the domain adaptation function.

[0015] Further optionally, the involved NLP semantic recognition engine supports intent recognition, entity extraction, context understanding, and domain adaptation functions. Through steps such as dimension metric recognition, data table matching, context tracking, entity disambiguation, and security verification, the user's natural language input is transformed into a structured semantic representation, driving the query logic of the hybrid SQL generation engine and providing semantic parameters for visual decision-making. This process specifically includes: Call the large model interface to perform intent recognition on the user's natural language input, accurately extract the dimensions, metrics, and query conditions contained therein, and provide the core basis for subsequent data queries. Match the recognized dimensions, metrics, and query conditions with the training data set in the vector library, screen out the top n data tables with the highest matching degree and return them, determine the source table range of the data query, and prepare for constructing the query statement. Adopt a dynamic attention mechanism to track the cross-turn conversation state, store historical query conditions, confirmed entities, and user visualization preferences through context vectors; use a gating mechanism to achieve dynamic context update, automatically merge when the user adds new query conditions, and delete the corresponding fields when encountering explicit correction instructions, ensuring the accurate inheritance and adjustment of query requirements in multi-turn conversations. Relying on the constructed business term knowledge graph, map the metric terms expressed by the user to the specific calculation formulas in the database, and identify synonyms to expand the scope of semantic understanding; for ambiguous entities, resolve them in combination with the context information of the user role or historical query records, and confirm with the user through the conversation to ensure the accuracy of semantic understanding. After semantic parsing, multi-level verification and security control are carried out, including: avoiding incorrect associations between fact tables and dimension tables through column-level lineage analysis, and using a business rule engine to automatically complete necessary query filtering conditions; at the security level, dynamically desensitizing sensitive fields according to user permissions and completely recording operation logs for subsequent auditing and traceability, and finally outputting an accurate and secure structured semantic representation to drive the query logic of the hybrid SQL generation engine and provide semantic parameters for visual decision-making.

[0016] Further optionally, the involved hybrid SQL generation engine adopts a two-stage architecture that combines a deep learning model and a rule engine. By constructing a training data set, designing a context-aware mechanism, performing column-level lineage analysis and error correction, optimizing performance, and strengthening security verification steps, based on the metadata and permission rules of the heterogeneous data management engine, it realizes the accurate generation of SQL statements from the semantic representation parsed by the NLP semantic recognition engine. This process specifically includes: Construct a training data set covering various query scenarios through a combination of manual annotation and automated generation. The training data set includes natural language descriptions, standard SQL statements, and database metadata. Adopt a two-stage hybrid architecture of "generation - optimization": ① In the generation stage, based on a large model, combine the data table, dimension, and metric information output by the NLP semantic recognition engine with the data table creation statement and table association relationship to parse natural language and generate candidate SQL; ② In the optimization stage, use the rule engine to optimize the structure of the candidate SQL, detect performance problems through a syntax tree parser and automatically rewrite it, and at the same time integrate business rule security verification to avoid unreasonable operations. In multiple rounds of conversations, track historical filtering conditions through a context manager, automatically merge new query conditions, adopt an explicit override strategy when dealing with condition conflicts, and at the same time support fuzzy condition completion, automatically improve the user's fuzzy query intention according to business rules, and ensure that the query conditions are accurate and complete. Construct a column-level lineage knowledge graph to record field sources, association relationships, and business meanings; perform real-time verification during the SQL generation process, prohibit direct JOIN of unassociated tables, detect misuses of aggregate functions, automatically complete preset association conditions, avoid multi-table association errors, and ensure the logical correctness of SQL statements. Integrate the database execution plan feedback into the generation stage, predict the execution time through a cost model, recommend index usage, cache the results of queries that meet preset conditions, and automatically add sharding strategies for large table queries to achieve performance-aware SQL optimization and improve query execution efficiency. Embed a security mechanism throughout the SQL generation process, detect and intercept illegal queries involving sensitive tables, dynamically desensitize according to the user role, defend against SQL injection attacks, and record complete audit logs to ensure the security and compliance of data queries. Establish a closed-loop optimization mechanism, recycle the failed SQLs and update the training set with the user's correction feedback. Release dedicated rule packages for different domains, evaluate the pros and cons of versions through A / B testing, use knowledge distillation technology to lightweight the model, adapt to edge device deployment, and continuously improve the adaptability and performance of the hybrid SQL generation engine.

[0017] Further optionally, the involved visual management console parameterizes the configuration of the large model, the training data set, and the data source information of the heterogeneous data management engine by solidifying the data query call process, and combines the NLP semantic recognition result with the SQL generation parameters to achieve the rapid visual construction of the data query agent. This process specifically includes: Solidify the data query call process: Standardize and fix the core process of the data query agent call, clarify the full-link operation steps from receiving the user's natural language query request to calling the NLP semantic recognition engine to parse the intention, triggering the hybrid SQL generation engine to generate a query statement, obtaining the result through the heterogeneous data management engine to connect to the data source, and finally feedback the visual analysis result, forming a reusable process framework; Parameterize the large model configuration: Parameterize the relevant configuration parameters of the large model through the visual interface, support selecting the model type and corresponding version, and configuring call parameters such as the interface address, maximum generation length, and temperature coefficient; the user directly modifies the parameters to dynamically adapt to the natural language processing requirements, and the parameter configuration result is synchronized to the NLP semantic recognition engine; Parameterize the training data set configuration: Specify the source, format, and range of the training data set in the visual interface, set data screening conditions and field mapping relationships, support obtaining metadata from the heterogeneous data management engine for field matching, and achieve flexible management and adjustment of the training data; Parameterize the data source information configuration: Configure basic connection information such as the data source type, connection address, port, username, and password in the visual interface, as well as positioning parameters such as the database schema, table name, and field mapping, directly associate with the protocol adaptation ability of the heterogeneous data management engine, and complete the rapid docking and permission verification of the data source; Visual construction and deployment: Based on the parameterized configuration results, in the visual interface, through the interactive methods of dragging or checking, combine the large model, training data set, data source elements with the logical rules for the NLP semantic recognition engine to convert the user's natural language input into a structured semantic representation, and the generation strategy of the hybrid SQL generation engine to convert the semantic representation parsed by the NLP semantic recognition engine into an SQL statement, and generate a data query agent with one key. Support automatically synchronizing the rules for the hybrid SQL generation engine to optimize the structure and perform security verification on the generated candidate SQLs, and complete the deployment and online of the data query agent.

[0018] An interactive question - based intelligent agent system based on large - language models of the present invention has the following beneficial effects compared with the prior art: Through multi - round conversations, the present invention realizes the intelligent conversion from natural language to structured query language SQL and data service interfaces, and automatically generates interactive data visualization results, solving problems such as high technical thresholds of traditional BI tools, inability to quickly respond to demand changes, single interaction methods, and fragmentation between data visualization and query processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG Figure 1 is a schematic diagram of the implementation architecture of the system described in an embodiment of the present invention; FIG Figure 2 is a block diagram of the heterogeneous data management engine structure of the system described in an embodiment of the present invention; FIG Figure 3 is a flowchart of the implementation of the NLP semantic recognition engine of the system described in an embodiment of the present invention; FIG Figure 4 is a flowchart of the implementation of the hybrid SQL generation engine of the system described in an embodiment of the present invention; FIG Figure 5 is a flowchart of the implementation of the visualization management console of the system described in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To make the technical solutions, technical problems to be solved, and technical effects of the present invention clearer and more understandable, the following describes the technical solutions of the present invention clearly and completely in combination with specific embodiments.

[0021] Embodiment: Referring to FIG Figure 1 , this embodiment proposes an interactive question - based intelligent agent system based on large - language models, which includes: A heterogeneous data management engine, responsible for realizing the unified access, integration, and secure and efficient access across databases, file systems, API interfaces, and real - time stream data sources, providing standardized data sources and metadata support for subsequent modules; An NLP semantic recognition engine, supporting functions such as intent recognition, entity extraction, context understanding, and domain adaptation. Through steps such as dimension and metric recognition, data table matching, context tracking, entity disambiguation, and security verification, it converts the user's natural language input into a structured semantic representation, drives the query logic of the hybrid SQL generation engine, and provides semantic parameters for visual decision - making; A hybrid SQL generation engine, adopting a two - stage architecture that combines a deep - learning model and a rule engine. Through steps such as constructing a training data set, designing a context - aware mechanism, performing column - level lineage analysis and error correction, optimizing performance, and strengthening security verification steps, based on the metadata and permission rules of the heterogeneous data management engine, it realizes the accurate generation of SQL statements from the semantic representation parsed by the NLP semantic recognition engine; The visualization management console is responsible for parameterizing the configuration of the large model, the training dataset, and the data source information of the heterogeneous data management engine by solidifying the data query call process. By combining the NLP semantic recognition results with the SQL generation parameters, it realizes the rapid visual construction of the data query intelligent agent.

[0022] In this embodiment, referring to the appendix Figure 2 The heterogeneous data management engine involved specifically includes four parts: a heterogeneous protocol adaptation connector, a metadata management module, a blockchain data security control module, and a virtualized federated query engine.

[0023] The heterogeneous protocol adaptation connector works in coordination through three major components: a data operation controller, a data type converter, and a metadata controller. It encapsulates the protocol differences of the underlying data sources, provides a unified upper-layer data access interface, and realizes the standardized access and efficient operation of relational databases, NoSQL, file systems, APIs, and real-time stream data. Among them, the data operation controller is responsible for processing the operation instructions received by the unified interface (such as SQL queries and data writes) and converting them into underlying protocol requests adapted to each data source; the data type converter is responsible for solving the data type incompatibility problems between different data sources (such as the format conversion between DATETIME in a relational database and ISODate in NoSQL) and realizes two-way conversion; the metadata controller is responsible for managing the structural information of the data source (such as table structure, field type, and index relationship), and supports automatic discovery and synchronization of metadata.

[0024] The metadata management module is based on metadata management technology and uses automated collection, ontology model supplementation, graph database storage, and semantic disambiguation technology to construct a unified metadata knowledge graph across data sources, realizing metadata information integration, lineage tracing, and semantic disambiguation. This process specifically includes: ① Using an automated scanning tool to scan each data source and collect basic metadata information such as table structure, field type, and primary and foreign key relationships; ② Based on the ontology model, deeply supplement the collected metadata; ③ Construct the supplemented and improved metadata into a unified knowledge graph and store it using a graph database (such as Neo4j). In the graph database, nodes represent metadata objects of data tables and fields, and edges represent the relationships between data tables, between data tables and fields, and between fields, thus clearly showing the data lineage relationship; ④ Using a semantic disambiguation module to automatically identify fields with the same name but different meanings and prompt the user to confirm in combination with the context.

[0025] The blockchain data security control module is built based on blockchain technology and combines the hierarchical and classified information of metadata to achieve fine-grained permission control over databases, tables, and fields, ensuring the security of the data usage process.

[0026] The virtualized federated query engine constructs a unified data view based on data virtualization technology. By connecting to the blockchain data security control module, it shields the physical differences of heterogeneous data sources and provides different users with a unified SQL / interface access method to achieve secure and efficient query of cross-source data.

[0027] In this embodiment, the involved NLP semantic recognition engine supports functions such as intent recognition, entity extraction, context understanding, and domain adaptation. Specifically: it judges the user query type through the intent recognition function, extracts metrics, dimensions, and filtering conditions through the entity extraction function, solves the integration and correction of historical conditions in multi-turn conversations through the context understanding function, and adapts to data in different industries for industry-specific term parsing through the domain adaptation function.

[0028] The involved NLP semantic recognition engine supports functions such as intent recognition, entity extraction, context understanding, and domain adaptation. Through steps such as dimension and metric recognition, data table matching, context tracking, entity disambiguation, and security verification, it converts the user's natural language input into a structured semantic representation, drives the query logic of the hybrid SQL generation engine, and provides semantic parameters for visual decision-making. Refer to the appendix Figure 3 , and this process specifically includes: Call the interface of the large model (GPT series or LLaMA series) to perform intent recognition on the user's natural language input, and accurately extract key elements such as dimensions (such as time, region), metrics (such as sales, gross profit margin), and query conditions (such as greater than a certain value, specific classification), etc., to provide a core basis for subsequent data queries; Match the identified dimensions, metrics, and query conditions with the training data set in the vector library, screen out the top 5 data tables with the highest matching degree and return them to determine the source table range for data queries, and prepare for constructing the query statement; Adopt a dynamic attention mechanism to track the cross-turn conversation state, store historical query conditions, confirmed entities, and user visualization preferences through context vectors; use a gating mechanism to achieve dynamic context update, automatically merge when the user adds new query conditions, and delete the corresponding fields when encountering explicit correction instructions to ensure accurate inheritance and adjustment of query requirements in multi-turn conversations; Relying on the constructed business term knowledge graph, map the metric terms expressed by the user to specific calculation formulas in the database, and identify synonyms to expand the scope of semantic understanding; for ambiguous entities, disambiguate them in combination with the context information of the user role or historical query records, and confirm with the user through dialogue to ensure the accuracy of semantic understanding; After semantic parsing, multi-level verification and security control are performed, including: avoiding incorrect associations between fact tables and dimension tables through column-level lineage analysis, and automatically completing necessary query filtering conditions using a business rule engine; at the security level, dynamically desensitizing sensitive fields based on user permissions and completely recording operation logs for subsequent auditing and traceability, and finally outputting an accurate and secure structured semantic representation for driving the query logic of the hybrid SQL generation engine and providing semantic parameters for visual decision-making.

[0029] In this embodiment, the involved hybrid SQL generation engine adopts a two-stage architecture that integrates a deep learning model and a rule engine. By constructing a training data set, designing a context-aware mechanism, performing column-level lineage analysis and error correction, optimizing performance, and strengthening security verification steps, based on the metadata and permission rules of the heterogeneous data management engine, accurate generation from the semantic representation parsed by the NLP semantic recognition engine to SQL statements is achieved. Refer to the appendix Figure 4 , and this process specifically includes: Construct a training data set covering various query scenarios through a combination of manual annotation and automated generation. The training data set includes natural language descriptions, standard SQL statements, and database metadata; for complex business scenarios, experts write standard SQL and reverse-generate natural language descriptions, use a template engine to batch-generate variant query statements, inject noise data to enhance robustness, and use WordNet for synonym expansion and sentence pattern expansion. At the same time, the training data set supports docking with multi-source heterogeneous data management modules to automatically obtain data from heterogeneous data sources; Adopt a two-stage hybrid architecture of "generation-optimization": ① In the generation stage, based on a large model, parse the data table, dimension, and metric information output by the NLP semantic recognition engine, combine the data table creation statements and table association relationships, and parse natural language to generate candidate SQL; ② In the optimization stage, use the rule engine to optimize the structure of the candidate SQL, detect performance problems through a syntax tree parser and automatically rewrite it, and at the same time integrate business rule security verification to avoid unreasonable operations; In multiple rounds of conversations, track historical filtering conditions through a context manager, automatically merge new query conditions, adopt an explicit overwrite strategy when dealing with condition conflicts, and at the same time support fuzzy condition completion, automatically improve the user's fuzzy query intent according to business rules, and ensure that the query conditions are accurate and complete; Construct a column-level lineage knowledge graph to record field sources, association relationships, and business meanings; perform real-time verification during the SQL generation process, prohibit direct JOIN of unassociated tables, detect misuse of aggregate functions, automatically complete preset association conditions, avoid multi-table association errors, and ensure the logical correctness of SQL statements; Integrate the database execution plan feedback into the generation stage, predict the execution time through the cost model, recommend index usage, cache the results of queries that meet the preset conditions, automatically add sharding strategies for large table queries, and achieve performance-aware SQL optimization to improve the query execution efficiency; Embed a security mechanism throughout the SQL generation process, detect and intercept illegal queries involving sensitive tables, perform dynamic desensitization according to user roles, defend against SQL injection attacks, and record complete audit logs to ensure the security and compliance of data queries; Establish a closed-loop optimization mechanism, recycle failed SQL executions and user correction feedback to update the training set, release dedicated rule packages for different domains, evaluate the pros and cons of versions through A / B testing, use knowledge distillation technology to lightweight the model, adapt to edge device deployment, and continuously improve the adaptability and performance of the hybrid SQL generation engine.

[0030] In this embodiment, the involved visual management console parameterizes the configuration of the large model, the training data set, and the data source information of the heterogeneous data management engine by solidifying the process of asking questions and making calls, combines the NLP semantic recognition results with the SQL generation parameters, and realizes the visual and rapid construction of the question-answering agent. Refer to Appendix Figure 5 This process specifically includes: Solidify the process of asking questions and making calls: Standardize and fix the core process of calling the question-answering agent, clarify the full-link operation steps from receiving the user's natural language query request to calling the NLP semantic recognition engine to parse the intent, triggering the hybrid SQL generation engine to generate query statements, accessing the data source through the heterogeneous data management engine to obtain results, and finally feeding back the visual analysis results, and form a reusable process framework; Parameterize the large model configuration: Parameterize the relevant configuration parameters of the large model through the visual interface, support selecting model types such as GPT series or LLaMA series and corresponding versions, and configure calling parameters such as the interface address, maximum generation length, and temperature coefficient; users can directly modify the parameters to dynamically adapt to the natural language processing requirements, and the parameter configuration results are synchronized to the NLP semantic recognition engine; Parameterize the training data set configuration: Specify the source (local file / database table / data warehouse), format (CSV / JSON / Parquet), and range of the training data set in the visual interface, set data screening conditions and field mapping relationships, support obtaining metadata from the heterogeneous data management engine for field matching, and realize flexible management and adjustment of training data; Parameterized data source information configuration: Configure basic connection information such as data source type (relational database / NoSQL / file system, etc.), connection address, port, username, and password in the visualization interface, as well as positioning parameters such as database schema, table name, and field mapping, directly associate with the protocol adaptation ability of the heterogeneous data management engine, and complete the quick docking and permission verification of the data source; Visualization construction and deployment: Based on the parameterized configuration results, in the visualization interface, through interactive methods such as dragging or checking, combine the large model, training data set, data source elements with the logical rules of the NLP semantic recognition engine to convert the user's natural language input into a structured semantic representation, and the generation strategy of the hybrid SQL generation engine to convert the semantic representation parsed by the NLP semantic recognition engine into SQL statements, and generate the question-answering intelligent agent with one click. Support automatically synchronizing the rules of the hybrid SQL generation engine to optimize the structure and perform security verification on the generated candidate SQL, and complete the deployment and online of the question-answering intelligent agent.

[0031] In summary, by using an interactive question-answering intelligent agent system based on a large language model of the present invention, through multi-round conversations, it realizes the intelligent conversion from natural language to structured query language SQL and data service interfaces, and automatically generates interactive data visualization results, solving problems such as high technical thresholds of traditional BI tools, inability to quickly respond to demand changes, single interaction methods, and fragmentation of data visualization and query processes.

[0032] The above application of specific examples has elaborated in detail the principle and implementation manner of the present invention. These embodiments are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, those skilled in the art of this technology, without departing from the principle of the present invention, any improvements and modifications made to the present invention shall fall within the scope of patent protection of the present invention.

Claims

1. An interactive question-and-number intelligent agent system based on a large language model, characterized in that, It includes: A heterogeneous data management engine, which is responsible for realizing the unified access, integration, and secure and efficient access across databases, file systems, API interfaces, and real-time stream data sources, and providing standardized data sources and metadata support for subsequent modules; An NLP semantic recognition engine, which supports functions such as intent recognition, entity extraction, context understanding, and domain adaptation. Through steps such as dimension index recognition, data table matching, context tracking, entity disambiguation, and security verification, it converts the user's natural language input into a structured semantic representation, drives the query logic of the hybrid SQL generation engine, and provides semantic parameters for visual decision-making; A hybrid SQL generation engine, which adopts a two-stage architecture integrating a deep learning model and a rule engine. By constructing a training data set, designing a context-aware mechanism, performing column-level lineage analysis and error correction, optimizing performance, and strengthening security verification steps, based on the metadata and permission rules of the heterogeneous data management engine, it realizes the accurate generation of SQL statements from the semantic representation parsed by the NLP semantic recognition engine; A visual management console, which is responsible for parameterizing the configuration of the large model, the training data set, and the data source information of the heterogeneous data management engine by solidifying the data query call process, and realizing the visual and rapid construction of the data query agent in combination with the NLP semantic recognition result and the SQL generation parameter; 2. The interactive question-answering intelligent agent system based on a large language model according to claim 1, wherein The heterogeneous data management engine includes a heterogeneous protocol adaptation connector, a metadata management module, a blockchain data security control module, and a virtualized federated query engine, where: The heterogeneous protocol adaptation connector works in cooperation with three major components: a data operation controller, a data type converter, and a metadata controller, encapsulates the protocol differences of the underlying data sources, provides a unified upper-layer data access interface, and realizes the standardized access and efficient operation of relational databases, NoSQL, file systems, APIs, and real-time stream data; The metadata management module, based on metadata management technology, uses technologies such as automatic collection, ontology model supplementation, graph database storage, and semantic disambiguation to construct a unified metadata knowledge graph across data sources, and realizes metadata information integration, lineage tracing, and semantic disambiguation; The blockchain data security control module is built based on blockchain technology, and combines the hierarchical and classification information of metadata to realize fine-grained permission control of databases, tables, and fields, and ensure the security of the data usage process; The virtualized federated query engine constructs a unified data view based on data virtualization technology. By docking with the blockchain data security control module, it shields the physical differences of heterogeneous data sources and provides a unified SQL / interface access method for different users to realize the secure and efficient query of cross-source data; 3. The interactive question-answering intelligent agent system based on a large language model according to claim 2, wherein The data operation controller is responsible for processing the operation instructions received by the unified interface and converting them into underlying protocol requests adapted to each data source; The data type converter is responsible for solving the data type incompatibility problem between different data sources and realizing two-way conversion; The metadata controller is responsible for managing the structure information of the data source and supporting the automatic discovery and synchronization of metadata; 4. The interactive question-answering intelligent agent system based on a large language model according to claim 2, characterized in that, The metadata management module specifically performs the following operations to realize metadata information integration, lineage tracing, and semantic disambiguation: Use an automated scanning tool to scan each data source and collect basic metadata information such as table structure, field type, and primary and foreign key relationships; Based on the ontology model, deeply supplement the collected metadata; Construct the supplemented and improved metadata into a unified knowledge graph and store it using a graph database; In the graph database, use nodes to represent metadata objects of data tables and fields, and use edges to represent the relationships between data tables, between data tables and fields, and between fields, so as to clearly display the data lineage relationship; Use a semantic disambiguation module to automatically identify fields with the same name but different meanings and prompt the user to confirm in combination with the context.

5. The interactive question-answering intelligent agent system based on a large language model according to claim 2, characterized in that, The NLP semantic recognition engine supports intent recognition, entity extraction, context understanding, and domain adaptation functions. Among them: judge the user query type through the intent recognition function, extract metrics, dimensions, and filtering conditions through the entity extraction function, solve the integration and correction of historical conditions in multi-turn conversations through context understanding, and adapt to data in different industries through the domain adaptation function for parsing industry-specific terms.

6. The interactive question-answering intelligent agent system based on a large language model according to claim 5, characterized in that, The NLP semantic recognition engine supports intent recognition, entity extraction, context understanding, and domain adaptation functions. Through steps such as dimension and metric recognition, data table matching, context tracking, entity disambiguation, and security verification, it converts the user's natural language input into a structured semantic representation, drives the query logic of the hybrid SQL generation engine, and provides semantic parameters for visual decision-making. This process specifically includes: Call the large model interface to perform intent recognition on the user's natural language input, accurately extract the dimensions, metrics, and query conditions contained therein, and provide a core basis for subsequent data queries; Match the identified dimensions, metrics, and query conditions with the training data set in the vector library, screen out the top n data tables with the highest matching degree and return them, determine the source table range of the data query, and prepare for constructing the query statement; Adopt a dynamic attention mechanism to track the cross-turn conversation state, store historical query conditions, confirmed entities, and user visualization preferences through context vectors; use a gating mechanism to achieve dynamic context update, automatically merge when the user adds new query conditions, and delete the corresponding fields when encountering explicit correction instructions, ensuring the accurate inheritance and adjustment of query requirements in multi-turn conversations; Relying on the constructed business term knowledge graph, map the metric terms expressed by the user to the specific calculation formulas in the database and identify synonyms to expand the semantic understanding range; for ambiguous entities, resolve them in combination with the context information of the user role or historical query records, and confirm with the user through the conversation to ensure the accuracy of semantic understanding; After semantic parsing, perform multi-level verification and security control, including: avoiding incorrect associations between fact tables and dimension tables through column-level lineage analysis, and automatically completing necessary query filtering conditions using a business rule engine; at the security level, perform dynamic desensitization processing on sensitive fields according to user permissions and completely record operation logs for subsequent auditing and traceability. Finally, output an accurate and secure structured semantic representation for driving the query logic of the hybrid SQL generation engine and providing semantic parameters for visual decision-making.

7. The interactive question-answering intelligent agent system based on a large language model according to claim 6, wherein, The hybrid SQL generation engine adopts a two-stage architecture that integrates a deep learning model and a rule engine. By constructing a training data set, designing a context-aware mechanism, performing column-level lineage analysis and error correction, optimizing performance, and strengthening security verification steps, based on the metadata and permission rules of the heterogeneous data management engine, it realizes the accurate generation from the semantic representation parsed by the NLP semantic recognition engine to the SQL statement. This process specifically includes: Construct a training data set covering various query scenarios through a combination of manual annotation and automated generation. The training data set includes natural language descriptions, standard SQL statements, and database metadata; Adopt a two-stage hybrid architecture of "generation-optimization": ① In the generation stage, based on a large model, combine the data table, dimension, and metric information output by the NLP semantic recognition engine with the data table creation statement and table association relationship to parse natural language and generate candidate SQL; ② In the optimization stage, use the rule engine to optimize the structure of the candidate SQL, detect performance problems through a syntax tree parser and automatically rewrite it, and at the same time integrate business rule security verification to avoid unreasonable operations; In multiple rounds of conversations, track historical filtering conditions through a context manager, automatically merge new query conditions, adopt an explicit overwrite strategy when dealing with condition conflicts, and at the same time support fuzzy condition completion, automatically improve the user's fuzzy query intent according to business rules, and ensure that the query conditions are accurate and complete; Construct a column-level lineage knowledge graph to record field sources, association relationships, and business meanings; perform real-time verification during the SQL generation process, prohibit direct JOIN of unassociated tables, detect misuse of aggregate functions, automatically complete preset association conditions, avoid multi-table association errors, and ensure the logical correctness of SQL statements; Integrate the database execution plan feedback into the generation stage, predict the execution time through a cost model, recommend index usage, cache the results of queries that meet preset conditions, automatically add a sharding strategy for large table queries, realize performance-aware SQL optimization, and improve the query execution efficiency; Embed a security mechanism in the entire process of SQL generation, detect and intercept illegal queries involving sensitive tables, perform dynamic desensitization according to user roles, defend against SQL injection attacks, and record complete audit logs to ensure the security and compliance of data queries; Establish a closed-loop optimization mechanism, recycle failed SQLs and user correction feedback to update the training set, release dedicated rule packages for different domains, evaluate the pros and cons of versions through A / B testing, use knowledge distillation technology to lightweight the model, adapt to edge device deployment, and continuously improve the adaptability and performance of the hybrid SQL generation engine.

8. An interactive question-and-number intelligent agent system based on a large language model according to claim 7, characterized in that, The visual management console realizes the visual and rapid construction of the data query agent by solidifying the data query call process, parameterizing the configuration of the large model, the training data set, and the data source information of the heterogeneous data management engine, and combining the NLP semantic recognition result and the SQL generation parameters. This process specifically includes: Solidify the Wenshu call process: standardize and fix the core process of the Wenshu agent call, clarify the full-link operation steps from receiving the user's natural language query request to calling the NLP semantic recognition engine to parse the intent, triggering the hybrid SQL generation engine to generate query statements, connecting to the data source through the heterogeneous data management engine to obtain results, and finally feeding back the visual analysis results, to form a reusable process framework; Parameterized large model configuration: parameterize large model related configurations through a visual interface, support selection of model type and corresponding version, configuration of interface address, maximum generation length and temperature coefficient and other calling parameters; users can directly modify parameters, dynamically adapt to natural language processing requirements, and synchronize parameter configuration results to the NLP semantic recognition engine; Parameterized training data set configuration: Specify the source, format, and range of the training data set in the visual interface, set data screening conditions and field mapping relationships, support obtaining metadata from heterogeneous data management engines for field matching, and achieve flexible management and adjustment of training data; Parameterized data source information configuration: Configure the basic connection information such as data source type, connection address, port, user name and password, as well as the positioning parameters of database mode, table name and field mapping in the visual interface, directly link the protocol adaptation capability of the heterogeneous data management engine, and complete the rapid connection of data sources and permission verification; Visual construction and deployment: Based on the parameterized configuration results, the large model, training data set, data source elements, and the logical rules of the NLP semantic recognition engine that converts user natural language input into structured semantic representation, and the hybrid SQL generation engine that converts the semantic representation parsed by the NLP semantic recognition engine into the generation strategy of SQL statements are combined through dragging or checking in the visual interface. The Question and Answer agent is generated with one click, and the rules of automatic synchronization of the hybrid SQL generation engine for structural optimization and security verification of the generated candidate SQL are supported to complete the deployment and launch of the Question and Answer agent.

Images