Multi-source data query system and query method

Abstract

The invention relates to the technical field of data processing, and particularly provides a multi-source data query system and method, and the system comprises an intention analysis module which is used for extracting a user query intention and extracting a query constraint slot position; the data source matching module is used for matching at least one target data source from a plurality of pre-configured heterogeneous data sources based on the query intention of the user; the query statement generation module is used for generating a query statement through an AI prompt project based on the query intention and the query constraint slot position; the query execution module is used for executing the query statement on the target data source to obtain a query result; and the result analysis module is used for carrying out validity analysis on the query result and sending a feedback signal to the query statement generation module when the query result is invalid. According to the method, user requirements can be accurately understood, the optimal data source can be intelligently selected, the query result can be corrected, the output form can be adaptively adjusted, and accurate and efficient data query service can be provided for users.

Description

A multi-source data query system and query method Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a multi-source data query system and query method. Background Technology

[0002] With the deepening of the digitalization process, data exhibits characteristics of being massive, multi-source, and heterogeneous, and may exist simultaneously in various systems such as relational databases, search engines, and data warehouses. In order to obtain valuable information from this data, users need an intelligent query system that can understand their natural language query intent and return results automatically, accurately, and efficiently. Current data query systems generally have the following significant drawbacks: (1) Lack of unified and intelligent query and management capabilities for multi-source heterogeneous data: Existing query systems often design fixed query interfaces for a single type of data source (such as relational databases or search engines), lacking a unified framework that can automatically identify and access different types of data sources (such as SQL for MySQL and DSL for Elasticsearch). Users need to understand the query language of the underlying data source and cannot achieve one-stop, cross-data source natural language query; (2) Low accuracy in understanding user natural language intent and slot extraction: Traditional natural language query systems mainly rely on rule-based or template-based methods to parse user queries, making it difficult to accurately understand the complex intent and precise query constraints in natural language. This results in a high error rate in intent extraction and query parameter extraction when facing complex or fuzzy queries, which seriously affects the quality of subsequent query statements (such as SQL / DSL); (3) The query statement generation and execution process lacks intelligence and adaptive optimization: Existing query statement (DDL) generation is either manually written or based on simple template mapping, lacking comprehensive consideration of intent, slots and data sources to dynamically generate DDL. More importantly, they lack intelligent verification and feedback mechanisms for query results. Once the query results are inaccurate or invalid, the system cannot automatically correct the DDL and re-execute it, and manual intervention is required; (4) The correlation judgment and formatted output of query results lack intelligent in-depth analysis: After acquiring data, existing systems usually only perform simple data cleaning and format conversion before directly outputting it to users. They lack intelligent analysis of the semantic correlation between the result dataset and the original query, which may lead to the output of a large amount of irrelevant or low-correlation "noise" data. At the same time, the presentation of results is limited to a single format, usually just raw data tables or simple JSON structures. It cannot intelligently prioritize and generate the most suitable visualization charts (such as trend charts and bar charts) based on data characteristics or potential user needs, resulting in poor intuitiveness of data analysis and user experience. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multi-source data query system and query method to realize an intelligent closed-loop query system from natural language input to high-value information output.

[0004] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0005] In a first aspect, embodiments of the present invention provide a multi-source data query system, comprising:

[0006] The requirement input module is used to receive user requirements and convert them into query text;

[0007] The intent analysis module is used to perform semantic analysis on the query text, extract the user's query intent, and extract query constraint slots;

[0008] The data source matching module is used to match at least one target data source from multiple pre-configured heterogeneous data sources based on the user's query intent; wherein, the heterogeneous data sources include relational databases and search engines;

[0009] The query statement generation module is used to generate a query statement based on the query intent, the query constraint slot, and the type of the target data source, through AI prompts.

[0010] The query execution module is used to execute the query statement on the target data source and obtain the query results;

[0011] The result analysis module is used to perform validity analysis on the query results and send a feedback signal to the query statement generation module to regenerate the query statement when the query results are invalid.

[0012] The output formatting module is used to render and output valid query results according to a preset output strategy.

[0013] In one embodiment of the present invention, a knowledge base storage module is also included; the knowledge base storage module is used to store valid query results to a graph database or an Elasticsearch index.

[0014] In one embodiment of the present invention, the demand input module includes:

[0015] A speech recognition unit is used to convert the user's speech input into text.

[0016] The text input unit is used to receive natural language text input by the user.

[0017] In one embodiment of the present invention, the intent analysis module includes:

[0018] The intent analysis unit is used to input the query text into the first natural language processing model and output an intent label representing the semantic category of the query text as the user's query intent.

[0019] The slot extraction unit is used to input the query text into the second natural language processing model and extract the key entity information in the query text as the query constraint slot.

[0020] In one embodiment of the present invention, the first natural language processing model is the BERT model; the second natural language processing model is a named entity recognition model.

[0021] In one embodiment of the present invention, the data source matching module includes:

[0022] The data source description vector acquisition unit is used to acquire the metadata description vector of the heterogeneous data source, wherein the metadata description vector represents the semantic content that the corresponding data source can support for querying.

[0023] An intent vector conversion unit is used to convert the query intent into an intent vector.

[0024] A similarity calculation unit is used to calculate the similarity between the intent vector and the metadata description vector;

[0025] The target data source determination unit is used to determine the data source with a similarity reaching a preset threshold as the target data source.

[0026] In one embodiment of the present invention, the result analysis module includes:

[0027] The result summary generation unit is used to summarize the query results and generate result summary text;

[0028] A vector mapping unit is used to map the query text into a query vector and the result summary text into a result summary vector.

[0029] A similarity score calculation unit is used to calculate the similarity score between the query vector and the result summary vector;

[0030] The judgment and feedback unit is used to compare the similarity score with a preset validity threshold. If the similarity score is lower than the validity threshold, the query result is determined to be invalid, and a feedback signal is sent to the query statement generation module.

[0031] In one embodiment of the present invention, the output formatting module includes:

[0032] Data standardization units are used to convert valid query results into a standardized data frame format.

[0033] The output strategy selection unit is used to select an output strategy based on the data type of the valid query result or the output format instruction specified in the query text;

[0034] The visualization rendering unit is used to call the data visualization engine and generate corresponding charts based on the standardized data frame format when the output strategy is visualization output;

[0035] The output strategy includes at least one of plain text output, structured data output, and visualization chart output.

[0036] In one embodiment of the present invention, the requirement input module, intent analysis module, data source matching module, query statement generation module, query execution module, result analysis module, and output formatting module communicate and transmit data asynchronously through message middleware and / or RESTful API interfaces.

[0037] Secondly, embodiments of the present invention provide a multi-source data query method, including:

[0038] Receive user requests and convert them into query text;

[0039] Semantic analysis is performed on the query text to extract the user's query intent and extract query constraint slots;

[0040] Based on the user's query intent, at least one target data source is matched from multiple heterogeneous data sources;

[0041] Based on the query intent, the query constraint slots, and the type of the target data source, a query statement is generated through AI-suggested engineering.

[0042] Execute the query statement on the target data source and obtain the query results;

[0043] The query results are subjected to validity analysis. If the validity analysis passes, the query results are rendered and output according to the preset output strategy. If the validity analysis fails, the query statement is regenerated.

[0044] This invention integrates intelligent semantic understanding, multi-source heterogeneous integration, self-optimization feedback, and flexible output to form a highly efficient, intelligent, and user-friendly multi-source data query system. The intent analysis module accurately understands complex and ambiguous user query intents without requiring users to possess professional query language or data structure knowledge, thus lowering the technical threshold. The data source matching module breaks down data silos, enabling dynamic selection and unified management across heterogeneous data sources. Users don't need to worry about the specific source and type of data; a single query automatically schedules multiple backend data sources, greatly improving the efficiency and breadth of data retrieval. The query statement generation module utilizes AI-powered prompting engineering technology to dynamically transform query intents and query constraint slots into executable query statements for specific target data sources, eliminating reliance on fixed templates and significantly improving the system's adaptability and scalability. The result analysis module introduces a feedback mechanism; when query results are unsatisfactory, the system can self-correct and regenerate query statements, forming a closed-loop optimization process that significantly improves query success rate and accuracy, ensuring the quality of query results. The output formatting module achieves intelligent and visual result output, enhancing the user experience. Attached Figure Description

[0045] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other embodiments based on these drawings without creative effort.

[0046] Figure 1 is a schematic diagram of the structure of a multi-source data query system provided in an embodiment of the present invention;

[0047] Figure 2 is a flowchart illustrating a multi-source data query method provided in an embodiment of the present invention. Detailed Implementation

[0048] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0049] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0050] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0051] Referring to Figure 1, this embodiment of the invention provides a multi-source data query system, the system comprising:

[0052] The requirement input module is used to receive user requirements and convert them into query text;

[0053] The intent analysis module is used to perform semantic analysis on the query text, extract the user's query intent, and extract query constraint slots;

[0054] The data source matching module is used to match at least one target data source from multiple pre-configured heterogeneous data sources based on the user's query intent; wherein, the heterogeneous data sources include relational databases such as MySQL and PostgreSQL, and search engines such as Elasticsearch;

[0055] The query statement generation module is used to generate a query statement based on the query intent, the query constraint slot, and the type of the target data source, through AI prompts.

[0056] The query execution module is used to execute the query statement on the target data source and obtain the query results;

[0057] The result analysis module is used to perform validity analysis on the query results and send a feedback signal to the query statement generation module to regenerate the query statement when the query results are invalid.

[0058] The output formatting module is used to render and output valid query results according to a preset output strategy.

[0059] In one embodiment of the present invention, the multi-source data query system further includes a knowledge base storage module, which is communicatively connected to the result analysis module and is used to store valid query results in a graph database (Neo4j) or a high-performance index (Elasticsearch).

[0060] Before storage, the query results need to be cleaned, categorized, and modeled. Then, the knowledge with model labels and relevance scores is stored in the knowledge base. Through the knowledge base storage module, effective query results can be transformed into structured, queryable, and categorized deep knowledge. The high-quality results of this query are sent to the knowledge building system to be accumulated as new knowledge material, providing an efficient and structured data foundation for subsequent knowledge reasoning and complex queries.

[0061] The input module of this invention includes a speech recognition unit and a text input unit. The speech recognition unit converts the user's speech input into text format; the text input unit receives the natural language text input by the user. The speech recognition unit of this invention overcomes the limitation of traditional query systems that only support text input, achieving intelligent query input and making it more convenient for users.

[0062] In addition, the demand input module of the present invention also includes an input cache queue for temporarily storing user input requests in high-concurrency scenarios.

[0063] The intent analysis module of the present invention includes an intent analysis unit and a slot extraction unit. The intent analysis unit is used to input the query text into a first natural language processing model and output an intent tag representing the semantic category of the query text as the user's query intent. The slot extraction unit is used to input the query text into a second natural language processing model and extract key entity information from the query text as the query constraint slot.

[0064] In this embodiment of the invention, the first natural language processing model employs the BERT (Bidirectional Encoder Representations from Transformers) model. The BERT model is a pre-trained language model based on the Transformer encoder, which can analyze the meaning of each word in the text and the relationships between words, thereby understanding the deep semantics of the entire sentence, rather than simply looking at keywords.

[0065] Specifically, the intent analysis unit of the present invention is configured to: input the query text into the BERT model, the BERT model outputs a semantic representation vector that can represent the semantics of the query text, the semantic representation vector is passed to the classification layer (a fully connected neural network layer), the classification layer calculates the probability that the semantic representation vector belongs to each predefined intent label based on the features of the semantic representation vector, and selects the intent label with the highest probability as the output. This selected intent label is the user's query intent.

[0066] In other embodiments, the first natural language processing model may also use various deep learning models such as RoBERTa, TinyBERT, ALBERT, LSTM, or TextCNN.

[0067] In this embodiment of the invention, the second natural language processing model adopts a named entity recognition (NER) model, which is constructed using the Spacy library.

[0068] Specifically, the slot extraction unit of this invention is configured to: input user query text into a named entity recognition model, which first performs word segmentation and vectorization on the query text, and utilizes its prior language knowledge obtained through pre-training on a large dataset, combined with contextual semantic association analysis, to accurately identify entity boundaries and semantic types with specific meanings in the text. During processing, the named entity recognition model determines the entity category of each word, and uses annotation systems such as BIO to distinguish the start, middle, and non-entity positions of entities, thereby completely capturing composite entities and nested structures. The extracted structured information constitutes the query constraint slots of this invention.

[0069] The data source matching module of this invention includes a data source description vector acquisition unit, an intent vector conversion unit, a similarity calculation unit, and a target data source determination unit. The data source description vector acquisition unit acquires the metadata description vector of the heterogeneous data source, the metadata description vector representing the semantic content that the corresponding data source can support for querying. The intent vector conversion unit converts the query intent into an intent vector. The similarity calculation unit calculates the similarity between the intent vector and the metadata description vector. The target data source determination unit determines data sources with similarity reaching a preset threshold as the target data source.

[0070] Specifically, the data source description vector acquisition unit of this invention is configured to generate a high-dimensional, numerical metadata description vector for each heterogeneous data source. This metadata description vector is a mathematical representation that summarizes the core semantic content of the data source, generated after a deep understanding of the data source's name, field names, field annotations, sample data, and data dictionary through semantic models such as the BERT model. The intent vector conversion unit of this invention converts query intent into vector form in a similar manner.

[0071] The similarity calculation unit of the present invention is configured to use a similarity algorithm to calculate the similarity between the intent vector and each metadata description vector. The calculation result is a value between 0 and 1. The higher the value, the more the user's query intent matches the content that the data source can provide.

[0072] In this embodiment of the invention, the similarity algorithm uses the cosine similarity algorithm.

[0073] The target data source determination unit of the present invention is configured to: set a preset threshold, determine data sources with similarity reaching the preset threshold as target data sources, and filter out one or more target data sources.

[0074] In this embodiment of the invention, the threshold is set to 0.8. It should be noted that the threshold can be adjusted according to actual needs, and there is no limitation thereon.

[0075] The query statement generation module of this invention is connected to the intent analysis module and the data source matching module respectively. Utilizing advanced AI-assisted prompting engineering technology, it dynamically transforms the query intent and query constraint slots into executable query statements (DDL / DML / DSL) for a specific target data source, and sends the generated query statements to the query execution module.

[0076] In this embodiment of the invention, a GPT-3 style prompting process is used, generating a prompt similar to: prompt = f"Generate SQL based on intent:{intent} and slots:{slots}". If the target data source is a relational database MySQL, an SQL statement is generated, such as: SELECT * FROM sales WHERE year=2023; if the target data source is a search engine Elasticsearch, a DSL query is generated, such as: {"query": {"match": {"year": 2023}}}.

[0077] Traditional methods require pre-writing a template for each data source and query intent, which is rigid and costly to maintain. Changes in data source structure or business needs necessitate manual modification and template additions, hindering scalability. This invention, however, employs AI-driven suggestion engineering, dynamically providing query intent, query constraint slots, and data source type as input to the AI ​​model. Based on a deep understanding of natural and programming languages, the AI ​​model generates the most suitable query in real-time, without relying on pre-defined templates. This method is highly flexible; whether facing new query requirements or adapting to a completely new data source, the core system logic does not need to be rewritten. Only the suggestions provided to the model need to be adjusted or expanded, significantly improving the adaptability and scalability of the query system.

[0078] The result analysis module of this invention includes a result summary generation unit, a vector mapping unit, a similarity score calculation unit, and a judgment and feedback unit. The result summary generation unit summarizes the query results and generates result summary text; the vector mapping unit maps the query text to a query vector and the result summary text to a result summary vector; the similarity score calculation unit calculates the similarity score between the query vector and the result summary vector; the judgment and feedback unit compares the similarity score with a preset validity threshold. If the similarity score is lower than the validity threshold, the query result is determined to be invalid, and a feedback signal is sent to the query statement generation module.

[0079] The result summary generation unit of the present invention is configured to: use text summarization technology, such as a Transformer-based generative summarization model, to refine and summarize the lengthy original query results and generate a concise result summary text.

[0080] The vector mapping unit and similarity calculation unit of the present invention are similar in principle to the intent vector conversion unit and similarity calculation unit in the data source matching module, and will not be described again here.

[0081] In this embodiment of the invention, the validity threshold is set to 0.8. It should be noted that the validity threshold can be adjusted according to actual needs, and there is no limitation on this.

[0082] When the query result is determined to be invalid, the judgment and feedback unit of the present invention sends a feedback signal to the query statement generation module to trigger the query statement generation module to regenerate the query statement, and then re-trigger the query execution module and the result analysis module until the query result is determined to be valid.

[0083] The result analysis module of this invention enhances the reliability and robustness of the query system. Firstly, by analyzing the similarity between the query results and the query text, the module automatically identifies and handles query failures, rather than directly presenting irrelevant or erroneous results to the user, significantly improving the professionalism and credibility of the query system. Furthermore, through closed-loop feedback, the query system can automatically learn from errors and adjust in real time, forming a continuously optimizing intelligent cycle and reducing reliance on external manual intervention. Secondly, the result analysis module's similarity analysis of the query results and query text goes beyond simple checks for empty results or grammatical errors; it delves into the semantic level to verify whether the "answer truly answers the question," representing a significant leap in quality assessment. The query system of this invention either provides high-quality, relevant results or proactively acknowledges the limitations of the current results and attempts to correct them, providing a smoother and more intelligent interactive experience.

[0084] The output formatting module of this invention includes a data standardization unit, an output strategy selection unit, and a visualization rendering unit. The data standardization unit converts valid query results into a standardized data frame format; the output strategy selection unit selects an output strategy based on the data type of the valid query results or the output format instruction specified in the query text; the visualization rendering unit, when the output strategy is visualization output, invokes a data visualization engine to generate corresponding charts based on the standardized data frame format; wherein, the output strategy includes at least one of plain text output, structured data output, and visualization chart output.

[0085] The data standardization unit of this invention converts valid query results from different data sources and in various forms into a unified, regular, and standardized data frame format, providing a foundation for subsequent visualization rendering.

[0086] The output strategy selection unit of this invention intelligently selects the most suitable output strategy based on the analysis of result data and user requirements.

[0087] Specifically, if the user explicitly specifies the output format in the query text, such as by inputting instructions like "draw as a bar chart," "represent as a line chart," "display as a table," or "just tell me the numbers," the output strategy selection unit will prioritize the user's explicit wishes and select the corresponding output strategy. This is the highest priority instruction. If the user does not explicitly specify the output format in the query text, the output will be based on the data type and characteristics of the data itself. This is the core of automatic decision-making.

[0088] In this embodiment of the invention, when the user does not explicitly specify the output format in the query text, the output strategy selection unit is configured as follows: if the valid query result is only a single numerical value or short text, then plain text output is selected; if the valid query result is a structured list of multiple records (e.g., a list containing various types of information), then structured data output, such as a table, is selected; if the valid query result contains numerical, aggregateable fields and clearly carries the intent of comparison, distribution, or trend analysis (e.g., monthly sales revenue, percentage of different categories), then visual chart output is selected.

[0089] The visualization rendering unit of this invention has a built-in or connected data visualization engine, such as Apache ECharts, Matplotlib, or G2Plot, to receive standardized data frame formats from the data processing unit. Based on the data characteristics of the valid query results, it selects the most suitable chart type; for example, a bar chart for comparing data, a line chart for analyzing trends, and a pie chart for viewing percentages. By calling the engine's API, attributes such as titles, legends, and axes are set, ultimately generating a complete, interactive, or static chart object.

[0090] For complex trend comparisons, charts are far more intuitive than numerical tables, while for simple queries, providing a direct text answer is faster than rendering a chart. This invention's output formatting module ensures that users always receive information in the most appropriate way, improving user experience and information retrieval efficiency, even when no output format is specified. It enables the query system to automatically select the best visualization scheme for the data, ensuring that the tables and charts presented to the user follow a unified style and specification regardless of the underlying data source. This reduces the tedious manual selection of chart types and configuration parameters, making the data analysis process more intelligent and automated, and supports multiple output strategies to adapt to diverse user needs.

[0091] Furthermore, the demand input module, intent analysis module, data source matching module, query statement generation module, query execution module, result analysis module, and output formatting module of this invention communicate and transmit data asynchronously through message middleware and RESTful API interfaces to achieve asynchronous connections and ensure smooth data transmission.

[0092] The modules in this invention do not communicate directly with each other. Instead, they exchange messages through message brokers such as RabbitMQ, Apache Kafka, and Redis. For example, after completing its work, module A publishes the result as a "message" to a message queue. Module A can then immediately return to process the next task without waiting. Module B listens to this message queue. When a new message arrives, the message broker will proactively push the message to module B, or module B will proactively pull the message, after which module B will begin processing.

[0093] As can be seen from the above, the demand input module, intent analysis module, data source matching module, query statement generation module, query execution module, result analysis module, and the built-in units of each module of the present invention have very low coupling. Whether adding or deleting modules or units, it will not affect other parts, and the maintainability is very high.

[0094] As shown in Figure 2, this application embodiment also provides a multi-source data query method. This multi-source data query method is implemented using the multi-source data query system described in the above embodiment. The solution provided by this method is similar to the implementation scheme described in the above multi-source data query system embodiment. Therefore, the specific limitations in the multi-source data query method embodiment provided below can be found in the limitations of the multi-source data query system above, and will not be repeated here. Figure 2 is a flowchart illustrating the multi-source data query method. This flowchart only shows the logical order of the method described in this embodiment. Under the premise of no conflict, in other possible embodiments of the present invention, the steps shown or described can be completed in a different order than that shown in Figure 2.

[0095] Referring to Figure 2, the method of this embodiment of the invention specifically includes the following steps:

[0096] Step S101: Receive user requests and convert them into query text.

[0097] User needs can be input via voice or text, overcoming the limitation of traditional query systems that only support text input, thus realizing intelligent query input and making it more convenient for users.

[0098] Step S102: Perform semantic analysis on the query text to extract the user's query intent and extract query constraint slots.

[0099] In this embodiment of the invention, step S102 specifically includes:

[0100] The query text is input into the first natural language processing model, and an intent label representing the semantic category of the query text is output as the user's query intent.

[0101] The query text is input into the second natural language processing model to extract key entity information from the query text, which is then used as the query constraint slot.

[0102] In this embodiment of the invention, the first natural language processing model adopts the BERT model, and the second natural language processing model adopts the named entity recognition model.

[0103] Step S103: Based on the user's query intent, match at least one target data source from multiple heterogeneous data sources.

[0104] The heterogeneous data sources include relational databases such as MySQL and PostgreSQL, as well as search engines such as Elasticsearch.

[0105] In this embodiment of the invention, step S103 specifically includes:

[0106] Obtain the metadata description vector of the heterogeneous data source, wherein the metadata description vector represents the semantic content that the corresponding data source can support for querying;

[0107] Transform the query intent into an intent vector;

[0108] Calculate the similarity between the intent vector and the metadata description vector;

[0109] Data sources with a similarity threshold are identified as the target data source.

[0110] Step S104: Based on the query intent, the query constraint slot, and the type of the target data source, generate a query statement through AI prompts.

[0111] This invention utilizes advanced prompting engineering technology to dynamically transform the query intent and query constraint slots into executable query statements (DDL / DML / DSL) for a specific target data source.

[0112] In this embodiment of the invention, a GPT-3 style prompting process is used, and the generated prompt is similar to: prompt = f"Generate SQL based on intent:{intent} and slots:{slots}".

[0113] Step S105: Execute the query statement on the target data source to obtain the query results.

[0114] Step S106: Perform a validity analysis on the query results; if the validity analysis passes, render and output the query results according to the preset output strategy; if the validity analysis fails, regenerate the query statement.

[0115] In this embodiment of the invention, step S106 specifically includes:

[0116] The query results are summarized to generate a result summary text;

[0117] Map the query text to a query vector, and map the result summary text to a result summary vector;

[0118] Calculate the similarity score between the query vector and the result summary vector;

[0119] The similarity score is compared with a preset validity threshold. If the similarity score is lower than the validity threshold, the query result is determined to be invalid, and steps S104 to S106 are re-executed until the query result passes the validity analysis.

[0120] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0121] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, the above terms have no special meaning and therefore cannot be construed as limiting the scope of protection of this application.

[0122] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A multi-source data query system, characterized in that, include: The requirement input module is used to receive user requirements and convert them into query text; The intent analysis module is used to perform semantic analysis on the query text, extract the user's query intent, and extract query constraint slots; The system includes a data source matching module for matching at least one target data source from multiple pre-configured heterogeneous data sources based on the user's query intent; wherein the heterogeneous data sources include relational databases and search engines; a query statement generation module for generating a query statement based on the query intent, the query constraint slots, and the type of the target data source, using AI-powered suggestions; a query execution module for executing the query statement on the target data source and obtaining query results; a result analysis module for performing validity analysis on the query results and sending a feedback signal to the query statement generation module to regenerate the query statement when the query results are invalid; and an output formatting module for rendering and outputting valid query results according to a preset output strategy.

2. The multi-source data query system according to claim 1, characterized in that, It also includes a knowledge base storage module; the knowledge base storage module is used to store valid query results to a graph database or an Elasticsearch index.

3. The multi-source data query system according to claim 1, characterized in that, The input module includes: a speech recognition unit for converting the user's speech input into text; and a text input unit for receiving natural language text input by the user.

4. The multi-source data query system according to claim 1, characterized in that, The intent analysis module includes: an intent analysis unit, used to input the query text into a first natural language processing model and output an intent label representing the semantic category of the query text as the user's query intent; and a slot extraction unit, used to input the query text into a second natural language processing model and extract key entity information from the query text as the query constraint slot.

5. The multi-source data query system according to claim 4, characterized in that, The first natural language processing model is the BERT model; the second natural language processing model is the named entity recognition model.

6. The multi-source data query system according to claim 1, characterized in that, The data source matching module includes: a data source description vector acquisition unit, used to acquire the metadata description vector of the heterogeneous data source, wherein the metadata description vector represents the semantic content that the corresponding data source can support for querying; an intent vector conversion unit, used to convert the query intent into an intent vector; a similarity calculation unit, used to calculate the similarity between the intent vector and the metadata description vector; and a target data source determination unit, used to determine the data source with a similarity reaching a preset threshold as the target data source.

7. The multi-source data query system according to claim 1, characterized in that, The result analysis module includes: a result summary generation unit, used to summarize the query results and generate result summary text; a vector mapping unit, used to map the query text to a query vector and the result summary text to a result summary vector; a similarity score calculation unit, used to calculate the similarity score between the query vector and the result summary vector; and a judgment and feedback unit, used to compare the similarity score with a preset validity threshold, and if the similarity score is lower than the validity threshold, determine that the query result is invalid and send a feedback signal to the query statement generation module.

8. The multi-source data query system according to claim 7, characterized in that, The output formatting module includes: a data standardization unit for converting valid query results into a standardized data frame format; an output strategy selection unit for selecting an output strategy based on the data type of the valid query results or the output format instruction specified in the query text; and a visualization rendering unit for calling a data visualization engine and generating corresponding charts based on the standardized data frame format when the output strategy is visualization output; wherein the output strategy includes at least one of plain text output, structured data output, and visualization chart output.

9. The multi-source data query system according to claim 1, characterized in that, The requirement input module, intent analysis module, data source matching module, query statement generation module, query execution module, result analysis module, and output formatting module communicate and transmit data asynchronously through message middleware and / or RESTful API interfaces.

10. A multi-source data query method, characterized in that, include: Receive user requests and convert them into query text; Semantic analysis is performed on the query text to extract the user's query intent and extract query constraint slots; Based on the user's query intent, at least one target data source is matched from multiple heterogeneous data sources; based on the query intent, the query constraint slots, and the type of the target data source, a query statement is generated through AI prompting; the query statement is executed on the target data source to obtain query results; and the validity of the query results is analyzed. If the validity analysis passes, the query results will be rendered and output according to the preset output strategy; if the validity analysis fails, the query statement will be regenerated.

Images