Multi-data-source joint query method, device and equipment based on large language model

Abstract

The invention provides a multi-data-source joint query method, device and equipment based on a large language model, and the method comprises the steps: in response to a natural language query request, extracting database table structure information of a target data source from a metadata knowledge base; inputting the natural language query request and the database table structure information into a large language model, and outputting a cross-database execution plan; disassembling the cross-library execution plan into an independent query subset which is executed concurrently and a sequential query subset which has a dependency relationship; concurrently executing multi-database dialect SQL (Structured Query Language) generation and query operation on the independent query subset to obtain a first query result; performing pre-query on the sequential query subset according to a dependency relationship sequence, and injecting a query result into a subsequent query to obtain a second query result; and fusing the first query result and the second query result and outputting a standardized query result. According to the method, the technical threshold is reduced, cross-library joint query is realized, error accumulation in an intermediate link is avoided, and the query accuracy is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data query, in particular to a multi-data source joint query method, device and equipment based on a large language model. BACKGROUND

[0002] Modern enterprises are facing the severe challenge of multi-data source heterogeneous query. Enterprise business data is scattered in different databases such as MySQL of CRM system, Oracle of ERP system, and PostgreSQL of supply chain management, forming a serious data island phenomenon.

[0003] Business analysts need to access multiple heterogeneous databases for cross-departmental collaborative analysis, but traditional solutions require mastering multiple database syntax, with a very high technical threshold. Intelligent reporting systems need to automatically aggregate information from multiple data sources, but related technologies require writing special adaptation logic for each data source, and the maintenance complexity increases exponentially with the number of data sources. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a multi-data source joint query method, device and equipment based on a large language model, which reduces the technical threshold, realizes cross-database joint query, reduces the maintenance complexity, avoids the error accumulation of intermediate links, and significantly improves the query accuracy.

[0005] In a first aspect, the present application provides a multi-data source joint query method based on a large language model, which comprises: extracting database table structure information of a target data source from a metadata knowledge base in response to a natural language query request input by a user; inputting the natural language query request and the database table structure information into a large language model to output a cross-database execution plan containing multiple sub-query tasks; based on the logical dependency relationship of the cross-database execution plan, decomposing the cross-database execution plan into multiple concurrent independent query subsets and sequential query subsets with dependency relationships; performing multi-database dialect SQL generation and query operations on the independent query subsets concurrently to obtain first query results; sequentially executing pre-query according to the dependency relationship of the sequential query subsets and injecting the query results of the pre-query into subsequent queries to obtain second query results; and fusing the first query results and the second query results and outputting standardized query results.

[0006] In a preferred embodiment of the present application, the above-mentioned inputting the natural language query request and the database table structure information into the large language model to output the cross-database execution plan containing multiple sub-query tasks comprises: constructing a structured Prompt template, the Prompt template containing database type identification, table structure field description and query requirement description; filtering table field information related to the query through a metadata dynamic injection mechanism; and calling the large language model to generate native SQL statements of the target database dialect.

[0007] In the preferred embodiment of the present application, the above-mentioned splitting the cross-database execution plan into multiple concurrent execution independent query subsets and sequential query subsets with dependencies includes: parsing the JOIN operation and nested query structure of the cross-database execution plan; identifying the pre-query that is executed in priority and the subsequent query that takes the query result of the pre-query as an input parameter; classifying the pre-query and the subsequent query into a sequential query subset; and classifying the query without data dependency into an independent query subset.

[0008] In the preferred embodiment of the present application, the above-mentioned identification of the pre-query is based on: the query result being referenced by the subsequent query as a filter condition; the query result containing a cross-database association key required by the subsequent query; and the query result being used for the aggregation calculation input of the subsequent query.

[0009] In the preferred embodiment of the present application, the above-mentioned concurrent execution of the independent query subset and the generation of the multi-database dialect SQL includes: identifying the target database type and loading the dialect feature configuration; adapting the database-specific syntax rules; and performing syntax verification and security check after the generation of the multi-database dialect SQL.

[0010] In the preferred embodiment of the present application, the above-mentioned fusion of the first query result and the second query result and the output of the standardized query result includes: performing data type uniformization conversion on the first query result and the second query result returned by the heterogeneous data sources; aligning the cross-database fields based on the field mapping relationship in the metadata knowledge base; performing JOIN operation according to the association logic of the cross-database execution plan; performing aggregation calculation and deduplication processing on the fused first query result and second query result to obtain the final result; and converting the final result into a standardized format for output.

[0011] In the preferred embodiment of the present application, the method further includes: establishing a multi-level result caching mechanism; and establishing the multi-level result caching mechanism includes: setting a caching strategy based on the data query frequency and the data update frequency; setting a cache invalidation strategy based on the data update frequency; and pre-executing high-frequency queries and caching hot data sets.

[0012] In a second aspect, the embodiments of the present application also provide a multi-data-source joint query device based on a large language model, comprising: a request response module configured to extract database table structure information of a target data source from a metadata knowledge base in response to a natural language query request input by a user; a cross-database execution plan output module configured to input the natural language query request and the database table structure information into the large language model and output a cross-database execution plan containing a plurality of sub-query tasks; a cross-database execution plan disassembly module configured to disassemble the cross-database execution plan into a plurality of concurrently executed independent query subsets and sequentially ordered query subsets with a dependency relationship based on a logical dependency relationship of the cross-database execution plan; a first query result obtaining module configured to perform multi-database dialect SQL generation and query operations on the independent query subsets concurrently to obtain a first query result; a second query result obtaining module configured to execute a preceding query on the sequentially ordered query subsets in a sequence according to the dependency relationship and inject a query result of the preceding query into a subsequent query to obtain a second query result; and a standardized query result output module configured to fuse the first query result and the second query result and output a standardized query result.

[0013] In a third aspect, the embodiments of the present application also provide an electronic device, comprising a processor and a memory, the memory storing computer executable instructions capable of being executed by the processor, and the processor executes the computer executable instructions to implement the multi-data-source joint query method based on the large language model of the first aspect.

[0014] In a fourth aspect, the embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium storing computer executable instructions, and the computer executable instructions, when invoked and executed by a processor, cause the processor to implement the multi-data-source joint query method based on the large language model of the first aspect.

[0015] The embodiments of the present application bring the following beneficial effects: The embodiment of the present application provides a multi-data source joint query method, device and equipment based on a large language model, extracts database table structure information of a target data source from a metadata knowledge base in response to a natural language query request input by a user, inputs the natural language query request and the database table structure information into the large language model, outputs a cross-database execution plan containing multiple sub-query tasks, based on logical dependency relationships of the cross-database execution plan, decomposes the cross-database execution plan into multiple independently queried subsets and sequentially queried subsets with dependency relationships, concurrently executes multiple database dialect SQL generation and query operations on the independently queried subsets to obtain a first query result, executes a pre-query on the sequentially queried subsets in a sequence according to the dependency relationships and injects a query result of the pre-query into a subsequent query to obtain a second query result, and fuses the first query result and the second query result and outputs a standardized query result. In this way, the technical threshold is reduced, cross-database joint query is realized, the maintenance complexity is reduced, error accumulation of intermediate links is avoided, and the query accuracy is significantly improved.

[0016] Other features and advantages of the present disclosure will be described in the following description, or can be inferred from the description, or can be determined without doubt, or can be known by implementing the above-mentioned technologies of the present disclosure.

[0017] In order to make the above-mentioned purposes, features and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0019] Figure 1 A flowchart of a multi-data source joint query method based on a large language model provided by the embodiment of the present application; Figure 2 A flowchart of another multi-data source joint query method based on a large language model provided by the embodiment of the present application; Figure 3 A structural schematic diagram of a multi-data source joint query device based on a large language model provided by the embodiment of the present application; Figure 4 A structural schematic diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described below in connection with the drawings, which are apparently only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0021] Modern enterprises are facing the severe challenge of multi-data source heterogeneous query. Enterprise business data is scattered in different databases such as MySQL of CRM system, Oracle of ERP system, and PostgreSQL of supply chain management, forming a serious data island phenomenon. Business analysts need to access multiple heterogeneous databases for cross-department collaborative analysis, but the traditional solution requires mastering multiple database syntax, with a very high technical threshold. Intelligent reporting systems need to automatically gather information from multiple data sources, but existing technologies require writing special adaptation logic for each data source, and the maintenance complexity increases exponentially with the number of data sources.

[0022] There are two types of technical solutions for current multi-data source query, but both have significant limitations. The first type is the traditional database gateway solution, which realizes multi-data source access through SQL proxy forwarding, but it is essentially a simple SQL transparent transmission and cannot handle cross-database joint queries. Each new data source requires the development of a special adapter. The second type is the Text-to-SQL solution, which generates SQL through natural language parsing, but is limited to a single database instance. It uses a multi-stage processing mode of "natural language → semantic parsing → AST construction → SQL generation", with serious error accumulation, and cannot support real cross-heterogeneous database joint analysis. Both of these two types of solutions cannot solve the complex query requirements in the multi-data source environment, especially in handling fuzzy semantics, database dialect adaptation, and real-time cross-database joint query.

[0023] There are four key technical defects in existing technical solutions. First, the database gateway solution can only do SQL transparent transmission and cannot realize real cross-database joint query. Each new data source requires the development of a special adapter, with high expansion cost. Second, the Text-to-SQL solution is limited to a single database and uses a multi-stage processing link (semantic parsing → AST construction → code generation). Each link can produce error accumulation, and cannot handle cross-database scenarios. Third, both types of solutions cannot effectively handle fuzzy semantics and uncertain query inputs, with too high technical requirements for users. Fourth, in terms of database dialect adaptation and dynamic expansion, existing solutions require a lot of manual configuration and code maintenance work, with exponential increase in operation and maintenance complexity with the number of data sources.

[0024] Based on this, the embodiment of the application provides a multi-data source joint query method, device and equipment based on a large language model, which can extract database table structure information of a target data source from a metadata knowledge base in response to a natural language query request input by a user, input the natural language query request and the database table structure information into a large language model, output a cross-database execution plan containing multiple sub-query tasks, based on a logical dependency relationship of the cross-database execution plan, decompose the cross-database execution plan into multiple concurrently executed independent query subsets and sequentially ordered query subsets with a dependency relationship, concurrently execute multi-database dialect SQL generation and query operations on the independent query subsets to obtain a first query result, sequentially execute a preceding query on the sequentially ordered query subsets according to the dependency relationship and inject a query result of the preceding query into a subsequent query to obtain a second query result, and fuse the first query result and the second query result and output a standardized query result. In this way, the technical threshold is reduced, cross-database joint query is realized, the maintenance complexity is reduced, error accumulation of intermediate links is avoided, and the query accuracy is significantly improved.

[0025] To facilitate the understanding of the present embodiment, first, a multi-data source joint query method based on a large language model disclosed by the present embodiment is introduced in detail.

[0026] Embodiment 1 The embodiment of the application provides a multi-data source joint query method based on a large language model, Figure 1 A flowchart of the multi-data source joint query method based on a large language model provided by the embodiment of the application is shown in the figure. Figure 1 As shown in the figure, the multi-data source joint query method based on a large language model can include the following steps: Step S101, in response to a natural language query request input by a user, extracting database table structure information of a target data source from a metadata knowledge base.

[0027] The natural language query request represents the query intention of the user, and the query preprocessor can perform basic cleaning, format standardization and sensitive information filtering on the natural language query request input by the user. The context manager maintains the historical context of multiple rounds of dialogues, and supports progressive query refinement. The intent classifier quickly identifies the query type, including data query, statistical analysis, report generation and different business scenarios.

[0028] Step S102, inputting the natural language query request and the database table structure information into a large language model, and outputting a cross-database execution plan containing multiple sub-query tasks.

[0029] Among them, the Prompt template containing metadata information can be constructed, the user query and the database schema information are injected into the template, the large language model is called to generate the SQL of the target database dialect, the generated SQL is syntax checked and security checked, and finally the executable SQL statement and the execution plan are returned.

[0030] Specifically, the table structure information of the target data source is obtained from the metadata cache, the field name, data type, constraint condition, and index information of the related table are extracted, the related tables and fields are intelligently filtered according to the user query content, the metadata is formatted into structured text that can be understood by the large model, and finally inserted into the Prompt of SQL generation.

[0031] Step S103, based on the logical dependency relationship of the cross-database execution plan, the cross-database execution plan is decomposed into multiple concurrent independent query subsets and sequential query subsets with dependency relationship.

[0032] Specifically, the cross-database execution plan is decomposed into multiple concurrent independent query subsets and sequential query subsets with dependency relationship, which can include: parsing the JOIN operation and nested query structure of the cross-database execution plan; identifying the pre-query to be executed first, and the subsequent query taking the query result of the pre-query as an input parameter; classifying the pre-query and the subsequent query into sequential query subsets; classifying the query without data dependency into independent query subsets.

[0033] Among them, the identification of the pre-query can include: the query result is referenced by the subsequent query as a filter condition; the query result contains the cross-database association key required by the subsequent query; the query result is used for the aggregation calculation input of the subsequent query.

[0034] Step S104, the independent query subset is executed concurrently to generate and query the multi-database dialect SQL, and the first query result is obtained.

[0035] Among them, the MySQL dialect features include supporting LIMIT syntax, using backticks to enclose field names, and supporting specific date functions and string functions. The Oracle dialect features include using ROWNUM for paging, supporting DUAL virtual table, and using TO_DATE and other Oracle-specific functions. The PostgreSQL dialect features include supporting LIMIT and OFFSET, using double quotes to enclose field names, and supporting array and JSON data types.

[0036] Step S105, the pre-query is executed in the order of the dependency relationship for the sequential query subset, and the query result of the pre-query is injected into the subsequent query to obtain the second query result.

[0037] Step S106, fuse the first query result and the second query result and output the standardized query result.

[0038] Among them, the distributed query executor executes the generated SQL statement on multiple data sources in parallel. The result set standardization processor unifies the format and data type of the results returned by different databases. The data fusion engine supports operations such as merging, deduplication, and aggregation of cross-data-source results. The multi-format output adapter supports multiple result output formats such as JSON, CSV, and Excel.

[0039] In practical applications, the embodiments of the present application can realize real cross-heterogeneous database joint query capability through intelligent metadata unified management and distributed query execution. By using the powerful semantic understanding capability of the large language model, fuzzy semantics and uncertain query inputs are supported, and the query accuracy is significantly improved. Through the carefully designed Prompt engineering technology, the adaptation time of new databases is shortened from several weeks to several hours, greatly reducing the system maintenance complexity.

[0040] The multi-data-source joint query method based on a large language model provided by the embodiments of the present application can extract database table structure information of a target data source from a metadata knowledge base in response to a natural language query request input by a user, input the natural language query request and the database table structure information into a large language model, output a cross-database execution plan containing multiple sub-query tasks, based on the logical dependency relationship of the cross-database execution plan, decompose the cross-database execution plan into multiple concurrent independent query subsets and sequential query subsets with dependency relationships, concurrently execute the independent query subsets to generate and query multiple database dialect SQLs, obtain a first query result, execute a pre-query in order according to the dependency relationship of the sequential query subset and inject the query result of the pre-query into a subsequent query to obtain a second query result, and fuse the first query result and the second query result and output a standardized query result. In this way, the technical threshold is reduced, cross-database joint query is realized, maintenance complexity is reduced, error accumulation in intermediate links is avoided, and query accuracy is significantly improved.

[0041] Embodiment 2 The embodiments of the present application also provide another multi-data-source joint query method based on a large language model; the method is implemented based on the method of the above-mentioned embodiments.

[0042] Figure 2 The flowchart of another multi-data-source joint query method based on a large language model provided by the embodiments of the present application is shown in Figure 2 The multi-data-source joint query method based on a large language model can include the following steps: Step 201, in response to a natural language query request input by a user, extracting database table structure information of a target data source from a metadata knowledge base.

[0043] Step S202, input the natural language query request and the database table structure information into the large language model, and output a cross-database execution plan containing multiple sub-query tasks.

[0044] Specifically, inputting the natural language query request and the database table structure information into the large language model to output a cross-database execution plan containing multiple sub-query tasks can include: constructing a structured Prompt template, the Prompt template containing database type identification, table structure field description and query requirement description; filtering table field information related to the query through a metadata dynamic injection mechanism; calling the large language model to generate native SQL statements of the target database dialect.

[0045] Step S203, based on the logical dependency relationship of the cross-database execution plan, the cross-database execution plan is decomposed into multiple concurrent independent query subsets and sequential query subsets with dependency relationship.

[0046] Step S204, concurrently executing multi-database dialect SQL generation and query operation on the independent query subset to obtain a first query result.

[0047] Specifically, concurrently executing multi-database dialect SQL generation on the independent query subset can include: identifying the target database type and loading the dialect feature configuration; adapting the database specific syntax rules; performing syntax verification and security check after generating the multi-database dialect SQL.

[0048] Wherein, adapting the database specific syntax rules can represent that the multi-dialect SQL generation engine generates native SQL statements for different databases such as MySQL, Oracle, PostgreSQL, etc.

[0049] Wherein, the metadata injector dynamically injects the data source meta information into the large model context. The SQL quality evaluator performs syntax checking and logical verification on the generated SQL.

[0050] Specifically, the post-generation verification process includes syntax checking, using a SQL parser to verify syntax correctness; security check, detecting SQL injection, dangerous operation and other security risks; performance evaluation, analyzing query complexity and estimating execution time; logical verification, checking field matching degree and query logic rationality; finally generating an execution report and risk prompt.

[0051] Step S205, executing the pre-query in the order of the dependency relationship for the sequential query subset and injecting the query result of the pre-query into the subsequent query to obtain a second query result.

[0052] Step S206, fusing the first query result and the second query result and outputting a standardized query result.

[0053] Specifically, fusing the first query result and the second query result and outputting the standardized query result can include: performing data type unification conversion on the first query result and the second query result returned by the heterogeneous data sources; aligning cross-database fields based on the field mapping relationship in the metadata knowledge base; performing a JOIN operation according to the association logic of the cross-database execution plan; performing aggregation calculation and deduplication processing on the fused first query result and second query result to obtain a final result; and converting the final result into a standardized format for output.

[0054] Step S207, a multi-level result cache mechanism is established.

[0055] Specifically, the multi-level result cache mechanism can include: setting a cache strategy based on data query frequency and data update frequency; setting a cache invalidation strategy based on data update frequency; pre-executing high-frequency queries and caching hot data sets.

[0056] Among them, the intelligent cache strategy is adopted, the cache strategy is formulated based on the query frequency and the data update frequency; the incremental update mechanism is adopted, when the data source changes, the related cache is intelligently invalidated; the cache preheating technology is adopted, the common query cache hot data is pre-executed; the hierarchical cache architecture is adopted, the multi-level structure of memory cache, Redis cache and database cache is constructed.

[0057] Further, through the database connection pool, an independent connection pool is maintained for each data source to support connection reuse; through the resource isolation mechanism, the query resources of different tenants and applications are isolated to prevent mutual influence; through dynamic scaling, the connection pool size and the number of model instances are automatically adjusted according to the load; through the fuse protection mechanism, when a data source fails, the overall system is automatically fused and protected.

[0058] Further, through the type cache strategy, the generated results of similar queries are cached to avoid repeated reasoning; through the batch processing mechanism, batch processing of multiple query requests is supported to improve throughput; through the reasoning acceleration technology, model quantization and reasoning optimization are used to reduce response time; through multi-model load balancing, multiple model instances are deployed to realize load dispersion and fault tolerance.

[0059] Further, through the AST analysis auxiliary module, log recording and problem tracing are performed. The SQL syntax tree generates an AST analysis of the SQL generated by the large model. The query intention extraction extracts semantic information such as query target, condition, and sorting from the AST. The execution track record records the complete conversion process from natural language to SQL. The problem diagnosis supports locating the problem cause when the SQL execution fails through AST analysis.

[0060] Further, the query history and learning mechanism includes query pattern recognition, analysis of user history queries to identify common query patterns; SQL template library construction, construction of reusable SQL templates based on successful query cases; feedback learning loop, collection of user feedback on generated SQL for continuous optimization of Prompt strategy; error case analysis, record of failed cases to avoid repeated errors.

[0061] The following Table 1 is a functional comparison table of the embodiment of the present application and related technologies: Table 1:

[0062] Based on Table 1 above, functional analysis is performed: Core limitations of the database gateway solution: The database gateway is essentially an SQL transparent proxy layer that can only route user SQL statements to the target database for execution, and cannot implement cross-database data correlation analysis. When faced with enterprise-level common requirements such as "correlation analysis of customer data in the sales department CRM system and order data in the finance department ERP system", the database gateway can only query the two databases separately, and the data integration work completely relies on manual processing by the user, and cannot provide core functions such as cross-database JOIN, data type unification, and field alignment.

[0063] Core limitations of the Text2SQL solution: Traditional Text2SQL technology is strictly limited to a single database environment, using a multi-stage processing link of "natural language → semantic analysis → AST construction → SQL code generation". Processing errors at each stage are accumulated and passed on, resulting in a final query accuracy of only 60-70%. More importantly, the Text2SQL architecture design cannot handle cross-database scenarios, and is completely powerless when faced with cross-database analysis requirements such as "comparing user behavior data in MySQL with transaction data trends in Oracle".

[0064] Technical breakthrough of the embodiment of the present application: The embodiment of the present application realizes end-to-end generation from natural language to multi-database dialect SQL, completely bypassing the traditional multi-stage processing link, effectively avoiding error accumulation problems. Through the deep semantic understanding ability of LLM, the system can intelligently analyze user intent and automatically disassemble query logic, generate corresponding dialect SQL statements for different databases, and finally realize the unified integration of cross-database results through intelligent fusion algorithms, truly solving the core problem of enterprise-level multi-data source collaborative analysis.

[0065] The following Table 2 is a comparison table of the quantitative improvement effects of the embodiment of the present application and related technologies: Table 2:

[0066] Based on Table 2 above, the technical innovation breakthrough dimension analysis is carried out: the embodiments of the present application realize four-dimensional fundamental breakthrough compared with the prior art. The first is the query processing mode breakthrough, which jumps from "single database transparent transmission" and "single database Text2SQL" to "cross-database intelligent joint query", solving the core problem of enterprise-level multi-data source collaborative analysis. The second is the technical architecture breakthrough, using end-to-end LLM to directly generate to replace the multi-stage processing link, which improves the query accuracy from 60-70% to 85-95%. The third is the scalability breakthrough, which realizes the automatic identification and adaptation of new data sources through the intelligent dialect adaptation mechanism, shortening the expansion cycle from weeks to hours. The fourth is the user experience breakthrough, which supports fuzzy and uncertain input through deep semantic understanding, and completely reduces the technical use threshold.

[0067] Embodiment 3 Corresponding to the above method embodiments, the embodiments of the present application provide a multi-data source joint query device based on a large language model, Figure 3 The structure diagram of the multi-data source joint query device based on a large language model provided by the embodiments of the present application is shown as Figure 3 The multi-data source joint query device based on a large language model can include: The request response module 301 is configured to extract the database table structure information of the target data source from the metadata knowledge base in response to the natural language query request input by the user. The cross-database execution plan output module 302 is configured to input the natural language query request and the database table structure information into the large language model and output a cross-database execution plan containing multiple sub-query tasks.

[0068] The cross-database execution plan decomposition module 303 is configured to decompose the cross-database execution plan into multiple concurrent independent query subsets and sequentially dependent query subsets based on the logical dependency relationship of the cross-database execution plan.

[0069] The first query result obtaining module 304 is configured to perform multi-database dialect SQL generation and query operations on the independent query subsets in parallel to obtain the first query result.

[0070] The second query result obtaining module 305 is configured to execute the pre-query in the order of the dependency relationship for the sequential query subsets and inject the query result of the pre-query into the subsequent query to obtain the second query result.

[0071] The standardized query result output module 306 is configured to fuse the first query result and the second query result and output the standardized query result.

[0072] The multi-data source joint query device based on the large language model provided by the embodiment of the application can extract database table structure information of a target data source from a metadata knowledge base in response to a natural language query request input by a user, input the natural language query request and the database table structure information into a large language model, output a cross-database execution plan containing multiple sub-query tasks, based on the logical dependency relationship of the cross-database execution plan, decompose the cross-database execution plan into multiple concurrent independent query subsets and sequential query subsets with a dependency relationship, perform multi-database dialect SQL generation and query operations on the independent query subsets concurrently to obtain a first query result, execute a preceding query in a sequential order according to the dependency relationship of the sequential query subsets and inject the query result of the preceding query into a subsequent query to obtain a second query result, and fuse the first query result and the second query result and output a standardized query result. In this way, the technical threshold is reduced, cross-database joint query is realized, the maintenance complexity is reduced, error accumulation in intermediate links is avoided, and the query accuracy is significantly improved.

[0073] In some embodiments, the cross-database execution plan output module is further configured to construct a structured Prompt template, the Prompt template containing database type identification, table structure field description and query requirement description; filter table field information related to the query through a metadata dynamic injection mechanism; and call the large language model to generate a native SQL statement of the target database dialect.

[0074] In some embodiments, the cross-database execution plan decomposition module is further configured to analyze JOIN operations and nested query structures of the cross-database execution plan; identify a preceding query to be executed preferentially and a subsequent query taking the query result of the preceding query as an input parameter; divide the preceding query and the subsequent query into a sequential query subset; and divide a query without data dependency into an independent query subset.

[0075] In some embodiments, the cross-database execution plan decomposition module is further configured to query a result referenced by a subsequent query as a filter condition; the query result contains a cross-database association key required by the subsequent query; and the query result is used for input of an aggregation calculation of the subsequent query.

[0076] In some embodiments, the first query result obtaining module is further configured to identify a target database type and load dialect characteristic configurations; adapt to database-specific syntax rules; and perform syntax verification and security checks after generating a multi-database dialect SQL.

[0077] In some embodiments, the standardized query result output module is further configured to perform data type uniformization conversion on the first query result and the second query result returned by the heterogeneous data sources; align cross-database fields based on field mapping relationships in the metadata knowledge base; perform a JOIN operation according to the associated logic of the cross-database execution plan; perform aggregation calculation and deduplication processing on the fused first query result and second query result to obtain a final result; and convert the final result into a standardized format for output.

[0078] In some embodiments, the standardized query result output module is further configured to set a cache strategy based on data query frequency and data update frequency; set a cache invalidation strategy based on data update frequency; pre-execute high-frequency queries and cache hot data sets.

[0079] The device provided by the embodiment of the application has the same implementation principle and technical effects as the foregoing method embodiment, and for brevity of description, the part not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiment.

[0080] Embodiment 4 The embodiment of the application further provides an electronic device for running the multi-data source joint query method based on a large language model. Figure 4 As shown in a structural schematic diagram of an electronic device, the electronic device includes a memory 400 and a processor 401, wherein the memory 400 is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor 401 to implement the multi-data source joint query method based on a large language model.

[0081] Further, Figure 4 As shown in the structural schematic diagram of the electronic device, the electronic device further includes a bus 402 and a communication interface 403, and the processor 401, the communication interface 403 and the memory 400 are connected through the bus 402.

[0082] The memory 400 can include a high-speed random access memory (RAM) and can also include a non-volatile memory, for example, at least one disk memory. The communication between the system network element and at least one other network element is realized through at least one communication interface 403 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used. The bus 402 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 4 In the foregoing description, only one bidirectional arrow is used to represent one bus or one type of bus.

[0083] The processor 401 can be an integrated circuit chip having a signal processing capability. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 401 or the instruction in the form of software. The processor 401 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiment of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiment of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 400, and the processor 401 reads the information in the memory 400, and combines the hardware to complete the steps of the method of the above embodiment.

[0084] The embodiment of the present application also provides a computer readable storage medium, the computer readable storage medium stores computer executable instructions, when the computer executable instructions are called and executed by the processor, the computer executable instructions cause the processor to implement the above-mentioned multi-data source joint query method based on a large language model. For specific implementation, please refer to the method embodiment, which will not be repeated here.

[0085] The computer program product for performing the multi-data source joint query method based on the large language model provided by the embodiment of the present application includes a computer readable storage medium storing non-volatile program code executable by the processor. The instructions included in the program code can be used to execute the method described in the foregoing method embodiment. For specific implementation, please refer to the method embodiment, which will not be repeated here.

[0086] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

[0087] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. The described device embodiments are merely schematic, and for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electric, mechanical or other forms.

[0088] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. In actual implementation, some or all of the units can be selected according to the actual needs to achieve the purposes of the embodiments of the present application.

[0089] In addition, each function unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit.

[0090] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art, or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0091] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the present application, and the protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still make modifications or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features therein, within the technical range disclosed by the present application. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for joint query of multiple data sources based on a large language model, characterized in that, The method comprises: extracting database table structure information of a target data source from a metadata knowledge base in response to a natural language query request input by a user; inputting the natural language query request and the database table structure information into a large language model to output a cross-database execution plan containing multiple sub-query tasks; based on the logical dependency relationship of the cross-database execution plan, the cross-database execution plan is disassembled into multiple concurrently executed independent query subsets and sequentially ordered query subsets with dependency relationships; performing multi-database dialect SQL generation and query operations on the independent query subsets concurrently to obtain first query results; performing pre-querying on the sequentially ordered query subsets according to the dependency relationship and injecting the query results of the pre-querying into subsequent querying to obtain second query results; fusing the first query results and the second query results and outputting standardized query results.

2. The method of claim 1, wherein, The inputting of the natural language query request and the database table structure information into the large language model to output the cross-database execution plan containing multiple sub-query tasks comprises: constructing a structured Prompt template, wherein the Prompt template contains database type identification, table structure field description and query requirement description; screening table field information related to querying through a metadata dynamic injection mechanism; calling a large language model to generate native SQL statements of a target database dialect.

3. The method of claim 2, wherein, The disassembling of the cross-database execution plan into multiple concurrently executed independent query subsets and sequentially ordered query subsets with dependency relationships comprises: parsing JOIN operations and nested query structures of the cross-database execution plan; identifying pre-querying to be executed preferentially and subsequent querying with the query results of the pre-querying as input parameters; classifying the pre-querying and the subsequent querying into sequentially ordered query subsets; classifying queries without data dependency into independent query subsets.

4. The method of claim 3, wherein, The identification basis of the pre-querying comprises: the query results are referenced by subsequent querying as filtering conditions; the query results contain cross-database association keys required by subsequent querying; the query results are used for the input of aggregation calculation of subsequent querying.

5. The method of claim 1, wherein, The concurrent execution of multi-database dialect SQL generation on the independent query subsets comprises: identifying a target database type and loading dialect characteristic configurations; adapting to database-specific syntax rules; performing syntax verification and security checks after generating multi-database dialect SQL.

6. The method of claim 1, wherein, The fusing of the first query results and the second query results and the outputting of standardized query results comprise: performing data type uniformization conversion on the first query results and the second query results returned by heterogeneous data sources; aligning cross-database fields based on field mapping relationships in a metadata knowledge base; performing JOIN operations according to association logic of the cross-database execution plan; performing aggregation calculation and deduplication processing on the first query results and the second query results after fusion to obtain final results; converting the final results into a standardized format for output.

7. The method of claim 1, wherein, The method further comprises establishing a multi-level result caching mechanism. The establishment of the multi-level result caching mechanism comprises: setting a caching strategy based on data query frequency and data update frequency; setting a cache invalidation strategy based on data update frequency; pre-executing high-frequency queries and caching hot data sets.

8. A large language model-based multi-data source joint query device, characterized in that, The device comprises: a request response module configured to extract database table structure information of a target data source from a metadata knowledge base in response to a natural language query request input by a user; a cross-database execution plan output module configured to input the natural language query request and the database table structure information into a large language model and output a cross-database execution plan comprising a plurality of sub-query tasks; a cross-database execution plan disassembling module configured to disassemble the cross-database execution plan into a plurality of independently queried subsets for concurrent execution and sequentially queried subsets with a dependency relationship based on a logical dependency relationship of the cross-database execution plan; a first query result obtaining module configured to perform multi-database dialect SQL generation and query operations on the independently queried subsets for concurrent execution to obtain a first query result; a second query result obtaining module configured to execute a preceding query on the sequentially queried subsets in a dependency relationship order and inject a query result of the preceding query into a subsequent query to obtain a second query result; a standardized query result output module configured to fuse the first query result and the second query result and output a standardized query result.

9. An electronic device, comprising: The device comprises a processor and a memory, the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the multi-data source joint query method based on a large language model according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions, when the computer executable instructions are called and executed by the processor, the computer executable instructions cause the processor to implement the multi-data source joint query method based on a large language model according to any one of claims 1 to 7.

Images