Database query method and electronic device

Abstract

This application discloses a database query method and electronic device. The method includes: connecting to multiple heterogeneous databases based on pre-stored database connection information, and reading the table structure information of all data tables in the multiple heterogeneous databases; inputting the target question and table structure information input by the user into a language processing model, performing analysis and reasoning to obtain a list of target data tables required to answer the target question; aggregating the target data in the target data tables, copying the target data to the aggregated database, and finally using the target query statement generated by the language processing model to query the aggregated database, obtaining a joint query result of multiple heterogeneous databases. This method solves the technical problem of low efficiency caused by overly complex data acquisition and processing when performing cross-database queries on heterogeneous databases in related technologies, and achieves the technical effect of improving the efficiency of cross-database queries.

Description

Technical Field

[0001] This application relates to the field of database processing technology, and in particular to a database query method and electronic device. Background Technology

[0002] At present, enterprise data is usually scattered across multiple and various types of database systems. When querying data based on business questions raised by users, it often involves joint statistical queries across databases.

[0003] In related technologies, it is usually necessary for professionals to write complex ETL scripts or connection programs. These scripts or programs extract data from various databases, perform field mapping, data type conversion, etc., and then load the transformed data into a unified data warehouse. Finally, the extracted cross-database data is merged in the data warehouse to obtain the joint query results. However, traditional cross-database query methods not only require users to understand the database structure and be able to write correct multi-table join query commands, but also have a complex and time-consuming query process, resulting in low efficiency in cross-database querying. Summary of the Invention

[0004] This application provides a database query method and electronic device to at least solve the technical problem of low efficiency in cross-database querying in related technologies.

[0005] According to one aspect of the embodiments of this application, a database query method is provided, comprising: connecting to multiple heterogeneous databases associated with a target question based on database connection information, and reading the table structure information of each data table in the multiple heterogeneous databases; obtaining a list of table names of target data tables required to answer the target question by inputting the target question and the table structure information into a language processing model; performing aggregation processing on the target data of each data table in the table name list to obtain an aggregated database containing the target data, wherein the aggregated data tables in the aggregated database have the same data as the target data tables in the multiple heterogeneous databases but different table names; generating a target query statement based on the table structure information, the target question, and some example data in the target data, and querying the aggregated database based on the target query statement to obtain a joint query result of multiple heterogeneous databases.

[0006] According to another aspect of the embodiments of this application, a service processing apparatus is also provided, comprising: a first processing unit, configured to connect to multiple heterogeneous databases associated with a target question based on database connection information, and read table structure information of each data table in the multiple heterogeneous databases; a second processing unit, configured to input the target question and table structure information into a language processing model to obtain a list of table names of target data tables required to answer the target question; a third processing unit, configured to aggregate the target data of each data table in the list of table names to obtain an aggregated database containing the target data, wherein the aggregated data tables in the aggregated database have the same data as the target data tables in the multiple heterogeneous databases but different table names; and a fourth processing unit, configured to generate a target query statement based on the table structure information, the target question, and some example data in the target data, and query the aggregated database based on the target query statement to obtain a joint query result of multiple heterogeneous databases.

[0007] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the steps of any of the above-described database query methods through the computer program.

[0008] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the steps of any of the above-described database query methods at runtime.

[0009] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the database query methods described above.

[0010] By employing the embodiments provided in this application, the target question and the table structure information of various data tables in multiple heterogeneous databases, obtained in advance, are input into the language processing model. This narrows the data query scope to at least a portion of the data tables associated with the target question (i.e., the target data tables). Using the list of table names comprised of the target data tables, the target data tables required to answer the target question are integrated into a unified, standardized aggregate database, providing an isolated and high-performance environment for the generation and execution of the target query statement. In other words, by creating an intermediate layer for data processing (the layer containing the aggregate database) and enabling seamless access and cross-database queries within the aggregate database using the language processing model, the cross-database query process is simplified, query time is saved, and the technical effect of improving the efficiency of cross-database queries is achieved. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram illustrating an application scenario of a database query method according to an embodiment of this application.

[0013] Figure 2 This is a flowchart of an optional database query method according to an embodiment of this application.

[0014] Figure 3 This is a schematic diagram of the overall architecture of an optional database query method according to an embodiment of this application.

[0015] Figure 4 This is a specific example of a database connection information table and a table structure information table.

[0016] Figure 5 This is an optional database connection initialization flowchart according to an embodiment of this application.

[0017] Figure 6 This is an optional overall dialogue flowchart according to an embodiment of this application.

[0018] Figure 7 This is a flowchart of extracting the table name of an optional data table according to an embodiment of this application.

[0019] Figure 8 This is a flowchart of extracting the table name of another optional data table according to an embodiment of this application.

[0020] Figure 9 This is a schematic diagram of an optional data aggregation and problem flow according to an embodiment of this application.

[0021] Figure 10 This is a structural block diagram of an optional database query device according to an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0023] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0024] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] According to one aspect of the embodiments of this application, a database query method is provided. Optionally, in this embodiment, the above-described database query method may be applied to, but is not limited to, [examples of database query methods]. Figure 1 In the hardware scenario shown, the server device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0026] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the database query method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to server devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0027] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0028] The embodiments of this application can be applied to, but are not limited to, multi-department collaborative data query and data analysis, and real-time query scenarios across databases. For ease of understanding, specific examples of several application scenarios are given below.

[0029] For example, in large enterprises, business modules such as financial services, sales, inventory, and customer relationship management typically use different database systems. For instance, financial services might use Oracle, while sales might use the open-source database MySQL. When cross-departmental data analysis is required, traditional query methods necessitate data engineers writing complex ETL scripts to acquire and integrate this data. However, with the technical solution presented in this application, questions can be directly posed using natural language. For instance, to view the total sales for the month, the system can automatically identify relevant data tables, aggregate the identified data, and automatically generate query statements using a large language model. This allows for rapid acquisition of joint query results, simplifying the cross-departmental data collaboration process and improving the timeliness of cross-database queries.

[0030] The database query method of this application embodiment can be executed by a server device, or by a server device in combination with at least one of a terminal device (which can also be understood as input / output device 108). Alternatively, the database query method of this application embodiment can be executed by a client installed on the terminal device.

[0031] Taking the database query method in this embodiment executed by the server as an example, Figure 2 This is a flowchart illustrating an optional database query method according to an embodiment of this application, such as... Figure 2 As shown, the process of this method may include steps S202 to S208.

[0032] Step S202: Based on the database connection information, connect to multiple heterogeneous databases related to the target problem, and read the table structure information of each data table in the multiple heterogeneous databases.

[0033] Step S204: By inputting the target question and table structure information into the language processing model, a list of table names of the target data tables required to answer the target question is obtained.

[0034] Step S206: Aggregate the target data of each data table in the table name list to obtain an aggregated database containing the target data. The aggregated data tables in the aggregated database have the same data as the target data tables in multiple heterogeneous databases, but the table names are different.

[0035] Step S208: Based on the table structure information, the target problem, and some example data in the target data, generate a target query statement, and query the aggregate database based on the target query statement to obtain the joint query results of multiple heterogeneous databases.

[0036] Before explaining the technical solution of this application, a brief introduction to the technical terms involved in the solution will be given first.

[0037] SQL: Structured Query Language.

[0038] LLM: Large Language Model, a large language model that can understand natural language problems and translate them into executable database query operations, such as generating SQL statements.

[0039] JSON: JavaScript Object Notation, a lightweight, easy-to-read and easy-to-write text data exchange format.

[0040] Heterogeneous databases refer to databases of different types and data formats, including but not limited to MySQL, Oracle, and PostgreSQL. These databases may run on different hardware platforms and use different data management systems, resulting in differences in how data is stored, processed, and queried.

[0041] Table structure information: This can refer to, but is not limited to, the architectural details of the data table, including metadata such as table name, column names, data types, comments, and indexes, which is crucial for understanding how the data is stored.

[0042] To facilitate understanding, let's first combine... Figure 3 The overall architecture diagram shown provides a brief introduction to the database query methods described above.

[0043] like Figure 3 As shown, the database query system in this application embodiment includes, but is not limited to, a dialogue client, a dialogue module, a large language model, a relational database, and multiple heterogeneous databases.

[0044] (1) Dialogue Client: User interaction layer. Users input natural language questions (such as viewing the total sales amount for this month) through this client. The client sends the questions to the dialogue module, triggering the subsequent processing flow.

[0045] (2) Dialogue Module: As the central coordinating component, it is responsible for receiving user input and coordinating and calling subsequent modules to complete parsing and querying. Its main functions include: 1) Connecting to the user database: reading the real-time table structure information of the user database (such as table name, column name, data type); 2) Question classification: that is, combining the built-in corpus in the vector database to determine whether the user input is a database query or statistical question; 3) Context integration: passing the user question, database structure information, prompt information, etc. to the coding model and the Large Language Model (LLM).

[0046] (3) Large Language Model (LLM): Based on the encoded input (user question + retrieval context hints + database structure), generate SQL statements that conform to the target database syntax.

[0047] (4) Relational Database (DB): The specific table structure information of the user database provides real-time metadata for the dialogue module, ensuring that the generated SQL statements conform to the actual database structure.

[0048] Among them, relational databases contain, for example Figure 4 The database connection information (connection) for storing each user's database is shown in (a), and as shown in (a)... Figure 4 The table structure (table_structure) shown in (b) is used to store data tables in each user's database. For details on each field and attribute, please refer to [reference needed]. Figure 4 The descriptions in (a) and (b) will not be repeated here.

[0049] (5) User database: Stores user data, i.e. dialogue data. The large model generates SQL queries for this database.

[0050] The overall process of the dialogue flow executed through the above modules is as follows: Figure 6 As shown, it includes the following content.

[0051] S602, the user selects one or more heterogeneous databases.

[0052] Based on the pre-acquired database connection information, connect to multiple heterogeneous databases, perform database initialization processing, and obtain full table structure information.

[0053] S604, Input the user's target problem (natural language problem).

[0054] S606: By inputting the target question and all table structure information into the language processing model, a list of specific data tables required to answer the target question (i.e., a list of target data tables) is obtained; and a list of table names is constructed based on the table names in the target data tables.

[0055] S608, with the help of DuckDB connection or DuckDB session (which can also be understood as in-memory database instance), performs aggregation processing on the data in the target data table, and forms an aggregated database containing the data in the target data table in the intermediate processing layer (which can also be understood as aggregation processing layer) where DuckDB is located.

[0056] The aggregation processing layer is an independent, lightweight, and high-performance data processing environment.

[0057] S610 generates SQL statements by inputting table structure information, the target problem, and some sample data into the language processing model.

[0058] S612 executes an SQL statement in the aggregate database to obtain the results of a cross-database query.

[0059] The results of the joint query include, but are not limited to, at least some of the data in the target data table, or the result data after processing at least some of the data in the target data table.

[0060] S614 delivers the results of the joint query to downstream business processes to assist in generating data statistics results or business reports, etc.

[0061] When a user poses a natural language question, hoping to perform a joint query across multiple databases of different types, the system first automatically establishes connections with each user's database (which can also be understood as multiple heterogeneous databases) based on the database connection information provided by the user, including database type, version, IP address, port, username, and password. This process is similar to building a bridge between databases in different languages, enabling them to be accessed and queried under a unified standard.

[0062] For example, suppose a user wants to analyze a company's internal financial reports and sales data, which are stored in a local MySQL database and a remote Oracle database, respectively. The system will establish connections with both the MySQL and Oracle databases based on the database connection information provided by the user, and automatically read the table structure information of their financial and sales data tables, including but not limited to table names, column names, data types, and indexes.

[0063] The user's natural language question and the previously retrieved table structure information are input into the language processing model. The model analyzes the semantics of the question and determines which data tables are key to answering the natural language question. In this way, the model intelligently filters out the data tables most relevant to the question and generates a list of table names. This list of table names indicates which database tables will participate in subsequent join queries.

[0064] After obtaining the list of table names, the data in each table is aggregated to enable seamless operation within the same data analysis engine (such as DuckDB), thus building an aggregated database containing the target data. In this database, although the aggregated tables contain the same data as the original tables, they use a unified aggregated table name for easier subsequent query operations.

[0065] Based on the previously obtained table structure information, user questions, and sample data from the aggregated database, the language processing model is invoked again to generate a specific query statement. Since the model is already familiar with the context of the question and the structure of the data tables, the generated SQL statement will accurately reflect the user's needs, taking into account all relevant tables and fields. The generated query statement will then be executed in the aggregated database to obtain the results of the join query.

[0066] In this embodiment, firstly, a connection is established with the target database based on pre-stored database connection information, and table structure information is read. Secondly, a list of table names is generated using a language processing model. Then, data aggregation processing and the construction of an aggregate database are performed. Finally, a query statement is generated based on the aggregate database, and a joint query is executed. These steps ensure that the entire process, from the user's natural language question to obtaining the joint query results needed to answer that question, is automated and intelligent, while also guaranteeing the accuracy and efficiency of the query. By introducing the concepts of a large language model and an aggregate database, the complexity and inefficiency of traditional joint queries are overcome.

[0067] By employing the embodiments provided in this application, the target question and the table structure information of various data tables in multiple heterogeneous databases, obtained in advance, are input into the language processing model. This narrows the data query scope to at least a portion of the data tables associated with the target question (i.e., the target data tables). Using the list of table names comprised of the target data tables, the target data tables required to answer the target question are integrated into a unified, standardized aggregate database, providing an isolated and high-performance environment for the generation and execution of the target query statement. In other words, by creating an intermediate layer for data processing (the layer containing the aggregate database) and enabling seamless access and cross-database queries within the aggregate database using the language processing model, the cross-database query process is simplified, query time is saved, and the technical effect of improving the efficiency of cross-database queries is achieved.

[0068] In an exemplary embodiment, the above-described method of connecting to multiple heterogeneous databases associated with the target problem based on database connection information and reading the table structure information of each data table in the multiple heterogeneous databases includes: obtaining pre-stored database connection information, wherein the database connection information includes database type, version, access address, and port information; when connecting to multiple heterogeneous databases based on the database connection information, reading the data table name, database connection identifier, and table structure description information from each data table, wherein the table structure description information includes field name, field type, and field format; and performing structuring processing on the data table name, database connection identifier, and table structure description information to obtain structured table structure information.

[0069] like Figure 5 As shown in the illustration, this application provides a user database connection initialization process. This process automatically extracts and structures the metadata (such as table names, column names, DDL statements, etc.) of each database when a user connects to the database for the first time, providing basic support for subsequently converting natural language questions into SQL query statements. DDL (Data Definition Language) is used to define or modify the database structure (such as tables, indexes, views, etc.), but does not directly manipulate the data in the tables.

[0070] S502, Enter database connection information.

[0071] This data is provided by the user and mainly includes basic information such as database type, version, IP address, port, username, and schema.

[0072] S504, Verify connection information.

[0073] The system attempts to establish a database connection using the parameters described above. If successful, it continues with subsequent steps; otherwise, it returns an error message.

[0074] S506, write the database connection information table (connection table).

[0075] Insert the above database connection information into, for example... Figure 4 In the connection table shown in (a), there are (name, db_type, version, host, port, username, schema_name, charset), and connection.id (primary key) is generated.

[0076] Wherein, name is the connection name, used to distinguish different connections; db_type is the database type, such as MySQL or Oracle; version indicates the database version; host indicates the database IP or domain name; post is the database port; username is the connection username; schema_name defaults to schema or database name.

[0077] It should be noted that after connecting to the aforementioned heterogeneous databases, information such as database version and database table name list can be read. The table name extraction method will be described in detail below with reference to specific embodiments.

[0078] S508, retrieve database table structure information.

[0079] Using the database connection information, connect to the target database (i.e., multiple heterogeneous databases) and read the structure of all tables in that database (table name, field name, data type, comments, indexes, etc.).

[0080] S510 converts table structure information into JSON format.

[0081] The goal is to organize the structure of each table in multiple heterogeneous databases into a unified JSON format.

[0082] S512, write the table structure information as follows Figure 4 The table_structure shown in (b) is an example.

[0083] By iterating through each data table, the table structure information is converted into JSON format and written to... Figure 4 The table shown in (b) is as follows. Here, connection_id represents the database connection ID, table_name is the table name, table_comment is the table comment, and table_structure is the table structure information, such as field names, types, and comments.

[0084] The above steps ensure that the system can accurately communicate with multiple heterogeneous databases and effectively read and understand their internal data table structures. By structuring this information into a unified format, a solid foundation is laid for subsequent natural language processing, data table recognition, and data aggregation operations. The efficient execution of these steps is crucial for ensuring a smooth and accurate query process, especially when handling complex, multi-database joint queries.

[0085] In an exemplary embodiment, the above-described method of inputting the target question and table structure information into a language processing model to obtain a list of table names of the target data tables required to answer the target question includes at least one of the following: inputting the target question into a language processing model to obtain a semantic analysis result after understanding the target question; performing analysis and reasoning based on the semantic analysis result and table structure information to obtain a list of data tables of the target data tables required to answer the target question, wherein each data table includes a target data table; parsing the table name of each data table in the target data table from the list of data tables to obtain a list of table names; or, in response to a trigger operation performed on the target interface, selecting the target data tables required to answer the target question from each data table in multiple connected heterogeneous databases; and constructing a list of table names based on the table names of each data table in the target data table.

[0086] After the user selects one or more database connections and submits the issue, the process proceeds to the crucial "table name extraction" stage. In this stage, the technical solution of this application provides two parallel implementation schemes. The first scheme is as follows: Figure 7 As shown.

[0087] S702, the user selects one or more heterogeneous databases.

[0088] S704: After connecting to the database based on pre-stored database connection information, the user submits a natural language question.

[0089] S706, read table structure information.

[0090] The system first automatically reads the organizational structure information of all data tables in the selected database.

[0091] S708 inputs the natural language problem and the read table structure information (full table structure information) into the language processing model (which can also be understood as a large model).

[0092] This means that the natural language questions raised by the user, along with the obtained full-scale structure information, are submitted to the Large Language Model (LLM) for subsequent model inference.

[0093] S710, LLM analyzes and reasons based on the semantics of the problem and the table structure information, and finally answers the specific list of data tables required to complete the data statistics (assuming that the natural language problem is a data statistics problem).

[0094] S712, the system parses the table names of each data table in the data table list from the LLM's response.

[0095] S714, Verify that the table names in the table name list are correct.

[0096] If correct, proceed to step S716; otherwise, proceed to step S718.

[0097] S716 returns the table names and generates a list of table names.

[0098] S718: Add a hint to the problem, resubmit to LLM, and start a new round of reasoning and analysis.

[0099] During the user review process, the table name is provided to the user for confirmation. This is an optional step; if enabled, the user needs to determine whether the system-recommended table name is accurate. If the user confirms it is accurate, the process ends, and the table name is successfully returned. If the user believes it is incorrect, the system will use this judgment result as a new prompt message, merge it with the original question, and resubmit it to step S708 to start a new round of LLM inference, forming an optimized closed loop.

[0100] The core of the above steps is to translate the understanding of the natural language question into specific operational guidelines for the database. This transformation process is led by an intelligent language processing model. Through three steps—semantic analysis, analytical reasoning, and table name resolution—the system can accurately identify the target data table associated with the user's question.

[0101] The second method for retrieving the list of table names is as follows: Figure 8 As shown. The specific steps are as follows: S802, the user selects one or more heterogeneous databases; S804, after connecting to the database based on pre-stored database connection information, the user submits a natural language question. S806, used to manually select the table name of the target data table; wherein, after the user submits the question, the system directly provides an interface allowing the user to manually browse and select the tables they believe need to query from all the data tables in the selected database. After the user completes the selection, these table names are directly returned, and the process ends. S808, the table name of the selected target data table is returned.

[0102] By employing these two parallel solutions, we can not only accurately capture the essence of the user's problem but also ensure that subsequent data processing steps are targeted, significantly improving the efficiency and accuracy of the entire query process. Furthermore, the second solution fully meets the personalized needs of users. Even non-technical users can easily initiate complex data query tasks without needing to concern themselves with the specific details of the underlying database. This not only greatly improves the accuracy and response speed of cross-database queries but also simplifies user operations.

[0103] In an exemplary embodiment, the method further includes: verifying the table names in the table name list and, if the verification result indicates that the table name is abnormal, using the verification result as a prompt message and merging the prompt message with the target question to obtain a merged question; and inputting the merged question and table structure information into a language processing model to execute a new round of reasoning to obtain an updated table name list.

[0104] After generating an initial table name, it needs to be validated to ensure its accuracy and suitability for subsequent data aggregation and query generation. This validation process includes checking if the table name exists, if it is spelled correctly, and if it matches the data requirements in the user's query.

[0105] If the verification finds any anomalies in the table names, the system will generate a corresponding message, clearly indicating which table names are marked as abnormal and the reason for the anomaly. This message will be merged with the original user question to form a new, more granular query command.

[0106] The updated question and database table structure information are then submitted to the language processing model again, triggering a new round of reasoning. Based on this more explicit instruction, the model will re-analyze the requirements and identify the relevant data tables, generating an updated and more accurate list of table names.

[0107] The execution logic in this embodiment is an important optimization mechanism in the technical solution of this application. Through feedback and correction mechanisms, it improves the accuracy of table name recognition and the reliability of query generation. It also enhances the user experience and ensures the efficiency and accuracy of data analysis.

[0108] In an exemplary embodiment, the above-described aggregation processing of the target data of each data table in the table name list to obtain an aggregated database containing the target data includes: creating a new in-memory database instance; loading at least a portion of the data from the target data tables in the plurality of heterogeneous databases into the aggregated database, and mapping the at least a portion of the data to the in-memory database instance based on database connection instructions, wherein the at least a portion of the data includes the full data of the target data tables or a portion of the data filtered from each heterogeneous database based on a structured query statement generated based on the target problem, the in-memory database instance is a database instance created in server memory, the plurality of heterogeneous databases are remote databases relative to the in-memory database instance, and the aggregated database includes the in-memory database instance; and performing aggregation processing on the at least a portion of the data in the target data tables based on the in-memory database instance to obtain the aggregated database.

[0109] This embodiment implements two data aggregation methods. One method involves directly migrating all data from the target data table in multiple heterogeneous databases to the aggregation database and performing aggregation processing to obtain the target query statement. The other method involves preliminary screening in each source database (i.e., each heterogeneous database) and migrating a portion of the data from the target data table to the aggregation database for aggregation processing. The two methods are described below.

[0110] For the first data aggregation method, a new database instance is created in the server's memory, which serves as the central location for subsequent data aggregation operations. In-memory database instances are widely used in scenarios requiring high-speed data access and processing due to their rapid data processing capabilities, especially in real-time analytics and decision support systems.

[0111] Create a database instance named DuckDB in the server's memory. A DuckDB instance is a lightweight, high-performance in-memory database instance (which can also be understood as a database session or database connection) capable of quickly loading and processing data, making it particularly suitable for data aggregation and statistical queries. The creation of this in-memory database instance aims to establish an independent, lightweight, high-performance sandbox computing environment for subsequent data integration and processing.

[0112] In other words, by creating an in-memory database instance, an intermediate processing layer is created to aggregate and perform joint queries on the data obtained from the target data table used to answer the target question. This can also be understood as an aggregation processing layer or an intermediate data processing layer.

[0113] After the in-memory database instance is created, the system loads the selected target data tables from multiple connected heterogeneous databases into the in-memory database instance. These remote databases may contain different data formats and storage methods, but through the unified interface of the in-memory database, this data can be seamlessly integrated.

[0114] Based on an in-memory database instance, aggregation processing will be performed on various target data tables obtained from multiple heterogeneous databases. This process may include merging data tables, mapping and transforming data fields, and pre-aggregation of data, ultimately forming an aggregated database containing all relevant data. Although the data tables in the aggregated database have the same data as those in the original database, they use a unified naming and format to facilitate subsequent joint queries and analysis.

[0115] For example, in the in-memory database instance DuckDB, the three data tables “sales”, “product_sales”, and “monthly_sales” can be aggregated and renamed to “sales_aggregated”, “product_sales_aggregated”, and “monthly_sales_aggregated”, respectively. It is important to ensure that the column names, data types, and data of these tables are consistent and standardized so that subsequent join queries can be performed smoothly.

[0116] In other words, in this embodiment, the full data of target tables from multiple heterogeneous databases is loaded into an in-memory database instance, and then aggregation processing is performed on the target tables based on the in-memory database instance. After loading, database connection commands (such as the Attach command) are used to map the target tables to the in-memory database instance. This mapping process is essentially an external link to the data; it does not require physical copying of the data, but rather establishes a reference relationship between the data tables and the in-memory database, ensuring that the in-memory database can read and manipulate this external data.

[0117] In other words, the system first identifies the data tables that are closely related to the user's problem, and then loads these target data tables into the in-memory database instance through a series of database operations. Then, through database connection commands, a mapping relationship is established between the target data tables and the in-memory database instance. This mapping can directly map or copy data from the target data tables selected in remote databases (i.e., multiple heterogeneous databases) to the aggregate database.

[0118] The second data aggregation processing method adopts a query pushdown mode, which pushes the calculation down to each source database as much as possible. DuckDB only serves as a coordination and merging layer for cross-database queries and no longer undertakes large-scale data transmission work. The specific steps are as follows (1) to (4).

[0119] (1) LLM generates structured query plans.

[0120] The system does not directly require the LLM to output executable SQL. Instead, it instructs the LLM to output a structured query intent description (JSON format) in the prompt, including the required filters, aggregation methods, grouping fields, and join keys for each data table involved in the target problem. In other words, in each source database, a structured query is generated based on the identified tables and the target problem. This query is used as the filtering condition to perform preliminary filtering in each heterogeneous database, which can also be understood as preliminary aggregation. The partially filtered data is then migrated to the aggregate database.

[0121] The purpose of performing the above preliminary screening in each heterogeneous database is to migrate more quantifiable data to the aggregated database, reducing data transfer resources. Secondly, it breaks down the originally complex task of generating the final SQL (target query statement) in one go into an intent description task that is easier for the model to accurately complete.

[0122] (2) The dialect translation layer is executed concurrently in each source database.

[0123] The system parses the JSON query plan generated in the previous step, adapts it to the SQL dialect of each source database type, translates the filtering conditions and aggregation logic into native SQL statements executable by that database, and submits them concurrently to each source database for execution. Each source database only returns an intermediate result set that has been aggregated, rather than the original full data.

[0124] For example, in the following scenario: the MySQL source only returns the order amount grouped and summarized by user_id (which may be aggregated from tens of millions of original records into thousands of results); the PostgreSQL source only returns a subset of users in a specific region after WHERE filtering.

[0125] (3) Register the intermediate results (i.e., part of the data in the target data table) to DuckDB. The system initializes a DuckDB in-memory instance and registers the aggregated result sets (in the form of DataFrame or Arrow Table) returned by each source database as virtual views in DuckDB, instead of physically copying the entire table. At this time, DuckDB only holds the small result sets calculated by each source database, and the memory usage is completely decoupled from the size of the original data table.

[0126] (4) Formation of a unified cross-database data view.

[0127] Once all the intermediate results are registered, a lightweight virtual database environment containing data from multiple heterogeneous databases is formed in DuckDB. The LLM and SQL execution engines in the subsequent question-and-answer phase only need to interact with this DuckDB instance, executing standard cross-view JOINs and final summary queries, without needing to concern themselves with the dialect differences and connection details of the underlying source databases. DuckDB's role changes from a data warehouse to a cross-database query coordinator, handling only the cross-source joins and final aggregation calculations that the source databases cannot perform independently.

[0128] The purpose of the above aggregation process is to ensure that all data requiring joint queries can be managed and queried in a unified environment. Through loading and mapping strategies, effective integration of data from multiple heterogeneous databases is achieved. By establishing connections with multiple heterogeneous databases within an in-memory database instance, a unified data access layer (i.e., middleware layer) is provided to users. Whether performing simple data viewing or complex joint queries, users can operate in a consistent manner, improving the efficiency of data analysis.

[0129] In an exemplary embodiment, the above-described aggregation processing of at least a portion of the data in the target data table based on the in-memory database instance to obtain the aggregated database includes: copying at least a portion of the data in the target data table to the in-memory database instance, wherein the target data is the at least a portion of the data; recreating aggregated data tables corresponding to each data table in the target data table by traversing the table name list, wherein there is a mapping relationship between the aggregated table name of the aggregated data table and the table names in the table name list; and determining the virtual database composed of the aggregated data tables containing the target data as the aggregated database.

[0130] At least a portion of the data from the target data table identified in the above steps is copied to the created in-memory database instance. This operation ensures that the data is available in a high-efficiency, low-latency environment, facilitating subsequent data aggregation and querying.

[0131] After the data is copied to the in-memory database, the previously constructed list of table names will be traversed to recreate aggregate data tables that correspond one-to-one with the target data tables. This reconstruction process may require adjustments to the table structure, including column names and data types, to ensure that all data tables are within a unified framework, facilitating subsequent join queries.

[0132] For example, based on the tables "sales", "users", and "products", aggregate data tables named "sales_aggr", "users_aggr", and "products_aggr" will be recreated in the in-memory database DuckDB. During the creation process, this includes, but is not limited to, adjusting the column names and data types of each table to conform to DuckDB's syntax and standards; for example, unifying date fields in all databases to the DATE type in DuckDB.

[0133] An aggregate database containing the target data is organized into a virtual database, which then serves as the aggregate database. The aggregate database can be a physically existing in-memory database containing the target data from the target tables, or it can be an in-memory database that maps to a series of aggregate tables mapped to the target tables. This database plays a crucial role in subsequent query generation and execution, providing a unified data access and query interface.

[0134] In an aggregated database, all tables use a unified naming convention and data format, ensuring the efficiency and accuracy of subsequent queries. Users can perform join queries directly within this aggregated database without needing to know which database the data originally came from.

[0135] The construction of aggregated databases simplifies the data query process for users, enabling even business personnel without database knowledge to easily perform complex joint queries, greatly improving the efficiency and quality of data analysis. Due to the high efficiency of in-memory databases in data processing, they also meet the needs of real-time cross-database analysis and statistics.

[0136] In an exemplary embodiment, the above-described method of recreating the aggregated data table corresponding to each data table in the target data table by traversing the list of table names includes: sequentially obtaining each table name from the list of table names as the current table name; based on the current table name, searching for the current data table in the target data table that matches the current table name; and creating a current aggregated table name corresponding to the current table name in the aggregated database, wherein the current aggregated table name is used to access the current data table in the aggregated database.

[0137] Each table name is extracted one by one from the list of table names constructed in the above embodiments. This operation is a fundamental step in the data aggregation process, providing precise instructions for subsequent data processing and access.

[0138] After obtaining the current table name, the system searches the target data table set for a table that matches the current table name. This search process ensures the accuracy of the data source for subsequent processing and avoids data errors caused by misidentification of the data table.

[0139] In the aggregate database, a new table corresponding to each current table name is created; this table name is referred to as the current aggregate table name. When creating aggregate tables, the system ensures that the structure and data of these tables match the tables in the original database, while using a unified standard to facilitate subsequent join queries.

[0140] This is achieved by sequentially searching for and creating matching aggregated data tables. The process begins by extracting table names from a list of table names, then finding data tables with identical names in various heterogeneous databases, and finally creating an aggregated data table with the same data and structure in the aggregated database, but using a unified data format and storage method.

[0141] By creating aggregated data tables corresponding to the original table names in the aggregated database, a unified data access and query platform is provided for users. Regardless of which heterogeneous database the data originally comes from, users can perform joint queries in a consistent manner, which greatly reduces the complexity of data queries and improves decision support capabilities.

[0142] In an exemplary embodiment, the above-described generation of a target query statement based on table structure information, target question, and partial example data from target data includes: extracting at least a portion of data from each data table in the table name list, wherein the partial example data includes at least a portion of data; integrating the table structure information, target question, and partial example data into target prompts based on contextual semantics, wherein the target prompts describe the query logic of performing a joint query on each data table based on the table structure information to obtain field attributes that answer the target question and satisfy the partial example data; and obtaining a structured target query statement by inputting the target prompts into a language processing model.

[0143] In other words, by inputting the table structure information, the target question, and some sample data into the language processing model, the target query statement is obtained.

[0144] In this embodiment, the first step involves constructing target prompts, which are formed by combining table structure information, the user's target question, and some sample data. Its design aims to provide the language processing model with a comprehensive context, including data metadata, the user's query requirements, and the actual form of the data, so that the model can more accurately understand and generate query statements.

[0145] The system then inputs the constructed target prompts into the language processing model. This model is typically a pre-trained Large Language Model (LLM) that has the ability to understand and process complex contexts and can generate SQL statements that meet the query requirements based on the provided contextual semantics.

[0146] Finally, the large language model performs deep reasoning and processing based on the information in the target prompt words, and outputs a structured target query statement, which is executed in the aggregate database to obtain the joint query data needed to answer the target question entered by the user.

[0147] This approach considers not only the static structure of the database but also dynamic sample data and specific user query needs. Through intelligent processing using a large language model, it achieves a high degree of customization and optimization of query statements. In practical applications, this process significantly reduces the time and cost of manually writing and debugging SQL, while also improving query accuracy.

[0148] Additionally, it should be noted that representative data, i.e., sample data, is extracted from the defined target data table. This process aims to enrich the context of the language processing model, providing specific numerical values ​​and data formats, thereby generating more accurate query statements.

[0149] In practical applications, the processing mechanism in this embodiment is particularly suitable for data-driven scenarios. Whether it is real-time data monitoring or historical data analysis, it can quickly and accurately obtain the required information, greatly improving the flexibility and response speed of data analysis.

[0150] The purpose of aggregation processing is to aggregate data from the target data table into an aggregate database formed by the DuckDB instance, based on the database names and table name list extracted in the above embodiments. Since data aggregation processing is crucial for implementing the aforementioned cross-database join query, the overall processing flow is given below for a clearer understanding of the aggregation process.

[0151] S11, Initialize the DuckDB instance (which can also be understood as an in-memory database instance), creating a new DuckDB connection or in-memory database instance in the service memory.

[0152] Its purpose is to establish an independent, high-performance sandbox computing environment for subsequent data integration and processing.

[0153] S12, Configure external data source connection.

[0154] Based on the database connection ID obtained in the above embodiments, the corresponding access credentials (including database type, host address, port, username, and password) are retrieved from the system's connection configuration pool. The system then prepares to establish a connection using the external data source extension capabilities of the DuckDB instance.

[0155] S13, mount the remote database to the DuckDB instance.

[0156] You can, but are not limited to, use DuckDB's Attach command or related extensions to map a remote database to a DuckDB session or DuckDB instance.

[0157] For example, for the source data MySQL, the way to mount it to the DuckDB instance using the above Attach command is as follows: ATTACH 'host=localhost, user=mysql_user, password='AS remote_mysql(TYPEMYSQL).

[0158] S14: Copy the target data table from the remote database to DuckDB, traverse the pre-obtained list of table names, and create a corresponding table or view in DuckDB for each table.

[0159] For example, CREATE TABLE orders AS SELECT The command FROM remote_mysql.orders creates an aggregate table or view corresponding to each target table in the intermediate processing layer or aggregation processing layer (DuckDB) using the DuckDB instance.

[0160] S15, A unified data view is formed.

[0161] Once all the specified target tables have been created in DuckDB, a virtual database (which can also be understood as an aggregate database) containing all the required data (i.e., the full data from the filtered target tables) is constructed. In the aggregate database, standard SQL join queries can be executed, just as if operating on a single local database.

[0162] S16 provides an access interface to access the DuckDB instance and its aggregated data tables, serving as the unified data access layer for subsequent SQL generation and data statistics steps. Subsequent processes only need to interact with this DuckDB instance, eliminating the need to concern themselves with the complex underlying data source differences.

[0163] In an exemplary embodiment, the above-mentioned querying of the aggregate database based on the target query statement to obtain the joint query result of multiple heterogeneous databases includes: if the target query statement passes the verification, locating the aggregate data table in the aggregate database based on the aggregate table name in the target query statement, wherein the aggregate data table has the same data as the target data table and the table names of each data table are different; based on the query logic of the target query statement, sequentially extracting the result data required to answer the target question from each data table in the aggregate data table, wherein the joint query result includes the result data, and the result data is delivered to downstream business and used to generate business data reports.

[0164] To ensure the correctness and feasibility of the generated target query statement, after the language processing model outputs the target query statement, its syntax and logic need to be validated to ensure that it can be executed normally in the target aggregate database without syntax errors or logical defects.

[0165] The system will perform syntax and logic checks on this statement, such as checking whether the column names exist and whether the JSON keywords are used appropriately, to ensure that it is accurate.

[0166] After the target query is validated, the system quickly locates the corresponding aggregate table in the aggregate database based on the aggregate table name mentioned in the query. Aggregate tables are created during the data aggregation phase; they contain complete information about the target table but use a standardized naming convention for ease of subsequent operations.

[0167] After locating the correct aggregate tables, the query logic in the target query statement will be executed to extract the result data that meets specific conditions from these aggregate tables. A join query is the process of combining data from multiple tables according to a certain logical relationship to obtain a more comprehensive view of the data.

[0168] For example, suppose you perform a target query in the in-memory database DuckDB to filter products with a sales quantity exceeding 100 and the number of unique buyers for each product. The query result would be a list of products sorted in descending order of sales quantity, with each product entry including the product name, total sales quantity, and the number of unique customers who purchased that product.

[0169] Finally, the final data extracted from the aggregated data table that meets user needs is delivered to downstream business modules, such as data visualization tools and report generation systems, for further data analysis and presentation. This step ensures that the data is used effectively to support decision-making.

[0170] The generated target query statement is validated to ensure its grammatical correctness and logical clarity. Based on the aggregate table name mentioned in the query statement, the aggregate data table in the aggregate database is located. Finally, the join query logic in the query statement is executed to extract the result data that meets specific conditions from the aggregate data table. This result data is then transmitted to downstream business modules to generate reports or perform data visualization, providing users with intuitive data display and decision-making basis.

[0171] By employing the above approach, the need for rapid and accurate information extraction from massive amounts of data is met. Furthermore, by delivering the results to downstream business processes, the system achieves a closed loop in data analysis, improving data processing efficiency and providing strong support for business decision-making. It also reduces the complexity of cross-database join queries.

[0172] Based on the analysis of the above embodiments, it can be seen that the core of the technical solution of this application lies in performing data aggregation processing, SQL statement generation, and multi-data joint query in the intermediate processing layer where the aggregation database is located. For ease of understanding, the following will be combined with... Figure 9 The data aggregation and question-and-answer process shown further describes these two parts. This process includes organizing and submitting LLM prompts, and LLM inference generating SQL that conforms to DuckDB syntax, which is then executed to obtain the required data.

[0173] S902, when connected to multiple heterogeneous databases, obtain the full data tables and table name list of each database in accordance with the method described in the above embodiment.

[0174] S904 uses a language processing model to filter out the target data table (which can be partial or full data from the full data table) from the full data table, and loads the data in the target data packet into the DuckDB instance to form an aggregate database.

[0175] Among them, the data tables in the aggregate database are aggregate data tables.

[0176] S906 extracts a small amount of sample data from the aggregated data tables. It extracts a small amount (e.g., 5 rows) of sample data from each of the aggregated data tables in the embedded OLAP database (DuckDB). The purpose is to allow the LLM to intuitively understand the actual form of the data (such as field formats, data enumeration values, etc.) to constrain the final generated data results to meet the requirements of field attributes, greatly improving the accuracy of the generated SQL.

[0177] S908 pushes the table structure information, some sample data, and the target question (natural language question) entered by the user to the LLM.

[0178] These three pieces of information are then integrated into a complete prompt word. The table structure information is obtained through database connection and table structure records in the database; some sample data is obtained from step S906, and the target question is user input. This prompt word is then submitted to the Large Language Model (LLM) to request the generation of an SQL query statement.

[0179] S910, LLM inference and generation of SQL query statements.

[0180] Based on the received context information, LLM performs logical reasoning to generate a target query statement that answers the user's question and conforms to the SQL syntax requirements of the aggregation processing layer (intermediate processing layer) where DuckDB is located.

[0181] S912, extract and verify the SQL.

[0182] The process involves precisely extracting SQL code snippets from the LLM's response. This stage typically includes a basic validation step to ensure that valid SQL text has been successfully obtained. If parsing fails, the process will terminate abnormally.

[0183] S914 executes SQL and retrieves the results.

[0184] In an embedded OLAP database (DuckDB or aggregate database), execute the SQL statements generated by the LLM. Upon successful execution, obtain the data statistics result set.

[0185] S916, Results Delivery and Follow-up Services.

[0186] The obtained statistical results are then passed to downstream business modules. Finally, based on these results, data summary reports or data visualization charts are generated and presented to the user, thus concluding the entire Q&A process.

[0187] In the various embodiments of this application, a complex "problem-to-SQL" generation process is broken down into two independent LLM call stages: "natural language problem to target data table" and "(table structure + sample data) to generated SQL statement". A data aggregation and preparation stage is inserted between the two stages or two processing steps. This architecture, through a "divide and conquer" strategy, significantly reduces the difficulty of a single LLM task and provides a precise data range for subsequent steps through intermediate results (table list), thereby improving the overall accuracy and reliability of the system.

[0188] Secondly, DuckDB is introduced as the unified data computation middleware layer for the entire system. Through its scalability, database tables from different SQL dialects and locations are quickly integrated into a unified, standardized computing environment. This provides an isolated, high-performance execution environment for the SQL generated by LLM, avoiding direct impact on the production database. It also provides efficient and convenient sample data sampling capabilities for the second-stage LLM prompts.

[0189] Furthermore, the project utilizes a dual contextual prompt term system—"table structure information + sample data"—for data semantic understanding. In the second phase of LLM invocation, it creatively treats the table structure (schema) and sample data as parallel and indispensable context. This directly solves the problem that LLM cannot understand the true meaning, format, and value range of data based solely on table and column names, and is crucial for generating correct WHERE conditions, query logic, and aggregate functions.

[0190] According to another aspect of the embodiments of this application, a database query device is also provided, the structural schematic diagram of which is shown below. Figure 10As shown, the system includes the following modules: a first processing unit 1002, used to connect to multiple heterogeneous databases associated with the target question based on database connection information, and read the table structure information of each data table in the multiple heterogeneous databases; a second processing unit 1004, used to input the target question and table structure information into a language processing model to obtain a list of table names of the target data tables required to answer the target question; a third processing unit 1006, used to aggregate the target data of each data table in the table name list to obtain an aggregated database containing the target data, wherein the aggregated data tables in the aggregated database have the same data as the target data tables in the multiple heterogeneous databases but different table names; and a fourth processing unit 1008, used to generate a target query statement based on the table structure information, the target question, and some example data in the target data, and query the aggregated database based on the target query statement to obtain a joint query result of multiple heterogeneous databases.

[0191] The specific execution steps involved in the various calculation processes and data processing processes in the above modules can be referred to the description in the above embodiments, and will not be repeated here.

[0192] Obviously, the above-described service processing apparatus can be used to implement the database query method provided in the above embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0193] It should be noted that the first processing unit 1002 in this embodiment can be used to execute the above step S202, the second processing unit 1004 in this embodiment can be used to execute the above step S204, the third processing unit 1006 in this embodiment can be used to execute the above step S206, and the fourth processing unit 1008 in this embodiment can be used to execute the above step S208.

[0194] In an exemplary embodiment, the first processing unit 1002 includes: a first acquisition module, configured to acquire pre-stored database connection information, wherein the database connection information includes database type, version, access address, and port information; a first reading module, configured to read table names, database connection identifiers, and table structure description information from each data table when connected to multiple heterogeneous databases based on the database connection information, wherein the table structure description information includes field names, field types, and field formats; and a first processing module, configured to perform structured processing on the table names, database connection identifiers, and table structure description information to obtain structured table structure information.

[0195] In an exemplary embodiment, the second processing unit 1004 includes: a second processing module configured to perform at least one of the following: inputting a target question into a language processing model to obtain a semantic analysis result after understanding the target question; performing analysis and reasoning based on the semantic analysis result and table structure information to obtain a list of target data tables required to answer the target question, wherein each data table includes a target data table; parsing the table name of each data table in the target data table from the list of data tables to obtain a list of table names; in response to a trigger operation performed on the target interface, selecting the target data table required to answer the target question from each data table in a plurality of connected heterogeneous databases; and constructing a list of table names based on the table names of each data table in the target data table.

[0196] In an exemplary embodiment, the above apparatus further includes: a verification unit, configured to, when verifying the table names in the table name list and the verification result indicates that the table names are abnormal, use the verification result as a prompt message and merge the prompt message with the target question to obtain a merged question; and a reasoning unit, configured to, by inputting the merged question and table structure information into a language processing model, execute a new round of reasoning process to obtain an updated table name list.

[0197] In an exemplary embodiment, the third processing unit 1006 includes: a creation module for creating a new in-memory database instance; a loading module for loading at least a portion of the data from the target data table in the plurality of heterogeneous databases into the aggregated database, and mapping the at least a portion of the data to the in-memory database instance based on database connection instructions, wherein the at least a portion of the data includes the full data in the target data table or a portion of the data filtered from each heterogeneous database based on a structured query statement generated based on the target problem, the in-memory database instance is a database instance created in server memory, the plurality of heterogeneous databases are remote databases relative to the in-memory database instance, and the aggregated database includes the in-memory database instance; and a third processing module for performing aggregation processing on the at least a portion of the data in the target data table based on the in-memory database instance to obtain the aggregated database.

[0198] In an exemplary embodiment, the third processing module includes: a copying submodule, configured to copy at least a portion of the data in the target data table to the in-memory database instance, wherein the target data is the at least a portion of the data; a traversal submodule, configured to recreate an aggregated data table corresponding to each data table in the target data table by traversing the table name list, wherein there is a mapping relationship between the aggregated table name of the aggregated data table and the table names in the table name list; and a first processing submodule, configured to determine the virtual database composed of the aggregated data table containing the target data as the aggregated database.

[0199] In one exemplary embodiment, the third processing module further includes: a first acquisition submodule, configured to sequentially acquire each table name from the table name list as the current table name; a search submodule, configured to search for a current data table matching the current table name from the target data table based on the current table name; and a creation submodule, configured to create a current aggregate table name corresponding to the current table name in the aggregate database, wherein the current aggregate table name is used to access the current data table in the aggregate database.

[0200] In an exemplary embodiment, the fourth processing unit 1008 includes: a first extraction module, configured to extract at least partial data from each data table in the table name list, wherein the partial example data includes at least partial data; a fourth processing module, configured to integrate table structure information, target question, and partial example data into target prompt words based on contextual semantics, wherein the target prompt words describe the query logic of performing a joint query on each data table based on the table structure information to obtain field attributes that answer the target question and satisfy the partial example data; and a fifth processing module, configured to obtain a structured target query statement by inputting the target prompt words into a language processing model.

[0201] In an exemplary embodiment, the fourth processing unit 1008 includes: a positioning module, configured to locate the aggregate data table in the aggregate database based on the aggregate table name in the target query statement when the target query statement passes verification, wherein the aggregate data table has the same data as the target data table and the table names of each data table are different; and a second extraction module, configured to extract the result data required to answer the target question sequentially from each data table in the aggregate data table based on the query logic of the target query statement, wherein the joint query result includes the result data, and the result data is delivered to downstream business and used to generate business data reports.

[0202] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0203] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described database query method embodiments.

[0204] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, which is configured to execute the steps in any of the above-described database query method embodiments when running.

[0205] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0206] According to another aspect of the embodiments of this application, a computer program product is also provided, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described database query method embodiments.

[0207] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described database query method embodiments.

[0208] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0209] The database query method provided in this application has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only intended to help understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A database query method, characterized in that, include: Based on database connection information, multiple heterogeneous databases associated with the target problem are connected, and the table structure information of each data table in the multiple heterogeneous databases is read. By inputting the target question and the table structure information into the language processing model, a list of table names of the target data tables required to answer the target question is obtained; Aggregate the target data of each data table in the list of table names to obtain an aggregated database containing the target data. The aggregated data tables in the aggregated database have the same data as the target data tables in the multiple heterogeneous databases, but the table names are different. Based on the table structure information, the target question, and some example data in the target data, a target query statement is generated, and the aggregated database is queried based on the target query statement to obtain the joint query results of the multiple heterogeneous databases.

2. The method according to claim 1, characterized in that, The step of connecting to multiple heterogeneous databases associated with the target problem based on database connection information and reading the table structure information of each data table in the multiple heterogeneous databases includes: Obtain the pre-stored database connection information, wherein the database connection information includes database type, version, access address, and port information; When connecting to the multiple heterogeneous databases based on the database connection information, the table name, database connection identifier, and table structure description information are read from each data table, wherein the table structure description information includes field name, field type, and field format; The data table name, the database connection identifier, and the table structure description information are processed into a structured table structure information.

3. The method according to claim 1, characterized in that, The step of inputting the target question and the table structure information into the language processing model to obtain a list of table names of the target data table required to answer the target question includes at least one of the following: By inputting the target question into the language processing model, a semantic analysis result after understanding the target question is obtained; based on the semantic analysis result and the table structure information, analysis and reasoning are performed to obtain a list of target data tables required to answer the target question, wherein each data table includes the target data table; the table name of each data table in the target data table is parsed from the list of data tables to obtain the list of table names; In response to a trigger operation performed on the target interface, the target data table required to answer the target question is selected from the data tables of the multiple connected heterogeneous databases; and the table name list is constructed based on the table names of the various data tables in the target data table.

4. The method according to claim 1, characterized in that, The method further includes: If the table name in the table name list is verified and the verification result indicates that the table name is abnormal, the verification result is used as a prompt message, and the prompt message is merged with the target problem to obtain the merged problem; By inputting the merged question and the table structure information into the language processing model, a new round of reasoning is executed to obtain an updated list of table names.

5. The method according to claim 1, characterized in that, The aggregation process of the target data of each data table in the table name list to obtain an aggregated database containing the target data includes: Create a new in-memory database instance; At least a portion of the data from the target data table in the plurality of heterogeneous databases is loaded into the aggregate database, and the at least a portion of the data is mapped to the in-memory database instance based on database connection instructions. The at least a portion of the data includes the full data in the target data table or a portion of the data filtered from each heterogeneous database based on the structured query statement generated based on the target problem. The in-memory database instance is a database instance created in the server memory. The plurality of heterogeneous databases are remote databases relative to the in-memory database instance. The aggregate database includes the in-memory database instance. Based on the in-memory database instance, at least a portion of the data in the target data table is aggregated to obtain the aggregated database.

6. The method according to claim 5, characterized in that, The aggregation process, based on the in-memory database instance, involves aggregating at least a portion of the data in the target data table to obtain the aggregated database, including: Copy at least a portion of the data from the target data table to the in-memory database instance, wherein the target data is the at least a portion of the data; By traversing the list of table names, an aggregated data table corresponding to each data table in the target data table is recreated, wherein there is a mapping relationship between the aggregated table name of the aggregated data table and the table name in the list of table names; The virtual database consisting of the aggregated data table containing the target data is defined as the aggregated database.

7. The method according to claim 6, characterized in that, The step of recreating the aggregated data table corresponding to each data table in the target data table by traversing the list of table names includes: Retrieve each table name from the list of table names in sequence as the current table name; Based on the current table name, find the current data table that matches the current table name from the target data table; Create a current aggregate table name corresponding to the current table name in the aggregate database, wherein the current aggregate table name is used to access the current data table in the aggregate database.

8. The method according to claim 1, characterized in that, The step of generating a target query statement based on the table structure information, the target question, and some example data in the target data includes: Extract at least a portion of the data from each of the data tables in the list of table names, wherein the partial sample data includes the at least a portion of the data; Based on contextual semantics, the table structure information, the target question, and the partial sample data are integrated into target prompt words. The target prompt words are used to describe the query logic of performing a joint query on each data table based on the table structure information to obtain field attributes that answer the target question and satisfy the partial sample data. By inputting the target prompt words into the language processing model, a structured target query statement is obtained.

9. The method according to claim 1, characterized in that, The step of querying the aggregated database based on the target query statement to obtain the joint query results of the multiple heterogeneous databases includes: If the target query statement passes the verification, the aggregate data table in the aggregate database is located based on the aggregate table name in the target query statement, wherein the aggregate data table has the same data as the target data table, and the table names of each data table are different; Based on the query logic of the target query statement, the result data required to answer the target question is extracted sequentially from each data table in the aggregated data table. The joint query result includes the result data, which is delivered to downstream business and used to generate business data reports.

10. An electronic device, comprising: include: Memory, used to store computer programs; A processor, configured to implement the steps of the database query method as described in any one of claims 1 to 9 when executing the computer program.

Images