Data query method, device, electronic device, storage medium and product

Abstract

The present application provides a data query method, device, electronic device, storage medium and product; the method includes: obtaining a query text for indicating a data query, and obtaining a target prompt template associated with the query text for generating a structured statement, wherein the target prompt template includes metadata information for describing a data source, and different prompt templates correspond to different data sources; based on the query text and the target prompt template, a structured query statement corresponding to the query text is generated; based on the generated structured query statement, in the target data source corresponding to the target prompt template, the target data corresponding to the query text is queried to obtain. Through the present application, the target data source of the query text can be more accurately located, thereby improving the efficiency of data query.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to a data query method, device, electronic device, storage medium and product. Background Art

[0002] With the development of the Internet and big data, data processing and analysis have become important links in all walks of life. Structured query language is a common language in the database field and is widely used in data analysis scenarios. However, since some non-technical personnel lack the skills to extract data from the database using structured query language, they usually need to submit data requirements to professional technicians to complete. This data analysis process is inefficient and has high communication costs. In response to this, natural language processing technology came into being.

[0003] Natural language processing technology aims to convert natural language queries into structured query statements, which can facilitate users to ask questions to the database through natural language and obtain corresponding query results. In related technologies, many automated data analysis products have also been developed based on natural language processing technology. As the application scenarios of data analysis become more and more extensive, more and more data sources are involved in data query, making data query more and more inefficient. Summary of the invention

[0004] The embodiments of the present application provide a data query method, device, electronic device, storage medium and product, which can improve data query efficiency.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present invention provides a data query method, which includes:

[0007] Acquire a query text for indicating a data query, and acquire a target prompt template associated with the query text for generating a structured statement, wherein the target prompt template includes metadata information for describing a data source, and different prompt templates correspond to different data sources;

[0008] Based on the query text and the target prompt template, generate a structured query statement corresponding to the query text;

[0009] Based on the generated structured query statement, the target data corresponding to the query text is queried in the target data source corresponding to the target prompt template.

[0010] The present application provides a data query device, including:

[0011] An acquisition module, used to acquire a query text for indicating a data query, and acquire a target prompt template associated with the query text for generating a structured statement, wherein the target prompt template includes metadata information for describing a data source, and different prompt templates correspond to different data sources;

[0012] A generating module, configured to generate a structured query statement corresponding to the query text based on the query text and the target prompt template;

[0013] A query module is used to query and obtain target data corresponding to the query text in a data source corresponding to the target prompt template based on the generated structured query statement.

[0014] In the above scheme, the acquisition module is also used to receive a data query instruction for performing a data query on the query text, the data query instruction carries a target query range corresponding to the query text; the target query range is obtained by selecting at least one query range displayed in the conversation interface; a mapping relationship between different query ranges and prompt templates is obtained; based on the mapping relationship and the target query range, a prompt template having a mapping relationship with the target query range is determined, and the prompt template having a mapping relationship with the target query range is used as the target prompt template for generating a structured statement.

[0015] In the above solution, the device further includes: a return module. After the query module obtains the target data corresponding to the query text, the return module is used to respond to the data query instruction and return the target data to the electronic device corresponding to the data query instruction; wherein the target data is used by the electronic device to output the target data corresponding to the query text in the conversation interface displaying the target query range.

[0016] In the above scheme, the device further includes: a relationship establishment module. Before the acquisition module acquires the mapping relationship between different query scopes and prompt templates, the relationship establishment module is used to determine the data source to which the data corresponding to each query scope belongs, and determine the prompt template of each data source; based on the data source to which the data corresponding to each query scope belongs, and the prompt template of each data source, establish the mapping relationship between each query scope and the corresponding prompt template.

[0017] In the above scheme, the device further includes: a template generation module. Before the acquisition module acquires the target prompt template associated with the query text for generating a structured sentence, the template generation module is used to acquire multiple data sources for data query and acquire attribute information of each of the data sources; convert the attribute information of each of the data sources into metadata information corresponding to each of the data sources; and generate a prompt template corresponding to each of the data sources based on the metadata information of each of the data sources.

[0018] In the above scheme, the device further includes: a format conversion module. The metadata information of each of the data sources has the same format. After the template generation module converts the attribute information of each of the data sources into the metadata information of the corresponding data source, the format conversion module is used to obtain the target metadata format and convert the format of the metadata information of each of the data sources into the target metadata format. The template generation module is also used to generate a prompt template corresponding to each of the data sources based on the metadata information of each of the data sources having the target metadata format.

[0019] In the above scheme, the generation module is further used to obtain a target language model, and call the target language model to predict a structured query statement corresponding to the query text based on the query text and the target prompt template.

[0020] In the above scheme, the metadata information includes the database type of the database corresponding to the target data source; the generation module is also used to obtain the target query syntax of the database corresponding to the target data source based on the database type; based on the query text and the target prompt template, generate a structured query statement corresponding to the query text that conforms to the target query syntax.

[0021] In the above scheme, the database includes at least one data table, and the metadata information also includes the table identifier of the target data table to which the target data belongs and the target field in the target data table to which the target data belongs. The generation module is further used to obtain the target query syntax of the target data table in the database based on the database type; based on the query text, the table identifier of the target data table and the target field, generate a structured query statement that corresponds to the query text and conforms to the target query syntax.

[0022] In the above scheme, the query module is also used to obtain the address information of the target data source associated with the target prompt template; based on the address information of the target data source, establish a communication connection with the target data source, and create a data source connection object of the target data source based on the communication connection; based on the data source connection object, execute the structured query statement in the target data source corresponding to the target prompt template to obtain the target data corresponding to the query text.

[0023] In the above scheme, the target prompt template also includes the data source identifier of the target data source. The query module is also used to extract the data source identifier of the target data source from the target prompt template; based on the data source identifier of the target data source, the generated structured query statement is executed in the target data source corresponding to the target prompt template to obtain the target data corresponding to the query text.

[0024] In the above scheme, the device further includes: a data operation module. The query text also includes an operation text, and the operation text is used to indicate the target data operation to be performed on the target data. After the query module obtains the target data corresponding to the query text through query, the data operation module is used to perform the target data operation on the target data to obtain the target data after the operation; and send the target data after the operation so that the target data after the operation is displayed on the view interface as the query result of the query text.

[0025] In the above scheme, the device further includes: a matching module. In the case where there are multiple target prompt templates, the matching module is used to obtain the data source corresponding to each target prompt template; according to the matching degree between the data source corresponding to each target prompt template and the query text, determine the target prompt template whose matching degree meets the expected matching. The generating module is also used to generate a structured query statement corresponding to the query text based on the query text, and the matching degree meets the expected matching target prompt template.

[0026] An embodiment of the present application provides an electronic device, which includes: a memory for storing computer-executable instructions; and a processor for executing the computer-executable instructions stored in the memory to implement the data query method provided in the embodiment of the present application.

[0027] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the data query method provided in the embodiment of the present application when executed by a processor.

[0028] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the data query method provided in the embodiment of the present application is implemented.

[0029] The embodiments of the present application have the following beneficial effects: there is an association relationship between the query text and the prompt template of the embodiments of the present application, and there is an association relationship between the prompt template and the data source. After obtaining the query text for indicating a data query, the target data source to which the data required to be queried by the query text belongs can be located by directly obtaining the target prompt template associated with the query text, so that the positioning of the target data source is more accurate, thereby improving the efficiency of data query. In addition, by using the metadata information included in the target prompt template that describes the target data source, a structured query statement for data querying the target data source corresponding to the query text can be generated more accurately, thereby improving the accuracy of extracting target data from the target data source corresponding to the query text. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the architecture of the data query system provided in the embodiment of the present application;

[0031] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0032] Figure 3A It is a first flow chart of the data query method provided in an embodiment of the present application;

[0033] Figure 3B It is a flowchart of a method for obtaining a target prompt template provided in an embodiment of the present application;

[0034] Figure 3C It is a first flow chart of a method for generating a structured query statement provided in an embodiment of the present application;

[0035] Figure 3D It is a second flow chart of the method for generating a structured query statement provided in an embodiment of the present application;

[0036] Figure 3E This is a first flow chart of a method for querying target data provided by an embodiment of the present application;

[0037] Figure 3F It is a second flow chart of the method for querying target data provided by an embodiment of the present application;

[0038] Figure 3G It is a second flow chart of the data query method provided in the embodiment of the present application;

[0039] Figure 3H is a third flow chart of the data query method provided in the embodiment of the present application;

[0040] Figure 4 is a schematic diagram of a conversation interface provided in an embodiment of the present application;

[0041] Figure 5 is a schematic diagram of the principle of the target language model provided in the embodiment of the present application;

[0042] Figure 6 is a schematic diagram of a target prompt template provided in an embodiment of the present application;

[0043] Figure 7 It is a schematic diagram of the association relationship between the prompt template and the data source identifier provided in the embodiment of the present application;

[0044] Figure 8 It is a schematic diagram of the architecture of the data query system provided in the embodiment of the present application;

[0045] Fig. 9 It is an interactive schematic diagram of the data query method provided in an embodiment of the present application.

[0046] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of superiority or inferiority of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0048] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0049] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0050] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0051] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0052] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0053] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0054] 1) Data source: refers to the source or storage location of data storage, management and provision. In the field of information technology and data processing, data sources can be diverse, including but not limited to: databases, file systems, data warehouses, public data sets, etc.

[0055] 2) Metadata information: It is data about data. It provides attribute information and descriptions about the data to help organize, manage and understand the data, and also facilitates the query, analysis and use of the data. Metadata information can include information such as data type, data format, data source, data structure, data label, data quality, data update time, data ownership and data version.

[0056] 3) Structured Query Language: It is a standardized language used to manage and operate relational databases. It is a query language used to access structured data (such as tables and databases) and can be used to query, insert, update, delete, and other operations on the database. For example: SQL query language.

[0057] The inventor discovered during the research that in the process of implementing natural language processing based on a large language model, prompt templates can better help the large language model understand the natural language text input by the user, thereby improving the output of more accurate structured query statements. In the related art, prompt templates mostly help the large language model to understand the structure and rules of structured query language by adding some prompt information related to the grammar of structured query language. In addition, the solutions in the related art are usually to match the corresponding prompt templates by analyzing the query text input by the user in natural language, thereby generating a structured query statement corresponding to the query text. However, as this technology is applied in more and more scenarios, more and more data sources are involved. Relying on the analysis of the query text to match the prompt template will result in matching deviations, resulting in inaccurate generated structured query statements, and thus the queried data source will also have deviations, making it difficult to locate the accurate data source, and the data query efficiency is low.

[0058] The embodiments of the present application provide a data query method, device, electronic device, storage medium and product, which can more accurately locate the data source when the user performs data query, thereby improving the data query efficiency.

[0059] See also Figure 1 , Figure 1 It is a schematic diagram of the architecture of the data query system 100 provided in an embodiment of the present application. To support a data query application, a terminal (terminal 401 and terminal 402 are shown as examples) is connected to a server 200 via a network 300. The network 300 may be a wide area network or a local area network, or a combination of the two.

[0060] The terminal (such as terminal 401, terminal 402) is used to display a conversation interface including at least one query range in a graphical interface (graphic interface 4011 and graphical interface 4012 are shown as examples), and the user can select the query range in the conversation interface of the terminal (such as terminal 401, terminal 402) and enter the query text for indicating data query. The terminal (such as terminal 401, terminal 402) is also used to output the target data corresponding to the query text in the conversation interface after the server 200 returns the target data. The server 200 is used to obtain the query text for indicating data query sent by the terminal (such as terminal 401, terminal 402), and obtain the target prompt template associated with the query text for generating structured sentences. The server 200 is also used to query the target data corresponding to the query text in the target data source corresponding to the target prompt template based on the query text and the target prompt template, and return the query-obtained target data to the terminal (such as terminal 401, terminal 402), so that the terminal (such as terminal 401, terminal 402) outputs the target data in the conversation interface.

[0061] The terminals (such as terminal 401 and terminal 402 ) may be implemented as various types of terminals such as laptop computers, tablet computers, desktop computers, set-top boxes, smart phones, smart speakers, smart watches, smart TVs, and vehicle-mounted terminals.

[0062] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0063] See also Figure 2 , Figure 2 4 is a schematic diagram of the structure of an electronic device 400 for implementing a data query method provided in an embodiment of the present application. The electronic device 400 may be a terminal (such as terminal 401 , terminal 402 ) or a server 200 . Figure 2 The electronic device 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the electronic device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .

[0064] The processor 410 may be an integrated circuit chip having signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0065] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0066] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0067] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0068] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0069] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0070] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless LAN (WiFi), and Universal Serial Bus (USB), etc.;

[0071] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0072] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0073] In some embodiments, the data query device provided in the embodiments of the present application can be implemented in a software manner. Figure 2 The data query device 455 stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, etc., and includes the following software modules: an acquisition module 4551, a generation module 4552 and a query module 4553. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0074] In other embodiments, the data query device provided in the embodiments of the present application can be implemented in hardware. As an example, the data query device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the data query method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other electronic components.

[0075] In some embodiments, the electronic device 400 can implement the data query method provided in the embodiment of the present application by running various computer executable instructions or computer programs. For example, computer executable instructions can be microprogram-level commands, machine instructions or software instructions. The computer program can be a native program or software module in the operating system; it can be a native application (APPlication, APP), that is, a program that needs to be installed in the operating system to run, such as an instant messaging APP; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0076] Below, the data query method provided by the embodiment of the present application will be described in conjunction with the accompanying drawings. As mentioned above, the electronic device 400 that implements the data query method of the embodiment of the present application can be a terminal (such as terminal 401, terminal 402), a server 200, or a combination of the two. Therefore, the execution subject of each step will not be repeated below.

[0077] The data query method of the embodiment of the present application is described by taking the execution subject as the server 200 as an example. Figure 3A , Figure 3A This is a first flow chart of the data query method provided in the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained.

[0078] In step 101, a query text for indicating a data query is obtained, and a target prompt template for generating a structured sentence associated with the query text is obtained.

[0079] The target prompt template includes metadata information for describing the data source, and different prompt templates correspond to different data sources.

[0080] Here, the query text can be a text for data query input by the user using natural language, such as Chinese text input using Chinese. The query text describes the query conditions for the user's query requirements. In an embodiment of the present application, the structured statement is a standardized statement written based on the structured query language for querying, inserting, updating, deleting, etc. for the operation library.

[0081] The data source in the embodiment of the present application can be a data table or a database. Different data sources correspond to different prompt templates. The prompt template includes metadata information for describing the data source. The metadata information can provide descriptive information about the data source, including but not limited to data type, data format, data source, data structure, data label, data quality, data update time, data ownership and data version. Metadata information can be used to understand the nature and characteristics of the data source.

[0082] Through the above embodiments, there is an association relationship between the query text and the prompt template, and there is an association relationship between the prompt template and the data source. After obtaining the query text for indicating data query, the target data source to which the data required to be queried by the query text belongs can be located by directly obtaining the target prompt template associated with the query text, so that the positioning of the target data source is more accurate, thereby improving the efficiency of data query.

[0083] In some embodiments, see Figure 3G Before step 101, corresponding prompt templates are pre-built for different data sources. The prompt templates corresponding to each data source can be built through steps 104 to 106.

[0084] In step 104, multiple data sources for data query are obtained, and attribute information of each data source is obtained.

[0085] In some embodiments, data for data query is pre-stored in at least one database, and the data may be stored in one or more data tables in the database. According to the content of the data stored in different data tables or different databases, the data tables or databases may be divided into multiple data sources. The divided data sources may be one data table or multiple tables in one of the databases, or one database or multiple databases. The data contents contained in different data sources are different.

[0086] Get the attribute information of each data source. The attribute information of the data source usually includes the data type, the structure information of the data source, the permission information to access the data, etc. Among them, the data type is used to determine the format and structure of the data in the data source. The structure information of the data source is used to determine the structure information of the data table corresponding to the data source, including the name of the data table, the name of the field included in the data table, and the data type of the field, etc. The permission information to access the data is used to determine whether the user has access rights to the data source.

[0087] In step 105, the attribute information of each data source is converted into metadata information corresponding to each data source.

[0088] In some embodiments, the attribute information of a data source usually characterizes the characteristics of the data source itself and describes the physical or logical structure of the data source. The metadata information of a data source includes not only the attribute information of the data source, but also the meaning of the data in the data source, the relationship between the data, the format of the data, instructions for use, etc. The metadata information can provide more comprehensive description information for the data source.

[0089] For any data source, the attribute information of any data source is converted into the metadata information corresponding to the any data source in the same format.

[0090] In some examples, if the attribute information of data source A includes: data table name "aa"; field name "b", field type "string"; field name "c", field type "string"; field name "d", field type "int". After the attribute information of data source A is converted into metadata information of data source A, the metadata information may include: data source A is data table "aa", and the data table "aa" is stored in database A. Data table "aa" represents xxx meaning. Data table "aa" includes the following three fields: field name "b", field type "string", this field represents xxx meaning; field name "c", field type "string", this field represents xxx meaning; field name "d", field type "int", this field represents xxx meaning. The above "xxx" are all referential descriptions, and the specific description needs to be based on the actual attribute information of the data source.

[0091] Through the above embodiment, the attribute information of the data source is converted into the corresponding metadata information of the data source, and descriptive information such as the data structure of the data source, the meaning of the data and the relationship between the data can be obtained, so that when generating structured statements based on the target prompt template, the data source can be understood more comprehensively and accurately, making the structured query statements generated for the data source corresponding to the query text more accurate.

[0092] In step 106, a prompt template corresponding to each data source is generated based on the metadata information of each data source.

[0093] In some embodiments, the prompt template is used to assist in generating a more accurate structured statement from the corresponding query text. Therefore, constructing a suitable prompt template, giving a reasonable description of the data source in the prompt template, and adding relevant prompts for generating structured statements can help better understand the data source and query text when subsequently generating structured statements corresponding to the query text, and improve the accuracy of generating structured statements.

[0094] In some examples, a job role can be defined in the prompt template, such as "You are an expert in writing structured sentences." Then, based on the metadata information of the data source, the relevant information of the data source is described. Finally, relevant prompts for generating structured sentences are given. For example, "Please combine the data source situation described above and generate structured sentences corresponding to the query text based on the query text." Rules and requirements for generating structured sentences can also be added to the prompt template. Through the above mode, the corresponding prompt template can be generated for any data source based on the metadata information of any data source.

[0095] Through the above embodiment, by pre-building a prompt template corresponding to the data source, when the user performs a data query, the prompt template can be used to directly locate the data source corresponding to the query text, thereby improving the data query efficiency. In addition, based on the metadata information describing the data source in the prompt template and the related prompts about the structured sentence, the accuracy of generating the structured sentence corresponding to the query text can be better improved.

[0096] In some embodiments, in step 105, when the attribute information based on the data source is converted into metadata information corresponding to the data source, the metadata information has the same format. However, in different usage scenarios, the metadata format used may be different. The choice of metadata format may depend on factors such as the type, scale, and usage scenario of the data in the data source. Metadata format generally refers to a specific format and standard for storing and organizing metadata.

[0097] Therefore, after step 105, the metadata information may also be processed in the following manner.

[0098] The target metadata format is obtained, and the format of the metadata information of each data source is converted into the target metadata format.

[0099] In some embodiments, a target metadata format that conforms to the current usage scenario is obtained, and the metadata information of each data source is formatted in advance based on the target metadata format. This can ensure that when a structured statement is generated through a prompt template constructed based on the metadata information, accurate relevant information about the data source can be obtained.

[0100] In some cases, if the data used for data query is stored in different data warehouses, and the formats and structures of data stored in different database warehouses may be different, the metadata information determined for different data sources may be incomplete or in inconsistent formats, resulting in an inability to better understand information such as the data storage structure of the data source based on the metadata information in the prompt template.

[0101] Therefore, the embodiment of the present application can migrate the data tables in different data warehouses to a pre-built storage layer in advance to perform unified physical storage of the data. First, the metadata information of each data source is obtained, and with the support of the data synchronization algorithm, the data of the source data source is read through the data synchronization engine according to the metadata information, and the data is written to the pre-built storage layer. During the data synchronization process, the data of different data sources can be associated to achieve data integrity and accuracy. Finally, according to the data structure and storage relationship of each data source in the storage layer, the metadata information of each data source is updated, and the format of the updated metadata information of each data source is converted to the target metadata format. Among them, the data synchronization algorithm can adopt any one of the full table synchronization algorithm, incremental synchronization algorithm, parallel synchronization algorithm or other synchronization algorithms, without limitation.

[0102] Through the above embodiment, the data in different data warehouses are migrated to the storage layer for management in advance, which can better standardize the metadata information corresponding to each data source and improve the accuracy of generating structured query statements using prompt templates. In addition, unified data management at the storage layer can solve the problem that different data warehouses cannot cross-extract data and improve the query efficiency when querying data.

[0103] In some embodiments, step 106 may also be implemented by the following steps.

[0104] Based on the metadata information of each data source in the target metadata format, a prompt template corresponding to each data source is generated.

[0105] The process of generating a prompt template corresponding to the data source based on the metadata information can be found in the above description of step 106, which will not be repeated here.

[0106] In some embodiments, see Figure 3B , Figure 3A The step 101 of "obtaining a target prompt template associated with the query text for generating a structured sentence" can be implemented through the following steps 1011 to 1013, which are described in detail below.

[0107] In step 1011, a data query instruction for performing a data query on a query text is received.

[0108] The data query instruction carries a target query scope corresponding to the query text. The target query scope is obtained by selecting at least one query scope displayed in the conversation interface.

[0109] In the embodiment of the present application, different query scopes can be divided according to the query requirements of relevant users in the downstream application to which the data query method is applied. Different query scopes correspond to different data sources, and the data required to be queried by the query text belonging to the same query scope all come from the data source corresponding to the query scope.

[0110] In some embodiments, the data query instruction may be generated by performing a query operation on the query text after the user inputs the query text based on natural language. Before or after the user inputs the query text, the user may select at least one query range displayed in the conversation interface to determine the target query range corresponding to the query text. The data query instruction carries the query text and the target query range corresponding to the query text.

[0111] In some examples, for a downstream application, different query ranges are divided according to the query requirements of each relevant user in the downstream application. Different query ranges correspond to different data sources. The example of a terminal (such as terminal 401 and terminal 402) generating a data query instruction and a server 200 receiving the data query instruction is used for explanation. Figure 4 , Figure 4 4 is a schematic diagram of a conversation interface provided by an embodiment of the present application. A conversation interface is displayed to a user in a terminal (such as terminal 401, terminal 402), and a drop-down list 4101 of at least one query range divided as described above, a text input box 4102, a control 4103 for indicating a query, and a display area 4104 are displayed in the conversation interface. The user can select a target query range that meets his or her query needs from at least one query range displayed in the drop-down list 4101 of the conversation interface, and enter a query text belonging to the target query range in the text input box 4102 of the conversation interface based on natural language, and then click the control 4103 for indicating a query operation to perform a data query. The terminal (such as terminal 401, terminal 402) generates a data query instruction in response to a click operation on the control 4103, and sends the data query instruction to the server 200. The data query instruction carries the query text and the target query range selected by the user, and the target query range has a corresponding relationship with the query text. When the server 200 receives a data query instruction sent by a terminal (such as the terminal 401 or the terminal 402 ), the server 200 may obtain the query text carried in the data query instruction and the target query range corresponding to the query text.

[0112] In step 1012, a mapping relationship between different query scopes and prompt templates is obtained.

[0113] In some embodiments, mapping relationships are pre-established for different query scopes and prompt templates. Before step 1012, the mapping relationships between different query scopes and prompt templates can be established through the following process.

[0114] Determine the data source to which the data corresponding to each query range belongs, and determine the prompt template of each data source. Based on the data source to which the data corresponding to each query range belongs, and the prompt template of each data source, establish a mapping relationship between each query range and the corresponding prompt template.

[0115] In some embodiments, the data corresponding to the query scope may be the data required to be queried by the query text belonging to the query scope. All data used for data query may be divided into different data sources according to the different data contents. The data corresponding to different query scopes belong to different data sources. Different data sources correspond to different prompt templates. The data sources corresponding to different query scopes may be determined according to the data corresponding to the query scope, and the prompt templates corresponding to different query scopes may be determined according to the prompt templates corresponding to the data sources. For any query scope, a mapping relationship between the any query scope and the prompt template of the data source to which the data corresponding to the any query scope belongs may be established according to the data source to which the data corresponding to the any query scope belongs and the prompt template of the data source to which the any query scope belongs. According to the above process of establishing a mapping relationship for any query scope, a mapping relationship between each query scope and the corresponding prompt template is established.

[0116] In an embodiment of the present application, when a data query instruction is received, a mapping relationship between different pre-established query scopes and prompt templates can be directly obtained.

[0117] In step 1013, based on the mapping relationship and the target query scope, a prompt template having a mapping relationship with the target query scope is determined, and the prompt template having a mapping relationship with the target query scope is used as a target prompt template for generating a structured sentence.

[0118] In some embodiments, after obtaining the target query scope corresponding to the query text, a search can be performed in the mapping relationship between the pre-established query scope and the prompt template to determine the prompt template that has a mapping relationship with the target query scope. Since the query text and the target query scope are in a corresponding relationship, the target query scope and the prompt template that has a mapping relationship are in a corresponding relationship. Therefore, based on the above two corresponding relationships, the corresponding relationship between the prompt template that has a mapping relationship with the target query scope and the query text can be determined, and the prompt template that has a mapping relationship with the target query scope can be used as the target prompt template corresponding to the query text for generating a structured statement.

[0119] Through the above-mentioned embodiments, a mapping relationship between the query scope and the prompt template is pre-established, and the prompt template and the data source also have a corresponding relationship. In this way, when a user performs a data query, he selects a corresponding query scope according to his own query needs, and inputs a query text belonging to the query scope, and then the target data source to which the data queried by the query text belongs can be accurately located according to the target prompt template corresponding to the query scope, thereby improving data query efficiency. The target data source is the data source corresponding to the target prompt template.

[0120] Continue reading Figure 3A , and the description will continue with step 101 above.

[0121] In step 102, a structured query statement corresponding to the query text is generated based on the query text and the target prompt template.

[0122] Here, the structured query statement belongs to the above structured statement, complies with the structured query language, and is used to perform query operations on data in the data source.

[0123] Through the above embodiment, by using the metadata information of the target data source corresponding to the query text included in the target prompt template, a structured query statement for data querying the target data source corresponding to the query text can be generated more accurately. The generated structured query statement is more in line with the query requirements of the query text in the target data source, and is suitable for data querying the target data source corresponding to the query text, thereby improving the accuracy of extracting target data from the target data source corresponding to the query text.

[0124] In some embodiments, see Figure 3C , Figure 3A Step 102 in the method can be implemented through step 1021.

[0125] Step 1021, obtain a target language model, call the target language model, and predict a structured query statement corresponding to the query text based on the query text and the target prompt template.

[0126] In some embodiments, the target language model can directly use a pre-trained large language model (LLM), such as the GPT-3 language model and the mixed large model. The large language model is an artificial intelligence model that can perform a wide range of tasks, including text summarization, translation, sentiment analysis, etc. In addition to directly using a pre-trained large language model, you can also fine-tune the open source large language model on a suitable data set based on your actual needs to adapt to the user's own language tasks and obtain the target language model.

[0127] For some companies, their own corporate data is confidential. Therefore, companies can use a fine-tuned large language model as the target language model, which can ensure their own data security and reduce the occurrence of data leakage. If the company directly uses a pre-trained large language model as the target language model, it can choose a trusted large language model.

[0128] In some embodiments, the above step 1021 can be implemented through the following process.

[0129] Get the query text and the target prompt template corresponding to the query text. The target prompt template is configured with a set of preset placeholders, which can be replaced by specific query information (for example, table name, field name, etc.).

[0130] Parse the query text to fill the query information (such as table name, field name, condition, etc.) in the query text into the target prompt template to form a structured input.

[0131] Call the target language model and pass the filled structured input to the target language model to understand the semantic content of the input.

[0132] The target language model generates a structured query statement corresponding to the query text based on the input semantic content and the metadata information in the target prompt template.

[0133] In this process, the target prompt template plays a guiding role, helping the target language model to better understand the user's natural language input (i.e., query text) and generate accurate structured query statements. The advantage of this method is that it can transform complex natural language processing problems into structured template matching and filling problems, thereby reducing the complexity of processing and improving the accuracy of converting query text into structured query statements.

[0134] Below, we take the target language model with Transformer structure as an example to explain the process of the target language model predicting the input query text and target prompt template. Figure 5 , Figure 5 It is a schematic diagram of the principle of the target language model provided in the embodiment of the present application.

[0135] based on Figure 5 The model structure of the Transformer-based target language model is shown, and the processing process of the target language model can be described as the following process.

[0136] The filled structured input obtained by using the query text and the target prompt template is passed to the input layer (Embedding) 510 of the target language model. The input layer 510 will perform appropriate preprocessing on the input, such as word segmentation, removal of stop words, etc., and convert the input into the format required by the model.

[0137] In the encoding layer (Encoding) 520 of the target language model, the target language model uses a self-attention mechanism to capture important information in the input sequence. The self-attention mechanism can model the relationship in the input sequence and calculate the representation of each position, so that the model can better understand the content of the input.

[0138] Using a multi-layer encoder (Encoder) 530 ( Figure 5 A three-layer encoder) and a multi-layer decoder (Decoder) 540 ( Figure 5 The three-layer decoder is schematically shown in Figure 3 to achieve sequence generation. These layers use self-attention mechanism to capture the relationship between the input sequence and generate corresponding output according to the target generation sequence.

[0139] Then, the output layer (Generate Output) 550 of the target language model will generate a vector representation representing the query, which can be used for subsequent query tasks. In an embodiment of the present application, the output layer will generate a vector representation representing a structured query statement, which is used in subsequent data query tasks.

[0140] In summary, the model structure of the Transformer-based target language model uses the self-attention mechanism and multi-layer Transformer structure to capture important information in the input sequence and generate corresponding output representations. The model continuously optimizes performance and can eventually convert the filled structured input into an accurate structured query statement.

[0141] In some embodiments, see Figure 3D , Figure 3A Step 102 in the method can also be implemented through steps 1022 to 1023.

[0142] Step 1022: Based on the database type, obtain the target query syntax of the database corresponding to the target data source.

[0143] In the embodiment of the present application, different data sources correspond to different prompt templates, and the target data source is the data source corresponding to the above target prompt template.

[0144] In some embodiments, the metadata information includes the database type of the database corresponding to the target data source. When the target data source is a data table, the database corresponding to the target data source is the database to which the data table belongs; when the target data source is a database, the database corresponding to the target data source is the target data source itself.

[0145] Common database types include relational databases, non-relational databases, object-oriented databases, graph-based databases, and the like. Relational databases include MySQL, Oracle, SQL Server, and the like, and non-relational databases include MongoDB, Redis, Cassandra, and the like. Different types of databases have different data structures, data models, and the like, so the query syntax of the structured query statements used when querying the database is different. Therefore, in an embodiment of the present application, the database type of the database corresponding to the target data source is configured in the metadata information. The database type configured in the metadata information of the target prompt template is used to indicate the target query syntax of the structured query statement for querying the database corresponding to the target data source.

[0146] In some examples, the metadata information in the target prompt template includes "Please convert the query text into a structured query statement for a MySQL database." In the metadata information, the database type is "MySQL database." "MySQL database" indicates the target query syntax of the structured query statement when performing a query operation on the MySQL database corresponding to the target data source.

[0147] Step 1023: Based on the query text and the target prompt template, a structured query statement corresponding to the query text and conforming to the target query grammar is generated.

[0148] In some embodiments, the target language model in step 1021 can be used to complete the process of generating a structured query statement. Exemplarily, the query text is parsed to fill the query information (such as table name, field name, condition, etc.) in the query text into the target prompt template to form a structured input. The structured input also includes the database type extracted from the metadata information. The target language model is called, and the filled structured input is passed to the target language model to understand the semantic content of the input. Based on the semantic content of the input and the metadata information in the target prompt template, the target language model generates a structured query statement that conforms to the target query grammar corresponding to the query text.

[0149] Through the above embodiment, by configuring the database type in the metadata information of the target prompt template, it is possible to determine the target query syntax for data query on the target data source when determining the target prompt template corresponding to the query text, thereby generating a structured query statement that conforms to the query operation on the database corresponding to the target data source, thereby improving the accuracy of the generation of the structured query statement.

[0150] In some embodiments, Figure 3D Step 1022 in can also be implemented in the following manner.

[0151] Based on the database type, obtain the target query syntax of the target data table in the database.

[0152] In some embodiments, the database corresponding to the target data source includes at least one data table. When the database corresponding to the target data source includes only one data table, the target data source is the data table. When the database corresponding to the target data source includes multiple data tables, the target data source can be the database or multiple tables in the database. Regardless of whether the target data source is a database or a data table, the target data belongs to the data table. Therefore, the metadata information also includes the table identifier of the target data table to which the target data belongs, and the target field in the target data table to which the target data belongs. Among them, the target data is the data to be queried corresponding to the query text. The target data is part of the data contained in the target field in the target data table. The table identifier of the target data table can adopt the table name of the target data table, and the target field can adopt the target field name in the target data table.

[0153] Based on the metadata information of the target prompt template and the query text, the target data table to which the target data corresponding to the query text belongs is determined. The target data table is the target data source to which the target data queried by the query text belongs. Therefore, based on the database type of the database to which the target data table belongs in the metadata information, the target query syntax corresponding to the target data table is determined, which can make the syntax of the structured query statement generated for the target data source corresponding to the query text more accurate.

[0154] In some embodiments, Figure 3D Step 1023 in can also be implemented in the following way.

[0155] Based on the query text, the table identifier of the target data table and the target field, a structured query statement that corresponds to the query text and conforms to the target query grammar is generated.

[0156] In some embodiments, the metadata information of the target prompt template describes the table identifiers of all data tables involved in the target data source and the field information in all data tables. By analyzing the query text, the target data table to which the target data corresponding to the query text belongs can be determined in the table identifier described in the metadata information. According to the query content described by the query text, the target field in the target data table to which the target data belongs is determined in the field information described in the metadata information for the target data table.

[0157] Based on the database type of the database to which the target data table belongs in the metadata information, the target query syntax of the structured query statement corresponding to the query text is determined. Then, based on the table identifier and target field of the target data table determined in the metadata information and the query text, a structured query statement that conforms to the target query syntax is generated. The structured query statement corresponding to the query text is a structured statement that searches for target data in the target field of the target data table based on a structured language description.

[0158] For some examples, see Figure 6 , Figure 6 It is a schematic diagram of a target prompt template provided in an embodiment of the present application. Figure 6 The target prompt template shown is a prompt template corresponding to the target query scope of "optimization of low-consumption groups in information flow". The target prompt template describes metadata information for the target data source corresponding to the target prompt template. The target data source corresponding to the target prompt template is the data table "ads_qb_user_indic_qimei36_mid_di" in database A.

[0159] If the generated structured query statement is a SQL query statement. Figure 6 , the metadata information of the target prompt template describes the target data source as follows:

[0160] “Question: You are an expert in writing SQL code. The database type of database A is "MySQL database". There is a table in database A named ads_qb_user_indic_qimei36_mid_di; this table represents the DAU of the market and the DAU of information flow consumption in different dimensions every day.

[0161] There are 13 fields in this data table, namely:

[0162] Field name: imp_date, bigint format, meaning: date partition;

[0163] Field name: gender, string format, meaning: gender;

[0164] **Hide other fields**

[0165] Field name: qb_dau, bigint format, represents: market DAU;

[0166] Field name: feed_cs_dau, bigint format, represents: information flow consumption DAU;

[0167] Please combine the above data table and generate a SQL query code for "MySQL database" according to the following query text, and return a complete SQL query statement. Do not use non-existent fields. SQL query statement requirements: Each subquery needs to limit the date partition condition, and the data is queried from the data table provided above.

[0168] Query text: In the past three days, the market DAU, consumer DAU, and message penetration rate of different age groups, city levels, and genders are aggregated as SUM, sorted in ascending order by consumer penetration rate, and the market DAU is greater than 10,000. "

[0169] It should be noted that the "hiding other fields" in the target prompt template provided above does not mean that other fields in the data table "ads_qb_user_indic_qimei36_mid_di" are not described, but for the convenience of Figure 6 The display schematically describes the four fields in the data table. The target prompt template actually constructed includes the description of the field name, format, and meaning of the 13 fields in the data "ads_qb_user_indic_qimei36_mid_di".

[0170] Figure 6The target data source corresponding to the target prompt template shown is the data table "ads_qb_user_indic_qimei36_mid_di". Therefore, the data table "ads_qb_user_indic_qimei36_mid_di" is the target data table corresponding to the query text. Based on the query information described in the query text and the description of the 13 fields of the data table in the metadata information, it can be determined that the target fields to which the target data corresponding to the query text belongs include: 6 fields corresponding to age group, city level, gender, market DAU, information flow consumption DAU and consumption penetration rate. Based on the field information provided by the metadata information in the target prompt template, the field names of the 6 target fields can be determined respectively. Based on the description of the database type in the target prompt template: "The database type of database A is "MySQL database"", it can be determined that the database type of database A to which the target data table "ads_qb_user_indic_qimei36_mid_di" belongs is "MySQL database".

[0171] Based on the query text, the target data table, and the target field in the target data table, the generated SQL query statement that is consistent with data query for the MySQL database is:

[0172] ""SELECT age_range,city_level,gender,SUM(qb_dau)AS total_dau,SUM(feed_cs_dau)AS total_feed_cs_dau,ROUND(SUM(feed_cs_dau) / SUM(qb_dau),4)ASfeed_cs_penetration

[0173] FROM ads_qb_user_indic_qimei36_mid_di

[0174] WHERE imp_date = 20231007

[0175] GROUP BY age_range, city_level, gender

[0176] HAVING SUM(qb_dau)>10000

[0177] ORDER BY feed_cs_penetration ASC;""

[0178] The comment explanation of this SQL code is: First, the required fields are selected, including age group (age_range), city level (city_level), gender (gender), market DAU (qb_dau), information flow consumption DAU (feed_cs_dau) and consumption penetration rate (total_feed_cs_dau). Next, query data from the data table "ads_qb_user_indic_qimei36_mid_di" and limit the date partition condition to 20231007. Then, group by age group, city level and gender, and aggregate the data in each group to calculate the market DAU, information flow consumption DAU and consumption penetration rate. Next, use the HAVING clause to filter out data with a market DAU greater than 10,000. Finally, sort in ascending order by consumption penetration rate.

[0179] Through the above-mentioned embodiments, the target data source is described in the target prompt template with metadata information, including the data table involved in the target data source and the field information in the data table, which can help the target language model better understand the structure of the target data source. The database type and related prompts described in the metadata information can help the target language model better understand the query syntax and generation rules of the structured query statement. In this way, the target language model can be guided to generate a structured query statement that is more in line with the user's intention, and the accuracy and stability of the target language model when generating structured query statements based on the query text can be improved. At the same time, through the metadata information described for the target data source, it can also help the target language model to more accurately determine the target data table and target field to which the target data corresponding to the query text belongs, improve the ability to locate the target data source corresponding to the query text and the accuracy of generating structured query statements for query operations on the target data source of the query text, thereby improving data query efficiency.

[0180] In some embodiments, the target data source corresponding to the target query scope to which the query text belongs may include multiple data sources divided. In this case, the target data source corresponds to multiple target prompt templates. Therefore, before step 102, the target prompt template for generating a structured query statement can also be determined in the following manner from the multiple target prompt templates.

[0181] The data source corresponding to each target prompt template is obtained. According to the matching degree between the data source corresponding to each target prompt template and the query text, the target prompt template whose matching degree meets the expected matching degree is determined.

[0182] In some embodiments, the target data corresponding to the query text comes from the target data source corresponding to the target query scope. However, in the case where the target data source corresponding to the target query scope includes multiple divided data sources, the data source to which the target data corresponding to the query text belongs may be only one of the multiple data sources corresponding to the target query scope. Therefore, in order to more accurately locate the data source to which the target data queried by the query text belongs, it is possible to select from the prompt templates corresponding to the multiple data sources associated with the target query scope to determine the target prompt template for generating a structured query statement.

[0183] In some embodiments, the data source corresponding to each target prompt template is determined according to the metadata information describing the corresponding data source in each target prompt template, and the description of the query condition in the query text is understood and analyzed to determine the matching degree between the data source corresponding to each target prompt template and the query text. The higher the matching degree between the data source and the query text, the greater the possibility that the target data corresponding to the query text comes from the data source.

[0184] In an achievable manner, the matching expectation may be set to the highest matching degree. Among the target prompt templates associated with the target query scope, the target prompt template whose corresponding data source has the highest matching degree with the query text is determined.

[0185] In another achievable manner, a matching threshold may be preset, and the matching expectation is set to be greater than or equal to the matching threshold. Among the target prompt templates associated with the target query scope, a target prompt template whose corresponding data source matches the query text greater than or equal to the matching threshold is determined.

[0186] The above step 102 can be implemented through the following process.

[0187] Based on the query text, the matching degree meets the expected target prompt template, and a structured query statement corresponding to the query text is generated.

[0188] For the process of generating a structured query statement corresponding to the query text based on the query text and the target prompt template, please refer to the above description of step 102, which will not be repeated here.

[0189] In some embodiments, when the match expectation is set as the match degree threshold, the determined target prompt templates whose match degree meets the match expectation may also be multiple. In this case, for any target prompt template whose match degree meets the match expectation, a structured query statement corresponding to the query text can be generated based on the query text and any target prompt template whose match degree meets the match expectation.

[0190] In some examples, if the target data source corresponding to the target query range to which the query text belongs has three target prompt templates, after matching processing, it is determined that the target prompt template 1 and the target prompt template 2 meet the matching expectations among the three target prompt templates. Then, based on the query text and the target prompt template 1, a structured query statement corresponding to the query text can be generated, and based on the query text and the target prompt template 2, another structured query statement corresponding to the query text can be generated.

[0191] Through the above embodiment, when there are multiple target prompt templates corresponding to the target data source corresponding to the target query range to which the query text belongs, the multiple target prompt templates can be screened by the matching degree, so that the target data source to which the target data corresponding to the query text belongs can be further accurately determined in the data source corresponding to the target query range.

[0192] In some embodiments, when the target data source corresponding to the target query scope to which the query text belongs includes multiple data sources, that is, when the target query scope corresponds to multiple target prompt templates, any one of the multiple target prompt templates can be selected as the target prompt template for generating a structured query statement. Alternatively, for multiple target prompt templates corresponding to the target query scope, based on the query text and any target prompt template, structured query statements corresponding to the query text are generated respectively.

[0193] Continue to see Figure 3A , and the description continues with step 102 above.

[0194] In step 103, based on the generated structured query statement, the target data source corresponding to the target prompt template is searched to obtain the target data corresponding to the query text.

[0195] In some embodiments, see Figure 3E , Figure 3A The illustrated step 102 may be implemented through steps 1031A to 1033A.

[0196] Step 1031A, obtaining address information of a target data source associated with the target prompt template.

[0197] In some embodiments, a correlation is pre-built between the prompt template and the address information of the data source corresponding to the prompt template. Based on the pre-built correlation between the prompt template and the address information, the address information of the target data source associated with the target prompt template can be determined.

[0198] In the case where the target data source is a data table, the address information of the target data source is the address information of the database to which the data table belongs. In the case where the target data source is a database, the address information of the target data source is the address information of the database.

[0199] The address information usually includes the following important information: Host Address, the host address or IP address of the database server, indicating the physical location of the database server; Port Number, used to identify the port number of the database service, which is the port on the database server used to detect database connections; Database Name, the name of the database to be connected, used to specify the database to be used when connecting to the database. Username and Password are used to verify the identity of the connection when connecting to the database.

[0200] Address information is the basic information required to establish a connection with the database corresponding to the data source. Providing accurate address information can successfully establish a communication connection with the database corresponding to the target data source. When querying the database corresponding to the data source, other information may need to be provided according to the specific database type and configuration, such as database type (Mysql, Oracle, SQL Server, etc.), connection protocol (such as TCP / IP), character set settings, SSL certificates, etc. In actual operation, database security measures should also be considered, such as encrypted connections, firewall settings, etc., to ensure the security and stability of the communication connection.

[0201] In some examples, the prompt template may establish an association relationship with a data source identifier of a data source corresponding to the prompt target. The data source identifier is a unique identifier that determines the identity of the data source. The data source identifier is used to point to or associate with the address information of the data source. Based on the association relationship between the prompt template and the data source identifier, the target data source identifier associated with the target prompt template is obtained, and the target address information pointed to or associated with the target data source identifier is used as the address information of the target data source associated with the target prompt template.

[0202] When the target data source is a data table, the data source identifier of the target data source is the data table identifier of the data table, and the data table identifier points to or is associated with the address information of the database to which the data table belongs. When the target data source is a database, the data source identifier of the target data source is the database identifier of the database, and the database identifier points to or is associated with the address information of the database.

[0203] In other examples, the prompt template may directly establish an association relationship with the address information of the data source corresponding to the prompt template. Target address information having a mapping relationship with the target prompt template is obtained as the address information of the target data source associated with the target prompt template.

[0204] The following explanation is based on the example of establishing an association between a prompt template and a data source identifier. Figure 7 , Figure 7 A schematic diagram of the association relationship between the prompt template and the data source identifier provided in the embodiment of the present application. The association relationship is described by way of example with prompt template 1. Prompt template 1 has an association relationship with data source identifier 1. Data source identifier 1 is the identity identifier of data source 1 corresponding to prompt template 1. Data source identifier 1 can point to the address information of data source 1. Figure 7 The target data source identifier associated with the target prompt template is determined to be data source identifier 3. The address information pointed to by data source identifier 3 is the address information of the target data source associated with the target prompt template.

[0205] In the case where the data source corresponding to the prompt template is multiple databases, the data source identifier associated with the prompt template includes the database identifiers corresponding to the multiple databases. The database identifier is a unique identifier that determines the identity of the database. The database identifier is associated with the address information of the database corresponding to the database identifier. Please continue to refer to Figure 7 The data source identifier n associated with the prompt template n includes the database identifier 1 and the database identifier 2. Therefore, the prompt template n is associated with two address information, namely, the address information of the database corresponding to the database identifier 1 and the address information of the database corresponding to the database identifier 2.

[0206] Through the above embodiment, by establishing an association relationship between the prompt template and the address information of the data source, when performing data query on the query text, there is no need to separately obtain the address information of the target data source, thereby improving data query efficiency.

[0207] Step 1032A: Establish a communication connection with the target data source based on the address information of the target data source, and create a data source connection object of the target data source based on the communication connection.

[0208] In some embodiments, to establish a communication connection with a target data source and create a data source connection object based on the communication connection, the following process may be used, wherein the data source connection object generally refers to a component or object used to implement data connection and data operation in various data processing and business logic applications.

[0209] The address information of the target data source is obtained, that is, the address information associated with the target prompt template obtained in step 1031A. The address information may include necessary information such as data source type (such as database, API interface, etc.), host address, port, database name, user name, password, etc.

[0210] According to the address information of the target data source, use the corresponding communication protocol (such as TCP / IP, HTTP, etc.) to establish a communication connection with the target data source. The specific connection method and parameter settings depend on the type and configuration of the target data source.

[0211] After successfully establishing a communication connection with the target data source, you can use the corresponding library or tool to create a data source connection object based on the actual situation of the target data source and the corresponding programming language. For example, in the Java programming language, you can use the Java Database Connectivity (Java DataBase Connectivity, JDBC library) to create a database connection object, and in the Python programming language, you can use a database toolkit (such as: SQLAlchemy library) to create a database connection object.

[0212] Step 1033A, based on the data source connection object, execute the structured query statement in the target data source corresponding to the target prompt template to obtain the target data corresponding to the query text.

[0213] In some embodiments, after creating a data source connection object, the data source connection object can be used to verify whether the communication connection with the target data source is successful, and a structured query statement corresponding to the query text can be executed in the target data source to perform a query operation on the target data source, and obtain the target data in the target field belonging to the target data table from the target data source.

[0214] It should be noted that the implementation process of establishing a communication connection with the target data source and creating a data source connection object will vary depending on the type of the target data source, data access method, and programming language. Ensure that data security is considered when establishing a communication connection with the target data source and operating the target data source to reduce the leakage of sensitive information and improper operations.

[0215] In some embodiments, the target prompt template also includes a data source identifier of the target data source. Figure 3F , Figure 3A The illustrated step 103 may also be implemented through steps 1031B to 1032B.

[0216] Step 1031B: extract the data source identifier of the target data source from the target prompt template.

[0217] In the embodiment of the present application, the data source identifier is a unique identifier for identifying the identity of the data source. The data source identifier points to or is associated with the address information of the data source corresponding to the data source identifier.

[0218] In some embodiments, the data source identifier of the target data source may be directly configured in the target prompt template corresponding to the target data source. After obtaining the query text, the data source identifier is extracted from the target prompt template corresponding to the query text, so that after generating the structured query statement, a communication connection is established with the target data source based on the extracted data source identifier.

[0219] In some examples, Figure 6 Taking the target prompt template shown in FIG. 1 as an example, a description of the data source identifier for the target data source may be added to the target prompt template. Figure 6 Add the following to the target prompt template: "The data source corresponding to this prompt template is the data table "ads_qb_user_indic_qimei36_mid_di", and the data source identifier is "xxxx"". Among them, "xxxx" is only a description of the example, and should actually be filled in with the data table identifier of the data table "ads_qb_user_indic_qimei36_mid_di".

[0220] Step 1032B, based on the data source identifier of the target data source, execute the generated structured query statement in the target data source corresponding to the target prompt template to obtain the target data corresponding to the query text.

[0221] In some embodiments, step 1032B may be implemented by the following process.

[0222] Get the address information pointed to or associated with the data source identifier of the target data source. Based on the address information of the target data source, establish a communication connection with the target data source, and create a data source connection object of the target data source based on the communication connection. Based on the data source connection object, execute a structured query statement in the target data source corresponding to the target prompt template to obtain the target data corresponding to the query text.

[0223] The process after obtaining the address information can refer to the above description of step 1032A and step 1033A, which will not be repeated here.

[0224] See also Figure 3A , Figure 3A is a flow chart of the data query method provided in the embodiment of the present application, which will be combined with Figure 3A The steps shown are further explained.

[0225] In step 101, a query text for indicating a data query is obtained, and a target prompt template for generating a structured sentence associated with the query text is obtained.

[0226] The target prompt template includes metadata information for describing the data source, and different prompt templates correspond to different data sources.

[0227] In step 102, a structured query statement corresponding to the query text is generated based on the query text and the target prompt template.

[0228] In step 103, based on the generated structured query statement, the target data source corresponding to the target prompt template is searched to obtain the target data corresponding to the query text.

[0229] See also Figure 3H ,exist Figure 3A After step 103 shown, step 107 may be performed, which is described in detail below.

[0230] In step 107, in response to the data query instruction, the target data is returned to the electronic device corresponding to the data query instruction. The target data is used by the electronic device to output the target data corresponding to the query text in the conversation interface displaying the target query range.

[0231] In some embodiments, the electronic device corresponding to the data query instruction may be a server 200, or a terminal (such as terminal 401, terminal 402). The following description is made by taking the electronic device corresponding to the data query instruction as a terminal (such as terminal 401, terminal 402), and the electronic device 400 implementing the data query method as a server 200 as an example.

[0232] The graphical interface (such as graphical interface 4011 and graphical interface 4012) of the terminal (such as terminal 401 and terminal 402) displays a session interface, and the target query range is displayed in the session interface. After the user selects the target query range and enters the query text belonging to the target query range, the terminal (such as terminal 401 and terminal 402) generates a data query instruction for the query text and sends the data query instruction to the server 200. The data query instruction includes the query text and the target query range corresponding to the query text. After the server 200 receives the data query instruction and obtains the target data based on the data query instruction, it also responds to the data query instruction and returns the target data obtained by the query to the terminal (such as terminal 401 and terminal 402), so that the terminal (such as terminal 401 and terminal 402) outputs the target data in the session interface in a specified display form. The specified display form can be at least one of data text, graphics, tables, files, codes, or other display forms that can be used to reflect the essence of the target data, and the specified display form is not limited here.

[0233] For some examples, see Figure 4 , the graphical interfaces (such as graphical interface 4011 and graphical interface 4012) of the terminals (such as terminal 401 and terminal 402) display Figure 4The conversation interface shown. The user can select the target query range "Optimization of low-consumption groups in information flow" from the multiple query ranges displayed in the drop-down list 4101 of the conversation interface, and enter the query text belonging to the target query range in the text input box 4102 "In the past three days, the market DAU, consumption DAU, and message penetration rate of different age groups, city levels, and genders are aggregated in SUM, sorted in ascending order by consumption penetration rate, and the market DAU is greater than 10,000". The terminal (such as terminal 401 and terminal 402) generates a data query instruction for the query text in response to the user's click operation on the control 4103 for indicating a query operation, and sends the data query instruction to the server 200. Continue to see Figure 4 In response to the user's click operation on the control 4103, the terminal (such as the terminal 401 and the terminal 402) displays the query text in the display area 4104 in the form of text, and displays the target data in the session interface in the specified display form after the server 200 returns the target data. Figure 4 As shown, the target data can be displayed in the display area 4104 in the conversation interface in the form of a table.

[0234] Through the above embodiment, the target query scope is selected in the conversation interface according to the user's own query needs, and the target data source to which the target data corresponding to the query text belongs can be directly located based on the target prompt template corresponding to the target query scope. In addition, based on the description of the metadata information of the target data source in the target prompt template, the query text can be more accurately generated into a corresponding structured query statement to query and obtain the target data in the target data source.

[0235] exist Figure 3A After step 103, the following steps may be performed, which are described in detail below.

[0236] Execute the target data operation on the target data to obtain the target data after the operation. Send the target data after the operation so that the target data after the operation is displayed in the view interface as the query result of the query text.

[0237] In some embodiments, the query text also includes an operation text, and the operation text is used to indicate the target data operation to be performed on the target data. The data operations performed on the target data include sorting, aggregation, deletion, format conversion, etc. Some display methods of the view interface may include: web page interface, applet interface, mobile application interface, etc. The view interface can also be a conversation interface that displays the target query range. The display method of the view interface is not limited here, and it can be selected according to user needs in actual application to provide better user experience and convenient operation.

[0238] In some embodiments, after the target data is obtained through query, a data operation is performed on the target data based on the target data operation indicated by the operation text in the query text, which can be achieved in the following manner.

[0239] Based on the operation text and the target prompt template in the query text, a structured query statement corresponding to the query text is generated, and the structured query statement includes a structured statement for performing a target data operation on the target data.

[0240] For some examples, see Figure 6 , Figure 6 The operation text in the query text shown is "Sort in ascending order of consumption penetration rate, and filter the market DUA greater than 10,000". The structured query statement converted based on the operation text in the query text includes a structured statement for performing data operations on the target data: "HAVING SUM (qb_dau)>10,000; ORDERBY feed_cs_penetration ASC". Through the data source connection object, the structured query statement is executed in the target data source corresponding to the target prompt template to obtain the target data corresponding to the query text and perform target data operations on the target data.

[0241] Through the above embodiments, target data operations are performed on target data, so that when the target data is output on a view interface or a session interface through a specified display method, the target data is more in line with the data format required by the specified display method.

[0242] Figure 8 The schematic diagram of the architecture of the data query system provided for the embodiment of the present application. The data query system is built on a tool library, such as the LangChain framework. The LangChain framework is designed to help developers build end-to-end applications using large language models. It provides a set of tools, components, and interfaces that simplify the process of creating applications powered by large language models (LLMs) and chat models. The LangChain framework can easily manage interactions with large language models, link multiple components together, and integrate additional resources such as APIs and databases.

[0243] See also Figure 8 The data query system includes an interaction layer 810, a service layer 820, a model layer 830 and a storage layer 840.

[0244] Among them, the interaction layer 810 can provide users with the Hunyuan large model through a user interface (UI). The user interface (UI) refers to the interface seen by the user when interacting with a computer system, mobile application, website or other electronic device. It includes the interaction design between the user's input device (such as a keyboard, mouse, touch screen, etc.) and the system's output device (such as a display, printer, etc.). In some embodiments, data query services can be provided to users such as business, R&D, and operations in the form of a WEB terminal, and users can get the desired data query results through a simple Chinese dialogue in the WEB terminal. During use, the user can interact in the form of a dialogue in the conversation interface of the WEB terminal, and use natural language to enter the query text, such as Chinese text, in the conversation interface. The service layer 820 of the data query system can output the target data corresponding to the query text to the user in different display forms such as text, data charts, and data files according to the conversation content (query text) in the conversation interface.

[0245] The service layer 820 is mainly based on the tool library (such as the LangChain framework) to achieve the combination with the interaction layer 810, the model layer 830, and the storage layer 840 built for different downstream applications. This data query system mainly uses the model component (Models component), prompt component (Prompts component), chain component (Chains component) and agent component (Agents component) of the LangChain framework. These four components can realize the coupling of the interaction layer 810, the model layer 830, and the storage layer 840. The following is a detailed introduction.

[0246] I. Model component: refers to the large language model (LLM) component, which is the core component of the LangChain framework and has a profound impact on the results of data queries. Here, the model component can provide a standard API interface for interacting with different large language models. Figure 8 In the data query system shown, the large language model can use the Hunyuan Large Model. The Hunyuan Large Model is a large-scale language model based on natural language processing and machine learning technology. It is a method based on deep learning. It is trained through a large corpus, can understand and generate natural language, and can complete various tasks such as text generation, question answering, summarization, etc. The Hunyuan Large Model is characterized by its large scale, usually trained with parameters at the level of hundreds of billions or even trillions, and has strong natural language processing capabilities. Since the Hunyuan Large Model uses a large number of Chinese corpora for training, the use of the Hunyuan Large Model can have a better understanding of query texts input in Chinese.

[0247] II. Prompts component: When a user talks to a large language model, if the query text (prompt) input by the user each time contains a large amount of repeated content, a prompt template (PromptTemplates) can be generated to extract the common parts. In some embodiments, the construction of the prompt template can refer to the embodiment of the prompt template described in the above data query method. The prompt template constructed in the data query method is an implementation of the prompt template here. The prompt template may include metadata information for describing the data source.

[0248] III. Chain component: This component is used for coupling with the interaction layer 810 and receives input from the user. The chain component can pass the response consisting of the prompt template and the query text entered by the user to the mixed large model for conversion of the text into structured statements. Here, the conversion of text into structured statements can adopt natural language processing (NLP) technology (such as Text-to-SQL technology). Text-to-SQL technology aims to convert natural language queries into structured query languages ​​(such as SQL query languages) for querying. The chain component can connect various components and combine multiple components of the service layer 820 to create a single, coherent application.

[0249] IV. Agent component: This component is the interface between the Hunyuan model and the tools related to the database operation corresponding to the data source, and is used to call the tools related to the database operation. After the Hunyuan model performs the Text-to-SQL conversion, the agent component will connect to the database corresponding to the data source according to the output results of the Hunyuan model and the relevant tools set in the Tools tool, and use the components or modules that execute the agent task (Agent Task), such as the Agent Executor component, to execute the SQL query statement output by the Hunyuan model, and realize the function of connecting to the database corresponding to the data source and completing the target data query in the database.

[0250] The model layer 830 is the Hunyuan Big Model used by the model component. By calling the API interface of the Hunyuan Big Model, the query text input by the user can be converted into an SQL statement.

[0251] The storage layer 840 can provide comprehensive metadata management services by importing and maintaining multiple data sources, improving the format of metadata information, and linking prompt templates with data sources.

[0252] Regarding the above Figure 8 The data query system shown can be implemented specifically through the following process.

[0253] Build a storage layer 840. Building the storage layer 840 is mainly for preparing the underlying data. In one possible implementation, a storage layer 840 for metadata management is pre-built, and the metadata format of the data in the storage layer 840 is standardized. Data from different data sources are migrated to the storage layer 840, and the metadata information of the data source is improved in a standardized format. For example, the database connection information, table information, field information, etc. of the data table corresponding to the data source are standardized. In another possible implementation, a storage layer 840 for metadata management is pre-built. When there are multiple target data sources corresponding to the user's query text, such as multiple databases, the required target data can be extracted from the multiple databases respectively, and then the extracted target data can be stored in the storage layer 840, and then the format of the target data can be standardized to facilitate other processing of the target data.

[0254] The model layer is configured 830. The data query system adopts the Hunyuan Big Model, so the communication link of the Hunyuan Big Model can be adjusted by configuring the API interface of the Hunyuan Big Model.

[0255] Build the service layer 820. Configure the model component (Models component), prompt component (Prompts component), chain component (Chains component) and agent component (Agents component). Specifically, the model component is configured the same as the model layer 830, and the communication link of the Hunyuan large model is adjusted by configuring the API interface of the Hunyuan large model. The configuration of the prompt component is mainly for the construction of the prompt template. For details, please refer to the construction process of the prompt template in the above-mentioned data query method. For different query scopes, according to the data source to which the data in the query scope belongs, determine the prompt template associated with the query scope. The prompt template is configured with metadata information of the corresponding data source, and the metadata information may include the database identifier of the database corresponding to the data source and the table identifier and target field of the data table corresponding to the data source. The prompt template can also be associated with the address information of the corresponding data source, for example, a mapping relationship between the prompt template and the data source identifier is established, and the data source identifier points to or associates the address information of the corresponding data source.

[0256] The construction of the proxy component is to realize the connection between the Hunyuan large model and the data source and to query the data source. Relying on the address information associated with the prompt template corresponding to the query text, a communication connection with the data source corresponding to the query text is established, and the SQL query statement corresponding to the query text is executed in the data source to correctly retrieve the target data. In the construction of the proxy component, the SparkSQLToolkit toolkit and the create_spark_SQL_agent function are required. The specific process can be: passing the address information of the data source and the SQL query statement returned by the Hunyuan large model to the SparkSQLToolkit toolkit. The code can be expressed as toolkit=SparkSQLToolkit(db,llm). Among them, db represents the address information and llm represents the SQL query statement. Then, the SparkSQLToolkit toolkit is executed using the create_spark_SQL_agent function to realize the query operation on the data source and retrieve the target data. The code can be expressed as agent_executor=create_spark_SQL_agent(llm,toolkit,verbose=True,prefix). The relevant result of the code execution is the target data extracted from the data source.

[0257] Construction of chain components. Since this data query system can realize the Text-to-SQL function, this data query system can realize the interaction with the database corresponding to the data source based on SQLDatabaseChain. The Chains component can connect the interaction layer 810, the Hunyuan large model and the database corresponding to the data source. It accepts a prompt template, formats the prompt template and the query text entered by the user and inputs the formatted content into the Hunyuan large model, obtains the response of the Hunyuan large model, and realizes the conversion of the query text into an SQL query statement. Then call the agent component to establish a communication connection with the database corresponding to the data source, and execute the SQL query statement in the database with the established communication connection to extract the target data. Since the subsequent agent create_spark_SQL_agent is implemented based on SQLDatabaseChain, the association relationship of the specific prompt template can be set in the prefix parameter in create_spark_SQL_agent.

[0258] Build interactive layer 810. Display on the user's WEB terminal Figure 4The conversation interface shown. The user can select a query range in the conversation interface, such as "Optimization of user groups with low information flow". Then, enter the query text belonging to the query range in the text input box 4102, such as the query text entered in Chinese: "In the past three days, the market DAU, consumption DAU, and consumption penetration rate of different age groups, city levels, and genders, the aggregation method is SUM, sorted in ascending order by consumption penetration rate, and the market DAU is greater than 10,000". After the chain component obtains the above query text, it formats the query text and the prompt template corresponding to the query range and inputs it into the Hunyuan large model to obtain the SQL query statement corresponding to the query text. Then, based on the address information associated with the prompt template corresponding to the query range, the database corresponding to the data source is connected, and the SQL query statement is executed to extract the target data from the data source and return the target data to the interaction layer 810. After obtaining the target data, the interaction layer 810 can convert the target data into text, data charts, generated code, data files, etc., so as to display the target data to the user in a specified display form in the conversation interface of the WEB terminal. See Figure 4 , the display area 4104 of the conversation interface of the WEB terminal displays the query text "In the past three days, the market DAU, consumption DAU, and consumption penetration rate of different age groups, city levels, and genders, the aggregation method is SUM, sorted in ascending order by consumption penetration rate, and the market DAU is greater than 10,000" and the target data corresponding to the query text displayed in a table format.

[0259] Through the above embodiments, by using a tool library (such as the LangChain framework) and accessing the Hunyuan large model to realize the Text-to-SQL function, the data query method of the embodiment of the present application is applied to the data analysis tasks of multiple downstream applications, and the full-link data analysis automation is realized. And the Text-to-SQL solution based on the LangChain framework is flexible, accurate and efficient.

[0260] I. Flexibility: The Text-to-SQL solution based on the LangChain framework can be easily integrated with other data sources to achieve a wider range of application scenarios.

[0261] II. Accuracy: The Text-to-SQL solution based on the LangChain framework can more accurately understand the meaning and structure of Chinese text, thereby generating more accurate SQL query statements.

[0262] III. Efficiency: The Text-to-SQL solution based on the LangChain framework can use the LangChain framework to achieve more efficient Text-to-SQL conversion and query execution.

[0263] Through the above embodiments, by combining the trusted Hunyuan large model with the technology of the LangChain framework, the problem of data leakage caused by accessing the large model in the existing solution can be solved. Figure 8 The data query system shown can provide personalized data analysis demand delivery, generate targeted data analysis results through dialogue based on the business needs proposed by product operations and technology, realize intelligent data analysis insights, respond quickly to user needs, and speed up analysis. In addition, through the automation of Text-to-SQL technology, some business-valuable but repetitive data analysis needs are made intelligent, which helps the data query system to continuously iterate and speed up the efficiency of strategy launch.

[0264] The electronic device that generates the data query instruction is a terminal (such as terminal 401, terminal 402), where the terminal can also be the above-mentioned WEB terminal, and the electronic device 400 that implements the data query method is a server 200. Fig. 9 ,based on Fig. 9 The data query system shown implements the data query method and may include the following steps:

[0265] In step 901, a metadata management service is constructed. Before the user performs a data query, the server 200 pre-constructs a storage layer 840 to perform a standardized metadata management service for each data source. The data source and metadata information of the data source are unified to help the hybrid model accurately generate SQL query statements based on the metadata information.

[0266] In step 902, the prompt template is configured. The server 200 pre-sets the prompt templates corresponding to different data sources through the LangChain framework to help the mixed model enter a specific role and obtain pre-information for the data source.

[0267] In step 903, the query scope is selected and the query text is input. The user selects the query scope and inputs the query text in the conversation interface displayed by the terminal (such as terminal 401, terminal 402). The terminal (such as terminal 401, terminal 402) sends a data query instruction to the server 200 for the input query text. After receiving the query text, the service layer 820 of the server 200 formats the query text and the corresponding prompt template and inputs them into the mixed-element model.

[0268] In step 904, an SQL query statement corresponding to the query text is generated. The hybrid model of the server 200 generates an SQL query statement corresponding to the query text by understanding the query text and the prompt template.

[0269] In step 905, a communication connection is established with the database corresponding to the data source. The server 200 uses the Tools component in the Agents component to set "database connection" to establish a communication connection with the database of the data source based on the address information associated with the prompt template.

[0270] In step 906, the target data is extracted. The server 200 calls the Agent Executor component of the Agents component to execute the SQL query statement to extract the target data from the database corresponding to the data source.

[0271] In step 907, the result is outputted. The server 200 returns the extracted target data to the terminal (such as the terminal 401, the terminal 402), and the terminal (such as the terminal 401, the terminal 402) outputs the target data on the conversation interface in a specified display form.

[0272] The following is a description of an exemplary structure of the data query device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the data query device 455 of the memory 450 may include:

[0273] The acquisition module 4551 is used to acquire the query text for indicating data query and acquire the target prompt template associated with the query text for generating structured sentences. The target prompt template includes metadata information for describing the data source, and different prompt templates correspond to different data sources.

[0274] The generation module 4552 is used to generate a structured query statement corresponding to the query text based on the query text and the target prompt template.

[0275] The query module 4553 is used to query the data source corresponding to the target prompt template based on the generated structured query sentence to obtain the target data corresponding to the query text.

[0276] In some embodiments, the acquisition module 4551 is further used to receive a data query instruction for performing a data query on a query text, wherein the data query instruction carries a target query scope corresponding to the query text. The target query scope is obtained by selecting at least one query scope displayed in the conversation interface. A mapping relationship between different query scopes and prompt templates is obtained; based on the mapping relationship and the target query scope, a prompt template having a mapping relationship with the target query scope is determined, and the prompt template having a mapping relationship with the target query scope is used as the target prompt template for generating a structured statement.

[0277] In some embodiments, the data query device further includes: a return module. After the query module 4553 obtains the target data corresponding to the query text, the return module is used to respond to the data query instruction and return the target data to the electronic device corresponding to the data query instruction. The target data is used for the electronic device to output the target data corresponding to the query text in the conversation interface displaying the target query range.

[0278] In some embodiments, the data query device further includes: a relationship establishment module. Before the acquisition module 4551 acquires the mapping relationship between different query scopes and prompt templates, the relationship establishment module is used to determine the data source to which the data corresponding to each query scope belongs, and determine the prompt template of each data source; based on the data source to which the data corresponding to each query scope belongs, and the prompt template of each data source, establish the mapping relationship between each query scope and the corresponding prompt template.

[0279] In some embodiments, the data query device further includes: a template generation module. Before the acquisition module 4551 acquires the target prompt template associated with the query text for generating a structured sentence, the template generation module is used to acquire multiple data sources for data query and acquire the attribute information of each data source; convert the attribute information of each data source into metadata information corresponding to each data source; and generate a prompt template corresponding to each data source based on the metadata information of each data source.

[0280] In some embodiments, the data query device further includes: a format conversion module. The metadata information of each data source has the same format. After the template generation module converts the attribute information of each data source into the metadata information of the corresponding data source, the format conversion module is used to obtain the target metadata format and convert the format of the metadata information of each data source into the target metadata format. The template generation module is also used to generate a prompt template corresponding to each data source based on the metadata information of each data source having the target metadata format.

[0281] In some embodiments, the generation module 4552 is further used to obtain a target language model and call the target language model to predict a structured query statement corresponding to the query text based on the query text and the target prompt template.

[0282] In some embodiments, the metadata information includes the database type of the database corresponding to the target data source. The generation module 4552 is further used to obtain the target query syntax of the database corresponding to the target data source based on the database type; and generate a structured query statement that conforms to the target query syntax corresponding to the query text based on the query text and the target prompt template.

[0283] In some embodiments, the database includes at least one data table, and the metadata information also includes a table identifier of the target data table to which the target data belongs and a target field in the target data table to which the target data belongs. The generation module 4552 is further used to obtain a target query syntax of the target data table in the database based on the database type; based on the query text, the table identifier of the target data table and the target field, generate a structured query statement that conforms to the target query syntax corresponding to the query text.

[0284] In some embodiments, the query module 4553 is also used to obtain the address information of the target data source associated with the target prompt template; based on the address information of the target data source, establish a communication connection with the target data source, and create a data source connection object of the target data source based on the communication connection; based on the data source connection object, execute a structured query statement in the target data source corresponding to the target prompt template to obtain the target data corresponding to the query text.

[0285] In some embodiments, the data query device further includes: a data operation module. The query text also includes an operation text, and the operation text is used to indicate the target data operation to be performed on the target data. After the query module 4553 obtains the target data corresponding to the query text through query, the data operation module is used to perform the target data operation on the target data to obtain the target data after the operation; and send the target data after the operation so that the target data after the operation is displayed in the view interface as the query result of the query text.

[0286] In some embodiments, the data query device further includes: a matching module. In the case where there are multiple target prompt templates, the matching module is used to obtain the data source corresponding to each target prompt template; according to the matching degree between the data source corresponding to each target prompt template and the query text, determine the target prompt template whose matching degree meets the expected matching. The generation module 4552 is also used to generate a structured query statement corresponding to the query text based on the query text and the target prompt template whose matching degree meets the expected matching.

[0287] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device 400 reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device 400 executes the data query method described above in the embodiment of the present application.

[0288] The present application embodiment provides a computer-readable storage medium in which computer executable instructions or computer programs are stored. When the computer executable instructions or computer programs are executed by a processor, the processor will execute the data query method provided by the present application embodiment, for example, Figure 3A The data query method is shown.

[0289] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0290] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0291] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0292] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0293] In summary, the embodiments of the present application can accurately locate the data source when the user performs a data query, and by using a trusted target language model, the problem of data leakage can be reduced, thereby ensuring data security. The prompt template constructed based on the metadata information of the data source can make the structured query statement corresponding to the generated query text more accurate. By migrating multiple data sources to the storage layer 840 in advance, the problem of cross-extraction of data can be solved, and the metadata information format of the data source can be standardized, which can help the target language model better understand the structure of the data source and generate accurate structured query statements.

[0294] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A data query method, characterized in that: The method comprises: Determine a data source corresponding to each of the at least one query scope displayed on the conversation interface, and determine a prompt template corresponding to each of the data sources; Wherein, the at least one query range is divided according to the query requirements, and different query ranges correspond to different data sources; Based on the data source corresponding to each query range and the prompt template corresponding to each data source, a mapping relationship between each query range and the corresponding prompt template is established; Acquire a query text for indicating a data query, wherein the query text corresponds to a target query range, and the target query range is obtained by selecting at least one query range displayed in the conversation interface; Get the mapping relationship between different query scopes and prompt templates; Based on the mapping relationship and the target query scope, determining a prompt template that has a mapping relationship with the target query scope, and using the prompt template that has a mapping relationship with the target query scope as a target prompt template for generating a structured statement, wherein the target prompt template includes metadata information for describing a data source, and different prompt templates correspond to different data sources; Based on the query text and the target prompt template, generate a structured query statement corresponding to the query text; Based on the generated structured query statement, the target data corresponding to the query text is queried in the target data source corresponding to the target prompt template.

2. The method according to claim 1, characterized in that Before obtaining the mapping relationship between different query scopes and prompt templates, the method further includes: A data query instruction for performing a data query on the query text is received, wherein the data query instruction carries a target query range corresponding to the query text.

3. The method according to claim 2, characterized in that After the query obtains the target data corresponding to the query text, the method further includes: In response to the data query instruction, returning the target data to the electronic device corresponding to the data query instruction; The target data is used by the electronic device to output the target data corresponding to the query text in a conversation interface displaying the target query range.

4. The method according to claim 1, characterized in that: Before determining the prompt template corresponding to each of the data sources, the method further includes: Acquire multiple data sources for data query, and acquire attribute information of each of the data sources; Converting the attribute information of each data source into metadata information corresponding to each data source; Based on the metadata information of each of the data sources, a prompt template corresponding to each of the data sources is generated.

5. The method according to claim 4, characterized in that The metadata information of each data source has the same format. After converting the attribute information of each data source into the metadata information of the corresponding data source, the method further includes: Acquire a target metadata format, and convert the format of the metadata information of each of the data sources into the target metadata format; The generating a prompt template corresponding to each data source based on the metadata information of each data source includes: Based on the metadata information of each of the data sources in the target metadata format, a prompt template corresponding to each of the data sources is generated.

6. The method according to claim 1, characterized in that The step of generating a structured query statement corresponding to the query text based on the query text and the target prompt template includes: A target language model is obtained, and the target language model is called to predict a structured query statement corresponding to the query text based on the query text and the target prompt template.

7. The method according to claim 1, characterized in that The metadata information includes a database type of a database corresponding to the target data source, and the generating of a structured query statement corresponding to the query text based on the query text and the target prompt template includes: Based on the database type, obtaining a target query syntax of the database corresponding to the target data source; Based on the query text and the target prompt template, a structured query statement corresponding to the query text and conforming to the target query grammar is generated.

8. The method according to claim 7, characterized in that The database includes at least one data table, and the metadata information also includes a table identifier of a target data table to which the target data belongs and a target field in the target data table to which the target data belongs; The step of acquiring a target query syntax of a database corresponding to the target data source based on the database type includes: Based on the database type, obtaining a target query syntax for the target data table in the database; The step of generating a structured query statement corresponding to the query text and conforming to the target query grammar based on the query text and the target prompt template includes: Based on the query text, the table identifier of the target data table and the target field, a structured query statement corresponding to the query text and conforming to the target query grammar is generated.

9. The method according to claim 1, characterized in that: The step of querying the target data source corresponding to the target prompt template based on the generated structured query statement to obtain the target data corresponding to the query text includes: Acquire the address information of the target data source associated with the target prompt template; Based on the address information of the target data source, establish a communication connection with the target data source, and create a data source connection object of the target data source based on the communication connection; Based on the data source connection object, the structured query statement is executed in the target data source corresponding to the target prompt template to obtain the target data corresponding to the query text.

10. The method according to claim 1, characterized in that The target prompt template also includes a data source identifier of the target data source; the target data source corresponding to the query text is queried in the target data source corresponding to the target prompt template based on the generated structured query statement, including: Extracting the data source identifier of the target data source from the target prompt template; Based on the data source identifier of the target data source, the generated structured query statement is executed in the target data source corresponding to the target prompt template to obtain the target data corresponding to the query text.

11. The method according to claim 1, characterized in that: The query text also includes an operation text, and the operation text is used to indicate the target data operation to be performed on the target data; After obtaining the target data corresponding to the query text through query, the method further includes: Performing the target data operation on the target data to obtain the target data after the operation; The target data after the operation is sent so that the target data after the operation is displayed on the view interface as the query result of the query text.

12. The method according to claim 1, characterized in that In the case where there are multiple target prompt templates, the method further includes: Obtaining the data source corresponding to each of the target prompt templates; According to the matching degree between the data source corresponding to each target prompt template and the query text, determining the target prompt template whose matching degree meets the expected matching degree; The step of generating a structured query statement corresponding to the query text based on the query text and the target prompt template includes: Based on the query text, the matching degree meets the expected target prompt template, and a structured query statement corresponding to the query text is generated.

13. A data query device, characterized in that: The device comprises: a relationship establishment module, used to determine the data source corresponding to each query scope in at least one query scope displayed on the conversation interface, and determine the prompt template corresponding to each data source; wherein the at least one query scope is divided according to the query requirements, and different query scopes correspond to different data sources; based on the data source corresponding to each query scope and the prompt template corresponding to each data source, a mapping relationship between each query scope and the corresponding prompt template is established; an acquisition module, configured to acquire a query text for indicating a data query, wherein the query text corresponds to a target query scope, and the target query scope is obtained by selecting the at least one query scope displayed in the conversation interface; acquire a mapping relationship between different query scopes and prompt templates; based on the mapping relationship and the target query scope, determine a prompt template having a mapping relationship with the target query scope, and use the prompt template having a mapping relationship with the target query scope as a target prompt template for generating a structured statement, wherein the target prompt template includes metadata information for describing a data source, and different prompt templates correspond to different data sources; A generating module, configured to generate a structured query statement corresponding to the query text based on the query text and the target prompt template; A query module is used to query and obtain target data corresponding to the query text in a data source corresponding to the target prompt template based on the generated structured query statement.

14. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions; A processor, used to implement the data query method according to any one of claims 1 to 13 when executing the computer executable instructions stored in the memory.

15. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the data query method according to any one of claims 1 to 13 is implemented.

16. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the data query method according to any one of claims 1 to 13 is implemented.

Images