Database operation statement generation method and generation performance evaluation method

Abstract

The embodiment of the invention provides a database operation statement generation method, a generation performance evaluation method, a data acquisition method, a data acquisition system, electronic equipment, a storage medium and a computer program product. The database operation statement generation method comprises the steps of generating a query text corresponding to a historical database operation statement of a target application scene through a pre-trained first generative large model, and constructing a first number of data pairs based on the historical database operation statement and the generated query text; based on the data pair, a pre-training generative large model is finely adjusted, a second generative large model is obtained, and the second generative large model is used for generating a database operation statement based on a query text in a target application scene. According to the method, the second generative large model has higher understanding capability on the database operation statement and the query text of the target application scene, and a more effective database operation statement can be generated based on the query text.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for generating a database operation statement, a method for evaluating the generation performance of a database operation statement, a data acquisition method, a data acquisition system, an electronic device, a storage medium and a computer program product. Background Art

[0002] In data-driven e-commerce scenarios, enterprise users have a growing demand for data query and analysis. NL2SQL (Natural Language to SQL, a technology that converts natural language into SQL statements) has attracted much attention because it can improve query efficiency and lower technical barriers. However, NL2SQL faces major challenges in enterprise-level applications. For example, the existing NL2SQL technical framework is mostly based on general data sets, which is difficult to adapt to complex application logic and domain-specific knowledge, resulting in reduced effectiveness of generated database query statements. In addition, for complex application scenarios, the lack of a large amount of annotated data for model fine-tuning will also lead to reduced effectiveness of database query statements generated by the fine-tuned large language model.

[0003] It can be seen that the data operation statement generation method in the prior art still needs to be improved. Summary of the invention

[0004] The embodiment of the present application provides a method for generating data operation statements, which can improve the effectiveness of generating database operation statements based on text.

[0005] Correspondingly, an embodiment of the present application also provides a method for evaluating the generation performance of a database operation statement, a data acquisition method, a data acquisition system, an electronic device, a storage medium and a computer program product to ensure the implementation and application of the above-mentioned database operation statement generation method.

[0006] In order to solve the above problems, the embodiment of the present application discloses a method for generating a database operation statement, the method comprising:

[0007] Generate query text corresponding to the historical database operation statement of the target application scenario through the pre-trained first generative large model;

[0008] constructing a first number of data pairs based on the historical database operation statement and the generated query text;

[0009] Based on the data, the pre-trained generative big model is fine-tuned to obtain a second generative big model, wherein the second generative big model is used to generate database operation statements based on the query text in the target application scenario.

[0010] The embodiment of the present application further discloses a data acquisition method, which is applied to a server. The data acquisition method is used to apply the above-mentioned method for generating data operation statements. The data acquisition method includes:

[0011] In response to a query request, a pre-established knowledge base is retrieved to obtain application knowledge matching the query text carried in the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of a workflow to be executed;

[0012] Based on the description information and / or the data table, dynamically generate database operation statements corresponding to the tasks to be executed in the workflow in sequence, and execute the database operation statements in real time to obtain data;

[0013] Based on the data acquired by executing the database operation statement in real time, data matching the query text is obtained.

[0014] The embodiment of the present application further discloses a data acquisition method, which is applied to a client. The data acquisition method is used to apply the above-mentioned method for generating data operation statements. The data acquisition method includes:

[0015] In response to the query text input by the user, a query request is sent to a preset server, and the query request is used to trigger the preset server to perform the following operations to obtain data: retrieve a pre-established knowledge base to obtain application knowledge matching the query text carried by the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of the workflow to be executed; based on the description information and / or the data table, dynamically generate database operation statements corresponding to the tasks to be executed in the workflow in sequence, and execute the database operation statements in real time to obtain data; based on the data obtained by executing the database operation statements in real time, obtain data matching the query text;

[0016] The data matching the query text is displayed.

[0017] The embodiment of the present application discloses a data acquisition system, including: a client and a server, wherein:

[0018] The client is used to obtain the query text input by the user and send a query request carrying the query text to the server;

[0019] The server is used to retrieve a pre-established knowledge base in response to the query request, and obtain application knowledge matching the query text carried by the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of a workflow to be executed;

[0020] The server is further used to dynamically generate database operation statements corresponding to the tasks to be executed in the workflow based on the description information and / or the data table, and execute the database operation statements in real time to obtain data;

[0021] The server is further configured to obtain data matching the query text based on the data obtained by executing the database operation statement in real time, and send the data matching the query text to the client;

[0022] The client is also used to receive and display data matching the query text.

[0023] The present application embodiment discloses a method for evaluating the generation performance of a database operation statement, the method comprising:

[0024] Obtain database operation statements generated by the pre-fine-tuned second generative large model for the target query text;

[0025] In the case where the target query text is annotated with a real database operation statement, based on the similarity between the database operation statement and the real database operation statement, and / or based on the consistency between the execution result of the database operation statement and the execution result of the real database operation statement, the performance of the second generative large model in generating the database operation statement is evaluated;

[0026] In the case that the target query text is not labeled with real database operation statements, the performance of the second generative large model in generating database operation statements is evaluated based on the correlation between the data obtained by executing the database operation statements and the target query text.

[0027] The embodiment of the present application also discloses an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0028] The memory stores computer-executable instructions;

[0029] The processor executes the computer-executable instructions stored in the memory to implement the method described in the embodiment of the present application.

[0030] The embodiment of the present application further discloses a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method described in the embodiment of the present application.

[0031] The embodiment of the present application also discloses a computer program product, including a computer program / computer executable instructions, characterized in that when the computer program / computer executable instructions are executed by a processor in an electronic device, the method described in the embodiment of the present application is implemented.

[0032] Compared with the prior art, the embodiments of the present application have the following advantages:

[0033] A query text corresponding to the historical database operation statements of the target application scenario is generated by a pre-trained first generative big model, a first number of data pairs are constructed based on the historical database operation statements and the generated query text, and a reverse data generation method is adopted to obtain a large amount of labeled data of the target application scenario as training data; thereafter, the pre-trained generative big model is fine-tuned based on the data pairs to obtain a second generative big model, wherein the second generative big model is used to: generate database operation statements based on the query text under the target application scenario, so that the second generative big model has a stronger understanding ability of the database operation statements and query text of the target application scenario, and can generate more effective database operation statements based on the query text. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flowchart of the steps of the method for generating a database operation statement disclosed in an embodiment of the present application;

[0035] Figure 2 It is one of the step flow charts of the data acquisition method disclosed in the embodiment of the present application;

[0036] Figure 3 This is the second step flow chart of the data acquisition method disclosed in the embodiment of the present application;

[0037] Figure 4 is an interactive schematic diagram of a data acquisition system disclosed in an embodiment of the present application;

[0038] Figure 5 It is a flowchart of the steps of the method for evaluating the generation performance of database operation statements disclosed in the embodiment of the present application;

[0039] Figure 6 It is a schematic diagram of the structure of an exemplary device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0041] The method for generating database operation statements disclosed in the embodiment of the present application is applicable to a variety of data query scenarios and data analysis scenarios that require operating data in the database, which helps to better utilize data resources. For example, the application scenarios include: querying market trends, consumption trends, supply chain status and other scenarios in e-commerce platforms, as well as intelligent product selection scenarios, business information question and answer scenarios, etc. In order to adapt to the professionalism of application scenarios and the complexity of user data acquisition needs, the generation scheme of database operation statements disclosed in the embodiment of the present application adopts the method of reverse data reasoning to generate sample data of corresponding application scenarios, and fine-tune the pre-trained generative large model based on the generated sample data, so that the fine-tuned generative large model can more accurately understand the query text of the corresponding application scenario, and accurately generate the corresponding database operation statements to efficiently and accurately obtain query data.

[0042] Reference Figure 1 The method for generating a database operation statement disclosed in the embodiment of the present application includes: steps 102 to 106.

[0043] Step 102, generating query text corresponding to the historical database operation statement of the target application scenario through the pre-trained first generative large model.

[0044] As we all know, enterprise-level application scenarios lack a large number of<Query,SQL> There is a lack of annotated data, that is, a large number of query text and database operation statement data pairs. Therefore, the generative large model used in the prior art to generate database operation statements based on query text is obtained by fine-tuning general data, which also leads to the low performance of database operation statements generated by the large model, and cannot solve the problem of generating query text to database operation statements in complex application scenarios.

[0045] However, the inventors have found that a wealth of original database query statements (hereinafter referred to as "SQL" in the embodiments of this application) are stored in enterprise-level application scenarios. In view of this data feature, the method for generating data operation statements disclosed in the embodiments of this application uses a general large-scale language model and a reverse generation model trained with a small amount of labeled data to fully explore the diverse application needs contained in the massive SQL, and constructs large-scale domain annotation data through the reverse data generation technology from database query statements to query texts.

[0046] Optionally, the first generative model is used to generate query text based on database operation statements. The first generative model is trained by the following method: obtaining a second number of data pairs consisting of sample database operation statements and sample query texts of the target application scenario; based on the second number of data pairs, training the first generative model, the first generative model is used to generate query text in the target application scenario based on data query statements. The first number is much larger than the second number.

[0047] For example, a data pair consisting of a small amount (about 200) of manually annotated query texts (i.e., sample query texts) and database operation statements (i.e., sample database operation statements) in the target application scenario can be expressed as:<SQL,Query> , train a large language model that can translate database query statements into natural language in the target application scenario, which is referred to as the "first generative large model" in the embodiment of the present application.

[0048] For the specific implementation method of obtaining the first generative large model based on fine-tuning of the marking data, please refer to the prior art and will not be repeated in the embodiments of the present application.

[0049] Next, the first generative large model is used to translate the application's large-scale original database operation statements into natural language, namely, query text.

[0050] Step 104: construct a first number of data pairs based on the historical database operation statements and the generated query text.

[0051] Furthermore, based on each original database operation statement and the query text generated based on the original database operation statement, a data pair of the original database operation statement and the query text is constructed respectively, and a large number of data pairs of the original database operation statement and the query text can be obtained. For example, the query text generated by the first generative large model can be used to annotate the original database operation statement, and a large number of database operation statements with query text tags can be obtained as annotated data, for example, represented as:<Query,SQL> .

[0052] Step 106, fine-tune the pre-trained generative big model based on the data to obtain a second generative big model, wherein the second generative big model is used to generate database operation statements based on the query text in the target application scenario.

[0053] Afterwards, based on the labeled data generated in the previous step, the large language models in the prior art (such as Qwen, LLaMA and other open source large models) are fine-tuned to obtain a vertical domain large model with powerful NL2SQL performance in the target application scenario, which is recorded as the "second generative large model" in the embodiment of this application.

[0054] For the specific implementation method of obtaining the second generative large model based on fine-tuning of the labeled data, please refer to the prior art and will not be repeated in the embodiments of the present application.

[0055] For the fine-tuned second generative large model, the prompt text is constructed with the query text of the target application field, which can trigger the second generative large model to generate corresponding database query statements.

[0056] In summary, the method for generating database operation statements disclosed in the embodiment of the present application generates query texts corresponding to historical database operation statements of the target application scenario through a pre-trained first generative big model, constructs a first number of data pairs based on the historical database operation statements and the generated query texts, and acquires massive labeled data of the target application scenario as training data by reversely generating data; then, fine-tunes the pre-trained generative big model based on the data pairs to obtain a second generative big model, wherein the second generative big model is used to generate database operation statements based on the query text under the target application scenario, so that the second generative big model has a stronger understanding ability of the database operation statements and query texts of the target application scenario, and can generate more effective database operation statements based on the query texts.

[0057] Based on the above embodiment, the present application also discloses a data acquisition method, which is applied to the server of the data acquisition system to implement the above method for generating database operation statements. Figure 2 As shown, the data acquisition method includes: step 202, step 204 and step 206.

[0058] Step 202, in response to the query request, retrieve a pre-established knowledge base to obtain application knowledge matching the query text carried in the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of the workflow to be executed.

[0059] In some optional embodiments, the server may obtain the query text by parsing the received data acquisition request, which may come from the client or other server. The query text is used to describe the user's data operation intention. For example, the query text may be: "Get the search exposure count of store X in the last 7 days".

[0060] In some optional embodiments, the query text input by the user can be obtained by a client corresponding to the server, and the query text can be used as a request parameter to generate a data acquisition request, and then the client sends the data acquisition request to the server. The generation method and receiving method of the query text in the embodiments of the present application are not limited.

[0061] In real application scenarios, users have complex data operation requirements. For example, when a user enters a query text: "Compare the traffic source countries of product 10001 and excellent products in the same industry", the user's needs not only involve the data acquisition and association operations of multiple data tables (such as user identity data tables, product data tables, etc.), but also require the support of domain knowledge and analysis logic (such as traffic acquisition methods, traffic statistics methods, traffic analysis logic, etc.). In the embodiments of the present application, in order to obtain scenario data under complex requirements, a data acquisition method is designed based on the idea of ​​divide and conquer.

[0062] First, by comprehensively combing the domain knowledge and data base information of the target application scenario, using index construction, vector recall and other technologies, a retrieval system with both accuracy and scalability is built as a knowledge base. The knowledge base at least includes: application knowledge for obtaining commonly used query text matching, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of the workflow to be executed when obtaining data matching the query text. Among them, the description information of the workflow is used to describe the domain knowledge and analysis logic associated with the query text, and the description information of the workflow includes but is not limited to one or more of the following information: the reply logic to the query text, the tasks to be performed in sequence when obtaining the data corresponding to the query text, and the task content description information of each task, the data table information to be operated when obtaining the data corresponding to the query text, etc. Among them, the data table information includes one or more of the following information: data table name, query data item corresponding to the data table name, etc. Among them, the knowledge base includes but is not limited to the RAG system (Retrieval-Augmented Generation).

[0063] For example, a RAG system can be established for common query texts. The RAG system is used to first retrieve relevant information from a wide range of document databases, and then use this information to guide the generation process of database operation statements, greatly improving the accuracy and relevance of the content. For example, the query text, data base table name, and domain data-related knowledge can be converted into vectors through the Chinese-text2vec-BERT model, and a vector retrieval service can be built for the vector based on tools in the prior art (such as Faiss, a tool for building an index of basic vector data), so that in the data acquisition process, the data table can be retrieved based on methods such as the cosine similarity of the vector, thereby obtaining an efficient and accurate retrieval system.

[0064] During the data acquisition process, after the user inputs the query text, the server first uses the Chinese-text2vec-BERT model to convert the query text entered by the user into a query vector. Then, based on the query vector, vector retrieval is performed in the RAG system, sorted according to cosine similarity, and the application knowledge corresponding to one or several query text indexes with the highest query vector similarity in the RAG system is recalled as the application knowledge matching the current query text.

[0065] Take the query text entered by the user as: "Why are the search exposures in the last 7 days less than those in the last 30 days? My store ID is 29099949637" as an example. Based on the word query text, the knowledge base is retrieved to obtain relevant knowledge to solve this problem, and the following application knowledge can be obtained:

[0066] #Reply summary logic

[0067] The traffic link of the international site is: exposure, click-through rate, click, business opportunity conversion rate, business opportunity, payment conversion rate, and payment.

[0068] #Workflow

[0069] step 1. Query the store's search exposure number in the last 7 days, the store's search exposure number in the last 30 days, the store's search exposure number in the last 7 days, the store's search exposure number in the last 30 days, the store's search exposure number in the last 7 days, the store's search exposure number in the last 30 days;

[0070] step 2. Get the peer ID;

[0071] Step 3. Query the group_type of the peers, the number of search exposures of the peers in the last 7 days, the number of search exposures of the peers in the last 30 days, the number of search exposures of natural traffic of the peers in the last 7 days, the number of search exposures of natural traffic of the peers in the last 30 days, the number of search exposures of direct trains of the peers in the last 7 days, and the number of search exposures of direct trains of the peers in the last 30 days;

[0072] Step 4. Obtain and compare the results of your own store traffic diagnostic tool;

[0073] #Table information

[0074] 1.mechanism_free_talk_traffic_info_comp_d: The online traffic effect table of the store, where you can query the store search exposure, search clicks, business opportunities, payments, etc. through the store ID;

[0075] 2.icbu_mechanism_free_talk_traffic_info_group_d: Traffic effect table of peers online. You can query the store search exposure, search clicks, business opportunities, payments, etc. through the peer ID.

[0076] It can be seen from the above application knowledge that by searching the knowledge base, information such as data acquisition logic, solutions, and related data tables that match the query text can be obtained. For example, in the above recalled application knowledge, the reply summary logic is used to describe the question answering logic, the workflow description information includes the four steps that need to be performed in sequence to answer the question, and the task description of the execution task of each step, and the data table is used to describe the data table name involved in the data acquisition process, as well as the data items that can be obtained in each data table.

[0077] In different application scenarios, the knowledge index established in the knowledge base matches the application scenario, wherein the application knowledge is constructed according to the domain knowledge, data table information, etc. of the corresponding application scenario. In the embodiments of the present application, the specific content of the application knowledge such as data table information and workflow description information in the knowledge base is not limited.

[0078] Step 204: Based on the description information and / or the data table, dynamically generate database operation statements corresponding to the tasks to be executed in the workflow in sequence, and execute the database operation statements in real time to obtain data.

[0079] After obtaining the application knowledge of the target application scenario, the current data acquisition operation is further split based on the application knowledge to obtain one or more tasks to simplify the complex problem. By generating a database operation statement corresponding to each task and executing the database operation statement corresponding to each task, the result of completing the complete data acquisition operation by executing all tasks is achieved.

[0080] In the embodiment of the present application, for a workflow including multiple tasks, wherein each task has data dependency and support relationship, each task needs to be executed in the order specified in the workflow.

[0081] In some embodiments of the present application, a pre-trained generative model such as GPT-4 can be guided to split a complex workflow into multiple sequentially executed tasks based on the application knowledge by dynamically generating prompt words. Then, a corresponding database operation statement is generated for each task in turn, and the generated database operation statement is executed to perform the task.

[0082] When dynamically generating the task description text of each task based on the description information and the data table, the pre-trained generative model will determine the task description of each task that needs to be generated in sequence according to the description information of the workflow, as well as the task description text of the task that needs to be generated in the current step. At the same time, it will generate the table name, table items and other information of the data table that needs to be operated during the execution of the current task.

[0083] The present application obtains the data ultimately required by splitting the data acquisition operation corresponding to the query text into multiple sequentially executed task processings, wherein each task performs different data processing operations, wherein some tasks need to read data from a specified data table, and some tasks need to further process the data obtained by other tasks, and the data to be processed by each task can be specified in a pre-established prompt word template, or can be automatically generated by the pre-trained generative model.

[0084] In some optional embodiments, based on the description information and / or the data table, dynamically generating database operation statements corresponding to the tasks to be executed in the workflow in sequence, and executing the database operation statements in real time to acquire data, includes: dynamically generating a first task description text of a first task based on the description information and the data table, and generating and executing a database operation statement corresponding to the first task based on the first task description text to acquire data, wherein the first task is the starting task of the workflow; dynamically generating a second task description text of a second task based on one or more of the description information, the data table, and the data acquired from the executed tasks, and generating and executing a database operation statement corresponding to the second task based on the second task description text to acquire data, wherein the second task is a task in the workflow other than the first task.

[0085] In some optional embodiments, dynamically generating a first task description text for the first task based on the description information and the data table includes: dynamically generating a first task description text for the first task based on the description information and the data table through a pre-trained generative model.

[0086] Taking the application knowledge obtained in the previous text as an example, when the GPT model is used to split the application knowledge obtained in the previous text into tasks and generate the task description of the first task to be performed, the following task description text can be obtained:

[0087] “{"plan":"1. Query the store's search exposure count in the last 7 days and the store's search exposure count in the last 30 days\n2. Get the peer ID\n3. Query the peer's group_type, the peer's search exposure count in the last 7 days and the peer's search exposure count in the last 30 days\n4. Get the results of the traffic diagnostic tool to compare your own store",

[0088] "current_plan_step":"1. Query the store's search exposure count in the last 7 days and the store's search exposure count in the last 30 days",

[0089] "send_to":"Executor",

[0090] "message":"Query the store's search exposure count in the last 7 days and the store's search exposure count in the last 30 days. The table name is icbu_mechanism_free_talk_traffic_info_comp_d, and the store id is 29099949637"}".

[0091] Among them, the string "plan" is used to identify the task content of each task that is executed sequentially by splitting the data acquisition problem corresponding to the query text, the string "current_plan_step" is used to identify the task content of the current task, the string "send_to" is used to identify the output object of the generated task description, and the string "message" is used to identify the query text that needs to be input into the database operation statement generation model.

[0092] When generating the task description text for the second or subsequent task, since the second task may need to operate the data table given in the description information of the workflow, and may also operate the data table obtained after the first task is completed, the generated description text may include one or more of the following data information: the data table given in the description information of the workflow, and the data table obtained from the task that has been executed.

[0093] For example, in some optional embodiments, based on one or more of the description information, the data table, and the data obtained from the executed task, a second task description text for the second task is dynamically generated, including: dynamically generating the second task description text for the second task based on the description information and the data table through a pre-trained generative model; or dynamically generating the second task description text for the second task based on the description information, the data table, and the data obtained from the executed task through a pre-trained text generative model; or dynamically generating the second task description text for the second task based on the description information and the data obtained from the executed task through a pre-trained text generative model.

[0094] In some embodiments of the present application, task splitting prompt word templates corresponding to workflows matching different types of query texts can be pre-set, so that in actual applications, the prompt word templates can be instantiated according to information in the query text acquired in real time, thereby dynamically generating prompt words for triggering the pre-trained generative model to generate description information for each task.

[0095] For example, a certain type of query text Query1, its application knowledge shows that it is necessary to execute N (N is an integer greater than or equal to 1) steps in sequence to obtain the data corresponding to the query text Query1, and defines the task content that needs to be completed in each step. At the same time, it defines the data table that needs to be operated to obtain the data corresponding to the query text Query1. In some embodiments, it is also possible to define N prompt word templates that trigger the pre-trained generative model to split the workflow into tasks to generate task description texts, and each prompt word template corresponds to a task in the workflow. In the prompt word template, the task content of all tasks and the task content of the current task can be described, as well as the data table that needs to be operated to execute the current task, the data dependency between tasks and other information. After receiving the query text, the server retrieves the corresponding application knowledge, and based on the information carried in the query text and the detected application knowledge, instantiates the prompt word template corresponding to each task, thereby dynamically generating the prompt words corresponding to each task.

[0096] In some other optional embodiments, only one general prompt word template may be defined to dynamically generate prompt words corresponding to each task.

[0097] In the embodiment of the present application, since the data acquisition process to be executed for a query text is split into multiple tasks, the task to be executed later has data dependency on the task to be executed earlier, therefore, the split tasks need to be executed in the order described in the workflow. That is, the task description text of each task needs to be generated in sequence, and after the task description text of the current task is generated, the current task is first executed based on the generated task description text, and then the task description text of the next task to be executed is generated.

[0098] In some optional embodiments, a database operation statement corresponding to the target task is generated and executed based on a target task description text, including: dynamically generating a prompt text based on the target task description text; calling a pre-fine-tuned second generative large model based on the prompt text to generate a database operation statement corresponding to the target task; and executing the database operation statement, wherein the target task includes: the first task or the second task.

[0099] Optionally, the target task description text includes one or more of the following: a task content description of the target task, a data description of data that needs to be accessed to execute the target task, and a data output format description. The dynamically generated prompt text based on the target task description text includes: based on the target task description text, generating prompt text in a preset format to guide the second generative large model obtained in advance by fine-tuning to generate database operation statements.

[0100] Optionally, the data description includes but is not limited to the data table name, table structure, user ID, product ID and other data table query parameters associated with the query text that need to be queried.

[0101] Taking the target task as the aforementioned first task as an example, firstly, according to the task description information of the first task and the preset prompt word template, a prompt text is dynamically generated, and the prompt text is used to trigger the second generative model to generate a database operation statement.

[0102] Optionally, the prompt word template includes but is not limited to one or more of the following prompt information: generation requirements, reference information, output requirements, etc. The reference information includes but is not limited to domain knowledge, data table structure, examples, etc. The reference information can come from the task description text generated in the above steps.

[0103] In some optional embodiments, the prompt word template may be defined in the following form:

[0104] "You are an intelligent assistant who is good at writing SQL. Do your best to generate accurate SQL statements based on all reference information and user questions.

[0105] #Reference Information:

[0106] ##Here is the structure information of the database table

[0107] 「Data table DDL information」

[0108] ##Here are some sample data in the data table

[0109] 「Some sample data」

[0110] ##Here are some similar user questions and corresponding SQL statements

[0111] 「Here are some NL2SQL examples for reference」

[0112] ##Domain knowledge related to user questions here

[0113] "Here is the domain knowledge related to the data table"

[0114] #Output content description and format

[0115] After SQL is executed, detailed results or execution logs will be returned to help debug SQL statements;

[0116] Please answer strictly according to the format below. Do not enter comments or any content other than the format given below;

[0117] The output must start with "Final Answer".

[0118] According to the task description text of the current task, the prompt word template is instantiated to obtain the prompt text corresponding to the task to be executed in the current step, and then the prompt text is used to trigger the second generative model to generate database operation statements according to the prompt text. Taking the task description text in the previous article as an example, the following database operation statements can be generated:

[0119] “SELECT self_7d_imps AS "Number of store search exposures in the last 7 days", self_30d_imps AS "Number of store search exposures in the last 30 days self_30d_imps AS "Number of store search exposures in the last 30 days" FROM icbu_mechanism_free_talk_traffic_info_comp_d WHERE comp_id=29099949637;”.

[0120] Next, execute the generated database operation statement to obtain the data required for the current task.

[0121] Starting from the task corresponding to the first step in the workflow, after the current task is completed, the data obtained by the current task is used as reference information for the next task to be executed. Referring to the method described above, the task description text of the next task is continued to be generated, and the prompt text is generated based on the task description text. Then, the second generative large model is triggered to gradually execute the tasks corresponding to each step until all tasks generated based on the workflow are completed.

[0122] In some optional embodiments, the second generative big model is fine-tuned by the following method: generate query text corresponding to the historical database operation statement of the target application scenario through the pre-trained first generative big model; construct a first number of data pairs based on the historical database operation statement and the generated query text; fine-tune the pre-trained generative big model based on the first number of data pairs to obtain the second generative big model. The second generative big model is used to generate database operation statements based on the query text in the target application scenario.

[0123] The training method of the second generative large model is described in the previous text and will not be repeated here.

[0124] Step 206: obtaining data matching the query text based on the data acquired by executing the database operation statement in real time.

[0125] Finally, data that meets the requirements of the query text is obtained based on the data obtained from each task. For example, the data obtained from the task corresponding to the last step in the workflow is used as the data that matches the query text.

[0126] In summary, the data acquisition method disclosed in the embodiment of the present application, after receiving the query text, first searches a pre-established knowledge base to obtain application knowledge matching the query text, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of the workflow to be executed, and then, based on the description information and / or the data table, dynamically generates database operation statements corresponding to the tasks to be executed in the workflow in sequence, and executes the database operation statements in real time to obtain data, simplifies complex problems, and splits them into simple data processing tasks. Finally, based on the data obtained by executing the database operation statements in real time, data matching the query text is obtained, which effectively improves the logical processing capabilities of complex data operations, reduces data query complexity and execution time, and improves query efficiency. Furthermore, through workflow splitting, multi-data table joint queries are optimized, further improving data query efficiency.

[0127] Based on the above embodiments, the present application also discloses a data acquisition method, which is a client of a data acquisition system. Figure 3 As shown, the data acquisition method includes: step 302 and step 304.

[0128] Step 302, in response to the query text input by the user, a query request is sent to a preset server, and the query request is used to trigger the preset server to perform the following operations to obtain data: retrieve a pre-established knowledge base to obtain application knowledge that matches the query text carried by the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of the workflow to be executed; based on the description information and / or the data table, dynamically generate database operation statements corresponding to the tasks to be executed in the workflow in sequence, and execute the database operation statements in real time to obtain data; based on the data obtained by executing the database operation statements in real time, obtain data matching the query text.

[0129] Step 304: display the data matching the query text.

[0130] The data acquisition method disclosed in the embodiment of the present application can be implemented by a data acquisition system, which may include: a client and a server, wherein the client is used to display a human-computer interaction interface to obtain a query text input by a user, and send the query text to the server, which performs a data acquisition operation.

[0131] After receiving the query text sent by the client, the server first searches the pre-established knowledge base to obtain application knowledge that matches the query text, wherein the application knowledge includes: a data table that needs to be queried when obtaining data that matches the query text, and description information of the workflow to be executed. Afterwards, the server dynamically generates database operation statements corresponding to the tasks that need to be executed in the workflow based on the description information and / or the data table, and executes the database operation statements in real time to obtain data. Finally, the server obtains data that matches the query text based on the data obtained by executing the database operation statements in real time, and sends the obtained data that matches the query text to the client.

[0132] The client is also used to receive data matching the query text sent by the server, and to display the data matching the query text locally on the client.

[0133] For example, after the user enters the query text "Why is the search exposure in the last 7 days less than that in the last 30 days? My store ID is 29099949637" through the client, the client generates a data acquisition request based on the query text, and sends the query text to the server through the data acquisition request. The server further performs a knowledge base search based on the query text to obtain domain knowledge related to the problem, as well as application knowledge such as the steps to solve the problem, the data that needs to be read, and other application knowledge, and performs task splitting based on the obtained application knowledge, and then sequentially executes the split tasks. Finally, the data obtained from each task is integrated to generate data that matches the query text, where the data includes but is not limited to: data tables, texts, graphs, etc. The server sends the acquired data that matches the query text to the client, which displays it to the user.

[0134] The server retrieves a pre-established knowledge base to obtain application knowledge that matches the query text, dynamically generates database operation statements corresponding to the tasks that need to be performed in the workflow based on the description information and / or the data table, and executes the database operation statements in real time to obtain data, and obtains data that matches the query text based on the data obtained by executing the database operation statements in real time. For a specific implementation method, please refer to the relevant description in the previous embodiment, which will not be repeated here.

[0135] In summary, in the data acquisition method disclosed in the example of the present application, after the client obtains the query text input by the user, the client sends the query text to the preset server, and the preset server further searches a pre-established knowledge base to obtain application knowledge matching the query text, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of the workflow to be executed; thereafter, the preset server dynamically generates database operation statements corresponding to the tasks to be executed in the workflow based on the description information and / or the data table, and executes the database operation statements in real time to obtain data; finally, the preset server obtains data matching the query text based on the data obtained by executing the database operation statements in real time, and sends the data matching the query text to the client for display, which effectively improves the logical processing capability of complex data operations, reduces the complexity and execution time of data queries, and improves query efficiency.

[0136] In order to implement the above embodiment, the present application also discloses a data acquisition system, such as Figure 4 As shown, the data acquisition system includes: a client 402 and a server 404, wherein:

[0137] The client 402 is used to obtain the query text input by the user and send a query request carrying the query text to the server 404;

[0138] The server 404 is used to retrieve a pre-established knowledge base in response to the query request, and obtain application knowledge matching the query text carried in the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of a workflow to be executed;

[0139] The server 404 is further used to dynamically generate database operation statements corresponding to the tasks to be executed in the workflow based on the description information and / or the data table, and execute the database operation statements in real time to obtain data;

[0140] The server 404 is further configured to obtain data matching the query text based on the data obtained by executing the database operation statement in real time, and send the data matching the query text to the client 402;

[0141] The client 402 is also used to receive and display data matching the query text.

[0142] In summary, in the data acquisition system disclosed in the example of the present application, after the client obtains the query text input by the user, the client sends the query text to the preset server, and the preset server further searches a pre-established knowledge base to obtain application knowledge matching the query text, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of the workflow to be executed; thereafter, the preset server dynamically generates database operation statements corresponding to the tasks to be executed in the workflow based on the description information and / or the data table, and executes the database operation statements in real time to obtain data; finally, the preset server obtains data matching the query text based on the data obtained by executing the database operation statements in real time, and sends the data matching the query text to the client for display, which effectively improves the logical processing capability of complex data operations, reduces the complexity and execution time of data queries, and improves query efficiency.

[0143] In order to further verify the performance of the method for generating database operation statements disclosed in the embodiment of the present application and provide a basis for optimizing the second generative model, the embodiment of the present application also discloses a method for evaluating the generation performance of database operation statements. Figure 5 As shown, the method includes: step 502, step 504 and step 506.

[0144] Step 502, obtaining a database operation statement generated by the pre-fine-tuned second generative model for the target query text;

[0145] Step 504, when the target query text is annotated with a real database operation statement, based on the similarity between the database operation statement and the real database operation statement, and / or based on the consistency between the execution result of the database operation statement and the execution result of the real database operation statement, evaluate the performance of the second generative large model in generating the database operation statement;

[0146] Step 506, when the target query text is not labeled with real database operation statements, the performance of the second generative large model in generating database operation statements is evaluated based on the correlation between the data obtained by executing the database operation statements and the target query text.

[0147] The target query text is a query text used to test the database operation statement generation performance of the second generative model. Preferably, the target query text is different from the query text in the data pair used when fine-tuning the second generative model.

[0148] First, a prompt text is generated based on the target query text, and the prompt text is used to trigger a second generative model to generate a database operation statement corresponding to the target query text.

[0149] Next, according to different test data conditions, corresponding methods are used to evaluate the effectiveness of the database operation statements generated by the second generative large model.

[0150] If the target query text is annotated with real database operation statements, the validity of the database operation statements generated by the second generative large model can be evaluated by the following two methods.

[0151] The first type is comparison statements.

[0152] In some optional embodiments, when the target query text is annotated with a real database operation statement, based on the similarity between the database operation statement and the real database operation statement, evaluating the performance of the second generative large model in generating the database operation statement includes: judging the consistency between the database operation statement and the real database operation statement through a pre-trained text generative model, and evaluating the performance of the second generative large model in generating the database operation statement according to the consistency judgment result. For example, a GPT model can be used to judge the consistency between each of the database operation statements used for testing and the corresponding real database operation statement. If the GPT model judges that all the database operation statements and the corresponding real database operation statements are consistent, it can be considered that the performance of the second generative large model in generating the database operation statement meets the preset requirements; otherwise, it is considered that the performance of the second generative large model in generating the database operation statement does not meet the preset requirements.

[0153] The second method is to compare the execution results

[0154] In some optional embodiments, the performance of the second generative large model in generating database operation statements is evaluated based on the consistency between the execution result of the database operation statement and the execution result of the real database operation statement, including: executing the database operation statement and the real database operation statement respectively in the same database environment to obtain the first data acquired by the database operation statement and the second data acquired by the real database operation statement; judging the similarity between the first data and the second data through a pre-trained text generative model; and evaluating the performance of the second generative large model in generating database operation statements based on the similarity.

[0155] For example, the database operation statement and the real database operation statement can be executed separately to access the same database, and the data obtained by the database operation statement accessing the database is obtained, which is recorded as "first data", and the data obtained by the real database operation statement accessing the database is obtained, which is recorded as "second data". Afterwards, the GPT model can be used to determine the consistency of each group of first data and second data. If the GPT model determines that the first data and the second data are consistent, it can be considered that the database operation statement generated by the second generative large model is correct; otherwise, it is considered that the database operation statement generated by the second generative large model is incorrect. Furthermore, by performing the above judgment on several database operation statements generated by the second generative large model and comprehensively obtaining the judgment results, the performance of the database operation statement generated by the second generative large model can be evaluated.

[0156] If the target query text is not labeled with real database operation statements, the validity of the database operation statements generated by the second generative large model can be evaluated by the following third method.

[0157] The third type is the correlation between the statement execution results and the query text.

[0158] In some optional embodiments, the performance of the second generative large model in generating database operation statements is evaluated based on the matching degree between the data obtained by executing the database operation statement and the target query text, including: executing the database operation statement to obtain query data; judging the relevance between the query data and the target query text through a pre-trained text generative model, and judging the performance of the second generative large model in generating database operation statements based on the relevance judgment result. For example, the relevance between the query data and the target query text can be judged through a GPT model, and if the judgment result is relevant, the database operation statement generated based on the target query text is considered to be accurate; otherwise, the database operation statement generated based on the target query text is considered to be inaccurate.

[0159] In summary, the method for evaluating the generation performance of database operation statements disclosed in the embodiment of the present application adopts corresponding evaluation methods for different data conditions after the database operation statements generated by the second generative big model for the target query text are generated. For example: when the target query text is annotated with real database operation statements, the performance of the database operation statements generated by the second generative big model is evaluated based on the similarity between the database operation statements and the real database operation statements, and / or, based on the consistency between the execution results of the database operation statements and the execution results of the real database operation statements; when the target query text is not annotated with real database operation statements, the performance of the database operation statements generated by the second generative big model is evaluated based on the correlation between the data obtained by executing the database operation statements and the target query text, thereby achieving the technical effect of reliably evaluating the performance of the database operation statements generated by the second generative big model, which is helpful to promote the rapid iteration of NL2SQL for application scenarios and promote the application of NL2SQL technology based on different application scenarios.

[0160] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0161] On the basis of the above embodiment, this embodiment further provides a device for generating a database operation statement, the device comprising:

[0162] A query text generation module, used to generate a query text corresponding to a historical database operation statement of a target application scenario through a pre-trained first generative large model;

[0163] A first data pair construction module, configured to construct a first number of data pairs based on the historical database operation statement and the generated query text;

[0164] The first model fine-tuning module is used to fine-tune the pre-trained generative large model based on the data to obtain a second generative large model, wherein the second generative large model is used to generate database operation statements based on the query text in the target application scenario.

[0165] Optionally, the first generative large model is trained by the following method:

[0166] Acquire a second number of data pairs consisting of sample database operation statements and sample query texts of the target application scenario;

[0167] Based on the second number of data pairs, the first generative big model is trained, and the first generative big model is used to generate query text in the target application scenario based on data query statements.

[0168] Optionally, the first quantity is much larger than the second quantity.

[0169] The device for generating database operation statements disclosed in the embodiment of the present application is used to implement the above-mentioned method for generating database operation statements. The specific implementation of each module of the device can be found in the specific implementation of the corresponding steps in the above-mentioned method embodiment, which will not be repeated here.

[0170] In summary, the device for generating database operation statements disclosed in the embodiments of the present application generates query texts corresponding to historical database operation statements of the target application scenario through a pre-trained first generative big model, constructs a first number of data pairs based on the historical database operation statements and the generated query texts, and acquires massive labeled data of the target application scenario as training data by reversely generating data; thereafter, fine-tunes the pre-trained generative big model based on the data pairs to obtain a second generative big model, wherein the second generative big model is used to: generate database operation statements based on the query text under the target application scenario, so that the second generative big model has a stronger understanding ability of the database operation statements and query texts of the target application scenario, and can generate more effective database operation statements based on the query texts.

[0171] On the basis of the above embodiment, this embodiment further provides a data acquisition device, which is applied to a server, and the device includes:

[0172] A knowledge retrieval module is used to retrieve a pre-established knowledge base in response to a query request, and obtain application knowledge matching the query text carried by the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of a workflow to be executed;

[0173] A task splitting and execution module, used to dynamically generate database operation statements corresponding to the tasks to be executed in the workflow based on the description information and / or the data table, and execute the database operation statements in real time to obtain data;

[0174] The data acquisition module is used to obtain data matching the query text based on the data acquired by executing the database operation statement in real time.

[0175] Optionally, the task splitting and execution module is further used to:

[0176] Based on the description information and the data table, dynamically generate a first task description text of the first task, and based on the first task description text, generate and execute a database operation statement corresponding to the first task to obtain data, wherein the first task is the starting task of the workflow;

[0177] Based on one or more of the description information, the data table, and the data obtained from the executed tasks, a second task description text for the second task is dynamically generated, and based on the second task description text, a database operation statement corresponding to the second task is generated and executed to obtain data, wherein the second task is a task in the workflow other than the first task.

[0178] Optionally, generating and executing a database operation statement corresponding to the target task based on the target task description text includes:

[0179] Dynamically generate prompt text based on the target task description text;

[0180] Based on the prompt text, a second generative model obtained by fine-tuning in advance is called to generate a database operation statement corresponding to the target task;

[0181] Execute the database operation statement, wherein the target task includes: the first task or the second task.

[0182] Optionally, the target task description text includes one or more of the following: a description of the task content of the target task, a description of the data required to access the target task, and a description of the data output format. The dynamically generating the prompt text based on the target task description text includes:

[0183] Based on the target task description text, a prompt text is generated in a preset format to guide the second generative large model obtained by pre-fine-tuning to generate database operation statements.

[0184] Optionally, the second generative large model is obtained by fine-tuning by the following method:

[0185] Generate query text corresponding to the historical database operation statement of the target application scenario through the pre-trained first generative large model;

[0186] constructing a first number of data pairs based on the historical database operation statement and the generated query text;

[0187] Fine-tune the pre-trained generative large model based on the first amount of data to obtain a second generative large model.

[0188] The second generative large model is used to generate database operation statements based on the query text in the target application scenario.

[0189] On the basis of the above embodiment, this embodiment further provides a data acquisition device, which is applied to a client, and the device includes:

[0190] A query text sending module is used to send a query request to a preset server in response to a query text input by a user, wherein the query request is used to trigger the preset server to perform the following operations to obtain data: retrieve a pre-established knowledge base to obtain application knowledge matching the query text carried by the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of a workflow to be executed; based on the description information and / or the data table, dynamically generate database operation statements corresponding to tasks to be executed in the workflow in sequence, and execute the database operation statements in real time to obtain data; based on the data obtained by executing the database operation statements in real time, obtain data matching the query text;

[0191] A data display module is used to display the data matching the query text.

[0192] The data acquisition device disclosed in the embodiment of the present application is used to implement the above-mentioned data acquisition method. The specific implementation of each module of the device refers to the specific implementation of the corresponding steps in the above-mentioned method embodiment, which will not be repeated here.

[0193] In summary, in the data acquisition device disclosed in the example of the present application, after the client obtains the query text input by the user, the query text is sent to the preset server, and the preset server further searches a pre-established knowledge base to obtain application knowledge matching the query text, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of the workflow to be executed; thereafter, the preset server dynamically generates database operation statements corresponding to the tasks to be executed in the workflow based on the description information and / or the data table, and executes the database operation statements in real time to obtain data; finally, the preset server obtains data matching the query text based on the data obtained by executing the database operation statements in real time, and sends the data matching the query text to the client for display, which effectively improves the logical processing capability of complex data operations, reduces the complexity and execution time of data queries, and improves query efficiency.

[0194] Based on the above embodiment, this embodiment further provides a generation performance evaluation device for database operation statements, the device comprising:

[0195] A database operation statement generation module, used to obtain the database operation statement generated by the pre-fine-tuned second generative large model for the target query text;

[0196] A first evaluation module is used to evaluate the performance of the second generative large model in generating database operation statements based on the similarity between the database operation statements and the real database operation statements, and / or based on the consistency between the execution results of the database operation statements and the execution results of the real database operation statements, when the target query text is annotated with real database operation statements;

[0197] The second evaluation module is used to evaluate the performance of the second generative large model in generating database operation statements based on the correlation between the data obtained by executing the database operation statements and the target query text when the target query text is not marked with real database operation statements.

[0198] Optionally, the evaluating the performance of the second generative large model in generating the database operation statement based on the consistency between the execution result of the database operation statement and the execution result of the real database operation statement includes:

[0199] Executing the database operation statement and the real database operation statement in the same database environment respectively to obtain first data acquired by the database operation statement and second data acquired by the real database operation statement;

[0200] Determine the similarity between the first data and the second data by using a pre-trained text generation model;

[0201] The performance of the second generative large model in generating database operation statements is evaluated based on the similarity.

[0202] The generation performance evaluation device for database operation statements disclosed in the embodiment of the present application is used to implement the generation performance evaluation method for the above-mentioned database operation statements. The specific implementation methods of each module of the device refer to the specific implementation methods of the corresponding steps in the above-mentioned method embodiment, which will not be repeated here.

[0203] In summary, the device for evaluating the generation performance of database operation statements disclosed in the embodiment of the present application adopts corresponding evaluation methods for different data conditions after the database operation statements generated by the second generative big model for the target query text are generated. For example, when the target query text is annotated with real database operation statements, the performance of the database operation statements generated by the second generative big model is evaluated based on the similarity between the database operation statements and the real database operation statements, and / or based on the consistency between the execution results of the database operation statements and the execution results of the real database operation statements; when the target query text is not annotated with real database operation statements, the performance of the database operation statements generated by the second generative big model is evaluated based on the correlation between the data obtained by executing the database operation statements and the target query text, thereby achieving the technical effect of reliably evaluating the performance of the database operation statements generated by the second generative big model, which is helpful to promote the rapid iteration of NL2SQL for application scenarios and promote the application of NL2SQL technology based on different application scenarios.

[0204] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0205] The embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method described in the embodiment of the present application.

[0206] The embodiment of the present application also provides an electronic device, comprising: a processor, and a memory connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method described in the embodiment of the present application. In the embodiment of the present application, the electronic device includes a server, a terminal device, and other devices.

[0207] The embodiment of the present application also discloses a computer program product, including a computer program / computer executable instructions, characterized in that when the computer program / computer executable instructions are executed by a processor in an electronic device, the method described in the embodiment of the present application is implemented.

[0208] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as a server (cluster), a terminal, etc. Figure 6An exemplary apparatus 600 that may be used to implement various embodiments described in this application is schematically illustrated.

[0209] For one embodiment, Figure 6 An exemplary apparatus 600 is shown having one or more processors 602, a control module (chip set) 604 coupled to at least one of the processor(s) 602, a memory 606 coupled to the control module 604, a non-volatile memory (NVM) / storage device 608 coupled to the control module 604, one or more input / output devices 610 coupled to the control module 604, and a network interface 612 coupled to the control module 604.

[0210] The processor 602 may include one or more single-core or multi-core processors, and the processor 602 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 600 can be used as a server, terminal, or other device described in the embodiments of the present application.

[0211] In some embodiments, the apparatus 600 may include one or more computer-readable media (e.g., memory 606 or NVM / storage device 608) having instructions 614 and one or more processors 602 combined with the one or more computer-readable media and configured to execute the instructions 614 to implement a module to perform the actions described in the present disclosure.

[0212] For one embodiment, control module 604 may include any suitable interface controller to provide any suitable interface to at least one of processor(s) 602 and / or any suitable device or component in communication with control module 604 .

[0213] The control module 604 may include a memory controller module to provide an interface to the memory 606. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0214] The memory 606 may be used, for example, to load and store data and / or instructions 614 for the device 600. For one embodiment, the memory 606 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 606 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0215] For one embodiment, control module 604 may include one or more input / output controllers to provide an interface to NVM / storage device 608 and input / output device(s) 610 .

[0216] For example, NVM / storage 608 may be used to store data and / or instructions 614. NVM / storage 608 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0217] NVM / storage device 608 may include storage resources that are part of the device on which apparatus 600 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 608 may be accessed via (one or more) input / output devices 610 over a network.

[0218] (One or more) input / output devices 610 may provide an interface for the apparatus 600 to communicate with any other appropriate device, and the input / output device 610 may include a communication component, an audio component, a sensor component, etc. The network interface 612 may provide an interface for the apparatus 600 to communicate through one or more networks, and the apparatus 600 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as Bluetooth, WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0219] For one embodiment, at least one of the processor(s) 602 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 604. For one embodiment, at least one of the processor(s) 602 may be packaged together with the logic of one or more controllers of the control module 604 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 602 may be integrated on the same die with the logic of one or more controllers of the control module 604. For one embodiment, at least one of the processor(s) 602 may be integrated on the same die with the logic of one or more controllers of the control module 604 to form a system-on-chip (SoC).

[0220] In various embodiments, the device 600 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 600 may have more or fewer components and / or different architectures. For example, in some embodiments, the device 600 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0221] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in a memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0222] The embodiment of the present application also provides an electronic device, including: a processor; and a memory, on which executable code is stored, and when the executable code is executed, the processor executes one or more methods as described in the embodiment of the present application. In the embodiment of the present application, the memory can store various data, such as target files, file and application association data, and user behavior data, so as to provide a data basis for various processing.

[0223] The embodiments of the present application also provide one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the embodiments of the present application.

[0224] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0225] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0226] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0227] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0228] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0229] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0230] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0231] The above is a detailed introduction to a method for generating a database operation statement, a data acquisition method, a data acquisition system, a method for evaluating the generation performance of a database operation statement, an electronic device, a storage medium and a computer program product provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for a person skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for generating a database operation statement, characterized in that: The method comprises: Generate query text corresponding to the historical database operation statement of the target application scenario through the pre-trained first generative large model; constructing a first number of data pairs based on the historical database operation statement and the generated query text; Based on the data, the pre-trained generative big model is fine-tuned to obtain a second generative big model, wherein the second generative big model is used to generate database operation statements based on the query text in the target application scenario.

2. The method according to claim 1, characterized in that The first generative large model is trained by the following method: Acquire a second number of data pairs consisting of sample database operation statements and sample query texts of the target application scenario; Based on the second number of data pairs, the first generative big model is trained, and the first generative big model is used to generate query text in the target application scenario based on data query statements.

3. The method according to claim 2, characterized in that The first number is greater than the second number.

4. A data acquisition method, applied to a server, characterized in that: The method comprises: In response to a query request, a pre-established knowledge base is retrieved to obtain application knowledge matching the query text carried in the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of a workflow to be executed; Based on the description information and / or the data table, dynamically generate database operation statements corresponding to the tasks to be executed in the workflow in sequence, and execute the database operation statements in real time to obtain data; Based on the data acquired by executing the database operation statement in real time, data matching the query text is obtained.

5. The method according to claim 4, characterized in that The step of dynamically generating database operation statements corresponding to the tasks to be executed in the workflow based on the description information and / or the data table, and executing the database operation statements in real time to acquire data includes: Based on the description information and the data table, dynamically generate a first task description text of the first task, and based on the first task description text, generate and execute a database operation statement corresponding to the first task to obtain data, wherein the first task is the starting task of the workflow; Based on one or more of the description information, the data table, and the data obtained from the executed tasks, a second task description text for the second task is dynamically generated, and based on the second task description text, a database operation statement corresponding to the second task is generated and executed to obtain data, wherein the second task is a task in the workflow other than the first task.

6. The method according to claim 5, characterized in that Generate and execute database operation statements corresponding to the target task based on the target task description text, including: Dynamically generate prompt text based on the target task description text; Based on the prompt text, a second generative model obtained by fine-tuning in advance is called to generate a database operation statement corresponding to the target task; Execute the database operation statement, wherein the target task includes: the first task or the second task.

7. The method according to claim 6, characterized in that The target task description text includes one or more of the following: a description of the task content of the target task, a description of the data required to access the target task, and a description of the data output format. The dynamically generating the prompt text based on the target task description text includes: Based on the target task description text, a prompt text is generated in a preset format to guide the second generative large model obtained by pre-fine-tuning to generate database operation statements.

8. The method according to claim 6, characterized in that The second generative large model is fine-tuned by the following method: Generate query text corresponding to the historical database operation statement of the target application scenario through the pre-trained first generative large model; constructing a first number of data pairs based on the historical database operation statement and the generated query text; Fine-tune the pre-trained generative large model based on the first amount of data to obtain a second generative large model. The second generative large model is used to generate database operation statements based on the query text in the target application scenario.

9. A data acquisition method, applied to a client, characterized in that: The method comprises: In response to the query text input by the user, a query request is sent to a preset server, and the query request is used to trigger the preset server to perform the following operations to obtain data: retrieve a pre-established knowledge base to obtain application knowledge matching the query text carried by the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of the workflow to be executed; based on the description information and / or the data table, dynamically generate database operation statements corresponding to the tasks to be executed in the workflow in sequence, and execute the database operation statements in real time to obtain data; based on the data obtained by executing the database operation statements in real time, obtain data matching the query text; The data matching the query text is displayed.

10. A method for evaluating the generation performance of database operation statements, characterized in that: The method comprises: Obtain database operation statements generated by the pre-fine-tuned second generative large model for the target query text; In the case where the target query text is annotated with a real database operation statement, based on the similarity between the database operation statement and the real database operation statement, and / or based on the consistency between the execution result of the database operation statement and the execution result of the real database operation statement, the performance of the second generative large model in generating the database operation statement is evaluated; In the case that the target query text is not labeled with real database operation statements, the performance of the second generative large model in generating database operation statements is evaluated based on the correlation between the data obtained by executing the database operation statements and the target query text.

11. The method according to claim 10, characterized in that The step of evaluating the performance of the second generative large model in generating database operation statements based on the consistency between the execution result of the database operation statement and the execution result of the real database operation statement includes: Executing the database operation statement and the real database operation statement in the same database environment respectively to obtain first data acquired by the database operation statement and second data acquired by the real database operation statement; Determine the similarity between the first data and the second data by using a pre-trained text generation model; The performance of the second generative large model in generating database operation statements is evaluated based on the similarity.

12. A data acquisition system, characterized in that: The system comprises: a client and a server, wherein: The client is used to obtain the query text input by the user and send a query request carrying the query text to the server; The server is used to retrieve a pre-established knowledge base in response to the query request, and obtain application knowledge matching the query text carried by the query request, wherein the application knowledge includes: a data table to be queried when obtaining data matching the query text, and description information of a workflow to be executed; The server is further used to dynamically generate database operation statements corresponding to the tasks to be executed in the workflow based on the description information and / or the data table, and execute the database operation statements in real time to obtain data; The server is further configured to obtain data matching the query text based on the data obtained by executing the database operation statement in real time, and send the data matching the query text to the client; The client is also used to receive and display data matching the query text.

13. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 11 when executed by a processor.

15. A computer program product comprising a computer program / computer executable instructions, characterized in that: When the computer program / computer executable instructions are executed by a processor in an electronic device, the method according to any one of claims 1 to 11 is implemented.

Images