Index acquisition method, device, electronic device and computer-readable storage medium

Abstract

The embodiments of the present application provide an indicator acquisition method, device, electronic device and computer-readable storage medium, which relate to the field of artificial intelligence. The method includes: obtaining information to be queried expressed in natural language; inputting the information to be queried into a pre-trained large language model to obtain the target indicator output by the large language model; inputting the target indicator into a preset indicator system to obtain the corresponding SQL statement, and calling the SQL statement to obtain the query result from the database. The embodiments of the present application use a large language model to split the information to be queried expressed in natural language into multiple sub-problems, and use operators to characterize each sub-problem. It has very good interpretability for natural language, and at the same time does not depend on business scenarios, which is obviously helpful to improve the accuracy and generalization of target indicators.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence. Specifically, the present application relates to an indicator acquisition method, device, electronic device and computer-readable storage medium. Background Art

[0002] Natural language to SQL statement conversion is the process of converting natural language used by humans (such as Chinese or English) into structured query language (SQL). In the process of natural language to SQL conversion, the system usually needs to parse and understand the questions or requests raised by users in natural language, and then convert them into equivalent SQL statements to be executed on the database and obtain the required data.

[0003] In the related art, the user's question can be converted into a corresponding SQL statement by directly converting natural language to SQL (NL2SQL) and converting natural language to indicator and then to SQL (NL2Metric2SQL).

[0004] Regarding the method of directly converting natural language into SQL, since users ask questions in a variety of ways and the same meaning can be expressed in multiple ways, and questions often contain business abbreviations or semantic ambiguity, the user's questions can only be understood by the large language model itself, and the accuracy of the resulting SQL statements is relatively limited; in addition, regarding the method of converting natural language into indicators and then into SQL statements, the task is decoupled into natural language through indicators as a medium, and then indicator parameters are extracted and converted into SQL, which loses a certain degree of flexibility. Summary of the invention

[0005] The embodiments of the present application provide an indicator acquisition method, device, electronic device and computer-readable storage medium, which can solve the above-mentioned problems of the prior art. The technical solution is as follows:

[0006] According to a first aspect of an embodiment of the present application, a method for obtaining an indicator is provided, comprising:

[0007] Obtaining the information to be queried expressed in natural language;

[0008] Inputting the information to be queried into a pre-trained large language model to obtain a target indicator output by the large language model;

[0009] Input the target indicator into a preset indicator system, obtain a corresponding SQL statement, and call the SQL statement to obtain a query result from a database;

[0010] The step of inputting the information to be queried into a pre-trained large language model to obtain a target indicator output by the large language model includes:

[0011] Inputting the information to be queried into the large language model to obtain at least one target operator, wherein the target operator includes a target operator and target operation data, and the target operation data is an entity in the information to be queried;

[0012] According to the conversion relationship between operators and indicators, the target operator and target operation data in each target operator are combined to obtain the target indicator output by the large language model.

[0013] As an optional implementation, the conversion relationship between the operator and the indicator includes a first mapping relationship between the indicator name and the indicator type, and a second mapping relationship between the indicator attribute and the operator; the large language model includes a first sub-model and a second sub-model;

[0014] The step of obtaining the target indicator output by the large language model based on the conversion relationship between the operator and the indicator and combining the target operator and the target operation data in each target operator includes:

[0015] Taking each operator including the preset first operator among all target operators as the first operator;

[0016] Inputting the target operation data of the first operator into the first sub-model, and obtaining a first indicator name corresponding to the target operation data of the first operator output by the first sub-model;

[0017] According to the first mapping relationship, obtain a first indicator type corresponding to the first indicator name;

[0018] Obtaining the indicator attribute preset for the first indicator type as a candidate indicator attribute;

[0019] According to the second mapping relationship, an operator corresponding to each candidate indicator attribute is obtained as the second operator, each second operator is matched with the target operator of each target operator, and the successfully matched target operator is used as the second operator; according to the second mapping relationship, the indicator attribute corresponding to the target operator of each second operator is obtained from the candidate indicator attribute;

[0020] Using an operator different from the first operator among the second operators as a third operator, inputting target operation data of each third operator into the second sub-model, and obtaining attribute values ​​of indicator attributes corresponding to target operators of each third operator output by the second sub-model;

[0021] Obtaining the indicator attribute corresponding to the target operator of the first operator and the indicator attribute corresponding to the target operator of each third operator from the indicator attributes corresponding to the target operator of each second operator; using the first indicator name as the attribute value of the indicator attribute corresponding to the target operator of the first operator;

[0022] The target indicator is generated according to the indicator attribute corresponding to the target operator of the first operator, the first indicator name, the indicator attributes corresponding to the target operators of each third operator, and the attribute value of the indicator attribute corresponding to the target operator of each third operator.

[0023] As an optional implementation manner, the first sub-model is trained using the operation data of the sample operator as training samples and the indicator name corresponding to the operation data of the sample operator as training labels;

[0024] The second sub-model is trained by using the operation data of the sample operator as training samples and the attribute values ​​of the indicator attributes corresponding to the operation data of the sample operator as training labels.

[0025] As an optional implementation, the large language model includes a third sub-model;

[0026] The step of inputting the information to be queried into the third sub-model to obtain at least one target operator, wherein the target operator includes a target operator and target operation data, comprises:

[0027] Inputting the information to be queried into the third sub-model, the third sub-model decomposes the information to be queried into at least one sub-question, and performs semantic analysis on each sub-question to obtain the grammatical structure and entity of each sub-question, as well as the reference relationship between the sub-questions;

[0028] For each sub-problem, the third sub-model determines a target operator corresponding to the sub-problem according to a grammatical structure and at least one of an entity of the sub-problem;

[0029] For each sub-problem, the third sub-model uses the target operator of the sub-problem that has a reference relationship with the sub-problem as the fourth operator according to the reference relationship between the sub-problems, obtains the output data of the fourth operator, and uses at least one of the entity of the sub-problem and the output data of the fourth operator as the target operation data of the target operator corresponding to the sub-problem;

[0030] For each sub-problem, the third sub-model outputs a target operator corresponding to the sub-problem in a one-to-one manner according to the target operator and target operation data of the sub-problem.

[0031] As an optional implementation manner, the using at least one of the entity of the subproblem and the output data of the fourth operator as the target operation data of the target operator corresponding to the subproblem includes:

[0032] If it is determined that all entities of the subproblem are irrelevant to the output data of the target operator corresponding to the subproblem that has a reference relationship with the subproblem, the entity of the subproblem is used as the target operation data of the target operator corresponding to the subproblem;

[0033] If it is determined that all entities of the subproblem are related to the output data of the target operator corresponding to the subproblem that has a reference relationship with the subproblem, the output data of the operator is used as the target operation data of the target operator corresponding to the subproblem;

[0034] If it is determined that the first entity of the subproblem is independent of the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship, and that the second entity of the subproblem is related to the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship, then the first entity and the output data of the target operator shall be used as the target operation data of the target operator corresponding to the subproblem.

[0035] As an optional implementation, it also includes:

[0036] For each target operator, a target operator, target operation data and output data of the target operator are respectively regarded as a node;

[0037] For each target operator, construct a directed edge between a node corresponding to the target operation data of the target operator and a node corresponding to the target operator of the target operator, and construct a directed edge between a node corresponding to the target operator of the target operator and a node corresponding to the output data of the target operator;

[0038] A directed acyclic graph is constructed according to each node and the directed edges between each node. The directed acyclic graph is used to characterize the relationship between the composite operations between the target operators.

[0039] As an optional implementation, the indicator system includes a third mapping relationship between preset indicator attributes and SQL statement keywords;

[0040] The step of inputting the target indicator into a preset indicator system to obtain a corresponding SQL statement includes:

[0041] The indicator system obtains the keyword of the SQL statement corresponding to each indicator attribute in the target indicator according to the third mapping relationship;

[0042] The first indicator name of the target indicator and the attribute value of each indicator attribute are used as parameters corresponding to the corresponding keywords;

[0043] According to the keywords of the SQL statement and the parameters corresponding to each keyword, the SQL statement corresponding to the target indicator is obtained.

[0044] According to a second aspect of an embodiment of the present application, there is provided an indicator acquisition device, including:

[0045] A first acquisition module is used to acquire the information to be queried expressed in natural language;

[0046] A second acquisition module is used to input the information to be queried into a pre-trained large language model to obtain a target indicator output by the large language model;

[0047] The third acquisition module is used to input the target indicator into a preset indicator system, obtain a corresponding SQL statement, and call the SQL statement to obtain a query result from the database;

[0048] The step of inputting the information to be queried into a pre-trained large language model to obtain a target indicator output by the large language model includes:

[0049] Inputting the information to be queried into the large language model to obtain at least one target operator, wherein the target operator includes a target operator and target operation data, and the target operation data is an entity in the information to be queried;

[0050] According to the conversion relationship between operators and indicators, the target operator and target operation data in each target operator are combined to obtain the target indicator output by the large language model.

[0051] According to a third aspect of an embodiment of the present application, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory, and when the processor executes the program, the steps of the method provided in the first aspect are implemented.

[0052] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method provided in the first aspect are implemented.

[0053] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0054] The present application uses a large language model to decompose the information to be queried into multiple sub-problems, thus overcoming the problem of low flexibility caused by directly converting the information to be queried input by the user into indicators in the prior art; at the same time, each sub-problem is converted into a target operator through the large language model. Since the target operator is closer to natural language, it has very good explanatory power for the sub-problem and is independent of the business scenario. It can be used as an independent task to assist the natural language understanding of the information to be queried input by the user, which is significantly helpful for improving the accuracy and generalization of the target indicator; the embodiment of the present application obtains the target indicator output by the large language model based on the conversion relationship between the operator and the indicator, combined with the target operator and the target operation data in each target operator, thus overcoming the problem of low flexibility caused by directly converting the information to be queried input by the user into indicators in the prior art, and effectively improving the accuracy of converting the information to be queried into the target indicator. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0056] Figure 1 A schematic diagram of the system architecture of the indicator acquisition method provided in an embodiment of the present application;

[0057] Figure 2 A flowchart of an indicator acquisition method provided in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of a directed acyclic graph provided in an embodiment of the present application;

[0059] Figure 4 A schematic diagram of a process for converting information to be queried into an SQL statement provided in an embodiment of the present application;

[0060] Figure 5 A schematic diagram of the structure of an indicator acquisition device provided in an embodiment of the present application;

[0061] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0063] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B".

[0064] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0065] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0066] First, several terms involved in this application are introduced and explained:

[0067] Natural language: Natural language is the main tool for communication in human society and the main carrier for transmitting information between people. It includes various languages ​​used by humans, such as Chinese, English, French, etc. These languages ​​are formed in the long-term historical development process of humans and have complex grammatical rules, vocabulary and expressions.

[0068] Entity: In the field of natural language processing, an entity usually refers to a nominal object or concept with a specific meaning or identity. An entity can be a word or phrase with a clear referential meaning, such as a person's name, a place name, an organization name, a date, or a time.

[0069] Large Language Model: A Large Language Model (LLM) is a deep learning model in the field of Natural Language Processing (NLP). It usually contains a large number of parameters and is pre-trained on a large corpus to be able to understand and generate human language. These models have powerful language representation capabilities and can handle a variety of complex language tasks.

[0070] Operator: In computer science, an operator is a symbol that operates on data. An operator consists of an operator and operation data.

[0071] Directed Acyclic Graph: A directed acyclic graph (DAG) is a special graph structure that contains directed edges but does not contain any loops. In a directed acyclic graph, each edge has a clear direction, and starting from any vertex and moving along the direction of the edge, it is impossible to return to the vertex, forming a closed loop.

[0072] Natural language to SQL statement conversion is the process of converting natural language (such as Chinese or English) used by humans into structured query language (SQL statement). In the process of natural language to SQL conversion, the system usually needs to parse and understand the questions or requests raised by users in natural language, and then convert them into equivalent SQL statements to be executed on the database and obtain the required data.

[0073] In related technologies, users' questions can be converted into corresponding SQL statements through the solutions of converting natural language directly to SQL (NL2SQL) and converting natural language to indicators and then to SQL (NL2Metric2SQL). However, no matter which solution is adopted, there are some difficulties in natural language understanding, mainly including:

[0074] Users ask questions in a variety of ways, and the same meaning can be expressed in multiple ways. Questions often contain business abbreviations or semantic ambiguity. If we do not perform structured analysis on natural language, we can only rely on the parsing model or the large language model's own capabilities to understand them, and the accuracy is relatively limited.

[0075] In addition, a complex problem needs to be split into multiple sub-problems to be solved step by step. How to judge whether a problem needs to be split and how to split it correctly not only tests the ability to understand natural language, but also the ability to understand SQL. Each sub-problem should be translated into a simple SQL. Existing solutions do not have a good solution to this problem.

[0076] The indicator acquisition method, device, electronic device, and computer-readable storage medium provided in this application are intended to solve the above technical problems in the prior art.

[0077] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0078] Figure 1 A schematic diagram of the system architecture of the indicator acquisition method provided in the embodiment of the present application, including a terminal 101 and a server 102. The terminal 101 and the server 102 are connected via a wired or wireless network. Optionally, the terminal 101 can be a terminal with data calculation, processing and storage capabilities such as a personal computer (PC), a vehicle terminal, a tablet computer, a smart phone, a wearable device, an intelligent robot, etc.; the server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0079] In the embodiment of the present application, the terminal 101 can be used for the information to be queried expressed in natural language input by the user, and the terminal 101 sends the information to be queried to the server 102. The server 102 includes a pre-trained large language model, and the information to be queried is input into the pre-trained large language model to obtain at least one target operator, and the target operator includes a target operator and target operation data, and the target operation data is an entity in the information to be queried; the server 102 also includes a conversion relationship between operators and indicators, and according to the conversion relationship between operators and indicators, combined with the target operator and target operation data in each target operator, the target indicator output by the pre-trained large language model is obtained.

[0080] The server 102 inputs the target indicator into a preset indicator system, obtains the SQL statement corresponding to the target indicator, calls the SQL statement to obtain the query result from the database; and sends the query result to the terminal 101 as the answer corresponding to the information to be queried input by the user.

[0081] Those skilled in the art will appreciate that the number of the servers 102 may be more or less. For example, there may be only one server, or there may be dozens or hundreds of servers, or a greater number. The embodiment of the present application does not limit the number and type of servers.

[0082] In some embodiments, server 102 is an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0083] In some embodiments, the server 102 undertakes the main computing work and the terminal 101 undertakes the secondary computing work; or, the server 102 undertakes the secondary computing work and the terminal 101 undertakes the main computing work; or, the server 102 and the terminal 101 adopt a distributed computing architecture to perform collaborative computing.

[0084] The indicator acquisition method in the embodiment of the present application is mainly completed by the server. The scheme in the embodiment of the present application is described below with the server as the execution subject.

[0085] Figure 2 A flowchart of an indicator acquisition method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method includes:

[0086] S201. Obtain information to be queried expressed in natural language.

[0087] In the embodiment of the present application, the user inputs the information to be queried in natural language at the terminal 101. The information to be queried can be a question, a description, or any other form of data, which is intended to obtain an answer corresponding to the information to be queried. For example, the information to be queried input by the user in Chinese is "What is the total operating expenses of branches with a number greater than 71?".

[0088] It should be noted that in the embodiments of the present application, the natural language can be not only Chinese, but also English, French, etc.; the information to be queried is a problem in a specific field, in addition to the banking field mentioned above, it can also be the manufacturing field, meteorological field, etc. This application does not limit the specific type of natural language and the field to which the information to be queried belongs.

[0089] S202: Input the information to be queried into a pre-trained large language model to obtain a target indicator output by the large language model.

[0090] In an embodiment of the present application, a pre-trained large language model can decompose the input information to be queried into multiple sub-questions. Taking the query information "What is the total operating expenses of branches with numbers greater than 71?" as an example, the large language model decomposes the query information into sub-questions and obtains two sub-questions, namely: Sub-question 1 "What is the total operating expenses?", and Sub-question 2 "What is the total operating expenses of branches with numbers greater than 71?".

[0091] In an embodiment of the present application, the large language model can obtain the grammatical structure and entity of each sub-problem, and according to at least one of the grammatical structure and entity of the sub-problem, the target operator corresponding to the sub-problem can be determined from the pre-set operator operators, wherein the operators of the pre-set operators in the embodiment of the present application are shown in Table 1.

[0092] Table 1 Operators of operators

[0093]

[0094] Specifically, Table 1 gives the operators of the operator and the behavior and function of each operator. According to the grammatical structure and entities of subproblems 1 and 2, it can be determined that the target operator corresponding to subproblem 1 is "SELECT" and the target operator of subproblem 2 is "FILTER". At the same time, the large language model can also determine the entity in each subproblem and use the entity as the target operation data corresponding to the subproblem, where the entity in subproblem 1 is "total operating expenses"; the entities in subproblem 2 are "branch number greater than 71" and "total operating expenses"; therefore, the target operation data corresponding to subproblem 1 is "total operating expenses", and the target operation data corresponding to subproblem 2 is "branch number greater than 71" and "total operating expenses"; according to the target operator and target operation data corresponding to the subproblem, the target operator corresponding to the subproblem can be obtained.

[0095] After obtaining the target operator corresponding to the sub-problem, the target operator is converted into the target indicator through the conversion relationship between the operator and the indicator; for example, the target operator corresponding to sub-problem 1 and the target operator corresponding to sub-problem 2 are converted into corresponding target indicators. The target indicator can be expressed as: [metricNames: operating expenses_SUM; where: branch number>71].

[0096] S203: Input the target indicator into a preset indicator system, obtain the corresponding SQL statement, and call the SQL statement to obtain the query result from the database.

[0097] In an embodiment of the present application, the indicator system includes a front-end graphical configuration interface and a back-end system; wherein, the front-end graphical configuration interface provides an input dialog box, the user can input the information to be queried into the dialog box, and the back-end system can translate the indicator into an SQL statement. The user inputs the information to be queried into the graphical interface, and the trained large language model generates indicators from the information to be queried, all of which are processed by the back-end system, and the user does not feel it; at the same time, the graphical configuration interface can also add, delete, check, and modify the generated indicators, including but not limited to adding indicator attributes and modifying the attribute values ​​of indicator attributes; in addition, the graphical configuration interface also includes a click button, when the user touches the button, the back-end system can convert the indicator into an SQL statement, call the SQL statement to obtain the query result from the database, and display the query result on the front-end graphical interface.

[0098] The embodiments of the present application solve the problems in the prior art of directly converting the information to be queried input by the user into SQL statements, which leads to low accuracy, and directly converting the information to be queried input by the user into indicators, which leads to low flexibility. The sub-problems are represented by target operators, which effectively improves the flexibility of converting the information to be queried into indicators. At the same time, through the conversion relationship between the operator and the indicator, the target operator is converted into the corresponding indicator, which improves the accuracy of converting the information to be queried into the target indicator.

[0099] Based on the above embodiments, as an optional embodiment, the conversion relationship between the operator and the indicator includes a first mapping relationship between the indicator name and the indicator type, and a second mapping relationship between the indicator attribute and the operator; the large language model includes a first sub-model and a second sub-model;

[0100] According to the conversion relationship between operators and indicators, combined with the target operators and target operation data in each target operator, the target indicators output by the large language model are obtained, including:

[0101] S301. Take each operator including a preset first operator among all target operators as a first operator.

[0102] In the embodiment of the present application, the first operator includes "SELECT" and "PROJECT". After obtaining the target operator corresponding to the sub-problem, the target operator including the first operator among all the target operators is used as the first operator.

[0103] S302: Input the target operation data of the first operator into the first sub-model, and obtain the first indicator name corresponding to the target operation data of the first operator output by the first sub-model.

[0104] In the embodiment of the present application, the target operation data of the first operator is input into the first sub-model to obtain the first indicator name. For example, if the target operation data of a first operator is "total operating expenses", the corresponding first indicator name may be "total expenses_SUM".

[0105] S303. Obtain a first indicator type corresponding to the first indicator name according to the first mapping relationship; and obtain an indicator attribute preset for the first indicator type as a candidate indicator attribute.

[0106] In an embodiment of the present application, there is a first mapping relationship between the first indicator name and the first indicator type. After the first indicator name is obtained, the first indicator type can be determined according to the first mapping relationship.

[0107] It should be noted that in the embodiment of the present application, multiple first indicator names may correspond to the same first indicator type. For example, when the first indicator name is profit margin or KPI achievement rate, it can be mapped to the proportion type, that is, there is a many-to-one mapping relationship between the first indicator name and the indicator type; the first indicator name and the first indicator type can also be a one-to-one relationship, that is, one first indicator name corresponds to one first indicator type.

[0108] It should also be noted that in the embodiment of the present application, after determining the first indicator type, the indicator attributes preset for the first indicator type can be obtained as candidate indicator attributes. For example, when determining that the first indicator type is a count type, the indicator attributes preset for the query type can be obtained, wherein the indicator attributes included in the query type include but are not limited to: metricNames, where, filter, groupBys. Therefore, the indicator attributes of metricNames, where, filter, and groupBys can be used as candidate indicator attributes.

[0109] S304. According to the second mapping relationship, obtain the operator corresponding to each candidate indicator attribute as the second operator, match each second operator with the target operator of each target operator, and use the successfully matched target operator as the second operator; according to the second mapping relationship, obtain the indicator attribute corresponding to the target operator of each second operator from the candidate indicator attributes.

[0110] In the embodiment of the present application, the second mapping relationship between indicator attributes and operators is shown in Table 2.

[0111] Table 2 Mapping relationship between indicator attributes and operators

[0112]

[0113] Specifically, as shown in Table 2, a mapping relationship table between some indicator attributes and operators in an embodiment of the present application is given. According to the mapping relationship in Table 2, the operator corresponding to each candidate indicator attribute can be obtained as the second operator. For example, the operator corresponding to the indicator attribute metricNames is SELECT and PROJECT, the operator corresponding to the indicator attribute groupBys is GROUP, the operator corresponding to the indicator attribute where is FILTER, PROJECT and INTERSECTION, and the operator corresponding to the indicator attribute orderBys is SORT and TOP.

[0114] In an embodiment of the present application, after obtaining the second operator, the second operator is matched with the target operator in the target operator, and the successfully matched target operator is used as the second operator, for example: when the second operator includes SELECT, FILTER and SORT; at this time, the target operator includes two, among which the target operator of the first target operator is SELECT, and the target operator of the second target operator is FILTER, then the first target operator matches the SELECT in the second operator, and the second target operator matches the FILTER in the second operator; therefore, the first target operator and the second target operator are used as the second operator.

[0115] Further, after the second operator is determined, the indicator attribute corresponding to the target operator of each second operator is determined from the candidate indicator attributes according to the second mapping relationship between the indicator attribute and the operator. For example, the indicator attribute corresponding to the target operator of the first target operator is metricNames, and the indicator attribute corresponding to the target operator of the second target operator is where.

[0116] S305. Use the operator different from the first operator in each second operator as the third operator, input the target operation data of each third operator into the second sub-model, and obtain the attribute value of the indicator attribute output by the second sub-model that corresponds one-to-one to the target operator of each third operator.

[0117] In an embodiment of the present application, the second operator includes the first operator. Since the target operation data in the first operator including the first operator has been converted into the first indicator name using a large language model, in order to avoid repeated operations, the first operator needs to be removed from the second operator to obtain a third operator; the target operation data of the third operator is input into the second sub-model, and the attribute value of the indicator attribute corresponding to the target operator of the target operator is output; for example: the third operator is #2.FILTER: #1 of "branch number is greater than 71", where FILTER is the target operator of the third operator. According to the second mapping relationship, it can be seen that the indicator attribute corresponding to the target operator FILTER is where, and the target operation data "branch number is greater than 71" is converted to obtain the attribute value "branch number>71" corresponding to the indicator attribute where.

[0118] S306, obtaining the indicator attribute corresponding to the target operator of the first operator and the indicator attribute corresponding to the target operator of each third operator from the indicator attributes corresponding to the target operator of each second operator; using the first indicator name as the attribute value of the indicator attribute corresponding to the target operator of the first operator;

[0119] A target indicator is generated according to the indicator attribute corresponding to the target operator of the first operator, the first indicator name, the indicator attributes corresponding to the target operators of each third operator, and the attribute value of the indicator attribute corresponding to the target operator of each third operator.

[0120] In the embodiment of the present application, the second operator includes the first operator and the third operator, so the indicator attribute corresponding to the target operator of the first operator and the indicator attribute corresponding to the target operator of each third operator can be obtained from the indicator attribute corresponding to the target operator of the second operator. In the embodiment of the present application, the first indicator name is the attribute value of the indicator attribute corresponding to the target operator of the first operator.

[0121] In an embodiment of the present application, a target indicator is generated based on the indicator attribute corresponding to the target operator of the first operator, the first indicator name, the indicator attribute corresponding to the target operator of each third operator, and the attribute value of the indicator attribute corresponding to the target operator of each third operator; for example, the target indicator can be: [metricNames: operating expenses_SUM; where: branch number>71].

[0122] Among them, metricNames is the indicator attribute corresponding to the target operator of the first operator, where is the indicator attribute corresponding to the target operator of the third operator; "operating expenses_SUM" is the first indicator name, and "branch number>71" is the attribute value of the indicator attribute where corresponding to the target operator of the third operator.

[0123] Based on the above embodiments, as an optional embodiment, the first sub-model is trained by using the operation data of the sample operator as the training sample and the indicator name corresponding to the operation data of the sample operator as the training label;

[0124] The second sub-model is trained by using the operation data of the sample operator as training samples, and the attribute values ​​of the indicator attributes corresponding to the operation data of the sample operator as training labels.

[0125] In an embodiment of the present application, the large language model includes a first sub-model and a second sub-model, wherein the first sub-model is used to convert the target operation data of the first operator into the first indicator name, and the second sub-model is used to convert the target operation data of the third operator into the attribute value of the indicator attribute corresponding to the target operator of the third operator. For example: the target operation data of the first operator is "sum of operating expenses", "sum of operating expenses" is input into the first sub-model, and the first sub-model outputs the first indicator name "operating expenses_SUM"; the target operation data of the third operator is "branch number greater than 71", "branch number greater than 71" is input into the second sub-model, and the second sub-model outputs the attribute value of the indicator attribute "branch number>71".

[0126] It should be noted that in an embodiment of the present application, the first sub-model and the second sub-model are trained using training samples until convergence. For example, when the loss values ​​of the first sub-model and the second sub-model meet the preset convergence conditions, the iterative training of the first sub-model and the second sub-model is stopped.

[0127] In some embodiments, the preset convergence condition may be: the total number of training rounds reaches a first threshold, or the number of times the calculated loss value of the loss function is continuously lower than the second threshold reaches a third threshold, wherein the values ​​of the first threshold, the second threshold and the third threshold are set according to actual processing needs.

[0128] Based on the above embodiments, as an optional embodiment, the large language model includes a third sub-model;

[0129] Input the information to be queried into the third sub-model to obtain at least one target operator, where the target operator includes a target operator and target operation data, including:

[0130] S401. Input the information to be queried into the third sub-model. The third sub-model decomposes the information to be queried into at least one sub-question, and performs semantic analysis on each sub-question to obtain the grammatical structure and entity of each sub-question, as well as the reference relationship between the sub-questions.

[0131] In an embodiment of the present application, the large language model also includes a third sub-model, wherein the third sub-model is used to decompose the information to be queried input by the user into at least one sub-problem, and output a target operator corresponding to each sub-problem. For example, the information to be queried input by the user is "What is the total operating expenses of branches with numbers greater than 71?", the information to be queried is input into the third sub-model, and two sub-problems related to the information to be queried are output, namely: sub-problem 1 "What is the total operating expenses?" and sub-problem 2 "What is the total operating expenses of branches with numbers greater than 71?".

[0132] It should be noted that the third sub-model needs to be trained before use. The third sub-model is trained with the information to be queried as a training sample and at least one target operator corresponding to the information to be queried as a training label. When the loss value of the third sub-model meets the preset convergence condition, the iterative training of the third sub-model is stopped. The preset convergence condition of the third sub-model is the same as the preset convergence condition of the first sub-model and the second sub-model.

[0133] S402: For each sub-question, the third sub-model determines a target operator corresponding to the sub-question according to the grammatical structure of the sub-question and at least one of the entities.

[0134] In the embodiment of the present application, the third sub-model can perform semantic analysis on each sub-question, obtain the subject, predicate, object, attributive, adverbial, complement and interrogative word of the sub-question, and determine the grammatical structure of the sub-question by analyzing these grammatical components. At the same time, the entity in the sub-question can also be combined to assist in determining the target operator corresponding to the sub-question. The following is an example of sub-question 2 "What is the total operating expenses of branches with a number greater than 71?"

[0135] The subject of sub-problem 2 is "the total operating expenses of the branch", the predicate is "is", the attributive is "branch number is greater than 71", and the complement is "how much". Therefore, the grammatical structure of sub-problem 2 is: subject (total operating expenses) + predicate (is) + attributive (branch number is greater than 71) + complement (how much); at the same time, the trained large language model can also determine the entities of sub-problem 2, where the entities in sub-problem 2 include "total operating expenses" and "branch number is greater than 71"; according to the grammatical structure and entities of sub-problem 2, it can be determined that sub-problem 2 needs to query the total operating expenses of branches with branch numbers greater than 71. Therefore, it can be determined that this sub-problem needs to aggregate the operating expenses of branches with branch numbers greater than 71, and the operator corresponding to "aggregation" is "FILTER", so it can be determined that the target operator corresponding to sub-problem 2 is FILTER.

[0136] S403. For each sub-problem, the third sub-model uses the operator of the sub-problem that has a reference relationship with the sub-problem as the fourth operator based on the reference relationship between the sub-problems, obtains the output data of the fourth operator, and uses at least one of the entity of the sub-problem and the output data of the fourth operator as the target operation data of the operator corresponding to the sub-problem.

[0137] In the embodiment of the present application, there is a reference relationship between sub-problems, for example: the entities of sub-problem 2 "What is the total operating expenses of branch numbers greater than 71?" include "total operating expenses" and "branch number greater than 71", and "total operating expenses" is exactly the output data of sub-problem 1 "What is the total operating expenses?" Therefore, there is a reference relationship between sub-problem 1 and sub-problem 2, and sub-problem 2 depends on the output data of the sub-problem. For sub-problem 2, sub-problem 1 is the fourth operator, so the output data of the target operator corresponding to sub-problem 1 can be obtained. For sub-problem 2, the entity "branch number greater than 71" is not included in sub-problem 1, so "branch number greater than 71" has nothing to do with the output data of the target operator corresponding to sub-problem 1; finally, the entity "branch number greater than 71" of sub-problem 2 and the output data of the target operator corresponding to sub-problem 1 can be used together as the target operation data of the target operator corresponding to sub-problem 2.

[0138] S404: For each sub-problem, the third sub-model outputs a target operator corresponding to the sub-problem in a one-to-one manner according to the target operator and target operation data of the sub-problem.

[0139] In an embodiment of the present application, after obtaining the target operator and target operation data of the sub-problem, the third sub-model can obtain the target operator corresponding to the sub-problem. Taking sub-problem 2 as an example, the target operator corresponding to sub-problem 2 is SELECT, and the target operation data of sub-problem 2 is "branch number is greater than 71" and the output data of the target operator corresponding to sub-problem 1. In the embodiment of the present application, "#" and Arabic numerals are used to represent the output data of the target operator corresponding to the sub-problem. Therefore, the target operator corresponding to sub-problem 2 can be expressed as: #2.FILTER: #1 of "branch number is greater than 71", where "#1" represents the output data of the operator corresponding to sub-problem 1.

[0140] Based on the above embodiments, as an optional embodiment, at least one of the entity of the sub-problem and the output data of the fourth operator is used as the target operation data of the operator corresponding to the sub-problem, including:

[0141] If it is determined that all entities of the subproblem are irrelevant to the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship, the entity of the subproblem is used as the target operation data of the target operator corresponding to the subproblem;

[0142] If it is determined that all entities of the subproblem are related to the output data of the target operator corresponding to the subproblem that has a reference relationship with the subproblem, the output data of the operator is used as the target operation data of the target operator corresponding to the subproblem;

[0143] If it is determined that the first entity of the subproblem is irrelevant to the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship, and the second entity of the subproblem is related to the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship, then the first entity and the output data of the target operator shall be used as the target operation data of the target operator corresponding to the subproblem.

[0144] In an embodiment of the present application, for each subproblem, if the subproblem does not need to use the output results of other subproblems, then all entities in the subproblem are independent of the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship; conversely, if the subproblem needs to use the output results of other subproblems, then the entities in the subproblem are related to the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship. It should be noted that, for the latter case, there may be a part of the entities in the subproblem that are related to the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship, while another part of the entities in the subproblem are independent of the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship. For each subproblem, after determining the relationship between the output data of the target operator corresponding to the subproblem with which the entity in the subproblem has a reference relationship, the target operation data of the target operator corresponding to the subproblem can be determined.

[0145] Based on the above embodiments, as an optional embodiment, it also includes:

[0146] For each target operator, a target operator, target operation data and output data of the target operator are respectively regarded as a node;

[0147] For each target operator, construct a directed edge between a node corresponding to the target operation data of the target operator and a node corresponding to the target operator of the target operator, and construct a directed edge between a node corresponding to the target operator of the target operator and a node corresponding to the output data of the target operator;

[0148] A directed acyclic graph is constructed based on each node and the directed edges between the nodes. The directed acyclic graph is used to represent the relationship between the composite operations between the target operators.

[0149] In the embodiment of the present application, a directed acyclic graph (DAG) is used to display the composite operations between operators. The following is a specific example to introduce how to construct a directed acyclic graph.

[0150] Figure 3 A schematic diagram of a directed acyclic graph provided in an embodiment of the present application is shown as follows: Figure 3 As shown, rectangular nodes are used to represent target operators, and circular nodes are used to represent target operation data and output data. The node "sum of operating expenses" is the target operation data of #1, the node "#1 SELECT" is the target operator of #1, and the node "#1 output data" is used to represent the output data of #1. Therefore, a directed edge from the target operation data of #1 to the target operator, and a directed edge from the target operator of #1 to the output data can be constructed. At the same time, the node "#2 FILTER" is the target operator of #2, the node "#1 output data" is also the target operation data of #2, and another target operation data of #2 is the node "branch number greater than 71", and the node "#2 output data" is used to represent the output data of #2; therefore, a directed edge from the target operation data of #2 to the target operator, and a directed edge from the target operator of #2 to the output data can be constructed. Through the directed edges between the above nodes and nodes, a directed acyclic graph of #1 and #2 can be generated. The directed acyclic graph can clearly show the relationship between the compound operations of #1 and #2.

[0151] It should be noted that, in the embodiment of the present application, in order to distinguish the nodes corresponding to the target operator and the nodes corresponding to the target operator data, different graphics can be used to represent them. For example, a rectangle can be used to represent the node corresponding to the target operator, and a circle can be used to represent the nodes corresponding to the target operation data and the output data.

[0152] It should also be noted that in the embodiment of the present application, the output data of a target operator may be the target operation data of another target operator. In this case, one node is used to represent both the output data and the target operation data, rather than using multiple nodes to represent them separately.

[0153] Based on the above embodiments, as an optional embodiment, the indicator system includes a third mapping relationship between preset indicator attributes and SQL statement keywords;

[0154] Input the target indicator into the preset indicator system to obtain the corresponding SQL statement, including the following steps:

[0155] S501, the indicator system obtains the keyword of the SQL statement corresponding to each indicator attribute in the target indicator according to the third mapping relationship;

[0156] In the embodiment of the present application, the indicator attributes and the keywords of the SQL statement have a corresponding relationship. For example, the indicator attribute "metricNames" corresponds to the keyword "SELECT" of the SQL statement, and the indicator attribute "where" corresponds to the keyword "where" of the SQL statement. Therefore, the indicator attributes can be converted into the corresponding keywords of the SQL statement.

[0157] S502: Use the first indicator name of the target indicator and the attribute value of each indicator attribute as parameters corresponding to the corresponding keyword;

[0158] In an embodiment of the present application, after obtaining the keyword of the SQL statement, the attribute value of the corresponding indicator attribute can be used as the parameter corresponding to the SQL statement keyword, where the parameters corresponding to the keyword include but are not limited to column name, table name, and expression.

[0159] S503: Obtain the SQL statement corresponding to the target indicator according to the keywords of the SQL statement and the parameters corresponding to each keyword.

[0160] After obtaining the keywords and parameters of the SQL statement, the keywords of the SQL statement can be concatenated to obtain the SQL statement. Assuming the target indicator is: [metricNames: operating expenses_SUM; where: branch number>71], the corresponding SQL statement is:

[0161] select Operating expenses_SUM

[0162] from table

[0163] where branch number>71;

[0164] Among them, select, from and where are all keywords of SQL statements, "operating expenses_SUM" is the name of the first indicator, which is the attribute value of the indicator attribute select; "branch number>71" is the attribute value of the indicator attribute where. It should be noted that in the embodiment of the present application, the data in the SQL database are all stored in the data table. When the indicator attribute is "select", it is necessary to specify which table to query the data from. Therefore, the from table statement will be added in the process of converting the target indicator into an SQL statement; among them, "from" is the keyword of the SQL statement, and table is the table name that contains both "operating expenses_SUM" and "branch number". In the embodiment of the present application, the table name that contains both "operating expenses_SUM" and "branch number" can be pre-set or determined by table selection logic. The embodiment of the present application does not limit how to determine the table name.

[0165] The following is a flowchart to introduce the process of converting the query information input by the user into SQL statements in this application.

[0166] Figure 4 A flow chart of converting information to be queried into an SQL statement is provided for an embodiment of the present application; as shown in the figure, the information to be queried is input into a pre-trained large language model, the pre-trained large language model decomposes the information to be queried, obtains at least one sub-problem, and generates a target operator corresponding to each sub-problem, each target operator includes a target operator and target operation data; according to the conversion relationship between the operator and the indicator and the pre-trained large language model, the target operator and the target operation data in each target operator are subjected to indicator parameter extraction to obtain the indicator name, indicator attribute and attribute value of the indicator attribute of the target indicator; according to the indicator name, indicator attribute and attribute value of the indicator attribute, the target indicator is generated; SQL escape is performed on the target indicator, specifically: according to the correspondence between the indicator attribute and the SQL statement keyword, the indicator attribute is converted into the keyword of the corresponding SQL statement, the obtained SQL statement keyword is spliced, and the attribute value of the indicator attribute is used as the parameter of the keyword of the corresponding SQL statement to obtain the corresponding SQL statement, wherein the parameter of the keyword can be at least one of a column name, a table name, and an expression.

[0167] In order to facilitate those skilled in the art to understand the technical solution in the present application, the indicator acquisition method of the present application is introduced below through a complete embodiment.

[0168] S601: Train the large language model to be trained to obtain a pre-trained large language model.

[0169] In an embodiment of the present application, a large language model to be trained is deployed on server 102. The large language model to be trained includes three sub-models, namely a first sub-model, a second sub-model and a third sub-model. The first sub-model, the second sub-model and the third sub-model are respectively trained for multiple rounds of iterations using training samples until convergence to obtain a pre-trained large language model.

[0170] S602: Obtain the information to be queried expressed in natural language input by the user.

[0171] In the embodiment of the present application, the information to be queried input by the user may be Example 0: What is the total operating expenses of branches with a number greater than 71?

[0172] S603: Input the information to be queried into a pre-trained large language model. The pre-trained large language model divides the information to be queried into multiple sub-questions and determines the reference relationship between the sub-questions.

[0173] In the embodiment of the present application, Example 0 can be broken down into the following two sub-problems:

[0174] Sub-question 1: What is the total amount of operating expenses?

[0175] Sub-question 2: What is the total operating expenses for branches with branch numbers greater than 71?

[0176] There is a reference relationship between sub-question 1 and sub-question 2. Sub-question 2 requires the calculation result of "total operating expenses" in sub-question 1.

[0177] S604: The pre-trained large language model converts each sub-problem into a target operator, and each target operator includes a target operator and target operation data.

[0178] Using a pre-trained large language model to transform each sub-problem into a target operator includes the following sub-steps:

[0179] S6041. The pre-trained large language model obtains the grammatical structure and entity of each sub-problem, and determines the target operator corresponding to the sub-problem according to at least one of the grammatical structure and entity of the sub-problem.

[0180] In an embodiment of the present application, a pre-trained large language model performs semantic analysis on each sub-question to obtain the subject, predicate, object, attributive, adverbial, complement and interrogative word of the sub-question, and determines the grammatical structure of the sub-question by analyzing these grammatical components. At the same time, the entities in the sub-question can also be combined to assist in determining the target operator corresponding to the sub-question.

[0181] For example: the subject of sub-problem 1 is "total operating expenses", the predicate is "is", and the complement is "how much", so the grammatical structure of sub-problem 1 is: subject (total operating expenses) + predicate (is) + complement (how much); at the same time, the pre-trained large language model can also determine that the entity in sub-problem 1 is "total operating expenses"; according to the grammatical structure and entity of sub-problem 1, it can be determined that sub-problem 1 needs to query the total operating expenses, and the operator corresponding to "query" is "SELECT", so it can be determined that the target operator corresponding to sub-problem 1 is SELECT.

[0182] The subject of sub-problem 2 is "the total operating expenses of the branch", the predicate is "is", the attributive is "branch number is greater than 71", and the complement is "how much". Therefore, the grammatical structure of sub-problem 2 is: subject (total operating expenses) + predicate (is) + attributive (branch number is greater than 71) + complement (how much); at the same time, the trained large language model can also determine the entities of sub-problem 2, where the entities in sub-problem 2 include "total operating expenses" and "branch number is greater than 71"; according to the grammatical structure and entities of sub-problem 2, it can be determined that sub-problem 2 needs to query the total operating expenses of branches with branch numbers greater than 71. Therefore, it can be determined that this sub-problem needs to aggregate the operating expenses of branches with branch numbers greater than 71, and the operator corresponding to "aggregation" is "FILTER", so it can be determined that the target operator corresponding to sub-problem 2 is FILTER.

[0183] S6042. Determine the target operation data of the target operator corresponding to each sub-problem according to the reference relationship between the sub-problems.

[0184] In the embodiment of the present application, sub-problem 1 is "What is the total operating expenses?", and the entity "total operating expenses" of sub-problem 1 is irrelevant to the output data of the operator corresponding to sub-problem 2; therefore, the entity of sub-problem 1 can be used as the target operation data of the target operator corresponding to sub-problem 1.

[0185] The entities of sub-problem 2 are "sum of operating expenses" and "branch number greater than 71". Obviously, the "sum of operating expenses" in sub-problem 2 is related to the output data of the target operator corresponding to sub-problem 1, and the "branch number greater than 71" in sub-problem 2 is irrelevant to the output data of the target operator corresponding to sub-problem 1. Therefore, the operation data of sub-problem 2 is the output data of the target operator corresponding to "branch number greater than 71" and sub-problem 1.

[0186] S6043. After determining the target operator and target operation data corresponding to each sub-problem, the target operator corresponding to the sub-problem may be generated.

[0187] The target operator corresponding to sub-problem 1 is "SELECT", and the target operation data is "sum of operating expenses". Therefore, the target operator corresponding to sub-problem 1 can be expressed as:

[0188] #1.SELECT: "Total operating expenses".

[0189] The target operator of subproblem 2 is "FILTER", and the target operation data are "sum of operating expenses" and "branch number greater than 71", and "sum of operating expenses" is the output data of the target operator corresponding to subproblem 1. Therefore, the target operator corresponding to subproblem 2 can be expressed as:

[0190] #2. FILTER: #1 of "branch number greater than 71".

[0191] It should be noted that, in this application, “#” and Arabic numerals are used to represent a target operator. In addition, “#” and Arabic numerals can also represent output data of the target operator.

[0192] S6044. Generate a directed acyclic graph of #1 and #2.

[0193] In the embodiment of the present application, the node "sum of operating expenses" is the target operation data of #1, the node "#1 SELECT" is the target operator of #1, the node "#1 output data" is used to represent the output data of #1, and is also one of the target operation data of #2. Therefore, a directed edge from the target operation data of #1 to the target operator, and a directed edge from the target operator of #1 to the output data can be constructed. At the same time, the node "#2 FILTER" is the target operator of #2, the node "#1 output data" is also the target operation data of #2, and another target operation data of #2 is the node "branch number greater than 71", and the node "#2 output data" is used to represent the output data of #2; therefore, a directed edge from the target operation data of #2 to the target operator, and a directed edge from the target operator of #2 to the output data can be constructed; through the directed edges between the above nodes and nodes, a directed acyclic graph of #1 and #2 can be generated.

[0194] S605. According to the conversion relationship between the operator and the indicator, the target operator and the target operation data in each target operator are combined to obtain the target indicator output by the large language model.

[0195] In the embodiment of the present application, obtaining the target indicator output by the large language model includes the following sub-steps:

[0196] S6051. Take the target operator including the first operator as the first operator, and use a pre-trained large language model to convert the operation data of the first operator into a first indicator name.

[0197] In the present application, the first operator includes "SELECT" and "PROJECT". In the target operator corresponding to Example 0, #1 contains the first operator "SELECT", so #1 is the first operator; the target operation data of the first operator is "total operating expenses", and the "total operating expenses" is input into the first sub-model to obtain the first indicator name "operating expenses_SUM".

[0198] S6052. Determine a first indicator type corresponding to the first indicator name according to a first mapping relationship between the indicator name and the indicator type.

[0199] In the embodiment of the present application, the first indicator type corresponding to the first indicator name "operating expenses_SUM" is a number-checking type.

[0200] S6053. Obtain indicator attributes preset for the first indicator type as candidate indicator attributes.

[0201] In the embodiment of the present application, the first indicator type is a count type, and the preset indicator attributes of the count type include but are not limited to: metricNames, where, filter, groupBys, as candidate indicator attributes of the count type.

[0202] S6054. Determine the second operator and the third operator according to the second mapping relationship between the indicator attribute and the operator, and obtain the indicator attribute corresponding to the target operator of each second operator and the attribute value of the indicator attribute corresponding to each third operator.

[0203] In the embodiment of the present application, the operators corresponding to each candidate indicator attribute can be known from Table 2. According to Table 2, the operators corresponding to metricNames are SELECT and PROJECT, the operators corresponding to where are GROUP, the operators corresponding to filter are FILTER, PROJECT and INTERSECTION, and the operators corresponding to groupBys are SORT and TOP. The operators corresponding to each candidate indicator attribute are used as the second operator. After obtaining the second operator, the second operator is used to match the #1 target operator, and the second operator is used to match the #2 target operator respectively, and both #1 and #2 are successfully matched; therefore, #1 and #2 are used as the second operator; according to the second mapping relationship between the indicator attribute and the operator, the indicator attributes corresponding to the target operators of #1 and #2 are divided into metricNames and where.

[0204] Since #1 is the first operator, the only target operator in the second operator that is different from the first operator is #2, so #2 is used as the third operator. The target operation data of #2 is input into the second sub-model, and the attribute value of the indicator attribute corresponding to the target operation data of #2 is obtained as "branch number>71".

[0205] S6055. Generate a target indicator according to the indicator attribute corresponding to the target operator of the first operator, the first indicator name, the indicator attributes of each third operator, and the attribute values ​​of the indicator attributes of each third operator.

[0206] In the embodiment of the present application, metricNames is the indicator attribute corresponding to the target operator "SELECT" in #1, "operating expense_SUM" is the first indicator name; where is the indicator attribute corresponding to the target operator "FILTER" in #2, and "branch number>71" is the attribute value of the indicator attribute corresponding to the target operator "FILTER" in #2. Therefore, the target indicator corresponding to Example 0 is as follows: [metricNames: operating expense_SUM; where: branch number>71].

[0207] It should be noted that, in the embodiment of the present application, the first indicator name can be regarded as an attribute value of a special indicator attribute.

[0208] S606: Use the indicator system to translate the generated target indicator into an SQL statement, and use the SQL statement to obtain the query result from the database as the answer to the information to be queried.

[0209] In the embodiment of the present application, the indicator attribute and the keyword of the SQL statement have a corresponding relationship. The indicator attribute "metricNames" corresponds to the keyword "SELECT" of the SQL statement, and the indicator attribute "where" corresponds to the keyword "where" of the SQL statement. Therefore, the indicator attribute can be converted into the corresponding SQL statement keyword, and then the SQL statement keywords are concatenated to obtain the SQL statement, wherein the attribute value of the indicator attribute is the parameter corresponding to the SQL statement keyword, and the parameter corresponding to the parameter keyword includes but is not limited to the column name, table name, and expression; therefore, the SQL statement corresponding to Example 0 can be obtained:

[0210] select Operating expenses_SUM

[0211] from table

[0212] where branch number>71;

[0213] It should be noted that in the embodiment of the present application, the data in the SQL database are all stored in the data table. When the indicator attribute is "select", it is necessary to specify which table to query the data from. Therefore, a from table statement will be added in the process of converting the target indicator into an SQL statement; wherein, "from" is the keyword of the SQL statement, and table is the table name that includes both "operating expenses_SUM" and "branch number". In the embodiment of the present application, the table name that includes both "operating expenses_SUM" and "branch number" can be pre-set or determined by table selection logic. The embodiment of the present application does not limit how to determine the table name.

[0214] Figure 5A schematic diagram of a structure of an indicator acquisition device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the indicator acquisition device 1000 includes: a first acquisition module 1001, a second acquisition module 1002 and a third acquisition module 1003.

[0215] A first acquisition module is used to acquire the information to be queried expressed in natural language;

[0216] The second acquisition module is used to input the query information into the pre-trained large language model to obtain the target index output by the large language model;

[0217] The third acquisition module is used to input the target indicator into the preset indicator system, obtain the corresponding SQL statement, and call the SQL statement to obtain the query result from the database;

[0218] The information to be queried is input into a pre-trained large language model to obtain target indicators output by the pre-trained large language model, including:

[0219] Inputting the information to be queried into a pre-trained large language model to obtain at least one target operator, where the target operator includes a target operator and target operation data, where the target operation data is an entity in the information to be queried;

[0220] According to the conversion relationship between operators and indicators, the target operator and target operation data in each target operator are combined to obtain the target indicator output by the large language model.

[0221] The indicator acquisition device of the embodiment of the present application can execute the indicator acquisition method provided by the embodiment of the present application, and the implementation principle is similar. The actions performed by each module in the indicator acquisition device of each embodiment of the present application correspond to the steps in the indicator acquisition method of each embodiment of the present application. For the detailed functional description of each module of the indicator acquisition device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0222] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the electronic device 4000 includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0223] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0224] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0225] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0226] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0227] Among them, the electronic equipment package may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0228] The embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented. Compared with the prior art, the following can be implemented:

[0229] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0230] The present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the above method embodiments when executed by a processor. Compared with the prior art, it can achieve:

[0231] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0232] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0233] The above are only optional implementation methods for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the scheme of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A method for obtaining indicators, It is characterized in that include: Obtaining the information to be queried expressed in natural language; Use a pre-trained large language model to decompose the query information into multiple sub-questions, and obtain the grammatical structure and entity of each sub-question; According to at least one of the grammatical structure and the entity, determining a target operator corresponding to each sub-problem from preset operators; Taking the entity as target operation data corresponding to the sub-problem; According to the target operator and target operation data corresponding to each sub-problem, determine the target operator corresponding to each sub-problem; according to the conversion relationship between the operator and the indicator and the pre-trained large language model, perform indicator parameter extraction on the target operator and target operation data in each target operator, obtain the indicator name, indicator attribute and attribute value of the target indicator, and generate the target indicator according to the indicator name, indicator attribute and attribute value of the indicator attribute; The target indicator is input into a preset indicator system, wherein the indicator system includes a third mapping relationship between preset indicator attributes and SQL statement keywords, and the indicator system obtains the keywords of the SQL statements corresponding to each indicator attribute in the target indicator according to the third mapping relationship; The first indicator name of the target indicator and the attribute values ​​of each indicator attribute are used as parameters corresponding to the corresponding keywords; according to the keywords of the SQL statement and the parameters corresponding to each keyword, the SQL statement corresponding to the target indicator is obtained, and the SQL statement is called to obtain the query result from the database.

2. The method according to claim 1, It is characterized in that The conversion relationship between the operator and the indicator includes a first mapping relationship between the indicator name and the indicator type, and a second mapping relationship between the indicator attribute and the operator; the large language model includes a first sub-model and a second sub-model; According to the conversion relationship between the operator and the indicator and the pre-trained large language model, the target operator and the target operation data in each target operator are subjected to indicator parameter extraction, the indicator name, indicator attribute and attribute value of the target indicator are obtained, and the target indicator is generated according to the indicator name, indicator attribute and attribute value of the indicator attribute, including: Taking each operator including the preset first operator among all target operators as the first operator; Inputting the target operation data of the first operator into the first sub-model, and obtaining a first indicator name corresponding to the target operation data of the first operator output by the first sub-model; According to the first mapping relationship, obtain a first indicator type corresponding to the first indicator name; Obtaining the indicator attribute preset for the first indicator type as a candidate indicator attribute; According to the second mapping relationship, an operator corresponding to each candidate indicator attribute is obtained as the second operator, each second operator is matched with the target operator of each target operator, and the successfully matched target operator is used as the second operator; according to the second mapping relationship, the indicator attribute corresponding to the target operator of each second operator is obtained from the candidate indicator attribute; Using an operator different from the first operator among the second operators as a third operator, inputting target operation data of each third operator into the second sub-model, and obtaining attribute values ​​of indicator attributes corresponding to target operators of each third operator output by the second sub-model; Obtaining the indicator attribute corresponding to the target operator of the first operator and the indicator attribute corresponding to the target operator of each third operator from the indicator attributes corresponding to the target operator of each second operator; using the first indicator name as the attribute value of the indicator attribute corresponding to the target operator of the first operator; The target indicator is generated according to the indicator attribute corresponding to the target operator of the first operator, the first indicator name, the indicator attributes corresponding to the target operators of each third operator, and the attribute value of the indicator attribute corresponding to the target operator of each third operator.

3. The method according to claim 2, It is characterized in that The first sub-model is trained by using the operation data of the sample operator as training samples and the indicator names corresponding to the operation data of the sample operator as training labels; The second sub-model is trained by using the operation data of the sample operator as training samples and the attribute values ​​of the indicator attributes corresponding to the operation data of the sample operator as training labels.

4. The method according to claim 1, It is characterized in that The large language model includes a third sub-model; The step of inputting the information to be queried into the third sub-model to obtain at least one target operator, wherein the target operator includes a target operator and target operation data, comprises: Inputting the information to be queried into the third sub-model, the third sub-model decomposes the information to be queried into at least one sub-question, and performs semantic analysis on each sub-question to obtain the grammatical structure and entity of each sub-question, as well as the reference relationship between the sub-questions; For each sub-problem, the third sub-model determines a target operator corresponding to the sub-problem according to a grammatical structure and at least one of an entity of the sub-problem; For each sub-problem, the third sub-model uses the target operator of the sub-problem that has a reference relationship with the sub-problem as the fourth operator according to the reference relationship between the sub-problems, obtains the output data of the fourth operator, and uses at least one of the entity of the sub-problem and the output data of the fourth operator as the target operation data of the target operator corresponding to the sub-problem; For each sub-problem, the third sub-model outputs a target operator corresponding to the sub-problem in a one-to-one manner according to the target operator and target operation data of the sub-problem.

5. The method according to claim 4, It is characterized in that The using at least one of the entity of the sub-problem and the output data of the fourth operator as the target operation data of the target operator corresponding to the sub-problem includes: If it is determined that all entities of the subproblem are irrelevant to the output data of the target operator corresponding to the subproblem that has a reference relationship with the subproblem, the entity of the subproblem is used as the target operation data of the target operator corresponding to the subproblem; If it is determined that all entities of the subproblem are related to the output data of the target operator corresponding to the subproblem that has a reference relationship with the subproblem, the output data of the operator is used as the target operation data of the target operator corresponding to the subproblem; If it is determined that the first entity of the subproblem is independent of the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship, and that the second entity of the subproblem is related to the output data of the target operator corresponding to the subproblem with which the subproblem has a reference relationship, then the first entity and the output data of the target operator shall be used as the target operation data of the target operator corresponding to the subproblem.

6. The method according to claim 5, It is characterized in that Also includes: For each target operator, a target operator, target operation data and output data of the target operator are respectively regarded as a node; For each target operator, construct a directed edge between a node corresponding to the target operation data of the target operator and a node corresponding to the target operator of the target operator, and construct a directed edge between a node corresponding to the target operator of the target operator and a node corresponding to the output data of the target operator; A directed acyclic graph is constructed according to each node and the directed edges between each node. The directed acyclic graph is used to characterize the relationship between the composite operations between the target operators.

7. An indicator acquisition device, It is characterized in that include: A first acquisition module is used to acquire the information to be queried expressed in natural language; The second acquisition module is used to decompose the query information into multiple sub-questions using a pre-trained large language model, and obtain the grammatical structure and entity of each sub-question; according to at least one of the grammatical structure and entity, determine the target operator corresponding to each sub-question from pre-set operators; Taking the entity as target operation data corresponding to the sub-problem; According to the target operator and target operation data corresponding to each sub-problem, determine the target operator corresponding to each sub-problem; according to the conversion relationship between the operator and the indicator and the pre-trained large language model, perform indicator parameter extraction on the target operator and target operation data in each target operator, obtain the indicator name, indicator attribute and attribute value of the target indicator, and generate the target indicator according to the indicator name, indicator attribute and attribute value of the indicator attribute; A third acquisition module, used to input the target indicator into a preset indicator system, the indicator system includes a third mapping relationship between preset indicator attributes and SQL statement keywords, and the indicator system obtains the keywords of the SQL statement corresponding to each indicator attribute in the target indicator according to the third mapping relationship; The first indicator name of the target indicator and the attribute values ​​of each indicator attribute are used as parameters corresponding to the corresponding keywords; according to the keywords of the SQL statement and the parameters corresponding to each keyword, the SQL statement corresponding to the target indicator is obtained, and the SQL statement is called to obtain the query result from the database.

8. An electronic device comprising a memory, a processor and a computer program stored in the memory, It is characterized in that The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images