Query method and device based on large language model, equipment and medium

Abstract

The invention discloses a query method and device based on a large language model, equipment and a medium. The method comprises the steps that a first index query intention of index query is obtained through the large language model; obtaining a first index result of index query through a large language model according to the first index query intention; obtaining a second index result of index query through a large language model according to the first index query intention and the constructed decomposition example data set; and verifying the first index result based on the second index result to obtain a final index result of the index query. According to the method, the first query result of the index query is determined according to the index query intention of the index query, then the second index result of the index query is obtained according to the priori knowledge provided by the reference queries, and the first index result is verified through the second index result; the problem of query errors or failure caused by non-professional terms contained in index query is effectively avoided, and the accuracy of index query is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a query method and device based on a large language model, equipment and medium. BACKGROUND

[0002] Industry economic data is an important reference for enterprise development decision-making. Comprehensive and accurate industry economic data can help enterprises seize the initiative in decision-making and market competition, and plays a crucial role in the development of enterprises. At present, when querying economic indicator data, a rule engine is generally used for operation. However, since users are usually not professionals in the economic field, the query information they input often contains non-standard terms, making it difficult for the rule engine to accurately identify the information the user needs to query, thereby causing query result errors or query failures.

[0003] Therefore, the prior art still needs to be improved and improved. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a query method, device, equipment and medium based on a large language model in view of the deficiencies of the prior art.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a query method based on a large language model, wherein the query method based on a large language model specifically includes:

[0006] In response to a user interaction indicator query, a first indicator query intent of the indicator query is obtained through a large language model;

[0007] According to the first indicator query intent, a first indicator result of the indicator query is obtained through a large language model;

[0008] According to the first indicator query intent and the constructed decomposition example data set, a second indicator result of the indicator query is obtained through a large language model;

[0009] Based on the second indicator result, the first indicator result is verified to obtain the final indicator result of the indicator query.

[0010] The query method based on a large language model, wherein the first indicator result of the indicator query is obtained through a large language model according to the first indicator query intent, specifically includes:

[0011] According to the first indicator query intent, the indicator query is decomposed into a structured indicator representation through a large language model, and the structured indicator representation is used to reflect the calculation content of the indicator query;

[0012] According to the structured index representation and a preset index query database, a query result of the index query is calculated to obtain a first index result.

[0013] The query method based on the large language model, wherein the second index result of the index query is obtained by the large language model according to the first index query intention and the constructed decomposition example dataset, and specifically includes:

[0014] In the constructed decomposition example dataset, a plurality of reference queries are selected for the index query.

[0015] According to the plurality of reference queries, the index query is decomposed into a structured index representation by the large language model according to the first index query intention, and the structured index representation is used to reflect the calculation content of the index query.

[0016] According to the structured index representation and a preset index query database, a query result of the index query is calculated to obtain a first index result.

[0017] The query method based on the large language model, wherein the index query is decomposed into a structured index representation by the large language model according to the first index query intention, and specifically includes:

[0018] When the first index query intention is a first index query, a semantic similarity between the index query and each formula in a preset index calculation formula library is calculated, and a preset number of candidate index calculation formulas are selected according to the semantic similarity.

[0019] The index query is decomposed into a first structured index representation by the large language model according to the preset number of candidate index calculation formulas, so as to obtain the structured index representation of the first index query.

[0020] The query method based on the large language model, wherein the query result of the index query is determined by the large language model according to the structured index representation and a preset index query database, and specifically includes:

[0021] The second structured index representation required by the first structured index representation is obtained by the large language model, and the second structured index representation reflects a structured index representation of a second index query.

[0022] The intermediate index result is obtained by the large language model according to the second structured index representation from the preset index query database.

[0023] The index result of the first index query is calculated according to the intermediate index result.

[0024] The method for querying based on a large language model, wherein the querying of the first index query intention through the large language model comprises specifically:

[0025] When the first index query intention is a second index query, the index query is decomposed into a second structured index representation through the large language model;

[0026] The second structured index representation is retrieved from a preset index query database through the large language model to obtain a structured index representation of the second index query.

[0027] The method for querying based on a large language model, wherein the first index result is verified based on the second index result to obtain a final index result of the index query, and the verification comprises specifically:

[0028] The second index result is compared with the first index result;

[0029] If the second index result is the same as the first index result, the first index result is taken as the final index result of the index query;

[0030] If the second index result is not the same as the first index result, the steps of obtaining the first index query intention of the index query through the large language model and obtaining the first index result of the index query through the large language model according to the first index query intention are re-executed.

[0031] The method for querying based on a large language model, wherein before the index query in response to the user interaction, the method further comprises:

[0032] An initial index query data set is constructed according to data tables in a plurality of historical reports, wherein the initial index query data set comprises a plurality of index query data binary tuples, and each index query data binary tuple comprises a historical index query and a historical query result;

[0033] A second index query intention of the historical index query is obtained through the large language model;

[0034] A third index result of the index query is obtained through the large language model according to the second index query intention;

[0035] The third index result is verified through the historical query result;

[0036] If the verification is passed, a decomposition example data set is constructed for the historical index query according to the third index result.

[0037] The query method based on the large language model, wherein the constructing a decomposition example dataset for the historical index query according to the third index result specifically comprises:

[0038] obtaining a corresponding correct data item in the third index result;

[0039] taking parameters of the correct data item as a decomposition example data to obtain a decomposition example dataset, wherein the parameters of the correct data item include a historical index query, a second structured index representation required by the historical index query, and an index calculation formula.

[0040] The query method based on the large language model, wherein the verifying the third index result through the historical query result further comprises:

[0041] if the verification fails, locating a corresponding error data item in the third index result and labeling the error data item;

[0042] taking parameters of the error data item as a fine-tuning data group to obtain a fine-tuning dataset, wherein the parameters of the error data item include a task content, an error result, an error reason, and a labeling result;

[0043] fine-tuning the large language model by using the fine-tuning dataset.

[0044] The second aspect of the present application provides a query device based on a large language model, wherein the query device based on the large language model specifically comprises:

[0045] an acquisition module, configured to acquire a first index query intention of an index query of a user interaction through a large language model;

[0046] a first query module, configured to acquire a first index result of the index query through the large language model according to the first index query intention;

[0047] a second query module, configured to acquire a second index result of the index query through the large language model according to the first index query intention and a constructed decomposition example dataset;

[0048] a verification module, configured to verify the first index result based on the second index result to obtain a final index result of the index query.

[0049] The third aspect of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the query method based on the large language model.

[0050] The fourth aspect of the present application provides a terminal device, comprising a processor and a memory;

[0051] The memory has stored thereon a computer readable program executable by the processor;

[0052] The processor implements the steps in the query method based on a large language model according to any one of the above when executing the computer readable program.

[0053] Advantages:

[0054] 1. The present application decomposes the index query into a structured index representation according to the index query intention, clearly defines the accurate path of the index query, and performs cross-level query on the index query according to the path, effectively avoiding query errors or failures caused by the inclusion of non-professional terms in the index query, thereby significantly improving the accuracy of the index query.

[0055] 2. The present application constructs a structured index representation of the first index query (composite index query) and a structured index representation of the second index query (atomic index query), and automatically switches the structured index representation according to the index query intention, realizes the three-level linkage of the index query intention, the first index query and the second index query, avoids the misjudgment risk of a single structured index representation, and further improves the accuracy of the index query.

[0056] 3. The present application constructs an automatic closed loop of formula retrieval, dynamic binding of the second structured index representation and function calculation for the first index query, breaks through the limitations of manual configuration of formulas, supports adaptive processing of complex nested calculation scenarios, and significantly improves the calculation efficiency and accuracy of complex indexes.

[0057] 4. The present application uses a large model to automatically construct a data table in a historical report into an initial index query data set, uses the initial index query data set to verify the query process to determine an error data set, and uses the error data set to fine-tune the large language model, optimizes the bias link in the large language model, and improves the model performance of the large language model.

[0058] 5. The present application uses the initial index query data set to construct a decomposition example data set, and uses the constructed decomposition example data set to verify the index result determined by the large language model, which on the one hand effectively restricts the logical bias caused by the free play of the model, and on the other hand solves the implicit calculation errors caused by the missing of the formula library or semantic ambiguity. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0060] Figure 1 An application scenario of the query method based on a large language model provided by the embodiments of the present application.

[0061] Figure 2 A flowchart of the query method based on a large language model provided by the embodiments of the present application.

[0062] Figure 3 A flowchart of the acquisition process of the first index result.

[0063] Figure 4 A flowchart of the acquisition process of the first structured index representation.

[0064] Figure 5 A flowchart of the acquisition process of the second index result.

[0065] Figure 6 A principle block diagram of the query device based on a large language model provided by the embodiments of the present application.

[0066] Figure 7 A principle block diagram of the terminal device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0067] The embodiments of the present application provide a query method, device, equipment and medium based on a large language model. In order to make the purpose, technical solutions and effects of the present application more clear and explicit, the present application will be further described in detail with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0068] It is to be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. It is further understood that the terms "comprise" (or comprise), "comprises" (or comprises) and "comprising" (or comprising) when used in this specification, specify the presence of stated features, integers, steps, operations, elements, or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof. It is further understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements can be present. In addition, the word "connected" or "coupled" as used herein can include wirelessly connected or wirelessly coupled. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0069] It is to be understood that the terms so used are intended to encompass the meanings commonly associated with the terms, as well as the meanings that are considered to be fresh in the art. It is further understood that the terms should not be interpreted in an overly literal or overly formal sense unless otherwise specifically so defined herein.

[0070] It is to be understood that the sequence of the steps in the embodiments and the size of the steps do not mean the order of execution, and the execution order of the processes is determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0071] It is found through research that industry economic data is an important reference for enterprise development decision-making. Comprehensive and accurate industry economic data can help enterprises seize the initiative in decision-making and market competition, and plays a crucial role in the development of enterprises. At present, when querying economic indicator data, a rule engine is generally used for operation. However, since users are usually not professionals in the economic field, the query information input by them often contains non-standard terms, which makes it difficult for the rule engine to accurately identify the information required by the user to query, and further causes the query result to be incorrect or the query to fail.

[0072] To solve the above problems, in the embodiment of the application, in response to a user interactive index query, a first index query intention of the index query is obtained through a large language model; a first index result of the index query is obtained through the large language model according to the first index query intention; a second index result of the index query is obtained through the large language model according to the first index query intention and a constructed decomposition example dataset; and the first index result is verified based on the second index result to obtain a final index result of the index query. The application determines the first query result of the index query according to the index query intention of the index query, then obtains the second index result of the index query according to the prior knowledge provided by the reference query, and verifies the first index result through the second index result, effectively avoiding the query error or failure problem caused by the non-professional terms in the index query, and improving the accuracy of the index query.

[0073] An application environment diagram of the query method based on the large language model provided by the embodiment of the application can be as shown in Figure 1 Referring to Figure 1 , the query system based on the large language model includes a user terminal 110 and a server 120. The user terminal 110 and the server 120 are connected through a network. The user terminal 110 can be a desktop user terminal 110 or a mobile user terminal 110, and the mobile user terminal 110 can be at least one of a mobile phone, a tablet computer, a notebook computer, etc. The server 120 can be implemented by an independent server 120 or a server 120 cluster composed of multiple servers 120. Among them, the user terminal 110 receives the index query input by the user. The server 120 obtains the index query from the user terminal 110. The server 120 responds to the user interactive index query, obtains the first index query intention of the index query through the large language model; obtains the first index result of the index query through the large language model according to the first index query intention; obtains the second index result of the index query through the large language model according to the first index query intention and the constructed decomposition example dataset; and verifies the first index result based on the second index result to obtain the final index result of the index query.

[0074] The application content will be further described through the description of the embodiments in combination with the drawings.

[0075] The embodiment of the application provides a query method based on a large language model, as shown in Figure 2 The query method based on the large language model specifically includes:

[0076] S10, in response to a user interactive index query, a first index query intention of the index query is obtained through a large language model.

[0077] Specifically, the index query is used to query an index result of an index to be queried corresponding to the index query. For example, the index query is "the number of points of manufacturing industry in XXX region in XXX year", and the index to be queried corresponding to the index query is the number of points, and the index result of the index query is the index result of the number of points. The index query can be an economic index query, a business index query, etc.

[0078] The first index query intention is used to reflect the index type of the index to be queried corresponding to the index query. The first index query intention can include a first index query and a second index query. The first index query can query a composite index that needs to be derived by using a second index (atomic index) to obtain a query result, such as a number of points, a contribution rate, a proportion, a growth rate, etc. The composite index needs to use the index result of the atomic index to calculate the index result. The second index query can query an atomic index that cannot be decomposed, such as GDP, industrial output value, etc. The atomic index can be a basic index that can directly query the index result from a preset index query database. The first index query intention can be obtained by using a large language model. Specifically, when responding to the index query of the user interaction, the index query can be input into the large language model, and the index query intention classification of the large language model is performed on the index query to obtain the first index query intention of the index query. When the large language model is used for index query intention classification, an intention classification prompt word can be constructed for the large language model. The intention classification prompt word can include an index query and a task requirement. The task requirement is used to inform the large language model of the task to be performed.

[0079] For example, the intention classification prompt word is:

[0080] "Index query: {index query}

[0081] Task requirement: As an economic analyst, please judge which of the following index queries the index query belongs to: 1. First index query (such as proportion, number of points, growth rate, etc.); 2. Second index query (such as GDP, industrial output value, etc.); 3. Other cases (non-data query)".

[0082] Embodiments of the present application classify the index query by using a large language model to obtain the first index query intention of the index query, so as to obtain the query result of the index query based on the first index query intention, and improve the accuracy of the query result of the index query.

[0083] S20, obtaining a first index result of the index query by using a large language model according to the first index query intention.

[0084] Specifically, the first index result is obtained by the large language model based on the first index query intention. When obtaining the first index result, the large language model will first convert the index query into a structured index representation, and then calculate the first index result of the index query according to the structured index representation.

[0085] As shown in an example, Figure 3 According to the first index query intention, the large language model obtains the first index result of the index query, specifically including:

[0086] S21, according to the first index query intention, the large language model decomposes the index query into a structured index representation;

[0087] S22, according to the structured index representation and the preset index query database, the query result of the index query is calculated to obtain the first index result.

[0088] In step S21, the structured index representation is used to reflect the calculation content of the index query, and the index result of the index query can be obtained according to the calculation content. Since the first index query intention includes the first index query for querying the composite index and the second index query for querying the atomic index, the atomic index can directly obtain the index result from the preset index query database, and the composite index needs to be decomposed into atomic indexes first, and then the index result of the atomic index is used to calculate to obtain the final index result. Thus, the structured index representation can be constructed for the first index query and the second index query respectively, wherein the structured index representation constructed for the first index query is denoted as the first structured index representation, and the structured index representation constructed for the second index query is denoted as the second structured index representation. In this way, when the index query is decomposed into a structured index representation according to the first index query intention, if the first index query intention is the first index query, the index query is decomposed into the first structured index representation, and if the first index query intention is the second index query, the index query is decomposed into the second structured index representation. In this way, the structured index representation can be automatically switched according to the first index query intention, realizing the three-level linkage of the first index query intention identification, the first index query and the second index query, avoiding the misjudgment risk of single calculation mode, and further improving the accuracy of the index query.

[0089] In one embodiment, as shown in an example, Figure 4 According to the first index query intention, the large language model decomposes the index query into a structured index representation, specifically including:

[0090] S211. When the first index query intention is a first index query, calculate semantic similarity of the index query and each formula in the preset index calculation formula library, and select a preset number of candidate index calculation formulas according to the semantic similarity;

[0091] S212. Decompose the index query into a first structured index representation according to the preset number of candidate index calculation formulas by using a large language model, to obtain a structured index representation of the first index query.

[0092] Specifically, the first structured index representation includes a second structured index representation and a composite index function required for calculating the first index query, wherein the composite index function is a calculation function for calculating the index result of the first index query, and the calculation function can include an index calculation formula for calculating the first index query using the second structured index representation and a code calculation function (such as a Python function).

[0093] The preset index calculation formula library is pre-constructed and includes a plurality of index calculation formulas of the first index query, by which the index result of the first index query can be calculated. For example, the first index query is "XXX year XXX region manufacturing industry to the industrial pull point number", and the index calculation formula of the first index query is "pull point number = (industry value added this year - industry value added last year) / industry value added last year * 100".

[0094] The semantic similarity can be semantic similarity of words included in the index query and words included in the index calculation formula, or can be coincidence degree of a second index included in the index query and a second index used in the index calculation formula, etc. The preset number can be pre-set to limit the number of candidate index calculation formulas selected from the preset index calculation formula library, and the preset number of candidate index calculation formulas are selected in descending order of semantic similarity. The embodiments of the present application automatically select candidate index calculation formulas according to semantic similarity, solve the low efficiency problem of formula matching depending on manual experience in the traditional method, and improve the query efficiency of the index query.

[0095] Further, after obtaining the preset number of candidate index calculation formulas, the preset number of candidate index calculation formulas are used as prior knowledge of the large language model, so that the large language model determines the second structured index representation and the composite index function required for the index query according to the preset number of candidate index calculation formulas, to obtain the first structured index representation of the index query. Through the large language model, the first index query is accurately disassembled into the second structured index representation and the composite index function required for calculating the first index query according to the first index query intention, ensuring that the calculation logic is strictly aligned with the business rules, and the second structured index representation can be automatically called through the composite index function, forming a closed-loop process of index calculation formula acquisition of the first index query-index result acquisition (second structured index representation) of the second index query-dynamic calculation of the index result of the first index query, which can further improve the query efficiency of the index query.

[0096] When determining the first structured index representation through the large language model, the first structured index representation prompt word is constructed for the large language model, which includes the index query, the preset number of candidate index calculation formulas, and the task requirements. In addition, in actual application, the first structured index representation prompt word can also include several example analyses to provide analysis prior knowledge through example analyses.

[0097] For example, the first structured index representation prompt word is:

[0098] Index query: {index query};

[0099] Candidate index calculation formula: {preset number of candidate index calculation formulas};

[0100] Task requirements: As an economic formula analysis expert, please determine the first structured index representation according to the following rules:

[0101] 1. Identify the composite index (such as the proportion, growth rate, pull point number, contribution rate, etc.) and the index calculation formula of the composite index;

[0102] 2. Clearly indicate the second index required for the index calculation formula (use the index name in the preset index query database), the parameters required for the index calculation formula;

[0103] 3. Output the index calculation formula, including the expression of the index calculation formula (including mathematical symbols and variable explanations);

[0104] 4. Output the Python code function.

[0105] Example analysis:

[0106] Input: "XXX year XXX region manufacturing industry pull point number of industry";

[0107] Output:

[0108] {

[0109] "Atomic indicators": ["XXX-year manufacturing value-added in XXX region", "XXX-year manufacturing value-added in XXX region", "XXX-year industrial value-added in XXX region"];

[0110] "Indicator calculation formula": "Pull point = (industry value-added this year - industry value-added last year) / industry value-added last year * 100";

[0111] "Python function": "XXXX"

[0112] }

[0113] Please output the parsing result in strict JSON format. If the query does not involve composite indicators, return an empty object.

[0114] The embodiments of the present application retrieve a preset number of candidate indicator calculation formulas from the preset indicator calculation formula library according to semantic similarity, and then use a large language model to parse the indicator query of the first indicator query based on the candidate indicator calculation formulas to convert it into a first structured indicator representation, accurately obtaining the calculation logic and data required for the first indicator query, and achieving efficient and accurate calculation of the first indicator.

[0115] Further, in step S22, the indicator result of the first indicator query is calculated according to the first structured indicator representation. When the first indicator query intent is the first indicator query, the query result of the indicator query is determined by the large language model according to the structured indicator representation and the preset indicator query database, specifically including:

[0116] Obtain the second structured indicator representation required by the first structured indicator representation through the large language model, and the second structured indicator representation is the structured indicator representation of the second indicator query;

[0117] Retrieve the intermediate indicator result from the preset indicator query database according to the second structured indicator representation through the large language model;

[0118] Calculate the indicator result of the first indicator query according to the intermediate indicator result.

[0119] Specifically, the second structured indicator representation is represented as a structured indicator representation of the second indicator query, and is also a second structured indicator representation required by the first structured indicator representation. The first structured indicator representation can call the decomposition process of the second structured indicator representation and retrieve the indicator result from the preset indicator query database based on the second structured indicator representation to realize the automatic calculation process of the indicator result of the first indicator query. For example, the composite indicator function in the first structured indicator representation can trigger the decomposition process of the second indicator query into the second structured indicator representation and the retrieval process of retrieving the intermediate indicator result based on the second structured indicator representation, and automatically input the retrieved indicator result into the composite indicator function. The indicator result of the first indicator query is automatically calculated through the composite indicator function, realizing the closed-loop process of the calculation of the indicator result of the first indicator query, and improving the query efficiency of the first indicator query.

[0120] The application constructs a query path of an indicator result for the first indicator query and the second indicator query respectively, then determines the query path corresponding to the indicator query by identifying the first indicator query intention of the indicator query, and determines the indicator result of the indicator query according to the query path, realizing the three-level linkage of the first indicator query intention identification, the first indicator query and the second indicator query, avoiding the misjudgment risk of a single query path, and further improving the accuracy of the indicator query. At the same time, the application constructs an automatic closed loop of formula retrieval, first indicator dynamic binding and function calculation for the first indicator query, breaks through the limitation of manual formula configuration, supports adaptive processing of complex nested calculation scenarios, and significantly improves the calculation efficiency and accuracy of complex indicators.

[0121] In one embodiment, the decomposition of the indicator query into a structured indicator representation according to the first indicator query intention specifically includes:

[0122] When the first indicator query intention is a second indicator query, the indicator query is decomposed into a second structured indicator representation by a large language model;

[0123] Retrieving from a preset indicator query database according to the second structured indicator representation by a large language model to obtain a structured indicator representation of the second indicator query.

[0124] Specifically, the second structured index representation includes a plurality of path nodes, the plurality of path nodes include a limit node and an index name node, the limit node is limit information of the index name, and the limit node can include one or more. The index name is gradually limited through the limit node to realize cross-level query of the atomic index, and avoid the case that the query error or failure is caused by the index query including non-professional terms. Wherein, the limit node includes one or more of a time node, a region node and an industry node, for example, the limit node includes a time node, a region node and an industry node.

[0125] When the index query is decomposed into the second structured index representation by the large language model, the corresponding structure specification of the second structured index representation can be obtained, and then the index query is structured and decomposed by the large language model according to the structure specification to convert it into the second structured index representation. For example, the structure specification is to split the index query of the second index query into the second structured index representation including the time node, the region node, the industry node and the index name node. For the index query whose first index query intention is the index query of the second index query: “XXX year XXX region industrial total output value”, it will be split into “time node: XXX year; region node: XXX region; industry node: industry; index name node: total output value”, to obtain the second structured index representation of the index query, which is (time node: XXX year; region node: XXX region; industry node: industry; index name node: total output value). The present application decomposes the index query whose index query intention is the index query of the second index query into the second structured index representation, which not only can avoid the risk of misprocessing of the composite index or the non-data query, but also can ensure the uniqueness and consistency of the second index query in the preset index query database through the standard field in the second structured index representation, improve the automation degree and speed of the index query. In addition, the present application sets the implicit time recognition when decomposing the second index query. The implicit time recognition is used to determine the time node of the index query without containing the time node (such as inferring the current year when the period is not clear). Through the combination of implicit time recognition and hierarchical path node extraction, the ambiguity defect of the index query can be eliminated, and the accuracy and integrity of the query result are further improved.

[0126] In addition, when the index query is decomposed into the second structured index representation by the large language model, the second structured index representation prompt word can be constructed for the large language model, the second structured index representation prompt word includes the index query and the task requirement, and of course it can also include a plurality of example analysis to provide analysis prior knowledge through the example analysis.

[0127] For example, the second structured index representation prompt word is:

[0128] Index query: {index query}

[0129] Task requirements: As an economic data preprocessing engineer, please strictly determine the second structured index representation according to the structure specification;

[0130] Structure specification:

[0131] Time range (accurate to year / season / month, if the implicit time period needs to be identified, this year is 2025)

[0132] Geographical scope (including province / city / district / county level)

[0133] Industry classification (using standard industry terms, normalized expression)

[0134] Index name (strictly query the index name in the preset index database)

[0135] Example analysis:

[0136] Input: XXX year XXX area industrial total output value

[0137] Output:

[0138] {

[0139] "Time node": "XXX year",

[0140] "Region node": "XXX area",

[0141] "Industry node": "Industry",

[0142] "Index name": "Total output value"

[0143] }

[0144] Please output the analysis result in strict JSON format.

[0145] Further, the index result of the second index query is directly obtained from the preset index query database. Accordingly, the query result of the index query determined by the large language model according to the structured index representation and the preset index query database is:

[0146] Retrieving index results from the preset index query database according to the second structured index representation by a large language model.

[0147] Specifically, when the first index query intent is a second index query, retrieve index results from the preset index query database according to the second structured index representation. Among them, the retrieval process can be hierarchical retrieval in the preset index query database according to the path node in the second structured index representation, to obtain the index result of the second index query.

[0148] S30, obtaining a second index result of the index query by the large language model according to the first index query intention and the constructed decomposed example dataset.

[0149] Specifically, the constructed decomposed example dataset includes a plurality of decomposed example data of the first index query, and the decomposed example data can include a historical index query, a second structured index representation required by the historical index query, and an index calculation formula, and a historical query result of the historical index query is the same as a query result obtained by taking the historical index query as an index query. In this way, the accuracy of the constructed decomposed example dataset can be improved. The constructed decomposed example dataset is used to provide prior knowledge of index query decomposition for the large language model, so as to improve the decomposition accuracy of the large language model for the index query.

[0150] Further, since the index result of the second index query is directly queried from the preset index query database, it is not affected by the query results of other index queries. Therefore, for the index query with the first index query intention being the second index query, the first index result can be directly taken as the final index result, and the second index result does not need to be obtained for the index query, and the first index result is verified based on the second index result. Based on this, before obtaining the second index result of the index query by the large language model according to the first index query intention and the constructed decomposed example dataset, it can include detecting whether the first index query intention of the index query is the first index query, if it is the first index query, executing the second index result of the index query by the large language model according to the first index query intention and the constructed decomposed example dataset, if it is not the first index query, taking the first query result as the final query result of the index query.

[0151] In one embodiment, as shown in Figure 5 obtaining a second index result of the index query by the large language model according to the first index query intention and the constructed decomposed example dataset, specifically includes:

[0152] S31, selecting a plurality of reference queries for the index query in the constructed decomposed example dataset;

[0153] S32, decomposing the index query into a structured index representation according to the first index query intention by the large language model according to the plurality of reference queries, the structured index representation being used to reflect the calculation content of the index query;

[0154] S33, calculating the index result of the index query according to the structured index representation and the preset index query database to obtain the second index result.

[0155] Specifically, the reference query is contained in the constructed decomposed example dataset, and the semantic similarity of the reference query and the index query meets the preset requirement, for example, the semantic similarity of the index query and each historical index query in the constructed decomposed example dataset is calculated respectively, and then the historical index query with a semantic similarity greater than a preset similarity threshold is selected as the reference query, or the historical index query of the specified data is selected as the reference query in descending order of semantic similarity, etc. In addition, after obtaining the reference query, the reference query is taken as a priori knowledge example of the large language model, and then the large language model is controlled to take the reference query as the priori knowledge, and the second index result is obtained by decomposing the index query into a structured index representation according to the implementation process of the first index query intention through the large language model. The process can refer to the above description, which will not be described in detail here. Only an example of the first structured index representation prompt word constructed with the reference query as the priori knowledge is given.

[0156] For example, the first structured index representation prompt word is:

[0157] Index query: {index query};

[0158] Reference decomposition result: {reference query and reference structured index representation};

[0159] Task requirement: As an economic formula analysis expert, please determine the first structured index representation according to the following rules:

[0160] 1. Identify composite indicators (such as proportion, growth rate, pull point number, contribution rate, etc.) and index calculation formulas of composite indicators;

[0161] 2. Clearly define the atomic indicators (use the index name in the preset index query database) and parameters required by the index calculation formula;

[0162] 3. Output the composite indicator function, including the expression (including mathematical symbols and variable explanations) and Python code function of the index calculation formula.

[0163] Example analysis:

[0164] Input: "XXX year XXX region manufacturing industry pull point number of industry";

[0165] Output:

[0166] {

[0167] "Atomic indicators": ["XXX year XXX region manufacturing industry added value", "XXX year XXX region manufacturing industry added value", "XXX year XXX region industrial added value"];

[0168] "Composite indicator function": "Pulling point number = (sub-industry this year added value - sub-industry last year added value) / industry last year added value * 100", "Python function": "XXXX"

[0169] }

[0170] Please output the parsing result in strict JSON format. If the query does not involve composite indicators, return an empty object.

[0171] S40, verifying the first indicator result based on the second indicator result to obtain the final indicator result of the indicator query.

[0172] Specifically, the second indicator result is used as a basis for determining whether the first indicator result can be the final indicator result. By the second indicator result, it can be determined whether the first indicator result can be the final indicator result, so as to improve the accuracy of the final indicator result.

[0173] In one embodiment, verifying the first indicator result based on the second indicator result to obtain the indicator result of the indicator query specifically includes:

[0174] Comparing the second indicator result with the first indicator result;

[0175] If the second indicator result is the same as the first indicator result, the first indicator result is taken as the final indicator result of the indicator query;

[0176] If the second indicator result is not the same as the first indicator result, the steps of obtaining the first indicator query intention of the indicator query through the large language model and obtaining the first indicator result of the indicator query through the large language model according to the first indicator query intention are re-executed.

[0177] Specifically, when the second indicator result is the same as the first indicator result, it means that the accuracy of the first indicator result meets the requirements, and the first indicator result can be taken as the final indicator result of the indicator query. On the contrary, when the second indicator result is not the same as the first indicator result, it means that the accuracy of the first indicator result does not meet the requirements, and the steps S10-S40 are re-executed to re-obtain the first indicator result. After re-obtaining the first indicator result, the first indicator result can be directly taken as the final result, or the process of verifying the first indicator result according to the second indicator result can be continued until the second indicator result is the same as the indicator result, or the re-execution times reach a preset number of thresholds.

[0178] The embodiment of the application provides a reference query selected from a constructed decomposition example data set according to semantic similarity as prior knowledge, and the prior knowledge is used for re-decomposing an index query by a large language model to obtain a second index result, and the second index result is used for verifying a first index result, so that logical deviation caused by free play of the large language model is effectively constrained, the risk of hidden error of a single calculation path is solved, and the accuracy of the index result is improved.

[0179] In one embodiment, before the index query responding to the user interaction, the first index query intention of the index query is obtained by the large language model, and the method further comprises the following steps of:

[0180] constructing an initial index query data set according to data tables in a plurality of historical reports;

[0181] obtaining a second index query intention of a historical index query by the large language model;

[0182] obtaining a third index result of the index query by the large language model according to the second index query intention;

[0183] verifying the third index result by the historical query result;

[0184] if the verification is passed, constructing a decomposition example data set for the historical index query according to the third index result.

[0185] Specifically, the historical report includes a plurality of historical indexes and historical index results of the historical indexes, and the historical indexes and the historical index results of the historical indexes are stored in the form of tables in the historical report. The initial index query data set includes a plurality of index query data binary tuples, each of which includes a historical index query and a historical query result, for example, the index query data binary tuple is (XXX year XXX region manufacturing industry to industry pull point number, 3.7), the XXX year XXX region manufacturing industry to industry pull point number is the historical index query, and 3.7 is the historical index result.

[0186] The index query data binary tuple is obtained by splitting the data table in the historical report, specifically, after the data table in the historical report is extracted, the data table can be split into a plurality of index query data binary tuples by the large model. When the data table is split into a plurality of index query data binary tuples by the large model, a splitting prompt word can be constructed for the large model, and the splitting prompt word can include a task requirement and a data table.

[0187] For example:

[0188] Splitting prompt word:

[0189] " Provide the data table in the preset format: {table data extracted from the historical report};

[0190] Task requirements:

[0191] Extract the index results of all indexes from the table data according to the following processing rules, and return them in the standard binary tuple format;

[0192] The processing rules are:

[0193] Data retention: strictly output according to the index name node (including time, region, industry, etc. Limiting words) and numerical precision in the data table

[0194] Format conversion: convert the data content in the data table into a binary tuple array containing historical index queries and historical query results, such as ("XXX year XXX region manufacturing industry pulling point number", 3.7);

[0195] Exception handling: if the data table does not contain the index or the data is ambiguous (such as "about 3.7"), return an empty list

[0196] Unit processing: percentage form (such as 25.5%) needs to be converted to decimal (such as 0.255)”.

[0197] After constructing the initial index query data set, construct the index calculation formula for the first index query in the historical index query to obtain the preset index calculation formula library; construct the second structured index representation for the second index query in the historical index query to obtain the preset index query database. The index calculation formula is the index calculation formula of the query result in the first index query in the historical index query, and the query result of the first index query in the historical index query can be calculated through the index calculation formula. In addition, in practical applications, after constructing the index calculation formula for the first index query in the historical index query, the index calculation formula can also be associated with the index query data binary tuple corresponding to the historical index query, to construct a high-quality training data set for the first index query, and improve the model performance of the large language model trained based on the training data set.

[0198] Further, since the index result of the second index query can be directly queried from the preset index query database, it does not need to be processed by the large language model. Therefore, in order to improve the fine-tuning efficiency, only the first index query in the historical index query is selected to construct the fine-tuning data set to fine-tune the large language model, that is, after obtaining the second index query intent of the historical index query, if the second index query intent is the first index query, the third index result of the index query is obtained through the large language model, wherein the acquisition process of the third index result is the same as that of the first index result, which will not be repeated here.

[0199] After obtaining the third index result, it is detected whether the historical query result of the historical query index is the same as the third index result to verify the third index result, wherein if the historical query result is the same as the third index result, it is determined that the third index result passes the verification, otherwise, if the historical query result is not the same as the third index result, it is determined that the third index result fails the verification. When the third index result passes the verification, a decomposition example data set can be constructed for the historical index query according to the third index result. Wherein, constructing a decomposition example data set for the historical index query according to the third index result specifically includes:

[0200] Obtaining the corresponding correct data item in the third index result;

[0201] Taking the parameters of the correct data item as a decomposition example data to obtain a decomposition example data set, wherein the parameters of the correct data item include a historical index query and a structured index representation of the historical index query.

[0202] Specifically, the correct data item is a data item for constructing a decomposition example data in a verification data group corresponding to the historical index query whose third index result passes the verification, and the verification data group is (historical index query, second index query, index calculation formula (if the first index query intention is the second index query, the index space is empty), query result of the second index query, third index result, historical query result). That is, after obtaining the third index result, the index calculation formula of the historical index query, the required second index query, the query result of the second index query, and the historical query result of the historical index query can be read, and the verification data group can be constructed according to the historical index query, the index calculation formula, the required second index query, the query result of the second index query, the third index result, and the historical query result of the historical index query. Then, the correct data item for constructing the decomposition example data is selected from the verification data group. Wherein, the correct data item includes the historical index query, the required second structured index representation of the historical index query, and the index calculation formula.

[0203] It should be noted that after obtaining the verification data group corresponding to each historical index query, the query accuracy of the large language model can be determined according to the verification data group, that is, the proportion of the verification data group whose query result is the same as the historical query result in all verification data groups. Wherein, the query accuracy can also be used as a basis for whether to fine-tune the large language model, such as when the accuracy is greater than a preset accuracy threshold, the large language model is not fine-tuned, when the accuracy is less than or equal to the preset accuracy threshold, the large language model is fine-tuned, etc. Here, the fine-tuning of the large language model is taken as an example for illustration.

[0204] In an embodiment, the verifying the third indicator result by the historical query result further includes:

[0205] If the verification fails, the corresponding error data item in the third indicator result is located, and the error data item is labeled;

[0206] The parameters of the error data item are taken as a fine-tuning data group to obtain a fine-tuning data set, wherein the parameters of the error data item include task content, error result, error reason, and labeled result;

[0207] The large language model is fine-tuned by using the fine-tuning data set.

[0208] Specifically, the error data item is a data item in the verification data group whose third indicator result does not pass the verification, and the error reason is a reason for the third indicator result to pass the verification. The error reason can be obtained by first obtaining the error data item in the verification data group that has an error, and then analyzing the reason for the error data item to have the error to obtain the error reason. For example, there is a query failure (i.e., the query result of the second indicator query is an error parameter) in the "query result of the second indicator query" in the error data item. The analysis of the reason for the error of the query result of the second indicator query finds that the second structured indicator representation of the second indicator query is incorrect, so the error reason of the error data group is the incorrect first structured indicator representation of the second indicator query.

[0209] Further, when constructing the fine-tuning data set according to the error reason, the labeled result of the error data item and the task content corresponding to the error data item can be obtained, and then a fine-tuning data group can be generated according to the task content corresponding to the error data item, the error result of the error data item, the error reason, and the labeled result of the error data item. The data set composed of all fine-tuning data groups is taken as the fine-tuning data set. That is, each fine-tuning data group in the fine-tuning data set includes the task content corresponding to the error data item, the error result, the error reason, and the labeled result. For example, when the atomic indicator result in the error data group is the error data item, the task content corresponding to the error data item is the first structured indicator representation decomposition, the error result of the error data item is the atomic indicator result in the error data group, the error reason is the incorrect first structured indicator representation of the atomic indicator, and the labeled result of the error data item is the labeled indicator result of the second indicator query. The fine-tuning data group corresponding to the error data group is (first structured indicator representation decomposition, query result of the second indicator query in the error data group, second structured indicator representation error, labeled indicator result).

[0210] It should be noted that after the error reason is obtained, the error data set can also be filtered based on the error reason first, and then the user is prompted to label the filtered error data item to obtain the labeling result of the error data item. The filtering basis can be a preset error reason data set, and the filtering process is to retain the error data set whose error reason belongs to the preset error reason data set, and delete the error data set whose error reason does not belong to the preset error reason data set. Of course, in actual application, other ways can also be used to filter the error data set, for example, a preset number of error data sets are randomly selected, and no specific limitation is made here.

[0211] The embodiment of the application calculates the query result of the initial index query data set, and generates a verification data set according to the query result, to realize the quality monitoring of the first index query from the second index query. Moreover, in the quality monitoring of the first index query from the second index query, the bias link (such as index calculation formula analysis or index result acquisition of the second index query) of the large language model can be located according to the error data set in the verification data set, and a fine-tuning data set is formed for the bias link of the large language model, and then the large language model is fine-tuned by using the fine-tuning data set, which can optimize the bias link of the large language model, improve the model performance of the large language model, and further improve the query accuracy of the index query.

[0212] In summary, the embodiment provides a query method based on a large language model, which includes responding to a user interactive index query, obtaining a first index query intent of the index query through a large language model; obtaining a first index result of the index query through a large language model according to the first index query intent; obtaining a second index result of the index query through a large language model according to the first index query intent and a constructed decomposition example data set; verifying the first index result based on the second index result to obtain a final index result of the index query. The application determines the first query result of the index query by analyzing the intent of the index query, then obtains the second index result of the index query according to the prior knowledge provided by a plurality of reference queries, and checks the first index result through the second index result, effectively avoiding the query error or failure problem caused by the inclusion of non-professional terms in the index query, and improving the accuracy of the index query.

[0213] Based on the above query method based on a large language model, the embodiment provides a query device based on a large language model, as shown in Figure 6 The query device based on a large language model specifically includes:

[0214] The obtaining module 100 is configured to respond to a user interactive index query, and obtain a first index query intent of the index query through a large language model.

[0215] The first query module 200 is configured to acquire a first index result of the index query by a large language model according to the first index query intention.

[0216] The second query module 300 is configured to acquire a second index result of the index query by a large language model according to the first index query intention and the constructed decomposition example data set.

[0217] The verification module 400 is configured to verify the first index result based on the second index result to obtain a final index result of the index query.

[0218] Based on the above-mentioned query method based on a large language model, the embodiment provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the query method based on a large language model.

[0219] Based on the above-mentioned query method based on a large language model, the present application further provides a terminal device, as shown in the accompanying drawings, which comprises at least one processor 20, a display screen 21, and a memory 22, and can further comprise a communications interface 23 and a bus 24. Wherein, the processor 20, the display screen 21, the memory 22 and the communications interface 23 can complete the communication among each other through the bus 24. The display screen 21 is set to display a user guide interface preset in an initial setting mode. The communications interface 23 can transmit information. The processor 20 can call the logical instructions in the memory 22 to execute the method in the above-mentioned embodiment. Figure 7

[0220] In addition, the logical instructions in the above-mentioned memory 22 can be realized in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer-readable storage medium.

[0221] The memory 22 as a computer-readable storage medium can be set to store software programs, computer executable programs, such as program instructions or modules corresponding to the method in the embodiment of the present disclosure. The processor 20 executes the function application and data processing by running the software programs, instructions or modules stored in the memory 22, that is, realizes the method in the above-mentioned embodiment.

[0222] ​The memory 22 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory. For example, various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. can also be a transitory storage medium.

[0223] In addition, the specific processes of the above-mentioned storage medium and the plurality of instruction processors in the terminal device load and execute have been described in detail in the above-mentioned method, and will not be described one by one here.

[0224] Finally, it should be pointed out that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A query method based on a large language model, characterized in that, The query method based on the large language model specifically comprises: In response to an index query of a user interaction, a first index query intention of the index query is obtained through a large language model; A first index result of the index query is obtained through the large language model according to the first index query intention; A second index result of the index query is obtained through the large language model according to the first index query intention and a constructed decomposition example dataset; The first index result is verified based on the second index result to obtain a final index result of the index query.

2. The query method based on a large language model according to claim 1, characterized in that, The first index result of the index query is obtained through the large language model according to the first index query intention, specifically comprising: The index query is decomposed into a structured index representation through the large language model according to the first index query intention, the structured index representation being used to reflect the calculation content of the index query; The query result of the index query is calculated according to the structured index representation and a preset index query database to obtain the first index result. 3.The query method based on a large language model according to claim 1, characterized in that, The second index result of the index query is obtained through the large language model according to the first index query intention and the constructed decomposition example dataset, specifically comprising: A plurality of reference queries are selected for the index query in the constructed decomposition example dataset; The index query is decomposed into a structured index representation through the large language model according to the first index query intention according to the plurality of reference queries, the structured index representation being used to reflect the calculation content of the index query; The index result of the index query is calculated according to the structured index representation and a preset index query database to obtain the second index result.

4. The large language model-based query method according to claim 2 or 3, characterized in that, The index query is decomposed into a structured index representation through the large language model according to the first index query intention, specifically comprising: When the first index query intention is a first index query, the semantic similarity of the index query and each formula in a preset index calculation formula library is calculated, and a preset number of candidate index calculation formulas are selected according to the semantic similarity; The index query is decomposed into a first structured index representation through the large language model according to the preset number of candidate index calculation formulas to obtain the structured index representation of the first index query.

5. The query method based on a large language model according to claim 4, characterized in that, The query result of the index query is determined through the large language model according to the structured index representation and a preset index query database, specifically comprising: The second structured index representation required by the first structured index representation is obtained through the large language model, the second structured index representation reflecting the structured index representation of a second index query; An intermediate index result is obtained by searching the preset index query database through the large language model according to the second structured index representation; The index result of the first index query is calculated according to the intermediate index result.

6. The query method based on a large language model according to claim 2 or 3, characterized in that, The index query is decomposed into a structured index representation through the large language model according to the first index query intention, specifically comprising: When the first index query intention is a second index query, the index query is decomposed into a second structured index representation through the large language model; retrieving, by the large language model, from a preset index query database according to the second structured index representation, to obtain a structured index representation of the second index query. 7.The query method based on a large language model according to claim 1, characterized in that, The first index result is verified based on the second index result to obtain a final index result of the index query, and the verification specifically includes: comparing the second index result with the first index result; if the second index result is the same as the first index result, taking the first index result as the final index result of the index query; if the second index result is not the same as the first index result, re-executing the steps of obtaining the first index query intention of the index query by the large language model and obtaining the first index result of the index query by the large language model according to the first index query intention. 8.The query method based on a large language model according to claim 1, characterized in that, Before the large language model obtains the first index query intention of the index query in response to the user interaction, the method further includes: constructing an initial index query data set according to data tables in a plurality of historical reports, wherein the initial index query data set includes a plurality of index query data pairs, and each index query data pair includes a historical index query and a historical query result; obtaining a second index query intention of the historical index query by the large language model; obtaining a third index result of the index query by the large language model according to the second index query intention; verifying the third index result by the historical query result; if the verification is passed, constructing a decomposition example data set for the historical index query according to the third index result. 9.The query method based on a large language model according to claim 8, characterized in that, The method of constructing the decomposition example data set for the historical index query according to the third index result specifically includes: obtaining a corresponding correct data item in the third index result; taking a parameter of the correct data item as a decomposition example data to obtain a decomposition example data set, wherein the parameter of the correct data item includes a historical index query, a second structured index representation required by the historical index query, and an index calculation formula.

10. The query method based on a large language model according to claim 9, characterized in that, The method of verifying the third index result by the historical query result further includes: if the verification is not passed, locating a corresponding error data item in the third index result and labeling the error data item; taking a parameter of the error data item as a fine-tuning data group to obtain a fine-tuning data set, wherein the parameter of the error data item includes a task content, an error result, an error reason, and a labeling result; fine-tuning the large language model by using the fine-tuning data set. 11.A query device based on a large language model, characterized in that, The query device based on the large language model specifically includes: an obtaining module configured to obtain, by a large language model, a first index query intention of an index query in response to a user interaction of the index query; a first query module configured to obtain, by the large language model, a first index result of the index query according to the first index query intention; a second query module configured to obtain, by the large language model, a second index result of the index query according to the first index query intention and a constructed decomposition example data set; and a third query module configured to obtain, by the large language model, a third index result of the index query according to the second index query intention. A verification module is configured to verify the first index result based on the second index result to obtain a final index result of the index query.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the query method based on the large language model according to any one of claims 1-10.

13. A terminal device, comprising: Comprise: A processor and a memory; The memory stores a computer readable program executable by the processor; The processor executes the computer readable program to implement the steps in the query method based on the large language model according to any one of claims 1-10.

Images