Question rewriting method, apparatus, device, medium, and product

Abstract

Embodiments of the present application disclose a question rewriting method, device, equipment, medium and product, the method comprising: obtaining a non-standardized expression type of an index in a data analysis task, and constructing a mapping relationship between the index and the non-standardized expression according to the non-standardized expression type of the index; generating each question pair based on an index analysis question template of the mapping relationship and standardized expression; performing index-based retrieval enhancement processing on the non-standardized expression of the question pair to obtain index enhancement information; training a to-be-trained model according to each question pair and the index enhancement information to obtain a question rewriting model, the question rewriting model being used to rewrite a target non-standardized expression index analysis question into a target standardized expression index analysis question. The technical solution of the embodiments of the present application accurately rewrites the non-standardized expression index analysis question into the standardized expression index analysis question, thereby improving the accuracy and reliability of data analysis.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a problem rewriting method, a problem rewriting apparatus, an electronic device, a computer-readable storage medium, and a computer product. Background Technology

[0002] In data analysis, objects are often expressed in natural language to express analytical needs, such as "How are the recent sales?" In such analytical questions, core indicator information (such as "sales") is the key to data analysis. However, due to the ambiguity and colloquial nature of natural language, the indicator information is often not accurately expressed in the analytical questions, which in turn affects the accuracy and reliability of data analysis. Summary of the Invention

[0003] Embodiments of this application provide a problem rewriting method, a problem rewriting apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which accurately rewrite non-standardized expression index analysis problems into standardized expression index analysis problems, thereby improving the accuracy and reliability of data analysis.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of the embodiments of this application, a problem rewriting method is provided, comprising: obtaining non-standardized expression types for indicators in a data analysis task, and constructing a mapping relationship from indicators to non-standardized expressions based on the non-standardized expression types for indicators; generating various problem pairs based on the mapping relationship and a standardized expression indicator analysis problem template, wherein each problem pair includes a non-standardized expression indicator analysis problem and a corresponding standardized expression indicator analysis problem; performing indicator-based retrieval enhancement processing on the non-standardized expression indicator analysis problem to obtain indicator enhancement information; and training a model to be trained based on each problem pair and the corresponding indicator enhancement information to obtain a problem rewriting model, wherein the problem rewriting model is used to rewrite a target non-standardized expression indicator analysis problem into a target standardized expression indicator analysis problem.

[0006] According to one aspect of the embodiments of this application, a problem rewriting apparatus is provided, comprising: an acquisition module, configured to acquire non-standardized expression types of indicators in a data analysis task, and construct a mapping relationship from indicators to non-standardized expressions based on the non-standardized expression types of indicators; a generation module, configured to generate various problem pairs based on the mapping relationship and a standardized expression indicator analysis problem template, wherein the problem pair includes a non-standardized expression indicator analysis problem and a corresponding standardized expression indicator analysis problem; an enhancement module, configured to perform indicator-based retrieval enhancement processing on the non-standardized expression indicator analysis problem to obtain indicator enhancement information; and a training module, configured to train a model to be trained based on each problem pair and the corresponding indicator enhancement information to obtain a problem rewriting model, wherein the problem rewriting model is used to rewrite a target non-standardized expression indicator analysis problem into a target standardized expression indicator analysis problem.

[0007] In one embodiment of this application, the non-standardized expression type for the indicator includes topic-based expressions for similar indicators; the acquisition module is further configured to perform similarity-based clustering processing on the indicators in the corresponding indicator database for different industries to obtain various similar indicator groups; acquire the indicator topics corresponding to each similar indicator group, and generate a general indicator expression corresponding to the similar indicator group based on the indicator topics; and establish a mapping relationship between similar indicator groups and general indicator expressions corresponding to the same indicator topic in the industry.

[0008] In one embodiment of this application, the non-standardized expression types for the indicators include colloquial expressions and rewritten expressions for the indicators; the acquisition module is further configured to input the indicator names of each indicator into a large language model to generate colloquial expressions corresponding to each indicator, and establish a mapping relationship between each indicator and the colloquial expressions; and / or, acquire the indicator meaning of each indicator, and semantically rewrite each indicator according to the indicator meaning to obtain rewritten expressions, and establish a mapping relationship between each indicator and the rewritten expressions.

[0009] In one embodiment of this application, the non-standardized expression type for the indicator includes the definition expression of the indicator; the acquisition module is further used to acquire the indicator definition of each indicator, and determine the complex indicator whose complexity of the indicator definition is greater than a preset complexity threshold; convert the calculation formula or business logic corresponding to the indicator definition of the complex indicator into a text expression, and establish a mapping relationship between each complex indicator and the text expression.

[0010] In one embodiment of this application, the generation module is further configured to replace the indicator elements in the indicator analysis problem template with the target indicator to obtain an indicator analysis problem with a standardized expression for the target indicator; retain the expression of the non-indicator elements in the indicator analysis problem template, and according to the mapping relationship from the target indicator to the non-standardized expression, replace the indicator elements with the non-standardized expression corresponding to the target indicator to obtain a non-standardized indicator analysis problem for the target indicator, so as to obtain the problem pair.

[0011] In one embodiment of this application, the apparatus further includes an update module, configured to perform enhancement processing based on diversified expression on the standardized expression index analysis problem to obtain a diversified standard expression problem; perform template processing on the diversified standard expression problem to obtain an extended index analysis problem; and update the index analysis problem template according to the extended index analysis problem.

[0012] In one embodiment of this application, the enhancement module is further configured to extract a semantic vector from the non-standardized expression of the indicator analysis problem to obtain a question semantic vector; search a pre-built indicator database of various industries based on the question semantic vector to obtain candidate indicator information related to the question semantic vector; and perform data enhancement processing on the candidate indicator information to obtain the indicator enhancement information.

[0013] In one embodiment of this application, the enhancement module is further configured to simulate multiple retrieval error scenarios to obtain multiple retrieval error types; modify the candidate indicator information based on the multiple retrieval error types to obtain indicator error information; and generate the indicator enhancement information based on the candidate indicator information and the indicator error information.

[0014] In one embodiment of this application, the training module is further configured to acquire thought prompting information for prompting the model to be trained to rewrite the problem according to intermediate thought reasoning steps. The intermediate thought reasoning steps include determining whether the index analysis problem contains non-standardized expressions; if it contains non-standardized expressions, rewriting the index according to the index enhancement information and the problem pair to obtain a standardized index analysis problem; and training the model to be trained according to each of the problem pairs, the corresponding index enhancement information, and the thought prompting information to obtain a problem rewriting model.

[0015] In one embodiment of this application, the training module is further configured to obtain the model to be trained and add a low-rank adaptive layer to the model to be trained; keep the original model parameters of the model to be trained frozen, take the indicator enhancement information, the thinking prompt information, and the non-standardized expression of the indicator analysis problem as input to the model to be trained, and take the standardized expression of the indicator analysis problem as output to the model to be trained, so as to adjust the model parameters of the low-rank adaptive layer of the model to be trained to obtain the problem rewriting model.

[0016] In one embodiment of this application, the training module is further configured to, if the number of adjustments to the model parameters of the low-rank adaptive layer of the model to be trained exceeds a preset threshold, perturb the new non-standardized expression of the index analysis problem to obtain an adversarial index analysis problem; use the adversarial index analysis problem, the corresponding new index enhancement information, and the thought prompt information as inputs to the model to be trained, and use the corresponding new standardized expression of the index analysis problem as output, so as to continue adjusting the model parameters of the low-rank adaptive layer of the model to be trained to obtain the problem rewriting model.

[0017] In one embodiment of this application, the apparatus further includes a query module, configured to obtain a target-standardized indicator analysis problem output by the problem rewriting model for the indicator analysis problem of the target non-standardized expression; perform intent recognition on the target-standardized indicator analysis problem, and generate a structured query statement based on the intent recognition result; perform data query based on the structured query statement to obtain query results, and generate response information for the indicator analysis problem of the target non-standardized expression based on the query results.

[0018] According to one aspect of the embodiments of this application, an electronic device is provided, including one or more processors; and a storage device for storing one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the problem rewriting method as described above.

[0019] According to one aspect of the embodiments of this application, an embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of an electronic device, causes the electronic device to perform the problem rewriting method as described above.

[0020] According to one aspect of the embodiments of this application, this application provides a computer program product, including a computer program stored in a computer-readable storage medium, wherein a processor of an electronic device reads from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the problem rewriting method as described above.

[0021] In the technical solution provided in the embodiments of this application, non-standardized expression types of indicators in data analysis tasks are obtained, and a mapping relationship from indicators to non-standardized expressions is constructed based on the non-standardized expression types of indicators. Based on the mapping relationship and the indicator analysis question template of standardized expression, various question pairs are generated. The question pairs include indicator analysis questions of non-standardized expression and corresponding indicator analysis questions of standardized expression. By dividing different non-standardized expression types, high-quality question pairs can be generated from multiple perspectives, thereby covering a wider range of expression scenarios. Then, indicator-based retrieval enhancement processing is performed on the indicator analysis questions of non-standardized expression to obtain indicator enhancement information, so as to supplement relevant indicator information and further improve the quality of training data, thereby optimizing the accuracy of question rewriting. Finally, the model to be trained is trained according to each question pair and the corresponding indicator enhancement information to obtain a question rewriting model. The question rewriting model can accurately rewrite the target non-standardized expression indicator analysis question into the target standardized expression indicator analysis question, ensuring the accurate execution of subsequent data analysis query tasks, thereby improving the accuracy and reliability of data analysis.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0024] Figure 1 This is a schematic diagram of one implementation environment involved in this application;

[0025] Figure 2 This is a flowchart illustrating a problem rewriting method in an exemplary embodiment of this application;

[0026] Figure 3 This is a schematic diagram illustrating a model training, rewriting, and response generation process, as shown in an exemplary embodiment of this application.

[0027] Figure 4 This is a schematic diagram of the interface for response information shown in an exemplary embodiment of this application;

[0028] Figure 5 This is a flowchart illustrating another problem rewriting method as shown in an exemplary embodiment of this application;

[0029] Figure 6This is a flowchart illustrating another problem rewriting method as shown in an exemplary embodiment of this application;

[0030] Figure 7 This is a schematic diagram illustrating the model training phase of an exemplary embodiment of this application;

[0031] Figure 8 This is a schematic diagram of query traffic shown in an exemplary embodiment of this application;

[0032] Figure 9 This is a schematic diagram illustrating data construction as shown in an exemplary embodiment of this application;

[0033] Figure 10 This is a schematic diagram illustrating model training in an exemplary embodiment of this application;

[0034] Figure 11 This is a schematic diagram illustrating model fine-tuning as shown in an exemplary embodiment of this application;

[0035] Figure 12 This is a structural block diagram illustrating a problem rewriting apparatus as shown in an exemplary embodiment of this application;

[0036] Figure 13 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0037] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying diagrams are merely illustrative and do not necessarily include all content and operations, nor do they necessarily have to be executed in the described order. For example, some operations may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0040] It should also be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0041] The technical solutions of the embodiments of this application will be described in detail below.

[0042] Please see Figure 1 , Figure 1 This is a schematic diagram of an implementation environment related to this application. The implementation environment includes a terminal 10 and a server 20.

[0043] The server is used to obtain the non-standardized representation types of indicators in data analysis tasks, and construct a mapping relationship from indicators to non-standardized representations based on the non-standardized representation types of indicators; generate various question pairs based on the mapping relationship and the indicator analysis question templates of standardized representations, including indicator analysis questions with non-standardized and standardized representations; perform indicator-based retrieval enhancement processing on the indicator analysis questions with non-standardized representations to obtain indicator enhancement information; and train the model to be trained based on each question pair and the corresponding indicator enhancement information to obtain the problem rewriting model.

[0044] Terminal 10 is used to acquire the index analysis problem of the target non-standardized expression to be processed, and send the index analysis problem of the target non-standardized expression to server 20.

[0045] Server 20 is used to rewrite the non-standardized expression of the target indicator analysis problem based on the problem rewriting model to obtain the standardized expression of the target indicator analysis problem.

[0046] The server can also send the target-standardized expression of the indicator analysis question to the terminal, so that the terminal can perform data query tasks based on the target-standardized expression of the indicator analysis question.

[0047] In some embodiments, the server 20 can pre-train a problem rewriting model and send the problem rewriting model to the terminal for storage, so that the terminal can also independently implement the problem rewriting process.

[0048] In some embodiments, the server may also obtain the target non-standardized expression of the index analysis problem itself, so as to rewrite the problem through a pre-trained problem rewriting model.

[0049] Among them, the aforementioned terminal 10 can be an electronic device such as a smartphone, tablet, laptop, computer, smart voice interaction device, smart home appliance, vehicle terminal, or aircraft. The server 20 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This document does not impose any restrictions on this.

[0050] Terminal 10 and server 20 establish a communication connection via a network beforehand, enabling them to communicate with each other. The network can be a wired network or a wireless network, and this is not a limitation.

[0051] It should be noted that the question rewriting model in this application embodiment can rewrite various analytical questions involving non-standardized expressions of indicators, and can be applied to various scenarios, including but not limited to AI (Artificial Intelligence) based question-answering dialogue scenarios, and data analysis scenarios in the fields of finance, e-commerce, medical care, and insurance.

[0052] For example, if the technical solution of this application embodiment is applied to a question-and-answer dialogue scenario, the terminal can be a smartphone, tablet, laptop, computer, smart voice interaction device, smart home appliance, vehicle terminal, etc. The terminal runs a question rewriting model. The terminal obtains the text information input by the object. The text information includes a conversational expression of an analytical question targeting an indicator, such as "How are sales of xx recently?". The question rewriting model rewrites the question as "Do you want to ask how efficient xx is recently?". The terminal can directly answer based on the standardized expression of the indicator analysis question after rewriting, or it can output the standardized expression of the indicator analysis question to the object for the object to confirm and then answer.

[0053] For example, applying the technical solution of this application embodiment to a data analysis scenario in the e-commerce field, in order to output a professional e-commerce analysis report, the object inputs the initial e-commerce analysis report into a problem rewriting model. The problem rewriting model rewrites the vague or colloquial expression of the indicator analysis problem in the initial e-commerce analysis report into a standardized expression of the indicator analysis problem, such as "recent user activity decline analysis" being rewritten as "daily active users (DAU) analysis" and "monthly active users (MAU) analysis". Then, based on the standardized expression of the indicator analysis problem, business decisions such as content recommendation optimization and recall strategies are made to output the final e-commerce analysis report, which includes the analysis problem and the business decisions made for the analysis problem.

[0054] It should be noted that in the specific implementation of this application, the issues involve objects. When the embodiments of this application are applied to specific products or technologies, permission or consent from the objects is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0055] The following describes in detail the various implementation details of the technical solutions in the embodiments of this application.

[0056] like Figure 2 As shown, Figure 2 This is a flowchart illustrating a problem rewriting method according to an embodiment of this application, which can be applied to... Figure 1 In the implementation environment shown, the method can be executed by a terminal or a server, or by both a terminal and a server. In this embodiment, the method is described as being executed by a server. The rewritten method can include steps S210 to S240, which are described in detail below.

[0057] S210. Obtain the non-standardized expression type of the indicator in the data analysis task, and construct the mapping relationship from the indicator to the non-standardized expression based on the non-standardized expression type of the indicator.

[0058] In this embodiment, the data analysis task can be a conversational data analysis task, such as when an object expresses its intention to analyze data through natural language, the server generates a data analysis task based on the conversational intention. It is understood that the data analysis task can be a task in any business domain, such as the financial field, e-commerce field, medical field, insurance field, etc.

[0059] Non-standardized expressions for indicators refer to the types of expressions used to describe indicators using non-standardized language. Non-standardized expressions include colloquial expressions and vague expressions. Colloquial expressions refer to casual and popular expressions achieved through everyday vocabulary and informal structures, while vague expressions refer to expressions that soften language by weakening specific information and retaining flexibility. In some examples, non-standardized expressions also include emotional expressions, which refer to expressions that contain emotions or subjective descriptions.

[0060] It is understandable that different types of non-standardized expressions differ in semantics and usage scenarios. Therefore, by classifying different types of non-standardized expressions for indicators, we can clarify the granularity of problem expression and then construct the mapping relationship from indicators to non-standardized expressions under different types.

[0061] In the embodiments of this application, the non-standardized expression types for indicators include at least one of the following: topic-based expression of similar indicators, colloquial expression of indicators, rewritten expression of indicators, and definition expression of indicators. For example, the non-standardized expression types for indicators may simultaneously include topic-based expression, colloquial expression, rewritten expression, and definition expression.

[0062] Optionally, for topic-based expressions of similar metrics, constructing mapping relationships includes:

[0063] For different industries, the indicators in the corresponding indicator database are clustered based on similarity to obtain various similar indicator groups; the indicator themes corresponding to each similar indicator group are obtained, and a general indicator expression corresponding to the similar indicator group is generated based on the indicator themes; a mapping relationship between similar indicator groups corresponding to the same indicator theme and the general indicator expression is established in the industry.

[0064] Different industries correspond to different indicator databases. For example, indicators for the e-commerce industry include repurchase rate, GMV, and order amount; indicators for the insurance industry include total policy amount, number of policyholders, and renewal payment conversion rate. For indicator databases of different industries, clustering is performed separately. The clustering process includes: performing similarity-based clustering on indicators in an industry's indicator database, grouping indicators with similarity greater than a preset similarity into similar indicator groups; text similarity algorithms, such as Term Frequency-Inverse Document Frequency (TF-IDF) + cosine similarity, semantic word vectors, etc., can be used for clustering, or clustering indicators with similar names and meanings can be performed; then, the clustered similar indicator groups are input into a large language model, which extracts the indicator themes of the similar indicator groups, generates general indicator expressions for the similar indicator groups based on the indicator themes, and establishes a mapping relationship between similar indicator groups belonging to the same indicator theme and general indicator expressions. This general indicator expression is the non-standardized expression of the similar indicator group, and the number of general indicator expressions can be one or more. For the e-commerce industry, similar indicator groups include "GMV", "sales revenue", and "order amount". The corresponding general indicators in the mapping relationship are expressed as transaction status, transaction details, etc.

[0065] Optionally, the mapping relationship can be constructed based on the colloquial and rewritten expressions of the indicators, including:

[0066] Input the names of each indicator into the large language model to generate the corresponding colloquial expressions for each indicator and establish the mapping relationship between each indicator and the colloquial expressions.

[0067] The formal metric names are generated into common and natural colloquial expressions using a large language model, thereby establishing a mapping relationship between each metric and colloquial expressions. For example, for the metric "sales volume", the large language model generates prompts such as "convert professional business metrics into everyday colloquial questions", and then the large language model generates the colloquial expression "how are sales?" based on the metric and prompts.

[0068] Optionally, for the rewritten expression of the metric, constructing the mapping relationship includes:

[0069] Obtain the meaning of each indicator, and semantically rewrite each indicator according to its meaning to obtain a rewritten expression, and establish a mapping relationship between each indicator and the rewritten expression.

[0070] Metric rewriting refers to replacing a metric with another expression that conveys the same meaning. Therefore, it requires obtaining the meaning of each metric. The meaning of a metric is an explanatory description used to help non-professionals, business personnel, or decision-makers understand its function more intuitively. In one example, for each metric, semantic hints are generated to express its meaning. These metric and hints can then be input into a large language model, allowing the model to semantically rewrite each metric based on its meaning, thus establishing a mapping relationship between each metric and its rewritten expression. For example, for the metric "number of purchases," the hint "express the meaning of the metric in another approximate way" is generated, leading to rewritten expressions such as "How many customers made a purchase?" or "How many people placed an order?". In another example, the explanatory expression of the metric's meaning is used as the rewritten expression. In yet another example, based on the metric's meaning, multiple equivalent or similar expressions are generated to obtain the rewritten expression. For example, the meaning of GMV is the total transaction amount of an object on the platform within a certain period. Equivalent terms are used to replace the corresponding words, such as replacing "total transaction amount" with "transaction amount," resulting in the rewritten expression: "The transaction amount of an object on the platform within a certain period."

[0071] Optionally, for the definition and expression of the indicator, the mapping relationship can be constructed as follows:

[0072] Obtain the definition of each indicator and identify complex indicators whose definition complexity exceeds a preset complexity threshold; convert the calculation formula or business logic corresponding to the definition of the complex indicator into a textual expression and establish a mapping relationship between each complex indicator and the textual expression.

[0073] The definition of an indicator is a formal and precise description of the indicator, including mathematical formulas and calculation rules. The complexity of each indicator's definition is calculated using a complexity calculation standard. This standard may include at least one of the following: character length (indicators exceeding a certain character limit), inclusion of mathematical formulas (involving division, weighted calculations, time windows, etc.), involvement of multiple variables (depending on multiple other indicators), or multiple calculation steps (e.g., LTV (Lifetime Value) requires accumulating revenue over multiple periods). Complex indicators are then selected based on this complexity calculation. Simple indicators include order amount and repurchase rate; complex indicators include decoration revenue ratio, landing page bounce rate, and LTV. For complex indicators, the calculation formulas or business logic are converted into detailed textual descriptions. For example, "advertising return on investment" corresponds to "the ratio of total revenue from advertising to advertising costs," and "landing page bounce rate" corresponds to "the ratio of page views leaving the homepage to page views," thus establishing a mapping relationship between complex indicators and their textual expressions.

[0074] Optionally, the colloquial, rewritten, and defined expressions of indicators can be generated without industry distinction, such as generating colloquial, rewritten, and defined expressions for each indicator in each industry; or they can be industry-specific, such as generating colloquial, rewritten, and defined expressions for each indicator in different industries for each indicator in the same industry.

[0075] The above scheme divides non-standardized expressions into four categories. This classification allows each category's mapping to address specific problems in a targeted manner, avoiding a "one-size-fits-all" approach and providing a more comprehensive coverage of various possible scenarios.

[0076] S220. Generate various question pairs based on the index analysis question template with mapping relationship and standardized expression. The question pairs include index analysis questions with non-standardized expression and index analysis questions with standardized expression.

[0077] In this embodiment, a pre-constructed indicator analysis question template is used. This template employs standardized expressions for the indicators and includes different types of questions, such as at least one of single-indicator trend analysis, multi-indicator comparative analysis, single-indicator percentage analysis, and multi-indicator correlation analysis. The indicator analysis template includes indicator elements and dimension elements. For example, the question template for single-indicator trend analysis under different dimensions is: "What is the trend of [index] under different [dim]?", and the question template for multi-indicator comparative analysis under the same dimension is: "What is the difference between [index]1 and [index]2 under the same [dim]?". [dim] represents the dimension (e.g., city, order type, time range), and [index] represents the specific indicator (e.g., order volume, sales volume). It is understood that the indicator dimension is not mandatory in the question template; for example, the question template could be: "What is the relationship between [index]1 and [index]2?".

[0078] Based on the mapping relationship formed by different non-standardized expression types of indicators, the indicator problem model is mapped to generate various problem pairs. For example, the indicator elements in the indicator analysis problem template are replaced with the target indicator to obtain the indicator analysis problem with the standardized expression of the target indicator. The expression of non-indicator elements in the indicator analysis problem template is retained, and the indicator elements are replaced with the non-standardized expression corresponding to the target indicator according to the mapping relationship from the target indicator to the non-standardized expression to obtain the non-standardized indicator analysis problem with the target indicator.

[0079] For example, the question template is: Under a certain [dim], the [index] of the last 30 days. Assuming the target indicator is "sales", the standardized expression is: Under a certain region, the sales of the last 30 days. By retaining the non-indicator element "Under a certain [dim], the sales of the last 30 days", we get the non-standardized expression: Under a certain region, how well did the sales perform in the last 30 days?

[0080] It should be understood that after replacing the indicator elements with the non-standardized expressions corresponding to the target indicator, if there are sentence structure errors or description errors between the non-standardized expressions and the non-indicator elements retained in the question template, it is necessary to adjust the sentences composed of the non-standardized expressions and the non-indicator elements in the question template to obtain the non-standardized indicator analysis questions for the target indicator.

[0081] As shown in Table 1 below, these are different types of problem pairs provided in the embodiments of this application.

[0082]

[0083] Table 1

[0084] The generated question pairs ensure that the entire training dataset contains both rigorous standardized queries and colloquial questions that people might ask in everyday life. These question pairs will be used to train the retrieval enhancement generative model, enabling the model to learn to flexibly match the correspondence between object queries and metrics.

[0085] Optionally, in this embodiment of the application, the indicator analysis problem can be continuously updated. Therefore, after generating the problem pair, it also includes:

[0086] The standardized expression of the indicator analysis problem is enhanced based on the diversified expression to obtain the diversified standard expression problem; the diversified standard expression problem is templated to obtain the extended indicator analysis problem, and the indicator analysis problem template is updated according to the extended indicator analysis problem.

[0087] Diverse expression refers to different ways of expressing the same problem. Among them, enhancement processing based on diversified expression can be to obtain the object habits of various types of objects, and enhance the sentence structure and vocabulary of the standardized expression index analysis problem according to the object habits to adapt to the object habits of different object types. For example, if the object habit of a certain region is to use inverted sentences, then the sentence structure of the standardized expression index analysis problem is adjusted to inverted sentences to obtain diversified standard expression problems.

[0088] Enhancement processing based on diversified expressions can also involve configuring extended prompts. Using a large language model, the standardized expression of the index analysis problem can be extended by configuring prompts. These extended prompts are used to prompt the large language model to express the problem in a diversified way.

[0089] Enhancement processing based on diversified expressions can also be applied to different industries by diversifying the expression of standardized expression indicators for analysis problems.

[0090] After obtaining the diverse standard expression problem, a template-based processing is performed. In one example, the indicator elements are set from the diverse standard expression problem to transform it into an extended indicator analysis problem that can be used for each indicator. The extended indicator analysis problem is then added to the indicator analysis problem model. In another example, the diverse standard expression problem is screened and summarized with the help of manual and model assistance, and then transformed into a standardized template.

[0091] The above approach enhances the standardized expression of indicator analysis by diversifying the expression, thereby improving the coverage and adaptability of the problem template and enhancing the ability of the subsequent model to process diverse natural language queries.

[0092] S230. For the analysis of non-standardized expression indicators, perform indicator-based retrieval enhancement processing to obtain indicator enhancement information.

[0093] In this embodiment of the application, the mapping relationship between non-standardized expressions and formal indicator expressions can be reflected through question pairs. In order to facilitate the subsequent model to more accurately identify the specific indicators referred to in the non-standardized expressions and further guide the rewriting of questions, indicator-based retrieval enhancement processing is performed on the indicator analysis questions of the non-standardized expressions in the question pairs to provide support for the mapping.

[0094] Among them, index-based retrieval enhancement processing refers to retrieving indices related to the index analysis problem of non-standardized expression, obtaining index enhancement information based on the retrieval results, which supplements the index analysis problem of non-standardized expression, so that the model can identify the specific indices referred to in the non-standardized expression.

[0095] S240. Based on each problem pair and the corresponding indicator enhancement information, train the model to be trained to obtain a problem rewriting model. The problem rewriting model is used to rewrite the indicator analysis problem with non-standardized expression of the target into an indicator analysis problem with standardized expression of the target.

[0096] It is understandable that each problem pair has a corresponding indicator enhancement information for non-standardized indicator analysis. Therefore, the problem rewriting model is obtained by training the model to be trained based on each problem pair and the corresponding indicator enhancement information.

[0097] Optionally, the model to be trained can be a pre-trained model or an untrained model. The non-standardized target index analysis problem and index augmentation information of the problem pair are combined and input into the model to be trained. The model to be trained learns the mapping between the non-standardized expression and the formal index expression to output the standardized index analysis problem of the problem pair, thereby training a problem rewriting model.

[0098] In one example, a problem rewriting model is used to rewrite a non-standardized expression of a target indicator analysis problem into a standardized expression of a target indicator analysis problem. This problem rewriting model can be applied to conversational intelligent assistants. When an object inputs a non-standardized expression of a target indicator analysis problem, the intelligent assistant rewrites it into a standardized expression of a target indicator analysis problem based on the problem rewriting model, and then performs indicator analysis and data query based on the standardized expression of the target indicator analysis problem.

[0099] Optionally, after training the problem rewriting model, the method further includes: obtaining the target standardized indicator analysis question output by the problem rewriting model for the target non-standardized expression indicator analysis question; performing intent recognition on the target standardized indicator analysis question and generating a structured query statement based on the intent recognition result; performing data query based on the structured query statement to obtain query results, and generating response information for the target non-standardized expression indicator analysis question based on the query results.

[0100] like Figure 3 As shown, the problem rewriting model is trained through steps S210 to S240. The user inputs a colloquial expression of an indicator analysis problem, and the problem rewriting model rewrites the problem to obtain a standardized expression of the indicator analysis problem. From the standardized indicator analysis problem, the specific query intent is identified to obtain the intent recognition result. Then, the intent recognition result is converted into a structured query statement (SQL or DSL query). The structured query statement is used to query relevant data from the database to obtain the query result. The query result is then converted into response information that conforms to the user's cognitive habits, such as generating data report-style response information, such as generating response information in the form of tables, charts, etc., to improve the accuracy and robustness of data query, thereby providing the user with a more intelligent and accurate data analysis experience.

[0101] When identifying intent from standardized indicator analysis questions, if the identified query intent lacks key information—which could be an indicator dimension or the query context—contextual session information related to the non-standardized indicator analysis question can be obtained. Based on this contextual session information, the missing key information in the current query can be identified and supplemented to obtain the intent identification result. Alternatively, the key information can be automatically supplemented by considering the object's business context to obtain the intent identification result. If the missing key information is an indicator dimension, all indicator dimensions of the corresponding industry can be used as supplementary information to obtain the intent identification result. Figure 4 As shown, for the colloquial expression of the input question "How are sales in xx province?", the standard indicator "sales volume" that the target needs to analyze is first identified, all "sales volume" dimensions are automatically added, and then the response information for querying sales volume is displayed in the form of a chart.

[0102] In this embodiment, non-standardized expression types of indicators in the data analysis task are obtained, and a mapping relationship from indicators to non-standardized expressions is constructed based on the non-standardized expression types. Based on the mapping relationship and standardized expression indicator analysis question templates, question pairs are generated. Each question pair includes a non-standardized expression indicator analysis question and its corresponding standardized expression indicator analysis question. By classifying different non-standardized expression types, high-quality question pairs can be generated from multiple perspectives, thus covering a wider range of expression scenarios. Then, indicator-based retrieval enhancement processing is performed on the non-standardized expression indicator analysis questions to obtain indicator enhancement information, supplementing relevant indicator information to further improve the quality of training data and optimize the accuracy of question rewriting. Finally, the question rewriting model is trained based on each question pair and the corresponding indicator enhancement information to obtain the question rewriting model. The question rewriting model can accurately rewrite the target non-standardized expression indicator analysis question into the target standardized expression indicator analysis question, ensuring the accurate execution of subsequent data analysis query tasks and thus improving the accuracy and reliability of data analysis.

[0103] In one embodiment of this application, another problem rewriting method is provided, which can be applied to... Figure 1 The implementation environment shown is illustrated using the example of the method being executed by a server. Figure 5 As shown, the method for rewriting this problem is in Figure 2 Based on S210 to S240 shown, S230 is extended to steps S510 to S530, which are described in detail below.

[0104] S510. Extract the semantic vector of the non-standardized expression index analysis problem to obtain the problem semantic vector.

[0105] S520. Based on the semantic vector of the question, search the pre-built index databases of various industries to obtain candidate index information related to the semantic vector of the question.

[0106] For a non-standardized expression index analysis problem, the problem is vectorized by a semantic vector model, and semantic features are extracted to obtain the problem semantic vector. This semantic vector model can be BERT (Bidirectional Encoder Representations from Transformers) or BGE (Bilingual General Embedding).

[0107] In this embodiment, to adapt to the professional indicator systems of different industries or businesses, an indicator database is pre-constructed for each industry. This indicator database includes a standardized list of indicators and a corresponding list of dimensions for that industry. Each indicator in the indicator database corresponds to an indicator vector. Therefore, the cosine similarity between the question semantic vector and all indicator vectors in the indicator database of each industry is calculated. The K indicator vectors with the highest cosine similarity are selected to obtain the candidate indicators corresponding to these K indicator vectors. Alternatively, the indicator dimensions corresponding to these K candidate indicators can be obtained, or the cosine similarity between the question semantic vector and the dimension vectors in the indicator database of each industry can be calculated, and the K dimension vectors with the highest cosine similarity can be selected. The selected candidate indicators and candidate dimensions are then used as candidate indicator information.

[0108] S530. Perform data augmentation processing on the candidate indicator information to obtain indicator enhancement information.

[0109] Candidate indicator information represents the search results under ideal conditions. In other embodiments of this application, candidate indicator information can be directly used as indicator enhancement information. However, in this embodiment, in order to improve the adaptability of the subsequent model to different situations, such as semantic retrieval errors, data enhancement processing is performed on the candidate indicator information to obtain indicator enhancement information.

[0110] Optionally, data augmentation can be performed by simulating retrieval error scenarios, such as... Figure 5 As shown, S530 includes: simulating multiple retrieval error scenarios to obtain multiple retrieval error types; modifying candidate indicator information based on multiple retrieval error types to obtain indicator error information; and generating indicator enhancement information based on the candidate indicator information and indicator error information.

[0111] Search error scenarios include missing, redundant, and erroneous cases. The corresponding search error types are missing search results, excessive search results, and incorrect search results. For each search error type, at least one of the candidate indicators and indicator dimensions in the candidate indicator information can be added, deleted, or replaced to obtain indicator error information, which is the imperfect search result.

[0112] Taking the modification of candidate indicators in the candidate indicator information as an example: For the missing search type, the candidate indicators in the candidate indicator information are deleted, such as randomly deleting one or more correct indicators. For example, in the non-standardized expression of indicator analysis question "How are sales this week?", the correctly matched "sales revenue" in the candidate indicator information is deleted. For the excessive search type, the candidate indicators in the candidate indicator information are added, such as adding some irrelevant indicators to the correct candidates found, such as adding "cost" and "profit" indicators. For the incorrect search type, the candidate indicators in the candidate indicator information are replaced, such as randomly replacing the correct indicator with other indicators, such as replacing the correct indicator with a similar but incorrect indicator, such as replacing "sales revenue" with "inventory".

[0113] Understandably, other error scenarios can be simulated based on actual conditions to add new error types, such as incorrect sorting of search indicators (key indicators are ranked after irrelevant indicators).

[0114] In this embodiment, both candidate indicator information and indicator error information are used as indicator enhancement information, so that the input of the model covers the retrieval of "ideal state", "missing case" and "interference case". That is, through adversarial training with erroneous data and correct data, the model learns how to find the correct non-standardized indicator correspondence under different retrieval quality during the training process, rather than simply relying on ideal candidate results.

[0115] The above approach uses a semantic vector model to retrieve candidate indicators, providing the model with more candidate information before mapping, thus determining which candidate best matches the meaning of the original spoken expression. Furthermore, after retrieving candidate indicators from the semantic vector model, different retrieval error scenarios are constructed by manually or automatically deleting, adding, or replacing them. This data augmentation enhances the model's adaptability to various retrieval error scenarios, ensuring that the model can still correctly output the standardized question even in real-world environments where incomplete retrieval or the presence of interfering items exists.

[0116] This application provides another problem rewriting method, which can be applied to... Figure 1 The implementation environment shown can be executed by a terminal or a server, or by both. In this embodiment, the method is described using the server as an example. Figure 6 As shown, the method for rewriting this problem is in Figures 2 to 5 Based on the above, S240 is extended to S610 to S620. S610 to S620 are described in detail below:

[0117] S610. Obtain thought prompts for the model to be trained to rewrite the problem according to the intermediate thought reasoning steps. The intermediate thought reasoning steps include determining whether the indicator analysis problem contains non-standardized expressions. If it contains non-standardized expressions, rewrite the indicators according to the indicator enhancement information and the problem pair to obtain the indicator analysis problem with standardized expressions.

[0118] Understandably, not all indicator analysis problems involve non-standardized expressions during the application of the problem rewriting model. If the model directly attempts to rewrite in a colloquial style, it may make mistakes. Therefore, this application uses the Chain of Thought (CoT) method during training to obtain thought prompts to guide the model to rewrite the problem according to intermediate thought reasoning steps. These intermediate thought reasoning steps help the model understand how to determine whether rewriting is necessary, how to match and rewrite appropriate indicators, etc.

[0119] Optionally, the intermediate reasoning steps include having the model first determine whether the problem includes a non-standardized expression; if it does, determining whether the indicator enhancement information includes candidate indicators related to the problem; if the indicator enhancement information includes candidate indicators related to the problem, replacing the non-standardized expression of the indicator with the corresponding standardized expression to output the indicator analysis problem in the standardized expression of the problem pair; if the problem is a standardized expression, not rewriting the problem; if the indicator enhancement information does not include candidate indicators related to the problem, outputting a prompt message that rewriting is not possible, or retrieving similar indicators from indicator databases of various industries to rewrite the problem.

[0120] If multiple candidate indicators related to the problem are retrieved from the indicator enhancement information, a self-correction process is performed, that is, a filtering mechanism is added to select the most suitable candidate indicator based on the context.

[0121] Optionally, the intermediate reasoning steps can be flexibly adjusted according to the actual situation. For example, the intermediate reasoning steps may include determining whether the problem contains non-standardized expressions. If non-standardized expressions are included, for multiple indicators, the problem may be split into sub-problems. If sub-problems are obtained, and if the indicator enhancement information includes candidate indicators related to the sub-problems, the problem may be modified.

[0122] S620. Based on each problem pair, the corresponding indicator enhancement information, and the thinking prompt information, train the model to be trained to obtain the problem rewriting model.

[0123] In this embodiment, the model input includes not only question pairs and indicator enhancement information, but also thought prompt information. By fine-tuning the model to be trained through three dimensions of input, a question rewriting model is obtained. Through thought prompt information, the model learns not only to make simple substitutions, but also to make layer-by-layer judgments and mappings based on question pairs and indicator enhancement information, based on understanding the intent of the question, thereby improving the accuracy and reliability of model rewriting.

[0124] In one example, the indicator enhancement information includes candidate indicator information and indicator error information. When training the model to be trained, staged training can be used. For example, in the first stage, the model is trained using question pairs, candidate indicator information, and thought prompts to rewrite the basic non-standardized problem into a standardized problem. In the second stage, the model is trained using question pairs, indicator error information, and thought prompts to adapt to retrieval errors, and different sets of indicator candidates are used for training. In the second stage, candidate indicator information and indicator error information can also be used simultaneously to form adversarial data.

[0125] Optionally, in order to train a large model that adapts to multiple industries, the model to be trained is fine-tuned using LORA (Low-Rank Adaptation). Therefore, step S620 includes: obtaining the model to be trained and adding a low-rank adaptive layer to the model to be trained; keeping the original model parameters of the model to be trained frozen, taking the indicator enhancement information and thinking prompt information, as well as the non-standardized expression of the indicator analysis problem as the input of the model to be trained, and taking the standardized expression of the indicator analysis problem as the output of the model to be trained, so as to adjust the model parameters of the low-rank adaptive layer of the model to be trained to obtain the problem rewriting model.

[0126] The model to be trained is a language model, including but not limited to Large Language Model (LLM), Multimodal Large Language Model (MLLM), and other generative models.

[0127] By introducing a low-rank adaptive LORA layer into the language model, a low-rank matrix is ​​introduced into the model. This low-rank matrix is ​​trainable. During forward propagation, the low-rank adaptive layer transforms the input data through the low-rank matrix before passing it to the next layer of the model. This transformation is equivalent to adding an extra layer of parameter adjustment on top of the language model, introducing new model parameters without changing the original model parameters. During training, the original model parameters of the language model remain frozen. The indicator enhancement information, thought prompts, and non-standardized expression indicator analysis problems are used as inputs to the model to be trained, so that only the new model parameters are adjusted without updating the original model parameters. In this way, fine-tuning of the model can be achieved while keeping the original model parameters unchanged.

[0128] In one example, when adjusting the new model parameters corresponding to the LORA layer, the language model outputs the index analysis question of the predicted standardized expression, calculates the text difference between the index analysis question of the predicted standardized expression and the index analysis question of the question in the question, constructs a loss function based on the text difference, and then adjusts the new model parameters corresponding to the LORA layer based on the loss function.

[0129] It should be noted that during the training process, the model may be affected by problems such as unclear non-standardized expressions and ambiguous metrics. Therefore, in this embodiment, adversarial training is adopted to allow the model to face more complex inputs with interference terms, thereby improving its robustness.

[0130] Optionally, step S620 further includes: if the number of times the model parameters of the low-rank adaptive layer of the model to be trained are adjusted is greater than a preset threshold, then the new non-standardized expression of the index analysis problem is perturbed to obtain an adversarial index analysis problem; the adversarial index analysis problem, the corresponding new index enhancement information, and the thinking prompt information are used as inputs to the model to be trained, and the corresponding new standardized expression of the index analysis problem is used as output, so as to continue to adjust the model parameters of the low-rank adaptive layer of the model to be trained to obtain a problem rewriting model.

[0131] When training the model to be trained, staged training can be used. When the number of times the model parameters of the low-rank adaptive layer of the model to be trained are adjusted is greater than the preset threshold, it means that the first stage and the second stage have been completed. The first stage completes the rewriting of the basic non-standardized problem into a standardized problem, and the second stage completes the robust optimization for retrieval error. Based on the first and second stages, the model is trained in the third stage of adversarial training, in which adversarial examples are added.

[0132] like Figure 7As shown, in the first stage, the model parameters of the model to be trained are adjusted to obtain the initial model. In the second stage, the model parameters of the initial model are adjusted to obtain the intermediate model. In the third stage, new question pairs are generated, including new non-standardized expression index analysis questions and new standardized expression index questions. The new non-standardized expression index analysis questions are perturbed to obtain adversarial index analysis questions. The perturbation process includes replacing some words to make them more ambiguous or unclear; adding irrelevant content to the questions to test whether the model can recognize the core information; deleting some key indicators in the questions so that the model can still infer the correct standardized expression with incomplete information; and changing the way the questions are expressed by adjusting the sentence structure to allow the model to learn diverse expression forms.

[0133] The problem of adversarial index analysis, the corresponding new index enhancement information, and the thinking prompt information are used as inputs to the intermediate model, and the corresponding new standardized expression of the index analysis problem is used as output. The model parameters of the low-rank adaptive layer after the second stage adjustment are further adjusted to obtain the problem rewriting model.

[0134] Optionally, after the number of iterations in the third stage reaches a preset number, the model's performance can be checked to see if it has reached the convergence condition, thus obtaining the problem rewriting model.

[0135] To facilitate understanding, this application also provides a method for rewriting questions, illustrated using colloquial expressions as an example, such as... Figure 8 As shown, the input object is a colloquial expression of the indicator analysis question query. It is transformed into a query statement through the question rewriting model and Text2SQL, and then the query data structure is output. The question rewriting model is combined with the indicator system to transform it into a standardized expression of the indicator analysis question. The question rewriting model enables Text2SQL to have more standardized input, thereby improving the accuracy of the final natural language query.

[0136] In this embodiment of the application, the model training process for rewriting the model for the problem includes two parts: data construction and retrieval enhancement generation training using a large language model.

[0137] like Figure 9 As shown, in order to transform colloquial natural language queries into precise data analysis indicator queries, this application embodiment designs an automated data construction process for subsequent model training.

[0138] Among them, the generation of typical question templates relies heavily on the experience of data analysts for the initial generation of question templates. A batch of basic question templates are generated, such as trend analysis questions for single indicators and comparative analysis questions for multiple indicators. Finally, typical question templates are obtained. The following are examples of question templates:

[0139] Under different [dim] values, what is the [index] for [date_dur]? (Where dim represents the dimension, [date_dur] represents the analysis time range, and [index] represents the metric. For example, what is the order volume for the most recent 3 days under different cities?)

[0140] Based on a set of seed question templates and combined with indicators, corresponding questions are generated. The generated questions are then expanded into more diverse questions using a large language model, such as using larger models like GPT4, to design prompts, diversify the expressions of the original questions, and template the diversified expressions of the questions (with the assistance of human intervention and large models), continuously expanding the typical question templates.

[0141] Multi-industry indicator system collection: The indicator system mainly includes indicators, dimensions, and the definitions of indicators and dimensions. For example, indicators for the e-commerce industry include repurchase rate, GMV, order amount, etc., and dimensions include order type, payment method, etc.; the insurance industry includes total policy amount, number of policyholders, renewal payment conversion rate, etc., and dimensions include vehicle type, vehicle brand, etc. For example, the indicator definition is: payment recipient ratio = recipients who have paid for the order / cumulative number of recipients. Among them, through web collection and crawling, dozens of classic indicator systems from various industries were obtained.

[0142] This application constructs a mapping from indicators to colloquial expressions based on different categories: For the data analysis field, this application categorizes colloquial indicator expressions into four types: 1. Indicator themes; 2. Colloquial expressions of indicators; 3. Indicator rewriting; 4. Indicator definitions. For each category, different prompts are used based on a large language model to construct the mapping from indicators to colloquial expressions.

[0143] 1) Indicator Theme Category

[0144] For different industry-specific indicator systems, clustering is performed separately. For example, the clustering results for an industry might include: "Total Matches," "Match Success Rate," "Number of Abnormal Matches," "Successful Match Rate," "Number of Successful Matches," and "Successful Match Duration." Indicators belonging to the same category are then input into a large language model to generate corresponding general expressions. For instance, the general expressions generated for matching might include: "Matching Status," "Object Matching Details," etc.; and for financing status, "Object Financing Details," etc.

[0145] 2) Colloquial expression of indicators

[0146] Different metrics are input into a large model (such as GPT4) to generate corresponding common colloquial expressions, and finally generate corresponding mappings, such as "sales volume" corresponding to "how well are sales", "profit" corresponding to "how much money have you made", "how much have you earned", etc., and "new customer percentage" corresponding to "how many are first-time customers".

[0147] 3) Indicator rewriting

[0148] Metric rewriting is more about giving a similar expression to the metric. For example, "number of buyers" corresponds to "how many customers made a purchase", and "visit-to-conversion rate" corresponds to "how many visitors converted into purchases", etc.

[0149] 4) Indicator Definition

[0150] A metric definition refers to expressing a complex metric as a formula, such as "decoration revenue ratio" which corresponds to "the ratio of decoration revenue to vehicle sales revenue", and "landing page bounce rate" which corresponds to "the ratio of page views that leave the homepage to the total page views".

[0151] Generate question pairs of standardized and colloquial indicator questions: Based on the indicator mappings of the four categories mentioned above and the corresponding classic question templates, generate question pairs. The indicator mappings are used to replace the [index] template, replacing [index] with the expressions before and after the indicator mapping, respectively, ultimately forming question pairs for subsequent model training.

[0152] like Figure 10 As shown, the training process for retrieval enhancement generation training using a large language model includes:

[0153] Based on the question pairs obtained in the first section, the model input consists of indicator analysis questions containing colloquial expressions within the question pairs, and the output consists of indicator analysis questions expressed in a standardized form within the question pairs. To adapt to the professional indicator systems of different industries or businesses, during the training process, this application first retrieves relevant indicators and dimensions based on the questions, combines the retrieved candidate dimensions and indicators, and uses questions containing colloquial indicator expressions as input, to output standardized indicator analysis questions.

[0154] For questions containing colloquial expressions, a semantic vector model (such as BGE) is used to retrieve relevant metrics and dimensions as candidate metrics and dimensions for that colloquial expression. For example, given e-commerce industry metrics such as repurchase rate, GMV, order amount, and sales volume, and dimensions including order type and payment method, a given analysis question such as "How are different orders selling?" would retrieve candidate metrics such as order amount, sales volume, and order type. Based on these retrieved candidates, combined with the question containing colloquial expressions as input, the question with standardized metrics is output.

[0155] Candidate metrics and candidate dimensions represent the ideal retrieval results. To more robustly train for retrieval augmentation, this application simulates various retrieval error scenarios. For example, if a retrieval metric is missing, the model cannot correctly rewrite the text in colloquial language given the candidate metrics. If more retrieval metrics are added, the model needs to select appropriate metrics from the candidate metrics for rewriting. By adding, deleting, and modifying candidate metrics and candidate dimensions of the retrieval results, data augmentation is performed to simulate various retrieval error scenarios, resulting in error metrics and error dimensions. These error metrics and error dimensions are then used as input to allow the model to adapt to different retrieval situations.

[0156] Meanwhile, considering that not all indicator questions contain colloquial expressions, this application adopts an intermediate thinking reasoning training process, such as 1. determining whether the question contains colloquial elements, 2. if it contains colloquial elements, determining whether the candidates contain relevant indicators, 3. if it contains colloquial elements and the candidates contain relevant indicators, then outputting the corresponding indicator mapping results and question rewriting results.

[0157] Taking all the above considerations into account, the final input example for the large language model is shown below:

[0158] Input: "You are an indicator analysis assistant. Given a list of indicators ['Average dwell time', 'Profit', 'Activity participation rate', 'Channel order amount', 'Number of times added to cart', 'Sales rate', 'Consumption frequency (weekly)', 'Number of successfully paid orders', 'Total amount of orders placed during the activity', 'Number of canceled orders', 'Registration conversion rate']. Next, consider the following steps: (1) Determine whether the indicators in the following question contain colloquial parts. If they do, return True; otherwise, return False. (2) If they contain colloquial parts, first map the colloquial parts in the question to the relevant indicators in the given indicator list. Determine whether the indicator list matches. If it matches, return True; otherwise, return False. (3) If there are relevant indicators, replace the colloquial parts that need to be mapped with the indicators, and then output the final rewritten result. Keep the parts that do not need to be replaced and only output the specified content. Question: How much money did we make in the promotion in the last month?"

[0159] Output: "[True,True,'[{How much money did we make:Profit},{How much profit did we make in the most recent month's promotion?}]']"

[0160] Based on the prepared pairwise training dataset, fine-tuning training is performed using an open-source large language model, employing models of 7b or 13b scale for data fine-tuning. The fine-tuning utilizes the efficient parameter fine-tuning technique LoRa. LoRa adds extra branches to the original model for parameter fine-tuning, such as... Figure 11As shown, the pre-trained weights represent the pre-trained parameters of the model. These weights have already learned some knowledge in the original model and will not change significantly during fine-tuning. The input x and the hidden representation h represent the hidden representation or intermediate state after processing by the pre-trained weights (and subsequent adjustments). B = 0 indicates that matrix B is initialized to 0; A = N(0, σ²) indicates that matrix A is randomly initialized using a normal distribution (mean of 0, variance of σ²). The input x is processed using the existing pre-trained weights W; new parameters A (randomly initialized) and B (initialized to 0) are introduced to form a low-rank update; the improved hidden state h is obtained by combining the information from the two parts.

[0161] In this embodiment of the application, for indicators in the field of data analysis, the fuzzy indicator expressions are divided into four categories: topic indicators, colloquial indicators, indicator rewriting, and indicator rewriting with formulas. For these four types of questions, through data construction, large language model retrieval and generation, and integration of thinking chain training, a question rewriting model is finally obtained, which realizes the effect of transforming colloquial natural language questions into professional data analysis questions, and further improves the accuracy of natural language to query language conversion.

[0162] The question rewriting model in this application demonstrates improved performance in both rewriting conversational questions and transforming them into query language for subsequent end-to-end data analysis. The beneficial effects of this application can be observed from two perspectives: 1. Rewriting accuracy; 2. End-to-end Text2SQL accuracy. A test set of 1000 rewritten data entries and a Text2SQL test set of 1000 entries (600 conversational and 400 non-conversational) were constructed. The accuracy on the rewritten test set reached 83%. Regarding end-to-end Text2SQL accuracy, as shown in Table 2, compared to the basic model, the addition of the question rewriting model in this application has little impact on the accuracy of queries with non-conversational expressions, but improves accuracy by 5% on the conversational test set, significantly enhancing the overall effectiveness.

[0163]

[0164] Table 2

[0165] This application describes an apparatus embodiment that can be used to execute the problem rewriting method described above in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the problem rewriting method described above in this application.

[0166] This application provides a problem rewriting apparatus, such as... Figure 12 As shown, the device includes:

[0167] The acquisition module 1210 is used to acquire the non-standardized expression type of the indicator in the data analysis task, and construct the mapping relationship from the indicator to the non-standardized expression based on the non-standardized expression type of the indicator;

[0168] The generation module 1220 is used to generate various question pairs based on the mapping relationship and the standardized expression of the indicator analysis question template. The question pairs include non-standardized expression of the indicator analysis question and the corresponding standardized expression of the indicator analysis question.

[0169] The enhancement module 1230 is used to perform index-based retrieval enhancement processing on the index analysis problem of the non-standardized expression to obtain index enhancement information;

[0170] The training module 1240 is used to train the model to be trained based on each of the problem pairs and the corresponding indicator enhancement information to obtain a problem rewriting model. The problem rewriting model is used to rewrite the indicator analysis problem with a non-standardized expression of the target into an indicator analysis problem with a standardized expression of the target.

[0171] In one embodiment of this application, based on the foregoing scheme, the non-standardized expression type for the indicators includes topic-based expressions for similar indicators; the acquisition module is further configured to perform similarity-based clustering processing on the indicators in the corresponding indicator database for different industries to obtain various similar indicator groups; acquire the indicator topics corresponding to each similar indicator group, and generate a general indicator expression corresponding to the similar indicator group based on the indicator topics; and establish a mapping relationship between similar indicator groups and general indicator expressions corresponding to the same indicator topic in the industry.

[0172] In one embodiment of this application, based on the foregoing scheme, the non-standardized expression types for the indicators include colloquial expressions and rewritten expressions for the indicators; the acquisition module is further configured to input the indicator names of each indicator into the large language model to generate colloquial expressions corresponding to each indicator, and establish a mapping relationship between each indicator and the colloquial expressions; and / or, acquire the indicator meaning of each indicator, and semantically rewrite each indicator according to the indicator meaning to obtain rewritten expressions, and establish a mapping relationship between each indicator and the rewritten expressions.

[0173] In one embodiment of this application, based on the foregoing scheme, the non-standardized expression type for the indicator includes the definition expression of the indicator; the acquisition module is further used to acquire the indicator definition of each indicator, and determine the complex indicator whose complexity of the indicator definition is greater than a preset complexity threshold; convert the calculation formula or business logic corresponding to the indicator definition of the complex indicator into a text expression, and establish a mapping relationship between each complex indicator and the text expression.

[0174] In one embodiment of this application, based on the foregoing scheme, the generation module is further configured to replace the indicator elements in the indicator analysis problem template with the target indicator to obtain an indicator analysis problem with a standardized expression for the target indicator; retain the expression of the non-indicator elements in the indicator analysis problem template, and according to the mapping relationship from the target indicator to the non-standardized expression, replace the indicator elements with the non-standardized expression corresponding to the target indicator to obtain a non-standardized indicator analysis problem for the target indicator, so as to obtain the problem pair.

[0175] In one embodiment of this application, based on the foregoing scheme, the apparatus further includes an update module, configured to perform enhancement processing based on diversified expression on the standardized expression index analysis problem to obtain a diversified standard expression problem; perform template processing on the diversified standard expression problem to obtain an extended index analysis problem; and update the index analysis problem template according to the extended index analysis problem.

[0176] In one embodiment of this application, based on the foregoing scheme, the enhancement module is further configured to extract the semantic vector of the non-standardized expression of the indicator analysis problem to obtain the problem semantic vector; search the indicator databases of various industries in advance according to the problem semantic vector to obtain candidate indicator information related to the problem semantic vector; and perform data enhancement processing on the candidate indicator information to obtain the indicator enhancement information.

[0177] In one embodiment of this application, based on the aforementioned scheme, the enhancement module is further configured to simulate multiple retrieval error scenarios to obtain multiple retrieval error types; modify the candidate indicator information based on the multiple retrieval error types to obtain indicator error information; and generate the indicator enhancement information based on the candidate indicator information and the indicator error information.

[0178] In one embodiment of this application, based on the aforementioned scheme, the training module is further configured to obtain thought prompting information for prompting the model to be trained to rewrite the problem according to intermediate thought reasoning steps. The intermediate thought reasoning steps include determining whether the index analysis problem contains non-standardized expressions; if it contains non-standardized expressions, rewriting the index according to the index enhancement information and the problem pair to obtain a standardized index analysis problem; and training the model to be trained according to each of the problem pairs, the corresponding index enhancement information, and the thought prompting information to obtain a problem rewriting model.

[0179] In one embodiment of this application, based on the foregoing scheme, the training module is further configured to obtain the model to be trained and add a low-rank adaptive layer to the model to be trained; keep the original model parameters of the model to be trained frozen, take the indicator enhancement information, the thinking prompt information, and the non-standardized expression of the indicator analysis problem as input to the model to be trained, and take the standardized expression of the indicator analysis problem as output to the model to be trained, so as to adjust the model parameters of the low-rank adaptive layer of the model to be trained to obtain the problem rewriting model.

[0180] In one embodiment of this application, based on the foregoing scheme, the training module is further configured to, if the number of adjustments to the model parameters of the low-rank adaptive layer of the model to be trained exceeds a preset threshold, perturb the new non-standardized expression of the index analysis problem to obtain an adversarial index analysis problem; use the adversarial index analysis problem, the corresponding new index enhancement information, and the thought prompt information as inputs to the model to be trained, and use the corresponding new standardized expression of the index analysis problem as output, so as to continue adjusting the model parameters of the low-rank adaptive layer of the model to be trained to obtain the problem rewriting model.

[0181] In one embodiment of this application, based on the foregoing scheme, the device further includes a query module, configured to obtain a target standardized indicator analysis problem output by the problem rewriting model for the indicator analysis problem of the target non-standardized expression; perform intent recognition on the target standardized indicator analysis problem, and generate a structured query statement based on the intent recognition result; perform data query based on the structured query statement to obtain query results, and generate response information for the indicator analysis problem of the target non-standardized expression based on the query results.

[0182] It should be noted that the apparatus provided in the above embodiments and the method provided in the above embodiments belong to the same concept, and the specific way in which each module and unit performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0183] The device provided in the above embodiments can be located in a terminal or in a server.

[0184] Embodiments of this application also provide an electronic device, including one or more processors and a storage device, wherein the storage device is used to store one or more computer programs, which, when executed by one or more processors, cause the electronic device to implement the above-described problem rewriting method.

[0185] Figure 13 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0186] It should be noted that, Figure 13 The computer system 1300 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0187] like Figure 13 As shown, the computer system 1300 includes a Central Processing Unit (CPU) 1301, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on a program stored in Read-Only Memory (ROM) 1302 or a program loaded from storage portion 1308 into Random Access Memory (RAM) 1303. The RAM 1303 also stores various programs and data required for system operation. The CPU 1301, ROM 1302, and RAM 1303 are interconnected via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0188] In some embodiments, the following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, mouse, etc.; an output section 1307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1310 as needed so that computer programs read from it can be installed into the storage section 1308 as needed.

[0189] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as a computer program. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1309, and / or installed from removable medium 1311. When the computer program is executed by processor (CPU) 1301, it performs various functions defined in the system of this application.

[0190] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory, flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0191] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and a computer program.

[0192] The units or modules described in the embodiments of this application can be implemented in software or hardware, and can also be located in a processor. The names of these units or modules do not necessarily limit the specific unit or module itself.

[0193] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the problem rewriting method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0194] Another aspect of this application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the problem rewriting method as described above in the various embodiments.

[0195] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0196] Other embodiments of this application will readily conceive of by considering the specification and practicing the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0197] The above content is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A method for rewriting a problem, characterized in that, include: Obtain the non-standardized expression type of the indicator in the data analysis task, and construct the mapping relationship from the indicator to the non-standardized expression based on the non-standardized expression type of the indicator; Based on the mapping relationship and the standardized expression of the indicator analysis question template, each question pair is generated, including the non-standardized expression of the indicator analysis question and the corresponding standardized expression of the indicator analysis question; The non-standardized expression index analysis problem is subjected to index-based retrieval enhancement processing to obtain index enhancement information; The problem rewriting model is obtained by training the model to be trained based on each of the problem pairs and the corresponding indicator enhancement information. The problem rewriting model is used to rewrite the indicator analysis problem with non-standardized target expression into the indicator analysis problem with standardized target expression.

2. The method according to claim 1, characterized in that, The non-standardized expression types for indicators include topic-based expressions for similar indicators; the step of constructing a mapping relationship from indicators to non-standardized expressions based on the non-standardized expression types for indicators includes: For different industries, the indicators in the corresponding indicator database are clustered based on similarity to obtain various groups of similar indicators. Obtain the indicator themes corresponding to each of the same type of indicator groups, and generate general indicator expressions corresponding to the same type of indicator groups based on the indicator themes; Establish a mapping relationship between similar indicator groups corresponding to the same indicator theme in the industry and general indicator expressions.

3. The method according to claim 1, characterized in that, The non-standardized expression types for indicators include colloquial expressions and rewritten expressions for indicators; the step of constructing a mapping relationship from indicators to non-standardized expressions based on the non-standardized expression types for indicators includes: Input the names of each indicator into the large language model to generate the colloquial expressions corresponding to each indicator, and establish a mapping relationship between each indicator and the colloquial expressions. And / or, Obtain the meaning of each indicator, and semantically rewrite each indicator according to the meaning to obtain a rewritten expression, and establish a mapping relationship between each indicator and the rewritten expression.

4. The method according to claim 1, characterized in that, The non-standardized expression type for the indicator includes the definition expression of the indicator; the step of constructing a mapping relationship from the indicator to the non-standardized expression based on the non-standardized expression type for the indicator includes: Obtain the definition of each indicator and determine the complex indicators whose complexity is greater than a preset complexity threshold. The calculation formulas or business logic corresponding to the definition of the complex indicators are converted into textual expressions, and a mapping relationship between each complex indicator and the textual expression is established.

5. The method according to claim 1, characterized in that, The indicator analysis question template based on the mapping relationship and standardized expression generates various question pairs, including: By replacing the indicator elements in the indicator analysis problem template with the target indicator, a standardized expression of the indicator analysis problem for the target indicator is obtained. The expression of non-indicator elements in the indicator analysis problem template is retained. Based on the mapping relationship from the target indicator to the non-standardized expression, the indicator elements are replaced with the non-standardized expression corresponding to the target indicator to obtain a non-standardized indicator analysis problem for the target indicator, thus obtaining the problem pair.

6. The method according to claim 1, characterized in that, After generating each question pair based on the index analysis question template according to the mapping relationship and standardized expression, the process further includes: The standardized expression index analysis problem is subjected to enhancement processing based on diversified expression to obtain a diversified standardized expression problem; The diverse standard expression problem is templated to obtain the extended index analysis problem, and the index analysis problem template is updated according to the extended index analysis problem.

7. The method according to any one of claims 1 to 6, characterized in that, The non-standardized expression index analysis problem is subjected to index-based retrieval enhancement processing to obtain index enhancement information, including: The semantic vector of the non-standardized expression index analysis problem is obtained by extracting the semantic vector of the problem. Based on the semantic vector of the question, a search is performed in a pre-built index database of various industries to obtain candidate index information related to the semantic vector of the question. The candidate indicator information is augmented to obtain the indicator enhancement information.

8. The method according to claim 7, characterized in that, The process of performing data augmentation on the candidate indicator information to obtain the enhanced indicator information includes: Simulate multiple retrieval error scenarios to obtain multiple retrieval error types; Based on the multiple retrieval error types, the candidate indicator information is modified to obtain indicator error information; The indicator enhancement information is generated based on the candidate indicator information and the indicator error information.

9. The method according to any one of claims 1 to 8, characterized in that, The step of training the model to be trained based on each of the aforementioned problem pairs and the corresponding indicator enhancement information to obtain a problem rewriting model includes: Obtain thought prompt information to prompt the model to be trained to rewrite the problem according to the intermediate thought reasoning steps. The intermediate thought reasoning steps include determining whether the index analysis problem contains non-standardized expressions. If it contains non-standardized expressions, rewrite the index according to the index enhancement information and the problem to obtain the index analysis problem with standardized expressions. The problem rewriting model is obtained by training the model to be trained based on the respective problem pairs, the corresponding indicator enhancement information, and the thinking prompt information.

10. The method according to claim 9, characterized in that, The step of training the model to be trained based on the respective question pairs, the corresponding indicator enhancement information, and the thinking prompt information to obtain the problem rewriting model includes: Obtain the model to be trained, and add a low-rank adaptive layer to the model to be trained; The original model parameters of the model to be trained are kept frozen. The indicator enhancement information, the thinking prompt information, and the non-standardized expression of the indicator analysis problem are used as inputs to the model to be trained. The standardized expression of the indicator analysis problem is used as the output of the model to be trained. The model parameters of the low-rank adaptive layer of the model to be trained are adjusted to obtain the problem rewriting model.

11. The method according to claim 10, characterized in that, After taking the indicator enhancement information, the thought prompting information, and the non-standardized expression of the indicator analysis problem as input to the model to be trained, the method includes: If the number of times the model parameters of the low-rank adaptive layer of the model to be trained is greater than a preset threshold, then the new non-standardized expression index analysis problem is perturbed to obtain the adversarial index analysis problem. The adversarial index analysis problem, the corresponding new index enhancement information, and the thinking prompt information are used as inputs to the model to be trained, and the corresponding new standardized expression of the index analysis problem is used as output, so as to further adjust the model parameters of the low-rank adaptive layer of the model to be trained to obtain the problem rewriting model.

12. The method according to claim 1, characterized in that, After training the problem rewriting model based on each of the problem pairs and the corresponding indicator enhancement information, the method further includes: Obtain the target-standardized index analysis problem output by the problem rewriting model for the index analysis problem of the non-standardized expression of the target; The intent is identified in the standardized indicator analysis problem of the target, and a structured query statement is generated based on the intent identification result; The data is queried according to the structured query statement to obtain the query results, and response information is generated based on the query results to analyze the indicator problem of the non-standardized expression of the target.

13. A problem rewriting device, characterized in that, include: The acquisition module is used to acquire the non-standardized expression type of the indicator in the data analysis task, and construct the mapping relationship from the indicator to the non-standardized expression based on the non-standardized expression type of the indicator; A generation module is used to generate various question pairs based on the mapping relationship and the standardized expression of the indicator analysis question template. The question pairs include non-standardized expression indicator analysis questions and corresponding standardized expression indicator analysis questions. An enhancement module is used to perform indicator-based retrieval enhancement processing on the indicator analysis problem of the non-standardized expression to obtain indicator enhancement information; The training module is used to train the model to be trained based on each of the problem pairs and the corresponding indicator enhancement information to obtain a problem rewriting model. The problem rewriting model is used to rewrite the indicator analysis problem with a non-standardized expression of the target into an indicator analysis problem with a standardized expression of the target.

14. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the method of any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the processor of the electronic device, causes the electronic device to perform the method of any one of claims 1 to 12.

16. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, wherein a processor of an electronic device reads from and executes the computer program to cause the electronic device to perform the method of any one of claims 1 to 12.

Images