A structured data insight generation method, system and medium
By fine-tuning the open source natural language model, data positioning and insight generation model are generated, locations related to problems in the table are identified and reasoned facts are generated, and privacy leakage, high labor consumption and model inference errors are solved in data insight generation, and the problems of privacy leakage, high labor consumption and model inference errors are improved. Generation efficiency and accuracy.
Patent Information
- Application Number
- CN202411046420.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-01
AI Technical Summary
The existing data insight generation methods have the risk of privacy leakage, high labor costs and high probability of model inference errors, especially when numerical inference, model hallucination problems are serious.
By fine-tuning the open source natural language model, a data positioning model and insight generation model are generated. The data positioning model is used to identify and annotate the locations related to the problem from the table, and generate relevant reasoning facts from the template using pre-generated facts to reduce the pressure of manual labeling. And improve the interpretability and accuracy of model predictions.
It reduces the pressure of manual labeling during data collection, improves the accuracy and interpretability of model predictions, reduces the model hallucination problem during numerical reasoning, and improves the generation efficiency of structured data insights.
Smart Images

Figure CN118626494B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data analysis technology, and in particular to a structured data insight generation method, system and medium. Background Art
[0002] Data insight refers to obtaining deep insights and understanding by analyzing and interpreting data. Structured data insight generation refers to generating data insights using structured data (i.e. tables). The generation of data insights needs to be based on different user needs. Data insights are answers to questions raised by users, but this answer requires generating a paragraph-style text summary containing the corresponding facts and reasoning in the table based on the given table and user query.
[0003] With the development of technology, large language models (such as GPT-3.5) have excellent ability to reason from tables and can be used to generate simple data insights. There are two common methods for using large models to generate data insights. The first is to directly use large models trained by others (such as GPT4), which requires sending data to a third party and has the risk of privacy leakage. The second is to download the open source large model and deploy it locally, and use relevant data for fine-tuning, but collecting and annotating data sets requires a lot of manpower. Moreover, when directly using large models for end-to-end predictions, the reasoning generated by the large model (such as averaging, summing, maximum value, and sorting) has a high probability of error. Summary of the invention
[0004] The main purpose of the present invention is to provide a structured data insight generation method, system and medium, aiming to reduce the pressure of manual labeling when collecting data, improve the interpretability and accuracy of model predictions, and reduce the model hallucination problem of language models during reasoning (especially numerical reasoning).
[0005] In order to achieve the above object, the present invention proposes a method for generating structured data insights, the method comprising the following steps:
[0006] Step S10, after fine-tuning the open source natural language model using the corresponding table, question and answer datasets, a data location model and an insight generation model are obtained;
[0007] Step S20, the data location model identifies and marks the location related to the problem from the table;
[0008] Step S30, the data location model generates relevant inference facts from the template using a method of pre-generating facts for the table;
[0009] Step S40, input the annotated tables, questions and generated reasoning facts into the insight generation model, and the insight generation model gives the final structured data insights.
[0010] A further technical solution of the present invention is that step S10 includes a data location model training task and an insight generation model training task, wherein:
[0011] (1) Data positioning model training task:
[0012] Input: Enter the table T and question Q in the form of prompt(T,Q);
[0013] Output: Table T* marked with *;
[0014] prompt: refers to a piece of text that provides input to the AI model to guide it to produce a specific output.
[0015] (2) Insight generation model training task:
[0016] Input: Enter the table T* and question Q marked with * in the form of prompt(T*,Q), where T* is generated from the original table T according to the algorithm;
[0017] Output: Answer to the question ;
[0018] When training the data location model and the insight generation model, you need to first obtain the table T* indicated by * from the table T, question Q and annotated answer A.
[0019] A further technical solution of the present invention is that the step of obtaining a table T* marked with * from the table T, the question Q and the marked answer A comprises:
[0020] Let the original table be T, question Q and the marked answer be A. There are n rows in table T, and the row numbers are 1, 2, 3, ..., n;
[0021] Define the answer generated by the pre-trained model according to the i-th row and question Q as ,Answer The semantic similarity with the question annotation answer A is recorded as (0,1],i=1,2,3…,n;
[0022] Remember the collection (0,1],i=1,2,3…,n has the maximum value max and the minimum value min;
[0023] Determine whether the difference between the maximum value max and the minimum value min is less than 0.1;
[0024] If it is less than 0.1, then the table is marked as T*, which is the original table T;
[0025] If it is greater than or equal to 0.1, the algorithm is used to obtain the final problem corresponding area , and mark it in the original table T to obtain T*.
[0026] A further technical solution of the present invention is that in step S30, the method for pre-generating facts comprises the following steps:
[0027] Parse the type of each column in the table and classify all columns in the table into three categories: value, date, and string;
[0028] Generate a series of facts based on predefined reasoning templates and tables;
[0029] Vectorize the original question and generated facts;
[0030] Filter out the facts relevant to the problem;
[0031] The selected facts, questions and tables are input into the language model.
[0032] A further technical solution of the present invention is that the step of performing type analysis on each column of the table and classifying all the columns in the table into three categories of numerical values, dates and strings comprises:
[0033] Determine whether the type of each column in the table is a numeric type;
[0034] If yes, then determine whether the type of each column in the table is a value;
[0035] If not, determine whether the type of each column in the table is a date type;
[0036] If yes, determine the type of each column in the table is date;
[0037] If not, determine whether the type of each column in the table is a string.
[0038] A further technical solution of the present invention is that in the step of generating a series of facts according to predefined reasoning templates and tables, five templates are defined, and each template generates at least three facts.
[0039] A further technical solution of the present invention is that the step of vectorizing the original question and the generated facts comprises:
[0040] Vectorize facts in natural language and screen several facts.
[0041] A further technical solution of the present invention is that the step of screening out facts related to the problem comprises:
[0042] After obtaining the vectorized representation of the question and the fact, calculate the cosine similarity between each fact and the question, and select facts with the highest cosine similarity to the question, among which,
[0043] ,rows refers to the number of columns in the table, cols refers to the number of rows in the table, and [x] represents the largest integer not greater than x, that is, x is rounded down.
[0044] To achieve the above objectives, the present invention also proposes a structured data insight generation system, which includes a memory, a processor, and a structured data insight generation program stored on the processor, and the structured data insight generation program executes the steps of the method described above when run by the processor.
[0045] To achieve the above objectives, the present invention further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a structured data insight generation program, and the structured data insight generation program executes the steps of the method described above when executed by a processor.
[0046] The beneficial effects of the structured data insight generation method, system and medium of the present invention are:
[0047] Through the above technical solution, the present invention uses the corresponding tables, questions and answer data sets to fine-tune the open source natural language model to obtain a data location model and an insight generation model; the data location model identifies and marks the locations related to the questions from the table; the data location model uses the method of pre-generating facts for the table to generate relevant reasoning facts from the template; the annotated table, question and generated reasoning facts are input into the insight generation model, and the insight generation model gives the final structured data insights, which reduces the pressure of manual labeling when collecting data, improves the interpretability and accuracy of model predictions, and reduces the model hallucination problem of the language model during reasoning (especially numerical reasoning). BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 It is a flow chart of a preferred embodiment of the structured data insight generation method of the present invention;
[0049] Figure 2 It is a schematic diagram of the overall process of an example of the structured data insight generation method of the present invention;
[0050] Figure 3 It is an overview of the supermarket's monthly sales;
[0051] Figure 4 It is a schematic diagram of the table after finding the relevant area;
[0052] Figure 5 It is a schematic diagram of the data localization model training task;
[0053] Figure 6 It is a schematic diagram of the insight generation model training task;
[0054] Figure 7 It is a flow chart of the algorithm for generating the labeled area;
[0055] Figure 8 It is an overview of the sales of goods;
[0056] Fig. 9 It is an overview of the sales of the marked products;
[0057] Fig.10 This is a table diagram after the final area Zone* is marked;
[0058] Fig.11 It is a pre-generated fact flow chart;
[0059] Fig.12 It is a type resolution flow chart;
[0060] Fig.13 It is a schematic diagram describing the input and output of the insight generation model.
[0061] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. DETAILED DESCRIPTION
[0062] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0063] The present invention proposes a structured data insight generation method, please refer to Figure 1 The preferred embodiment of the structured data insight generation method of the present invention comprises the following steps:
[0064] Step S10, after fine-tuning the open source natural language model using the corresponding table, question and answer datasets, a data location model and an insight generation model are obtained.
[0065] Step S20: the data location model identifies and marks locations related to the problem from the table.
[0066] Step S30: the data location model generates relevant inference facts from the template using a method of pre-generating facts for the table.
[0067] Step S40, input the annotated tables, questions and generated reasoning facts into the insight generation model, and the insight generation model gives the final structured data insights.
[0068] Furthermore, in this embodiment, step S10 includes a data location model training task and an insight generation model training task.
[0069] Among them, (1) data positioning model training task:
[0070] Input: Enter the form T and question Q through prompt(T,Q);
[0071] Output: Table T* marked with *.
[0072] (2) Insight generation model training task:
[0073] Input: Enter the table T* and question Q marked with * through prompt(T*,Q), where T* is generated from the original table T according to the algorithm;
[0074] Output: Answer to the question .
[0075] When training the data location model and the insight generation model, you need to first obtain the table T* indicated by * from the table T, question Q and annotated answer A.
[0076] Furthermore, in this embodiment, the step of obtaining a table T* marked with * from the table T, question Q and the marked answer A includes:
[0077] Let the original table be T, question Q and the marked answer be A. There are n rows in table T, and the row numbers are 1, 2, 3, ..., n;
[0078] Define the answer generated by the pre-trained model according to the i-th row and question Q as ,Answer The semantic similarity with the question annotation answer A is recorded as (0,1],i=1,2,3…,n;
[0079] Remember the collection (0,1],i=1,2,3…,n has the maximum value max and the minimum value min;
[0080] Determine whether the difference between the maximum value max and the minimum value min is less than 0.1;
[0081] If it is less than 0.1, then the table is marked as T*, which is the original table T;
[0082] If it is greater than or equal to 0.1, the algorithm is used to obtain the final problem corresponding area , and mark it in the original table T to obtain T*.
[0083] Furthermore, in this embodiment, in step S30, the method for pre-generating facts includes the following steps:
[0084] Perform type analysis on each column in the table and classify all columns in the table into three categories: value, date, and string.
[0085] A series of facts are generated according to the predefined reasoning templates and tables. In this embodiment, five templates are defined, and each template generates at least three facts.
[0086] Vectorize the original question and the generated facts.
[0087] Filter out the facts that are relevant to the question.
[0088] The selected facts, questions and tables are input into the language model.
[0089] The step of performing type analysis on each column of the table and classifying all columns in the table into three categories: value, date and string includes:
[0090] Determine whether the type of each column in the table is a numeric type.
[0091] If so, determine whether the type of each column in the table is a value.
[0092] If not, determine whether the type of each column in the table is a date type.
[0093] If so, determine whether the type of each column in the table is a date.
[0094] If not, determine whether the type of each column in the table is a string.
[0095] The step of vectorizing the original question and the generated facts comprises:
[0096] Vectorize facts in natural language and screen several facts.
[0097] The steps of filtering out facts related to the problem include:
[0098] After obtaining the vectorized representation of the question and the fact, calculate the cosine similarity between each fact and the question, and select facts with the highest cosine similarity to the question, among which,
[0099] ,rows refers to the number of columns in the table, cols refers to the number of rows in the table, and [x] represents the largest integer not greater than x, that is, x is rounded down.
[0100] The following examples are combined Figures 2 to 13 The structured data insight generating method of the present invention is further elaborated.
[0101] The overall flow chart of the structured data insight generation method of the present invention is as follows: Figure 2 shown.
[0102] After fine-tuning the open-source natural language model using the corresponding tables, questions, and answer datasets, we obtained the data location model and insight generation model. Medium Problem The area is located and marked with * in the table. Then, use the pre-generated facts method to generate relevant inference facts from the template. Finally, the annotated table ,question After being input into the insight generation model together with the generated inference facts, the insight generation model gives the final structured data insights.
[0103] After the model is fine-tuned, the data location model and the insight generation model collaborate to complete the final structured data insight generation task. The following example illustrates their collaboration process.
[0104] like Figure 3 The table shown shows the best-selling products in a supermarket in a month, including product name, product category, sales and profit. Different types of products are classified into different categories, such as groceries, electronic products, clothing, shoes and hats, household goods, and cosmetics and skin care products. The table lists the sales and profit of each product in yuan. The user's Query is: "Which product classified as household goods has the highest profit? How does it compare with the product with the highest profit in all categories?"
[0105] First, the fine-tuned data location model locates the rows of furniture and home decoration based on the keyword "home furnishings" in the entire table. Then add an asterisk to the cell contents in these rows to get the table Table* Figure 4 shown.
[0106] Then, yes Figure 3 The table in the example uses the pre-generated fact method to obtain the relevant facts, as follows:
[0107] "1. The names of the products sorted from high to low by profit (yuan) are TV, mobile phone, grain and oil, cosmetics... shirts
[0108] 2. After sorting by profit (yuan) from high to low, the order of commodity classification of household goods is furniture, home decoration
[0109] 3. The average profit (RMB) of products that meet the household goods category is 5,000
[0110] …
[0111] 6. The difference between 7890 and 20678 in profit (yuan) is
[15234] "
[0112] Finally, the relevant facts, the table with * and the user question Query are input into the insight generation model, and the insight generation model generates the answer. The above table Table* and the user question Query are input into the insight generation model separately, and the model output is "Among the products classified as household goods, the one with the highest profit is "home decoration", with a profit of 7,890 yuan. Compared with the product with the highest profit in all categories, that is, "mobile phone" in electronic products (profit of 15,234 yuan), it can be seen that the profit of "mobile phone" is higher than that of "furniture", with a difference of 7,344 yuan."
[0113] In the generated answer above, the insight generation model mistakenly identified the product with the highest profit as a “mobile phone.”
[0114] After the above table, user questions, and generated facts are input into the insight generation model, the model output is: "Among the products classified as household goods, the one with the highest profit is "home decoration", with a profit of 7,890 yuan. Compared with the product with the highest profit in all categories, that is, "TV" in electronic products (profit of 20,678 yuan), it can be seen that the profit of "mobile phone" is higher than that of "furniture", with a difference of 12,788 yuan." The model corrects this problem through the input facts.
[0115] The structured data insight generation method of the present invention mainly involves three aspects: data construction method of fine-tuning model, pre-generated fact input strategy and arrangement method between models.
[0116] (1) Data construction method for fine-tuning the model. In the process of model fine-tuning, a large amount of data usually needs to be labeled, which requires a lot of manpower and time. The present invention proposes an algorithm for automatically generating answer-related areas based on tables, questions and answers to reduce the pressure of manual labeling.
[0117] a. Data location model training tasks, for example Figure 5 Shown:
[0118] Input: Enter the form T and question Q through prompt(T,Q);
[0119] Output: Table T* marked with *.
[0120] b. Insight generation model training tasks, for example Figure 6 Shown:
[0121] Input: Enter the table T* and question Q marked with * through prompt(T*,Q), where T* is generated from the original table T according to the algorithm;
[0122] Output: Answer to the question .
[0123] During training, we need to first obtain the table T* indicated by * from the table T, question Q and the annotated answer A. The method for obtaining T* is as follows: Figure 7 shown.
[0124] First, suppose there are n rows in the original table T, and their row numbers are 1, 2, 3, ..., n starting from the first row (excluding the header). Define the pre-trained model According to The answer generated by the row and question Q is ,Answer The semantic similarity with the question annotation answer A is recorded as (0,1],i=1,2,3…,n. The pre-trained model here can be any model with table question answering capabilities.
[0125] by Figure 3 For example, it has 11 rows. The user's query is "Which product classified as household goods has the highest profit? How does it compare to the product with the highest profit in all categories?". The marked answer A is "Among the products classified as household goods, the one with the highest profit is furniture, with a profit of 7,890 yuan. However, the profit of television is 20,678 yuan, which is higher than that of furniture. In fact, the profit of television is 12,788 yuan higher than that of furniture, which is nearly twice the profit of furniture."
[0126] The pre-trained model generates the answer based on the table header plus the content of the third row "Product Name Product Category Sales (RMB) Profit (RMB) | Snacks Food Groceries 43,567 9012" and the question Query "In this table, there is only one row of data, which shows that the sales of snacks are 43,567 yuan and the profit is 9,012 yuan. Since there is only one row of data, it is impossible to determine which product classified as household goods has the highest profit." Semantic similarity with the query .
[0127] The data insight generation model generates the answer based on the table header plus the 4th row "Product Name Product Category Sales (RMB) Profit (RMB) | TV Electronics 52,345 20678" and the question Query "In this table, there is only one row of data, that is, the sales of TVs are 52,345 yuan and the profit is 20,678 yuan. Since there is only one row of data, it is impossible to determine which product classified as household goods has the highest profit.", Semantic similarity with the query .
[0128] Similarly, we can get the pre-trained model generated according to each line of content , their semantic similarity with the question Query They are {0.3, 0.3, 0.64, 0.6, 0.3, 0.3, 0.3, 0.64, 0.6, 0.3, 0.3}, .
[0129] if , that is, the answer generated by each row and question Query If the difference between the maximum and minimum values of the semantic similarity with answer A is less than 0.1, the entire table is considered to be a related area and each row is marked with an *.
[0130] Take the following example to illustrate, Figure 8 Table shown , the query is "Which product classified as household goods has the highest profit? How does it compare to the product with the highest profit in all categories?". The marked answer A is "Among the products classified as household goods, the one with the highest profit is furniture, with a profit of 7,890 yuan. However, the profit of television is 20,678 yuan, which is higher than that of furniture. In fact, the profit of television is 12,788 yuan higher than that of furniture, which is nearly twice the profit of furniture.
[0131] Answers generated by the pre-trained model The semantic similarity with the marked answer A is {0.64, 0.64, 0.6}, and the difference between the maximum and minimum values is 0.64-0.6=0.04<0.1, so add * to all the cells in the table to get the table *,like Fig. 9 shown.
[0132] if , then use the search algorithm shown in Table 1 below to filter out the relevant areas.
[0133] Table 1 Related area algorithm description table
[0134]
[0135] The algorithm for generating the final region is shown in the following example. Figure 3 For the table Table, as well as the above question Query and the marked answer A, please refer to Figure 3 .
[0136] The pre-trained model generates , their semantic similarity with question Q They are {0.3, 0.3, 0.34, 0.6, 0.3, 0.3, 0.3, 0.64, 0.64, 0.3, 0.3}, . After sorting, {0.64,0.64,0.6,0.3,0.3,0.3,0.3,0.3,0.3,0.3,0.3}, the corresponding row number sequence is {8,4,9,1,2,3,5,6,7,10,11}, these rows are , ,…, ,for example It is line 8, It is line 4.
[0137] Marked area Initially, only the table header "Product Name Product Category Sales (RMB) Profit (RMB)" is included, and the pre-trained model is used. The pre-trained model uses the table header and the question Query to generate the answer. "Due to a lack of specific product data, it is not possible to determine which products in the home furnishings category have the highest profit margins. Similarly, it is not possible to compare the highest profit margin products in home furnishings with the highest profit margin products in all categories."
[0138] Use of pre-trained models That is, "Product Name Product Category Sales (RMB) Profit (RMB) | Furniture Household Goods 53,789 7,890" and the question Query generate the answer ,Answer It is "In this table, among the products in the household goods category, the one with the highest profit is furniture, with a profit of 7,890 yuan. Compared with the product with the highest profit in all categories, that is, the profit of furniture, it is the same as itself, because furniture is the product with the highest profit in household goods.".
[0139] Calculated The semantic similarity with the marked answer A is 0.2. The semantic similarity with the marked answer A is 0.64, 0.64>0.2, so the 8th row of table T is added to the region In .
[0140] Next, the pre-trained model uses the current Region, i.e. "Product Name Product Category Sales (RMB) Profit (RMB) | Furniture and Household Goods 53,789 7,890 | Television Electronic Products 52,345 20678", Question Query Generates Answer "In the table provided, furniture is the most profitable item in the household goods category, with a profit of $7,890. Television is the most profitable item in all categories, with a profit of $20,678. Therefore, the profit of television is higher than that of furniture, with a difference of $12,788." Semantic similarity with the marked answer A is 0.9, 0.9>0.64, so add the 4th row of table T to the region In .
[0141] Generate The process is similar, using Generated Yes "In the table provided, among the items classified as household goods, the most profitable item is furniture, with a profit of 7,890 yuan. And among all categories, the most profitable item is television, with a profit of 20,678 yuan. Therefore, the profit of television is higher than that of furniture, with a difference of 12,788 yuan." Semantic similarity ,so Remain unchanged.
[0142] By analogy, we finally get yes{ }, marked in the figure, such as Fig.10 shown.
[0143] (2) Pre-generated fact input strategy. In the process of the model generating text based on tables and questions, the generated content is often inconsistent with the actual information in the table, especially when numerical operations are involved. Therefore, this method generates actual facts in advance and uses them as input to the model in the prediction stage.
[0144] like Fig.11 As shown in the figure, the method of pre-generating facts can be broken down into five steps. First, all columns in the table are divided into three categories: values, dates, and strings; then, a series of facts are generated based on the predefined reasoning template and table; then, the original question and the generated facts are vectorized; then, the facts related to the question are filtered out; finally, the filtered facts, question, and table are input into the language model together.
[0145] First, perform type analysis on each column of the table. The type analysis defined in this article refers to classifying each column in the string form into three categories: value, date, and string. The type analysis process for each column is as follows: Fig.12 shown.
[0146] by Figure 3Take the table in the example to illustrate the process of type resolution. The table shows the best-selling commodities in a supermarket in a month, including information about commodity name, commodity category, sales volume and profit. The table lists the sales volume and profit of each commodity in yuan.
[0147] After parsing the four columns, the corresponding numeric types for the columns named {Product Name, Product Category, Sales (Yuan) and Profit (Yuan)} are {String, String, Numeric, Numeric}.
[0148] After the type resolution is completed, the second step is to generate several facts based on the reasoning template. A total of 5 templates are defined, and each template generates at least 3 facts. The fact templates defined in this article are shown in Table 2 below. Each fact template includes several symbols that need to be filled with information retrieved from the table. Multiple facts can be generated based on the fact template. [Operator] can be [sum] operation, [average] operation, [difference] operation of two numbers, [maximum] and [minimum] operation, which means to find the [rate of change] of two values (assuming that the two numbers are a and b, and the rate of change of b relative to a is ba / a).
[0149] The type parsed in the previous step is used to generate [Condition]. [Condition] (1) When the column is a string type, it is a statement of string inclusion. For example, [the month contains the word "April"] (2) When the column is a numeric or date type, it is a statement of numeric or date comparison. (For example, the score is greater than 93 or the creation time is later than 1993). [Execution result] is the value of some elements in a column that meet the [Condition] or the result after the [Operator] is executed.
[0150] Table 2 Fact template definition table
[0151]
[0152] For example, in "Connect", the template is "[column name] of [condition] is [execution result]". Figure 7 In the table T in the example, [Column Name] is filled with "Product Name", and [Condition] is selected as the "Profit (Yuan)" column of the numeric type, and is filled with the numeric comparison "Profit (Yuan) exceeds 13789". [Execution Result] is filled with the value in the "Product Name" column that satisfies "Profit (Yuan) exceeds 13789". The final fact is "The product name with profit (Yuan) exceeding 13789 is [TV, mobile phone].".
[0153] In "Numeric Operation (Total, Average)", "[Operator] of [Column Name 1] of [Column Name 2] of [Condition 2] is [Execution Result]." The filled result is "[Profit (Yuan)] of [Product Category] [Is Household Products] [Average Value] is
[5673] ".
[0154] The third step is to vectorize the facts in natural language form and screen several facts. To directly compare the similarity of two natural sentences, the common practice in the field of NLP is to use the Embedding model to convert the two natural sentences into two vectors, and then calculate their cosine values. For each table, this article converts the generated natural language facts and natural language questions into vectors using the same Embedding model. This Embedding model is not restricted. The Embedding model refers to a model that maps high-dimensional data (such as text, pictures, videos) to a low-dimensional space (vector).
[0155] For example, the fact "Years with more than 33 matches are 1975, 1977, 1978, 1979, 1980, 1981, 1972, 1973, 1974, 1976." is converted into an m-dimensional vector by the AI model. .
[0156] The fourth step is to calculate the cosine similarity between each fact and question after obtaining the vectorized representation of the question and fact. The facts with the highest cosine similarity to the question. , rows refers to the number of columns in the table, cols refers to the number of rows in the table, and [x] represents the largest integer not greater than x, that is, x is rounded down. Suppose two m-dimensional vectors for , the cosine similarity of x and y is , and These two m-dimensional vectors are converted by the embedding model based on two different natural texts. and The cosine similarity value represents the semantic proximity of the two natural texts. Indicates yes The word vector bit element, Representing word vectors No. Bit element.
[0157] For example, the original table has 11 rows and 4 columns. , 20 facts are generated using the above template. So finally, the top 5 facts with the highest cosine similarity to the question are selected from these 20 facts.
[0158] Finally, the table marked with *, the question, and the k facts most similar to the question are input into the insight generation model in the form of prompts in natural language to obtain the final answer, such as Fig.13shown.
[0159] (3) A method of arranging models. The present invention introduces a method of arranging models, which includes two models: one model is responsible for marking the areas related to the question with * in the table, and the other model is responsible for asking and answering questions based on the marked table. This strategy divides the process of the model answering table questions into two steps: information search and reasoning. Traditional methods mainly improve the information search and reasoning capabilities of the model during the training phase. During the prediction phase, the model still implicitly performs these two steps. However, this new training strategy allows the model to explicitly perform these two processes step by step during the prediction phase, ensuring that the model pays more attention to information related to the question during the generation process, thereby improving the accuracy of the generated results. This makes the reasoning process of the model more transparent and improves the interpretability of the model.
[0160] Models are arranged as processes Figure 2 As shown, the following example illustrates how the data location model and insight generation model work together:
[0161] like Figure 3 As shown in the figure, the table shows the best-selling products in a supermarket in a month, including product name, product category, sales and profit. Different types of products are classified into different categories, such as groceries, electronic products, clothing, shoes and hats, household goods, and cosmetics and skin care products. The table lists the sales and profit of each product in yuan. The user's question Query is: "Which product classified as household goods has the highest profit? How does it compare with the product with the highest profit in all categories?"
[0162] First, the data location model locates the rows of furniture and home decoration based on the keyword "home furnishings" in the entire table. Then add an asterisk in front of the cell contents in these rows, such as Figure 4 shown.
[0163] Then, yes Figure 3 The table in uses the pre-generated fact method described in the previous summary to obtain relevant facts.
[0164] “1. The [product names] sorted from high to low by [profit (yuan)] are [TV, mobile phone, grain and oil, cosmetics…shirts],
[0165] 2. After sorting by [Profit (Yuan)] from high to low, the order of [Companies] whose product category is household items is furniture, then home decoration.
[0166] 3. The average value of [profit (yuan)] for [product category] [household goods] is 5000
[0167] 4…”
[0168] The above relevant facts, the *-edged table and the questions are input into the insight generation model, and finally the insight generation model generates data insights, such as Fig.13 shown.
[0169] The key points of the structured data insight generation method of the present invention are:
[0170] 1. A dataset annotation generation algorithm: In the process of model fine-tuning, a large amount of data usually needs to be annotated, which requires a lot of manpower and time. The present invention proposes an algorithm that automatically generates answer-related areas based on tables, questions and answers to reduce the pressure of manual annotation.
[0171] 2. Pre-generated fact input strategy: In the process of the model generating text based on tables and questions, the generated content is often inconsistent with the actual information in the table, especially when numerical operations are involved. Therefore, this method generates the actual facts in advance and uses them as input to the model in the prediction stage.
[0172] 3. Model orchestration strategy: This strategy contains two models: one model is responsible for marking the areas related to the question with * in the table, and the other model is responsible for answering questions based on the marked table. This strategy divides the process of the model answering table questions into two steps: information search and reasoning. Traditional methods mainly improve the information search and reasoning capabilities of the model during the training phase. During the prediction phase, the model still performs these two steps implicitly. However, this new training strategy enables the model to explicitly perform these two processes step by step during the prediction phase, ensuring that the model pays more attention to information related to the question during the generation process, thereby improving the accuracy of the generated results. It also makes the reasoning process of the model more transparent and improves the interpretability of the reasoning process.
[0173] The following is an explanation of the terms involved in the structured data insight generation method of the present invention.
[0174] Natural Language Model: An AI model used to generate natural sentences.
[0175] Model hallucination: The phenomenon where an AI model generates information that is inconsistent with reality. In table Q&A, it refers to the phenomenon where the model generates information that is inconsistent with the facts in the table. For example, the average of several numbers in the table is 10, but the text generated by the model says that their average is 9.87.
[0176] Training corpus: Text data used to train natural language models.
[0177] Pretrained Model: A model that has been trained to perform a specific task on a large amount of data.
[0178] Model fine-tuning: refers to continuing to train an existing pre-trained model to improve the model's capabilities in certain aspects.
[0179] End-to-end: End-to-end means that the output result is directly obtained from the input data. In this article, it means that the model generates answers directly from the input table and questions.
[0180] Non-end-to-end: corresponds to multi-step problem solving, that is, splitting a problem into multiple steps and solving them step by step.
[0181] The beneficial effects of the structured data insight generation method of the present invention are:
[0182] Through the above technical solution, the present invention uses the corresponding tables, questions and answer data sets to fine-tune the open source natural language model to obtain a data location model and an insight generation model; the data location model identifies and marks the locations related to the questions from the table; the data location model uses the method of pre-generating facts for the table to generate relevant reasoning facts from the template; the annotated table, question and generated reasoning facts are input into the insight generation model, and the insight generation model gives the final structured data insights, which reduces the pressure of manual labeling when collecting data, improves the interpretability and accuracy of model predictions, and reduces the model hallucination problem of the language model during reasoning (especially numerical reasoning).
[0183] To achieve the above objectives, the present invention also proposes a structured data insight generation system, which includes a memory, a processor, and a structured data insight generation program stored on the processor. When the structured data insight generation program is run by the processor, the steps of the method described in the above embodiment are executed, which will not be repeated here.
[0184] To achieve the above objectives, the present invention also proposes a computer-readable storage medium, which stores a structured data insight generation program. When the structured data insight generation program is executed by a processor, the steps of the method described in the above embodiment are executed, which will not be repeated here.
[0185] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A structured data insight generation method, characterized in that: The method comprises the following steps: Step S10, after fine-tuning the open source natural language model using the corresponding table, question and answer datasets, a data location model and an insight generation model are obtained; Step S20, the data location model identifies and marks the location related to the problem from the table; Step S30, the data location model generates relevant inference facts from the template using a method of pre-generating facts for the table; Step S40, inputting the annotated table, questions and generated reasoning facts into the insight generation model, and the insight generation model gives the final structured data insights; The step S10 includes a data location model training task and an insight generation model training task, wherein: (1) Data positioning model training task: Input: Enter the table T and question Q in the form of prompt(T,Q); Output: Table T* marked with *; (2) Insight generation model training task: Input: Enter the table T* and question Q marked with * in the form of prompt(T*,Q), where T* is generated from the original table T according to the algorithm; Output: Answer to the question ; When training the data location model and insight generation model, you need to first obtain the table T* indicated by * from the table T, question Q and annotated answer A; The step of obtaining a table T* marked with * from the table T, question Q and the marked answer A comprises: Let the original table be T, question Q and the marked answer be A. There are n rows in table T, and the row numbers are 1, 2, 3, ..., n; Define the answer generated by the pre-trained model according to the i-th row and question Q as ,Answer The semantic similarity with the question annotation answer A is recorded as (0,1],i=1,2,3…,n; Remember the collection (0,1],i=1,2,3…,n has the maximum value max and the minimum value min; Determine whether the difference between the maximum value max and the minimum value min is less than 0.1; If it is less than 0.1, then the table is marked as T*, which is the original table T; If it is greater than or equal to 0.1, the algorithm is used to obtain the final problem corresponding area , and mark it in the original table T to obtain T*; In step S30, the method for pre-generating facts includes the following steps: Parse the type of each column in the table and classify all columns in the table into three categories: value, date, and string; Generate a series of facts based on predefined reasoning templates and tables; Vectorize the original question and generated facts; Filter out the facts relevant to the problem; Input the selected facts, questions and tables into the language model; The step of performing type analysis on each column of the table and classifying all columns in the table into three categories: value, date and string includes: Determine whether the type of each column in the table is a numeric type; If yes, then determine whether the type of each column in the table is a value; If not, determine whether the type of each column in the table is a date type; If yes, determine the type of each column in the table is date; If not, determine whether the type of each column in the table is a string.
2. The structured data insight generation method according to claim 1, characterized in that: In the step of generating a series of facts according to the predefined reasoning templates and tables, five templates are defined, and each template generates at least three facts.
3. The structured data insight generation method according to claim 1, characterized in that: The step of vectorizing the original question and the generated facts comprises: Vectorize facts in natural language and screen several facts.
4. The structured data insight generation method according to claim 1, characterized in that: The steps of filtering out facts related to the problem include: After obtaining the vectorized representation of the question and the fact, calculate the cosine similarity between each fact and the question, and select facts with the highest cosine similarity to the question, among which, ,cols refers to the number of columns in the table, rows refers to the number of rows in the table, and [x] represents the largest integer not greater than x, that is, x is rounded down.
5. A structured data insight generation system, characterized in that: The system includes a memory, a processor, and a structured data insight generation program stored on the processor, wherein the structured data insight generation program executes the steps of the method according to any one of claims 1 to 4 when executed by the processor.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a structured data insight generation program, and the structured data insight generation program, when executed by a processor, performs the steps of the method described in any one of claims 1 to 4.
