A prompt-based table fact detection method
By transforming the table fact detection task into a masked word prediction task using the prompt method, and leveraging the word prediction capabilities of the pre-trained model, the high resource consumption problem in existing technologies is solved, and efficient detection is achieved under small or zero sample conditions.
Patent Information
- Application Number
- CN202211402592.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-10
- Publication Date
- 2026-01-06
- Estimated Expiration
- 2042-11-10
AI Technical Summary
In existing table fact detection tasks, fine-tuning methods based on pre-trained models may disrupt the original network of the model, and the large number of parameters leads to high resource consumption, making it difficult to apply effectively in cases with small or zero samples.
The prompt method is adopted to transform the table fact detection task into a masked word prediction task. By defining the prompt template and mapping relationship, the word prediction ability of the pre-trained model is directly utilized, avoiding retraining or requiring only a small part of training, and the pre-trained model is used directly for prediction.
In cases with small or zero samples, it improves the judgment effect of table fact detection, reduces resource consumption, and avoids the negative impact of model retraining.
Smart Images
Figure CN115618004B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of natural language processing technology, and in particular relates to a table fact detection method based on prompt. Background Technology
[0002] In the field of natural language processing, there is a task involving tables called tabular fact detection. Tabular fact detection is a classification task that requires you to determine whether a statement is true based on the content of a table, given a textual statement and an associated table.
[0003] Existing methods for tabular fact detection tasks typically rely on fine-tuning of pre-trained models, which can be roughly described as follows:
[0004] 1. Train a pre-trained model using a large amount of text, tables and other related corpora. This model will learn some general representations of tables and text.
[0005] 2. Apply the pre-trained model to the table fact detection task. The usual approach is to input the table and related text statements into the pre-trained model, obtaining a vector representation of the table and text. This vector will contain the knowledge learned by the model during the pre-training phase.
[0006] 3. After obtaining the vector representations of the tables and text output by the pre-trained model, use the vectors to continue the table fact detection task. Finally, use the labels of the corresponding data in the dataset for supervised learning, calculate the loss, and use the loss to fine-tune the model network.
[0007] 4. Obtain the fine-tuned model and use it to predict the table fact detection task.
[0008] However, the above implementation process has the following drawbacks:
[0009] 1. The pre-trained model already contains common features of tables and text. Inappropriate fine-tuning may damage the original network of the model.
[0010] 2. The approach of fine-tuning the pre-trained model requires retraining on the downstream task. Since the number of parameters of the pre-trained model is generally large, this overhead will be unacceptable. Summary of the Invention
[0011] To address the aforementioned issues, this invention proposes a prompt-based table fact detection method. For table fact detection tasks, it adopts the prompt method's processing mode and optimizes the use of pre-trained models. This allows for good results even with small or zero sample sizes, without requiring significant resources for retraining; the pre-trained model alone can achieve satisfactory results.
[0012] The technical solution of this invention is as follows:
[0013] A prompt-based method for detecting tabular facts includes the following steps:
[0014] S1. Process the dataset used for tabular fact detection to obtain the first data, which is in the format of (table, text, label).
[0015] S2. Extract table data from the first data, flatten it, and use it as the second data;
[0016] S3. Extract text data from the first data and input it into the prompt module to obtain the third data;
[0017] S4. After concatenating the second and third data, input them into the pre-trained prediction model to obtain the prediction result;
[0018] S5. Using the obtained prediction results and the mapping relationship, obtain the table fact detection results.
[0019] Furthermore, the specific method of step S3 is as follows: by defining a prompt template, the table fact detection task is transformed into a masking word prediction task; according to the defined prompt template, the input text is modified by injecting the corresponding prior knowledge of the table fact detection task.
[0020] Furthermore, the specific method of step S5 is as follows: First, the training data is vectorized and processed by the pre-trained model to obtain a hidden state containing table and text information. The corresponding masked word position will also have a hidden layer vector. Softmax is applied to the hidden layer vector to obtain the probability of which words may be at that position. All probabilities of a certain class are summed, and the class with the largest sum is found based on the mapping relationship, which is the corresponding class.
[0021] The mapping relationship is established by manually creating the mapping relationship based on the mutual exclusion relationship, and then continuously adding words to the mapping table based on the similarity relationship between words until adding more word groups no longer improves the effect.
[0022] The beneficial effects of this invention are as follows: Compared with traditional methods, this invention is the first to use the Prompt method on a table fact detection task, which effectively improves the judgment effect in the case of small sample or zero sample. Attached Figure Description
[0023] Figure 1 This invention relates to a table-based fact detection task graph.
[0024] Figure 2 A schematic diagram of the table pre-training model designed for this invention.
[0025] Figure 3 This is a prompt template diagram for the present invention.
[0026] Figure 4 A mapping module diagram is designed for this invention. Detailed Implementation
[0027] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0028] Table fact detection task such as Figure 1 As shown, the goal is to provide a table and several related statements, and for each statement, determine whether it is true or false, with results categorized as "entailed" (true) or "refuted" (false). However, current practices typically involve fine-tuning a pre-trained model, as shown in... Figure 2 As shown, the [CLS] vector is first obtained through a pre-trained model, and then used to continue the table fact detection task. However, this usually requires retraining the pre-trained model and additional network structures, while the Prompt method does not require retraining or only requires a small portion of training. Prompt adopts a task modification approach, transforming the downstream task into a structure that is the same as or similar to the pre-trained model's task, and directly obtaining the final result through the pre-trained model's built-in word prediction ability.
[0029] Example:
[0030] This example specifically includes the following steps:
[0031] S1. Construct a data reading module to read the corresponding table, text, and label data from the dataset and process them into several training data sets (table, text, label). Since the TabFact dataset for table fact detection usually provides data in the format of (a table, several statements, several labels), but in order to facilitate the prompting of the input table, it is processed into the form of (a table, a statement, and a label).
[0032] S2. Extract tables from the training data, flatten them, and use them as input to the pre-trained model, i.e., [table]. This is because during prompting, only the text statement part needs to be modified, and the table part does not need to be processed.
[0033] S3. Extract text from the training data as input to the prompt module. The prompt module provides relevant table fact detection task information as prompts for the text statements, which are usually described as "[statement]" or "[query]". The prompt module transforms the extracted text and outputs the second input to the pre-trained model. Specifically, the prompt process involves adding prompts to the input statement text, which is in simple "[statement]" form, and reserving masked words for the pre-trained model to predict. The specific method is as follows: First, define several prompt templates to transform the table fact detection task into a masked word prediction task (the pre-training task used by the pre-trained model); according to the transformation templates in the prompt module, transform the input statement text and inject the corresponding prior knowledge of the table fact detection task; the resulting statement text with added table fact detection task information is then used as input to the pre-trained model. Figure 3 These are some prompt templates. Taking [query][the statement was___] as an example, "the statement" is the prompt injected into the text statement, and the empty space is the masked word. The pre-trained model will predict the word at that position.
[0034] S4. Extract features from the table and text using a pre-trained model. Subsequent features are used for table fact detection. Specifically, the text, prompt, and table are concatenated to form the input format [statement][prompt][table]. These are then input into the pre-trained model for vectorization using word2vec. This vector is then processed by the pre-trained model to obtain a hidden state containing table and text information. The masked word position corresponding to [prompt] will also have a hidden layer vector of vocab_size*hidden_state. Applying softmax to this hidden layer vector will give the probability of which words might be at that position.
[0035] S5. Using the predicted words, the mapping module is used to obtain the final table fact detection result. At this point, it is only necessary to sum all the probabilities of a certain class according to the correspondence of the mapping processing module. The class with the largest sum is the class of the corresponding [statement]. Figure 4 It is an instance corresponding to the mapping relationship. The predicted word "wrong" corresponds to the "refuted" tag, and the predicted word "right" corresponds to the "entailed" tag.
[0036] After prompting the input text and table, the data can be directly fed into the pre-trained model for prediction, as the pre-trained model used in this invention is specifically designed for word prediction. Since the word prediction result from the pre-trained model is the probability of all words in the vocabulary, a mapping module is needed to associate the predicted words with the table fact detection classification results.
[0037] The mapping module primarily associates predicted words with classification results, summing the predicted probabilities of one or more words as the classification result. The mapping module specifically includes:
[0038] Manually create mapping relationships based on mutual exclusion, such as the word "wrong" -> "entailed" label, and the word "right" -> "refuted" label. Initialize the mapping table to group 1.
[0039] By continuously adding words to the mapping table based on their similarity relationships, the effect continues to improve when adding more word groups no longer improves the result.
[0040] The mapping module has been established and can be used for subsequent prompt training and prediction.
[0041] The method of this invention firstly transforms the form of the tabular fact detection task to be consistent with the pre-trained tabular model, so that no further training is required on the basis of the pre-trained model; secondly, it defines a multi-set mapping relationship, which can conveniently convert the prediction results of the prompt into tabular fact detection results.
Claims
1. A prompt-based table fact detection method, characterized in that, The method comprises the following steps: S1, processing a data set for table fact detection to obtain first data, the format of the first data being (table, text, label); S2, extracting table data from the first data, and flattening the table data to obtain second data; S3, extracting text data from the first data and inputting the text data into a prompt module to obtain third data, the specific method being: defining a prompt template, converting the table fact detection task into a masked word prediction task; and according to the defined prompt template, modifying the input text by injecting prior knowledge of the table fact detection task; S4, concatenating the second data and the third data and inputting the second data and the third data into a pre-trained prediction model to obtain a prediction result; S5, obtaining a table fact detection result according to a mapping relationship by using the obtained prediction result, the specific method being: first, vectorizing the training data, processing the vectorized training data by using the pre-trained model to obtain a hidden state containing table and text information, and a hidden layer vector corresponding to the position of the masked word, performing softmax on the hidden layer vector to obtain the probability that the position is a certain word, summing the probabilities of all words in a certain category, and finding the category of the maximum value based on the mapping relationship, so as to obtain the corresponding category; The mapping relationship is established in the following manner: manually creating the mapping relationship according to the mutual exclusion relationship, and constantly adding words to the mapping table according to the similarity relationship between the words until the effect of adding the words is not improved.