Open world sparse corpus-oriented few-sample attribute extraction method and system

Abstract

The invention relates to a few-sample attribute extraction method and system for open world sparse corpora. According to the method, firstly, an unsupervised meta-mode extraction method is applied to guide training from a sequence to a sequence generation model, so that the sparsity of a training set is reduced; then, designing a'filtering after generation 'process, so as to extract all valuable attribute triples from the input corpus; beam search is used to obtain a plurality of candidates, and a high quality attribute triad is selected from the candidates by a screening module. In addition, the invention further provides a reference data set named as OSN-515 so as to assist in evaluating the effect of extracting the open world few-sample attributes from the sparse corpus. Experimental results show that the framework is obviously superior to a baseline framework, and ablation research proves the effectiveness of each part of the model.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a method and system for extracting a small number of attributes from open-world sparse corpus. Background Art

[0002] Open-world attribute extraction is one of the important tasks of information extraction, which aims to automatically identify and extract all valuable attributes and corresponding values ​​of an entity from natural text. It is very useful in highlighting the key attributes of an entity and has made significant contributions to downstream tasks such as knowledge graph construction and recommendation systems. Attribute triples are valuable if the attribute name can describe the intrinsic properties of the entity and conform to the construction of attributes in the knowledge graph. The attribute information of an entity enriches the description of the entity itself and the understanding of the human perspective. Compared with relation extraction, which aims to extract the semantic relationship between entity pairs in text, the particularity of attribute extraction lies in the multiple forms of attribute values. Attribute values ​​exist not only in the form of entities, but also in the form of numbers, phrases, and special tokens, which means that attribute extraction is not easy to treat as a classification task similar to relation extraction. One of the most common forms of attribute extraction is to extract attribute triples (entity, attribute, value) and their corresponding attribute values ​​describing entity attributes from natural language text. In this work, since the goal of open-world extraction is to obtain all valuable attribute information for each possible entity, precise attribute names are not required.

[0003] Most cutting-edge work mainly implements attribute extraction through sequence labeling methods based on neural networks. However, these methods can only extract attribute triplets when the attribute names are given or constrained to a fixed set, while in a large number of practical situations, the attribute names are always uncertain. This leaves a non-negligible gap between these methods and practical applications.

[0004] Some existing methods have contributed to attribute extraction in open-world settings, basically in a weakly supervised or unsupervised manner. A series of traditional methods extract semantically rich patterns, called meta-patterns, from typed corpora, and then use the obtained meta-patterns to match target attribute triples. These methods usually use three techniques to extract meta-patterns: context-aware segmentation with learning pattern quality assessment, synonymous meta-pattern grouping, and type-level adjustment for accuracy. Another method considers open-world settings, but focuses more on generalization to unseen attributes. This method usually generates candidate attributes given a seed attribute set and generalizes them to new attribute types. For example, OA-mine proposes a principal framework that first generates attribute value candidates with weak supervision using a seed attribute value set and then groups them into attribute clusters. Some recent studies have focused on few-shot attribute extraction, which adopt a new paradigm of prompts or propose a knowledge-enhanced attention framework to solve the problem of long-tail data.

[0005] However, most of these methods are implemented on datasets for product attribute extraction. They only extract attribute information from information-dense product titles or summaries, but ignore extracting attribute information from the original sparse corpus, where only a few texts contain valuable attribute triples. Extracting information from large sparse corpora, such as news corpora, has the potential to be widely used in practical applications, especially in few-shot scenarios, where manual annotation is always expensive. Therefore, methods for open-world few-shot attribute extraction from sparse corpora are urgently needed.

[0006] The following introduces the existing research on attribute extraction from three aspects: supervised sequence labeling, unsupervised meta-pattern extraction, and weakly supervised generative extraction.

[0007] Sequence labeling is a commonly used supervised attribute extraction method, which labels attribute values ​​by type and trains sequence labeling models for extraction. OpenTag first introduced sequence labeling into the attribute value extraction task based on BiLSTM-CRF model and attention mechanism. Several works have been devoted to improving it, including SuOpenTag, AVEQA and AdaTag. These works usually require fully supervised data of given product attribute names, ignoring open world conditions.

[0008] Unsupervised meta-pattern extraction Meta-pattern extraction methods usually require manual design of some templates to extract or score constraints, which are originally unsupervised. MetaPAD first extracts semantically rich information meta-patterns from the corpus through context-aware segmentation and grouping and refinement processes. TruePIE proposes a self-training framework to discover more reliable meta-patterns containing correct information through an iterative process based on MetaPAD. These methods are traditional rule-based methods and do not require supervisory signals. DeepTruth further trains a combined network of CNN and LSTM to improve the reliability of extracted meta-patterns under supervision. These works are not limited to any specific type of corpus, but have weak semantic analysis capabilities and poor generalization capabilities.

[0009] Weakly supervised generative extraction methods mainly focus on the generalization ability of unknown attributes, while giving some seed attribute values ​​for open-world attribute extraction. OAmine noticed the emerging characteristics of e-commerce products and proposed the problem of attribute extraction in an open-world environment. It generates candidate attribute values ​​from product titles with some seeds and then groups them into attribute clusters. Amacer improves few-shot generalization by leveraging product context and implements implicit semantic signals as additional supervision. Another work also applied a new paradigm of prompts, introducing only a small number of trainable parameters to improve few-shot attribute extraction. KEAF further used their prototype network for multi-label few-shot attribute value extraction. Finally, AtTGen[6] proposed attribute trees to uniformly model closed, open, and semi-open world attribute extraction.

[0010] However, most of the above related works focus on attribute extraction in the field of e-commerce products, especially in information-intensive product titles or product descriptions. Existing studies rarely focus on attribute extraction from open-world sparse corpora, and most studies cannot accurately and comprehensively extract all valuable attribute triplets due to insufficient model learning, especially in the few-shot setting. Summary of the invention

[0011] In view of the above problems, the present invention provides a meta-pattern enhanced few-sample attribute extraction method and system for open-world sparse corpus.

[0012] The technical solution adopted by the present invention is as follows:

[0013] A method for extracting attributes from a few samples of open-world sparse corpus, comprising the following steps:

[0014] Extract meta-patterns from sparse corpora to obtain meta-patterns that may contain attribute information;

[0015] Enhance the training text with the extracted meta-patterns to obtain a training data set;

[0016] Fine-tune the sequence-to-sequence model using the training dataset;

[0017] Generate attribute triplet candidates for the input sentence using a fine-tuned sequence-to-sequence model;

[0018] The scoring function is used to select high-quality attribute triples from the attribute triplet candidates as the target attribute extraction results.

[0019] Furthermore, the meta-pattern extraction method is MetaPAD, and the sequence-to-sequence model is a T5 model.

[0020] Furthermore, the method of enhancing the training text using the extracted meta-pattern includes:

[0021] According to the few-shot setting, several samples are labeled from the corpus under the guidance of the meta-pattern extraction results, and the relational information of the results is filtered out;

[0022] Select N sentences with information value from the initial corpus and mark these N sentences as the training data set D; a sentence is called information valuable if and only if MP(s i )≠φ, where MP(s i ) represents the result obtained after the meta-pattern extraction process; the training data set D is as follows:

[0023]

[0024] in, From the sentence s i An attribute triplet label extracted from , if there are multiple attribute triples in a single sentence, separate the multiple attribute triplet labels of the sentence into multiple training samples, so as to learn the equal tendency of multiple attribute triples to sequence-to-sequence models during fine-tuning; if the sentence s i does not contain any valid attribute triples, just set ε represents the empty string; sentences s with unique ε labels cannot be discarded i .

[0025] Furthermore, the loss function used in fine-tuning the sequence-to-sequence model using the training dataset is expressed as:

[0026]

[0027] Among them, l ij ={y j1 ,y j2 ,...,y jR}, where y jr Indicates l ij The rth token, y j，<r represents the token sequence that has been generated before the rth token, R represents the total number of tags for each tag, and P represents the number of tokens generated before the rth token. i Generate y under the condition of the first r-1 tags jr probability.

[0028] Furthermore, the generating attribute triplet candidates for the input sentence using the fine-tuned sequence-to-sequence model includes:

[0029] We use a fine-tuned sequence-to-sequence model to transform the sequence-to-sequence generation task into a unified text conversion task with an encoder-decoder transformer architecture.

[0030] In the generation phase, beam search is used to obtain multiple candidate attribute triples for each sentence, and the first k generated results are regarded as candidate attribute triples.

[0031] Furthermore, the selecting of high-quality attribute triples from attribute triple candidates using a scoring function includes:

[0032] First, invalid attribute triples are defined: if a candidate attribute triple is not in the form of “subject#attribute#value”, or it is an empty string ε, then the attribute triple is invalid;

[0033] Then, the scoring function is designed according to the following heuristic rules:

[0034] Rule 1: If a sentence s i The result of the first sort is an invalid attribute triple. i There is a high probability that there is no valid attribute triple in ;

[0035] Rule 2: A correct attribute triple should not contain tokens other than the attribute name that do not appear in the original sentence to ensure that the extraction result does not have illusions;

[0036] Rule 3: For candidate attribute triples extracted from the same sentence, if there are two candidate attribute triples with the same entity and attribute name but different attribute values, the candidate attribute triple with a higher ranking is considered to be correct and the other candidate attribute triple is discarded;

[0037] Rule 4: The reliability of the generated candidate attribute triples is negatively correlated with the distance between the entity position and the attribute value in the original input text;

[0038] The scoring function consists of four parts: generation score, effectiveness score, specificity score and credibility score;

[0039] After scoring all the generated attribute triplet candidates, candidates with a total score less than a threshold hyperparameter are discarded, and other candidates are selected as the final extraction results.

[0040] A few-sample attribute extraction system for open-world sparse corpus, comprising:

[0041] The meta-pattern extraction module is used to extract meta-patterns from sparse corpora and obtain meta-patterns that may contain attribute information;

[0042] A meta-pattern enhancement module is used to enhance the training text using the extracted meta-patterns to obtain a training data set;

[0043] A fine-tuning module, which is used to fine-tune the sequence-to-sequence model using the training dataset;

[0044] A candidate generation module for generating attribute triplet candidates for input sentences using a fine-tuned sequence-to-sequence model;

[0045] The attribute triplet filtering module is used to select high-quality attribute triples from the attribute triplet candidates using a scoring function as the target attribute extraction result.

[0046] The beneficial effects of the present invention are as follows:

[0047] The present invention adopts the meta-pattern extraction method to enhance the training data in the sparse corpus, solving the problem of attribute extraction in the open world sparse corpus. Under the guidance of the meta-pattern extraction results, the generative T5 model is used to introduce the generation-filtering process to accurately and comprehensively implement the few-sample attribute extraction. Experiments show that on the benchmark dataset OSN-515, the present invention achieves the best performance compared with the baseline model, and further detailed analysis proves the effectiveness of each part of the framework of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is the overall framework of the few-sample attribute extraction method (MEGFAE) of the present invention.

[0049] Figure 2 It is the process of meta-pattern extraction. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below through specific embodiments and drawings.

[0051] The present invention aims to enhance sparse corpora with unsupervised metapattern extraction methods and realize open-world few-shot attribute extraction through generative post-filtering paradigm. Sentences without attribute information are filtered out from a large sparse corpus by unsupervised metapattern extraction method MetaPAD (Jiang, M., Shang, J., Cassidy, T., Ren, X., Kaplan, LM, Hanratty, TP, Han, J.: Metapad: Metapattern discovery from massive text corpora. In: Proceedings of the 23rd ACMSIGKDDInternational Conference on Knowledge Discovery and Data Mining. p. 877–886. KDD'17, Association for Computing Machinery, New York, NY, USA (2017).), and some remaining sentences (about 0.001% of the entire corpus) are manually annotated. Then, under the guidance of the meta-pattern extraction results, the generative T5 model (Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, PJ: Exploring the limits of transfer learning with a unified text-to-text transformer (2023)) is used to fine-tune the annotated sentences. In addition, the present invention introduces a generative model of first generating and then filtering to accurately and fully extract all valuable attribute triples from sparse text. Specifically, using the trained T5 model, multiple attribute triple candidate words are generated for each input sentence in the original corpus through beam search, and then a scoring function is used to select high-quality attribute triples as the target attribute extraction results.

[0052] The following first introduces the open-world few-shot attribute extraction task from a sparse corpus. Then, the meta-pattern enhanced generation of few-shot attribute extraction framework MEGFAE ( Figure 1 ).

[0053] 1. Problem Formulation

[0054] Given an input corpus S = {s1,s2,s3,...,s n}, length is n, where s iis the i-th sentence in S. The purpose of extracting open-world attributes from a sparse corpus is to extract all valuable attribute triples T = t1∪t2∪t3∪...∪t n , where t i Indicates from s i Specifically, if s i Does not contain attribute triples, then t i =φ, otherwise t i ={t i1 ,t i2 ,...,t im}, where m means that it can be obtained from s i The number of attribute triples extracted from . Each attribute triple is in the form of (Entity, Attribute, Value). Wherein, Entity represents entity, Attribute represents attribute name, and Value represents attribute value. In the invention, the attribute name Attribute is not constrained by any predefined set, and only a small number of annotations are allowed for training.

[0055] 2. Model Architecture

[0056] Figure 1 The meta-pattern enhancement of the present invention generates a few-sample attribute extraction framework. The framework consists of two parts: meta-pattern enhancement and post-generation filtering (or post-generation screening) process, represented by blue dotted boxes and green dotted boxes respectively. The first part is meta-pattern enhancement, which completes the training process by enhancing the training text used to fine-tune the generation model through meta-pattern extraction. The meta-pattern enhancement part consists of a meta-pattern extraction module, an attribute filtering and a small amount of annotation module, and a meta-pattern-guided fine-tuning module. The second part is a two-stage "post-generation filtering" process. In this part, several candidate words are generated for each sentence, and then a scoring function is used to select high-quality attribute triples from the filter. The post-generation filtering process part consists of a candidate generation module and a high-quality attribute triple filtering module.

[0057] 2.1) Meta-mode enhancement

[0058] In order to reduce the sparsity of few-sample training texts, the present invention adopts the existing meta-pattern extraction method MetaPAD and uses endogenous supplementary resources to enhance the training texts.

[0059] Metapattern: Metapatterns are typed information text pattern structures originally proposed by MetaPAD. They are semantic patterns that contain rich information about entity types (e.g., $PERSON, $POLITICIAN, $COUNTRY), numeric types (e.g., $DIGIT, $MONTH, $YEAR), words and phrases, and punctuation, which are also complete semantic units in a given text. An example of a metapattern is "$PERSON, age$DIGIT", which represents the age attribute of the person and the corresponding value. Metapatterns usually contain attribute information or relationship information.

[0060] Meta-pattern extraction: The goal of meta-pattern extraction is to mine meta-patterns from a typed corpus, which is achieved by the MetaPAD method. The method includes three modules: Figure 2 ), firstly, a context-aware segmentation strategy and learned pattern quality evaluation functions (e.g., frequency, consistency, informativeness, and other indicators) are adopted to accurately locate the boundaries of the pattern, avoid expensive dependency analysis, and generate high-quality patterns. Next, synonymous meta-patterns are identified and grouped by several aspects (e.g., type, context, and extraction method). Finally, the entity type distribution of the extracted instances in each pattern group is checked, and the appropriate type level is selected to ensure the accuracy of the discovered pattern. Through this process, the present invention is able to obtain all meta-patterns that may contain attribute information from a sparse corpus in an unsupervised manner. For each sentence s input from the corpus C i , the result obtained after the meta-pattern extraction process is formalized as follows:

[0061] MP(s i )=MetaPAD(s i ,C)

[0062] MP(s i ) means from sentence s i The set of meta-patterns extracted from .

[0063] Attribute filtering and few-shot annotation: After completing the meta-pattern extraction of the sparse corpus, according to the few-shot setting, under the guidance of the meta-pattern extraction results, several samples are labeled from the corpus. Since the results of meta-pattern extraction may contain some unexpected relational triples, the relational information of the results will be filtered out during the manual annotation process. Select N sentences with information value from the initial corpus and label these N sentences as the training dataset D. A sentence is said to be informative if and only if MP(s i )≠φ. The training data set D is as follows:

[0064]

[0065] in From the sentence s i An attribute triplet label extracted from . If there are multiple attribute triples in a single sentence, separate the multiple attribute triplet labels of the sentence into multiple training samples so that the equal tendency of these multiple attribute triples to the sequence-to-sequence model can be learned during fine-tuning. If the sentence s i does not contain any valid attribute triples, just set Sentences s with unique ε labels cannot be discarded i , because the sequence-to-sequence model is required to generate an empty string in the attribute extraction process in the next section. Valid attribute triples are in the form of "subject#attribute#value".

[0066] After preprocessing the few-shot training set D based on meta-pattern enhancement, we can use this training set to fine-tune the sequence-to-sequence model, especially T5, and obtain an enhanced model that extracts few-shot attributes from a sparse corpus.

[0067] Meta-pattern guided fine-tuning: For each sentence s in the training set D i ,s i The corresponding attribute tag is l i = {l i1 ,l i2 ,...,l iw}, where w represents s i The number of attribute triplets contained. During fine-tuning, the loss function for each attribute triplet label is expressed as a sequence-to-sequence loss:

[0068]

[0069] Where l ij ={y j1 ,y j2 ,...,y jR}, where y jr Indicates l ij The rth token, y j，<r represents the token sequence that has been generated before the rth token, R represents the total number of tags for each tag, and P represents the number of tokens generated before the rth token. i Generate y under the condition of the first r-1 tags jr Specifically, the cross entropy loss is used to measure the loss of sequence-to-sequence generation. Then the sentence s i The total loss is:

[0070]

[0071] 2.2) Post-generation screening module

[0072] The post-generation filtering module is a two-stage process that first generates several attribute triplet candidates using the fine-tuned T5 model and then selects high-quality triples as the final results through filters.

[0073] Candidate generation: The T5 (text-to-text Transfer Transformer) model transforms the sequence-to-sequence generation task into a unified text conversion task with an encoder-decoder transformer architecture. This unified paradigm allows the T5 model to flexibly handle a variety of text generation tasks, and it also shows the potential to process structured text.

[0074] In the generation phase, the existing beam search method is used to obtain multiple candidate attribute triples for each sentence. The first k (k can be an integer value between 1 and 4 based on the requirements) generated results will be regarded as candidate attribute triples c i ={c i1 ,c i2 ,c i3 ...}, as shown below:

[0075] c i =T5[beam_search(s i ,k)]

[0076] High-quality attribute triple filtering: After obtaining the candidate attribute triples of all sentences, high-quality attribute triples are selected from the candidate attribute triples through filtering rules. Note that since the rules are not customized for a specific corpus, this module can also work on other corpora.

[0077] First, we define invalid attribute triples: if a candidate attribute triple is not in the form of “subject#attribute#value” or it is the empty string ε, then the attribute triple is invalid.

[0078] Then, the scoring function is designed according to the following heuristic rules.

[0079] Rule 1: If a sentence s i The result of the first sort is an invalid attribute triple. i There is a high probability that there is no valid attribute triple in . Therefore, to a certain extent, other subsequent extraction results can be ignored.

[0080] Rule 2: A correct attribute triple should not contain tokens other than the attribute name that do not appear in the original sentence to ensure that the extraction result does not have illusions.

[0081] Rule 3: For candidate attribute triples extracted from the same sentence, if there are two candidate attribute triples with the same entity and attribute names but different attribute values, the candidate attribute triple with the higher ranking probability is considered to be correct and the other candidate attribute triple is discarded.

[0082] Rule 4: The reliability of the generated candidate attribute triples is negatively correlated with the distance between the entity position and the attribute value in the original input text.

[0083] Based on the above rules, the present invention proposes the following scoring function for screening candidate objects while considering the score of the T5 model generation result. Therefore, the scoring function consists of four parts: generation score, effectiveness score, specificity score and credibility score.

[0084] Generation score: It is the raw score of the T5 generation result, expressed as follows:

[0085]

[0086] P(y jr ∣y j,<r ,s i )=Softmax(w o h r +b o )

[0087] In the formula, c ij ={y j1 ,y j2 ,...,y jR} means s i The jth candidate triplet, h r So i is the hidden state of the input through the feedforward network layer and the self-attention layer output of the decoder at step r, w o 、b o is the weight matrix and bias of the linear mapping, and R represents the total number of tags for each candidate triplet.

[0088] Validity score: Set the validity score to filter invalid attribute triple candidates. If the candidate c ij is an invalid attribute triple, the score is 0, otherwise it is 1.

[0089] Specificity score: The specificity score is proposed according to Rule 3, because we want to constrain the generation of duplicate or similar candidates. The specificity score is a penalty attached to the total score when candidate triplets from the same sentence share most of the tokens.

[0090] Confidence score: The confidence score focuses on the relevance between the candidate entity and the attribute value. This relevance is measured by calculating the distance between the positions of the entity token and the value token in the original text. If any entity or value token is missing in the original text, its position is set to a much larger number. Specifically, c ij The position variance of is taken as the reliability score, which can be expressed as:

[0091]

[0092] Where Loc(y jr ) indicates token y in the input text jr , and R represents the total number of tags of candidate attribute triples.

[0093] Finally, the scoring function is obtained as shown below:

[0094] Score(s i ,c ij )=λ1Socre generation (s i ,c ij )+λ2Score validity (s i ,c ij )+λ3Score specificity (s i ,c ij )+λ4Score reliability (s i ,c ij )

[0095] Among them, weights λ1, λ2, λ3, λ4 are hyperparameters, Score validity is the effectiveness score, Score specificity is the specificity score.

[0096] After successfully scoring all generated candidates, those candidates whose total scores are less than a threshold hyperparameter γ are discarded, and the other candidates are selected as the final extraction results.

[0097] 3. Experiment

[0098] This section introduces the datasets used and the experimental settings. The experimental results are then analyzed in detail. First, the main experiments are conducted with the proposed benchmark dataset to evaluate the performance of extracting open-world attributes from a sparse corpus. Ablation experiments are then performed to verify the effectiveness of each part of the model. In addition, the experiments are explored in different settings to provide a complete understanding of the model of the present invention.

[0099] 3.1 Dataset and Settings

[0100] a) Datasets and metrics. The research goal of this paper is to extract all valuable attribute triplets from an open-world sparse corpus. However, most of the datasets used in recent works focus on the extraction of e-commerce product titles or descriptions, which are information-intensive and not suitable for the goal of this paper. Therefore, this paper proposes a lightweight benchmark dataset Open-world SparseNews 515 (OSN-515) based on the APR news corpus mentioned in MetaPAD, which has no open source dataset. The APR corpus has more than 4 million sentences. OSN-515 contains a test input set consisting of 515 sentences randomly selected from the APR news corpus and a test set containing all valuable attribute triplets that can be extracted from the 515 sentences. The dataset used for training is dynamically selected from OSN-515. Considering the few-sample scenario, no validation set is made, but the approximate best epoch is selected based on the convergence of the loss during training. Macro average precision (P), recall (R), and F1 score (F1) are used as metrics. Specifically, for a pair of extracted attribute triples token set S1 = set(t) and label set S2 = set(l), the p, r, and f1 indicators of this pair are as follows

[0101]

[0102] Where len(x) represents the length of set x, and P, R, and F1 are the average values ​​of p, r, and f1 of all pairs respectively.

[0103] b) Settings. We use “t5-base” as the backbone sequence-to-sequence model of the framework. The hyperparameters λ1,λ2,λ3,λ4 are set to 1, 2000, 1000, 1 respectively, and the threshold γ is 1000. For the main experiment, during evaluation, in the few-shot scenario, beam k is set to 2 and the size of the training dataset N is set to 50. During training, the learning rate is set to 1.5×10 -4 , when the training dataset size is 50, the number of epochs is set to 25 and the batch size is only set to 1, which is sufficient for a small number of trainings. AdamW is used as the optimizer and the warm up step is 2000.

[0104] Since the training dataset is dynamically selected from OSN-515, in order to make the results convincing, three experiments with different dynamically selected training sets are conducted for each model.

[0105] c) Baselines. Since there are few models suitable for the open-world sparse corpus extraction task, four models are selected as baselines: (1) Two meta-pattern extraction methods: MetaPAD extracts semantically rich and informative meta-patterns from the corpus through a context-aware segmentation, grouping, and refinement process; Deeptruth further improves the reliability of the extracted meta-patterns using a CNN-LSTM network. (2) Two sequence-to-sequence models fine-tuned on randomly selected training sets: BART consists of a bidirectional and autoregressive transformer and has superior performance in various NLP tasks including summarization and translation; T5 transforms the sequence-to-sequence generation task into a unified text-to-text conversion task, showing the ability to handle structured text generation. In the experiments, the two models are trained with randomly selected training sets and the output results of the first ranking are evaluated.

[0106] 3.2 Main results

[0107] Table 1 shows the overall results of the OSN-515 benchmark dataset in the few-shot scenario setting with 50 training instances. To ensure the persuasiveness of the experimental results, three different dynamic selections of the training set for each model were performed. Each model was experimented on three different dynamically selected training sets. All experiments were performed 5 times and the average was taken.

[0108] Table 1 shows that:

[0109] (1) The model of the present invention significantly outperforms all baselines in F1 score on the OSN-515 dataset. MEGFAE exceeds the best baseline by 16.4% in average F1 score and all baselines on all dynamically selected training sets.

[0110] (2) The significant improvement in F1 score mainly comes from the improvement in recall score, while there is a trade-off between precision and recall. The recall score exceeds about 25% at the cost of a loss of about 15% in precision score compared to the highest precision score. However, the reason for the higher precision is the extremely conservative generation strategy. Table 2 gives the number of triplets successfully extracted by each model. As can be seen from Table 2, the valid attribute triplets generated by MEGFAE from OSN-515 far exceed any other baseline model, while the accuracy of MEGFAE is only 15% lower than the highest accuracy in the baseline. This shows the advantage of the framework of the present invention in extracting open-world few-shot attributes from sparse corpora, and is able to extract as many valuable attribute triplets as possible.

[0111] (3) Both Tables 1 and 2 show the instability of the methods except the meta-pattern extraction method which is not affected by the dynamic selection of the training data set. For BART and T5, the difference in F1 scores between the three dynamic training sets can reach more than 20%, showing weakness when processing sparse text. The model of the present invention has strong stability, with the F1 score fluctuating within only 2%, which shows that the meta-pattern enhancement module of the present invention is effective in reducing the sparsity of the corpus. Table 2 shows the number of triples successfully extracted from OSN-515, while the total number of attribute triple labels in OSN-515 is 286.

[0112] Table 1 Overall performance on the OSN-515 dataset with 50 training instances

[0113]

[0114] Table 2 Number of triplets successfully extracted from OSN-515

[0115]

[0116] 3.3 Ablation Experiment

[0117] Ablation studies were performed to illustrate the impact of each part of the framework of the present invention. The results are listed in Table 3. The ablation experiments include no meta-pattern enhancement, no beam search, no post-generation filtering, and no all the above modules. "T num" represents the number of successfully extracted attribute triples. Each result is the average result from three dynamically selected training sets.

[0118] From Table 3, it can be inferred that the F1 score decreases by 5.7%, 3.8%, and 4.7% when the meta-pattern enhancement part is removed, the generation-then-filter part is removed, and only the filtering module is removed. For the meta-pattern enhancement module, the overall decrease in its precision, recall, F1 score, and the number of successfully extracted attribute triples indicates that it is effective in reducing the sparsity of attribute extraction in the open-world sparse corpus. For the generation-then-filter module, the results imply a trade-off between precision and recall. However, when both the beam search and filtering modules are retained in the model, the main F1 score is higher. In addition, it also shows that the generation-then-filter module provides users with the flexibility to choose whether to pay more attention to the precision score or the recall score. Although a large number of extracted attribute triples are fake, the large number of attribute triples without filtering means a more flexible generation strategy. When all the above modules are removed, the overall results are the same as those of the original T5.

[0119] Table 3 Results of ablation studies

[0120]

[0121] 3.4 Detailed Analysis

[0122] In order to more fully understand the model of the present invention, several experiments were further performed, including experiments with different training set sizes and exploration of different beam numbers.

[0123] Training set size N: Table 4 shows the results of different training set sizes in the few-shot scenario. “MEG FAE w / o MPE” represents the MEGFAE model without meta-pattern enhancement, and the training set size N is set to 10, 30, 50, and 70. Each result is obtained by the average of three dynamic training sets of size 50. The model of the present invention was also experimented without meta-pattern enhancement under different training set size settings. From the results in Table 4, it can be seen that with the increase of N, the overall performance of MEGFAE and MEGFAE without meta-pattern enhancement is improving. The performance of MEGFAE is always better than that of MEGFAE without meta-pattern enhancement, indicating that the training text enhanced by meta-pattern extraction collects more effective attribute information than the latter. Even the comprehensive score of MEGFAE with N = 50 is about 1% higher than that of MEGFAE with N = 70. Moreover, when N = 30, the F1 score of MEGFAE is still 30% and 13% higher than that of BART and T5 when N = 50, indicating that the out method is well adapted to the task of extracting few-shot attributes from sparse corpus.

[0124] Table 4 Results of different training set sizes in the few-shot scenario

[0125]

[0126]

[0127] Number of bundles k: Further experiments with different numbers of bundles k are shown in Table 5. Each result is the average of three dynamic training sets. Multi-triplet generation experiments are conducted on some input texts. The increase in recall and decrease in precision with the growth of k is not surprising. However, it can be inferred from Table 5 that the F1 score when k = 3 is only 1.1% higher than that when k = 2, and the F1 score when k = 4 is almost the same as that when k = 3. Handling texts with more than 4 triplets without affecting the overall effect becomes a problem worth solving.

[0128] Table 5 Results of beam search under different beam numbers k

[0129]

[0130] In summary, the present invention raises the problem of extracting open-world attributes from an information-sparse corpus in a few-sample scenario, and proposes a meta-pattern-enhanced generate-then-filter framework (MEAGFAE). The present invention applies the unsupervised meta-pattern extraction method MetaPAD to guide the fine-tuning of the sequence-to-sequence model, and then extracts attribute triplets under the "generate-filter" paradigm from the given text. A large number of experiments on the APR news corpus demonstrate the superiority of the model, and ablation studies verify the effectiveness of each part of the model. The model of the present invention can also work well on other corpora because no part of it is tailored to a specific corpus.

[0131] Another embodiment of the present invention provides a few-sample attribute extraction system for open-world sparse corpus, comprising:

[0132] The meta-pattern extraction module is used to extract meta-patterns from sparse corpora and obtain meta-patterns that may contain attribute information;

[0133] A meta-pattern enhancement module is used to enhance the training text using the extracted meta-patterns to obtain a training data set;

[0134] A fine-tuning module, which is used to fine-tune the sequence-to-sequence model using the training dataset;

[0135] A candidate generation module for generating attribute triplet candidates for input sentences using a fine-tuned sequence-to-sequence model;

[0136] The attribute triplet filtering module is used to select high-quality attribute triples from the attribute triplet candidates using a scoring function as the target attribute extraction result.

[0137] The division of the above modules is only an example. In actual applications, the above functions can be assigned to different functional modules as needed to complete all or part of the functions described in the above method. The specific working process of each module can refer to the corresponding process in the above method embodiment, which will not be repeated here.

[0138] Another embodiment of the present invention provides a computer device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0139] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0140] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. A few-sample attribute extraction method for open-world sparse corpus, characterized in that: The following steps are involved: Extract meta-patterns from sparse corpora to obtain meta-patterns that may contain attribute information; Enhance the training text with the extracted meta-patterns to obtain a training data set; Fine-tune the sequence-to-sequence model using the training dataset; Generate attribute triplet candidates for the input sentence using a fine-tuned sequence-to-sequence model; The scoring function is used to select high-quality attribute triples from the attribute triplet candidates as the target attribute extraction results.

2. The method according to claim 1, characterized in that The meta-pattern extraction method is MetaPAD, and the sequence-to-sequence model is the T5 model.

3. The method according to claim 1, characterized in that The method of enhancing the training text by using the extracted meta-pattern comprises: According to the few-shot setting, several samples are labeled from the corpus under the guidance of the meta-pattern extraction results, and the relational information of the results is filtered out; Select N sentences with information value from the initial corpus and mark these N sentences as the training data set D; a sentence is called information valuable if and only if MP(s i )≠φ, where MP(s i ) represents the result obtained after the meta-pattern extraction process; the training data set D is as follows: in, From the sentence s i An attribute triplet label extracted from , if there are multiple attribute triples in a single sentence, separate the multiple attribute triplet labels of the sentence into multiple training samples, so as to learn the equal tendency of multiple attribute triples to sequence-to-sequence models during fine-tuning; if the sentence s i does not contain any valid attribute triples, just set ε represents the empty string; sentences s with unique ε labels cannot be discarded i .

4. The method according to claim 3, characterized in that The training dataset is used to fine-tune the sequence-to-sequence model, and the loss function used is expressed as: Among them, l ij ={y j1 ,y j2 ,...,y jR }, where y jr Indicates l ij The rth token, y j，<r represents the token sequence that has been generated before the rth token, R represents the total number of tags for each tag, and P represents the number of tokens generated before the rth token. i Generate y under the condition of the first r-1 tags jr probability.

5. The method according to claim 1, characterized in that The method of generating attribute triplet candidates for an input sentence using a fine-tuned sequence-to-sequence model includes: We use a fine-tuned sequence-to-sequence model to transform the sequence-to-sequence generation task into a unified text conversion task with an encoder-decoder transformer architecture. In the generation phase, beam search is used to obtain multiple candidate attribute triples for each sentence, and the first k generated results are regarded as candidate attribute triples.

6. The method according to claim 1, characterized in that The method of selecting high-quality attribute triples from attribute triple candidates by using a scoring function includes: First, invalid attribute triples are defined: if a candidate attribute triple is not in the form of "subject#attribute#value" or it is the empty string ε, then the attribute triple is invalid; Then, the scoring function is designed according to the following heuristic rules: Rule 1: If a sentence s i The result of the first sort is an invalid attribute triple. i There is a high probability that there is no valid attribute triple in ; Rule 2: A correct attribute triple should not contain tokens other than the attribute name that do not appear in the original sentence to ensure that the extraction result does not have illusions; Rule 3: For candidate attribute triples extracted from the same sentence, if there are two candidate attribute triples with the same entity and attribute name but different attribute values, the candidate attribute triple with a higher ranking is considered to be correct and the other candidate attribute triple is discarded; Rule 4: The reliability of the generated candidate attribute triples is negatively correlated with the distance between the entity position and the attribute value in the original input text; The scoring function consists of four parts: generation score, effectiveness score, specificity score and credibility score; After scoring all the generated attribute triplet candidates, candidates with a total score less than a threshold hyperparameter are discarded, and other candidates are selected as the final extraction results.

7. The method according to claim 1, characterized in that The scoring function is calculated using the following steps: Calculate the generated score: Where P(y jr ∣y j,<r ,s i )=Softmax(w o h r +b o ), c ij ={y j1 ,y j2 ,...,y jR } means s i The jth candidate triplet, h r So i is the hidden state of the input through the feedforward network layer and the self-attention layer output of the decoder at step r, w o , b o is the weight matrix and bias of the linear mapping, R represents the total number of tags for each candidate triple; Calculate the validity score: Set the validity score to filter invalid attribute triple candidates. If the candidate c ij If it is an invalid attribute triple, the score is 0, otherwise it is 1; Calculate the specificity score: The specificity score is proposed according to Rule 3. The specificity score is a penalty attached to the total score when candidate triples from the same sentence share most of the tokens; Calculate the credibility score: The credibility score focuses on the correlation between the candidate entity and the attribute value. ij The position variance of is taken as the reliability score, which is expressed as: Where Loc(y jr ) indicates token y in the input text jr The index of , R represents the total number of tags of candidate attribute triples; Finally, the scoring function is obtained as shown below: Score(s i ,c ij )=λ1Score generation (s i ,c ij )+λ2Score validity (s i ,c ij )+λ3Score specificity (s i ,c ij )+λ4Score reliability (s i ,c ij ) The weights λ1, λ2, λ3, λ4 are hyperparameters, and Score validity is the effectiveness score, Score specificity is the specificity score.

8. A few-sample attribute extraction system for open-world sparse corpus, characterized in that: include: The meta-pattern extraction module is used to extract meta-patterns from sparse corpora and obtain meta-patterns that may contain attribute information; A meta-pattern enhancement module is used to enhance the training text using the extracted meta-patterns to obtain a training data set; A fine-tuning module, which is used to fine-tune the sequence-to-sequence model using the training dataset; A candidate generation module for generating attribute triplet candidates for input sentences using a fine-tuned sequence-to-sequence model; The attribute triplet filtering module is used to select high-quality attribute triples from the attribute triplet candidates using a scoring function as the target attribute extraction result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images