Triple text information extraction method and system

Abstract

The invention discloses a triple text information extraction method and system, and relates to the technical field of natural language processing, a ZEHD model is preset, and the method comprises a generation training stage and a discrimination training stage; in the generation training stage, input information and a template are input into the ZEHD model, training of the ZEHD model is carried out, and the quality of a generated output sequence text is improved in combination with a reverse cross entropy loss function; in the discriminant training stage, a candidate relation triple is generated through a beam search decoding method, the candidate relation triple is evaluated through a preset scoring standard, a high-score candidate triple sequence is screened, a loss function of the ZEHD model is optimized through the high-score candidate triple sequence, and then the ZEHD model is adjusted through the loss function. And sorting the output sequence through the adjusted ZEHD model. According to the method, the more accurate relation triad can be generated, and the triad information is accurately sorted.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and more specifically to a method and system for extracting triple text information. Background Art

[0002] Relation triple extraction is an important task in information extraction, which aims to extract relation triples from unstructured text. It has a wide range of practical applications, such as knowledge graph construction, enriching knowledge bases, and building question-answering systems. Currently, there are many method paradigms for relation triple extraction tasks, ranging from tag-based methods, span-based methods, table-based methods to sequencing methods.

[0003] However, the above model is only applicable to extracting relation triplets in a dataset with predefined relations, and cannot be extended to actual application scenarios with many unknown relations. In order to solve the problems of the above traditional triple extraction, Chia et al. proposed the Zero-shot Relation Triplet Extraction (ZeroRTE) task. This task aims to extract relation triplets from unstructured text in a zero-shot setting, where the training phase and the test phase contain disjoint sets of relations respectively. Figure 1 Let's take an example to illustrate the main differences between traditional RTE and ZeroRTE. Specifically, Figure 1 (a) shows two disjoint sets of relations. Traditional RTE models only need to extract triplets with the same visible relations in the test phase ( Figure 1 (b) and Figure 1 (c)). The zero-shot relation triple extraction model is trained on a visible relation dataset and extracts relation triplets with unknown relations in the test phase ( Figure 1 (b) and Figure 1 (d)). However, most existing zero-shot relation triple extraction models fail to fully exploit the generative capabilities of the model and also lack the accurate ranking of relation triples.

[0004] Therefore, how to extract and accurately sort triple data that lacks pair relationships is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a method and system for extracting triplet text information, which can generate more accurate relation triples and sort the triplet information.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A method for extracting triple text information, comprising:

[0008] Preset a ZEHD model, wherein the ZEHD model includes a generation training phase and a discrimination training phase;

[0009] In the generation training phase, the input information and the template are input into the ZEHD model, the ZEHD model is trained, and an output sequence is generated; wherein, during the ZEHD model training process, a reverse cross entropy loss function is combined to improve the text quality of the generated output sequence;

[0010] In the discriminant training stage, a beam search decoding method is used to generate candidate relationship triples according to the input information, the candidate relationship triples are evaluated according to a preset scoring standard, a high-scoring candidate triple sequence and a low-scoring candidate triple sequence are generated, the loss function of the ZEHD model is optimized by the high-scoring candidate triple sequence, the ZEHD model is adjusted by the loss function, and the output sequence is sorted by the adjusted ZEHD model;

[0011] The information to be input is obtained, and the information to be input is processed by the trained ZEHD model to generate and sort the sequence to be output.

[0012] In a specific embodiment, the training of the ZEHD model specifically includes:

[0013]

[0014] Where l is the length of the sequence s, θ is the learning parameter, represents the decoding result of position j, is the autoregressive decoding method of the reference sequence, and represents the sum of the decoding probabilities before position j, s is the output sequence, x is the input information, t(r s ) as the template, Q θ To minimize the model distribution, P is the data distribution, E x～p [logQ θ ] is to minimize the forward cross entropy.

[0015] In a specific embodiment, the combined reverse cross entropy loss function specifically includes:

[0016]

[0017] Among them, L gen is the mixed cross entropy function, η is a hyperparameter; when the hyperparameter η=1, L gen Equivalent to an independent forward cross entropy loss function; when the hyperparameter η = 0, L gen Equivalent to a single inverse cross entropy loss function.

[0018] In a specific embodiment, the preset scoring standard is Sentence-bleu.

[0019] In a specific embodiment, the loss function specifically includes:

[0020] L dis =max(0,f(s A )-f(s B )+α(M(s A )-M(s B )));

[0021]

[0022] Among them, α is the boundary weight term, l represents the length of sequence s, and s A is a high-scoring candidate triple sequence, s B is a low-scoring candidate triple sequence, t(r s ) is a template, s<l represents the sum of the decoding probabilities before position l, M is the manual scoring standard, i∈(A, B) is an index, and represents the calculation of high-scoring candidate sequences s respectively. A and low-scoring candidate sequences s B ; A and B represent high-scoring text and low-scoring text respectively.

[0023] In a specific embodiment, the optimizing the loss function of the ZEHD model by the high-scoring candidate triple sequence specifically includes:

[0024] L dis =max(0,f(s A )-f(s B )+α(M(s A )-M(s B )))-f(s A ).

[0025] A triple text information extraction system, comprising:

[0026] A training module is generated, and a ZEHD model is preset, wherein the ZEHD model includes a generation training phase and a discrimination training phase; in the generation training phase, input information and a template are input into the ZEHD model, and the ZEHD model is trained to generate an output sequence; wherein, in the ZEHD model training process, a reverse cross entropy loss function is combined to improve the text quality of the generated output sequence;

[0027] A discriminant training module, in the discriminant training stage, generates candidate relationship triples according to the input information by using a beam search decoding method, evaluates the candidate relationship triples by using a preset scoring standard, generates a high-scoring candidate triple sequence and a low-scoring candidate triple sequence, optimizes the loss function of the ZEHD model by using the high-scoring candidate triple sequence, adjusts the ZEHD model by using the loss function, and sorts the output sequence by using the adjusted ZEHD model;

[0028] The output module obtains the information to be input, processes the information to be input through the trained ZEHD model, generates a sequence to be output, and sorts it.

[0029] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method and system for extracting triplet text information, which can not only generate more accurate relation triplets, but also rank candidate triplets more accurately. A hybrid cross entropy loss function is designed in the generative training stage to solve the over-generalization problem associated with general maximum likelihood estimation; in addition, a re-ranking task is incorporated into the discriminative training stage to improve the ranking quality of candidate triplets; the proposed model has achieved significant improvements on two classic zero-shot relational datasets, fully demonstrating the effectiveness of the hybrid cross entropy loss function as a generative task objective and a discriminative training stage based on the re-ranking task. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0031] Figure 1 The difference diagram between the traditional RTE and ZeroRTE provided by the present invention; (a) is a disjoint set of relations; (b) is the training result on the visible set of relations; (c) is the test result on the visible set of relations; (d) is the training result on the invisible set of relations;

[0032] Figure 2 The overall framework diagram of the ZEHD model provided by the present invention;

[0033] Figure 3 This is a flow chart of the testing phase of the ZEHD model provided by the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] The embodiment of the present invention discloses a method for extracting triple text information, comprising:

[0036] The ZEHD model is preset. The ZEHD model includes a generation training phase and a discrimination training phase.

[0037] In the generation training phase, the input information and template are input into the ZEHD model, the ZEHD model is trained, and the output sequence is generated; wherein, the reverse cross entropy loss function is combined in the ZEHD model training process to improve the text quality of the generated output sequence;

[0038] In the discriminative training phase, the beam search decoding method is used to generate candidate relationship triplets according to the input information, and the candidate relationship triplets are evaluated by the preset scoring criteria to generate high-scoring candidate triple sequences and low-scoring candidate triple sequences. The loss function of the ZEHD model is optimized by the high-scoring candidate triple sequence, and the ZEHD model is adjusted by the loss function. The output sequence is sorted by the adjusted ZEHD model.

[0039] Obtain the input information, process it through the trained ZEHD model, generate the output sequence and sort it.

[0040] Figure 2The basic structure and process of the ZEHD model are shown. The ZEHD model is divided into a generative training phase (phase one) and a discriminative training phase (phase two). First, the goal of phase one is to generate high-quality relation triplets. In this phase, the ZEHD model inputs the text and its corresponding relation template into the encoder, and then obtains the predicted output through the decoder. Specifically, the ZEHD model uses the T5 model, a deep learning model based on the Transformer architecture, as an encoder and decoder. The T5 model consists of an encoder and a decoder. The main function of the encoder is to understand the input text semantically, encode the text into a unified implicit semantic representation, capture global context information, and provide a basis for subsequent generation. The decoder is based on the implicit semantic representation generated by the encoder and the currently generated context information to gradually generate an output sequence. At the same time, it can generate multiple candidate sequences for the sorting module to screen. The entire T5 model plays the role of a core framework in the new model: the encoder is responsible for semantic understanding, and the decoder is used to generate target triplets and candidate sequences, realizing the functions of the generative and discriminative stages of the relation extraction task. The ZEHD model masks entity tags in the template, and then fine-tunes the model to predict the masked entity tags. In addition, the ZEHD model designs a hybrid cross entropy loss function in stage one, which not only does not increase the complexity of calculation, but also can effectively improve the generation quality of relation triples. Secondly, the purpose of stage two is to integrate the sorting process into the ZEHD model as an optimization goal, and enable the model of the present invention to assign higher probability values ​​to better candidate triples. In this stage, the ZEHD model that has completed stage one training will generate a sequence of candidate triples, and then manually score and sort the candidate triples. Finally, the ZEHD model extracts a pair of triplets with higher and lower manual scores from the sorted candidate triplets, and performs learning on the re-ranking task, thereby enhancing the ZEHD model's ability to sort multiple triplets.

[0041] In a specific embodiment, Figure 2 The training process of the entire generation training phase (phase 1) is shown. In terms of model input, the ZEHD model uses artificially constructed relationship templates as natural language prompt templates. Table 1 shows some examples of relationship tags and relationship templates. For example, for the relationship "publisher", the module can obtain the corresponding relationship template "[H]is publishedby[T]", where [H] and [T] are identifiers of the head entity and tail entity positions. The ZEHD model connects the input original sentence X and the template t(r s ) to generate the input sequence, and then use the mask markers in the relation template <x>and <y>Replace [H] and [T]. Since there are differences between head entities and tail entities in the input sentence, ZEHD replaces [H] and [T] according to the order of [H] and [T] in the template. <x>and <y>The shielding marks correspond to the head entity position and the tail entity position respectively. Finally, the ZEHD model can obtain the output sequence s:" <x> e 1 <y> e 2 <z>”, where the predicted entity span e 1 and e 2 Immediately after the mask mark <x>and <y>Afterwards generated.

[0042] Table 1 Examples of some relation templates in the dataset

[0043]

[0044]

[0045] In a specific embodiment, for the model proposed previously for the zero-shot triple extraction task, the task goal is to train the model G under the input of the original sentence x and finally generate a suitable relation triple sequence s. The training method of these models is to minimize the model distribution Q θ The cross entropy relative to the data distribution P, that is, minimizing the forward cross entropy E x～p [logQ θ ], including:

[0046]

[0047] Where l is the length of the sequence s, θ is the learning parameter, represents the decoding result of position j, is the autoregressive decoding method of the reference sequence, and represents the sum of the decoding probabilities before position j, s is the output sequence, x is the input information, t(r s ) as the template, Q θ To minimize the model distribution, P is the data distribution, E x～p [logQ θ ] is to minimize the forward cross entropy.

[0048] In the specific calculation process, the above model will check each identifier in the sequence to see if its predicted value is consistent with the reference sequence. If the two are consistent, the state of the identifier is set to 1, otherwise the state of the identifier is set to 0. However, in actual operation, the above model will use label smoothing instead of the original state setting in the formula to prevent overfitting:

[0049]

[0050] Here, β is a small hyperparameter and N represents the length of the sequence identifier.

[0051] In a specific embodiment, since minimizing the above forward cross entropy loss function often causes the trained model to fall into the dilemma of over-generalization, the present invention proposes a loss function for the generative training stage. The loss function combines the characteristics of the forward cross entropy loss function and the reverse cross entropy loss function, and further combines label smoothing to achieve the design of the generative task goal in zero-sample relation triple extraction. ZEHD combines the reverse cross entropy loss function on the basis of the forward cross entropy loss function to form a hybrid cross entropy function L gen , as shown in the following formula, this function aims to improve the text quality of the extracted relation triples while avoiding the over-generalization problem caused by the forward cross entropy loss function. Combined with the reverse cross entropy loss function, it specifically includes:

[0052]

[0053] Among them, L gen is the mixed cross entropy function, η is a hyperparameter; when the hyperparameter η=1, L gen Equivalent to an independent forward cross entropy loss function; when the hyperparameter η = 0, L gen Equivalent to a single inverse cross entropy loss function.

[0054] Usually, when the true distribution is unknown, the reverse cross entropy loss function cannot be directly obtained. Therefore, the ZEHD model uses the forward cross entropy loss function to represent it, as follows:

[0055]

[0056] Among them, Q θ (x|x ＜j ) represents the data distribution of self-decoding regression.

[0057] In a specific embodiment, the training process of the discriminant training phase (phase 2) of the model ZEHD proposed by the present invention is as follows: Figure 2 As shown. In this training phase, the present invention designs a complete discriminant re-ranking training process, so that ZEHD can assign higher quality probabilities to high-quality candidate triples, thereby achieving more accurate sorting. When evaluating the candidate triple sequence generated by ZEHD, the present invention believes that the measurement of precision value is an effective way to evaluate the quality of triples. Therefore, ZEHD selects Sentence-bleu as the manual scoring criterion M to calculate the correlation between candidate relation triples and reference relation triples.

[0058] In addition, after determining the manual scoring criteria, the ZEHD model uses the beam search decoding method to generate candidate relationship triplets after completing the generative training phase. Afterwards, ZEHD uses the manual scoring criteria to score the candidate triplets and obtains the high-scoring candidate triple sequences s A and low-scoring candidate triple sequences s B Then in the second stage, the rearrangement loss function L is used dis , which is intended to encourage ZEHD to assign higher quality probabilities to better candidate triples. In a specific embodiment, the loss function specifically includes:

[0059] L dis =max(0,f(s A )-f(s B )+α(M(s A )-M(s B )));

[0060]

[0061] Among them, α is the boundary weight term, l represents the length of sequence s, and s A is a high-scoring candidate triple sequence, s B is a low-scoring candidate triple sequence, t(r s ) is a template, s<l represents the sum of the decoding probabilities before position l, M is the manual scoring standard, i∈(A, B) where i is an index, indicating that high-scoring candidate sequences sA and low-scoring candidate sequences sB are calculated respectively; A and B represent high-scoring texts and low-scoring texts respectively.

[0062] The above re-ranking loss function L dis It can be used as a reference-free evaluation model to alleviate the inherent inconsistency between the objective function and the evaluation metric. However, since the automatic regression generation depends on the prediction accuracy at the identifier level and the coordination at the sequence level, fine-tuning ZEHD using only the re-ranking loss will lose the text generation ability of ZEHD itself. Therefore, ZEHD optimizes the loss function L by using the sequence of high-scoring candidate relation triples as the reference evaluation. dis , thereby enhancing ZEHD's reordering capabilities while maintaining ZEHD's triple generation quality, including:

[0063] L dis =max(0,f(s A )-f(s B )+α(M(s A )-M(s B )))-f(s A ).

[0064] A triple text information extraction system, comprising:

[0065] A generation training module is provided, and a ZEHD model is preset. The ZEHD model includes a generation training phase and a discrimination training phase. In the generation training phase, input information and templates are input into the ZEHD model, and the ZEHD model is trained to generate an output sequence. In the ZEHD model training process, a reverse cross entropy loss function is combined to improve the text quality of the generated output sequence.

[0066] Discriminative training module: In the discriminative training stage, the beam search decoding method is used to generate candidate relationship triples according to the input information, and the candidate relationship triples are evaluated according to the preset scoring criteria to generate high-scoring candidate triple sequences and low-scoring candidate triple sequences. The loss function of the ZEHD model is optimized by the high-scoring candidate triple sequence, and the ZEHD model is adjusted by the loss function. The output sequence is sorted by the adjusted ZEHD model.

[0067] The output module obtains the input information, processes the input information through the trained ZEHD model, generates the output sequence and sorts it.

[0068] like Figure 3 As shown in the figure, during the test phase, the ZEHD model performs well on the relation set Y containing unknown relations. u The triple extraction capability is tested on the above. First, the ZEHD model divides m unknown relations from the unknown relation set Next, the ZEHD model constructs m input sequences by concatenating the input sentence x and the relation template Use mask at the same time <x>and <y>To replace the original identifiers [h] and [t]. After that, ZEHD generates the final output sequence for the m input sequences And the score value of s is calculated by the hybrid cross entropy function. For the prediction of the final triple, ZEHD selects the relation triple As the result output, and is the entity span of the predicted output. In the output result, the present invention uses beam search to decode and search the triplet sequences of multiple outputs.

[0069] When the number of relations is very large, it is very inefficient to calculate the score of each potential candidate relation in all output triples and then sort them. Therefore, ZEHD uses the restrictions of relation descriptions to filter out relation labels that are irrelevant to the input text. First, ZEHD obtains the embedding vector representation of the input sentence and the corresponding relation description. For the relation description, ZEHD uses Sentence-BERT to encode the relation description. Then, ZEHD calculates the similarity between the input text embedding vector and the relation description embedding vector. After filtering out irrelevant relations, the remaining restricted relations are scored and those with The relation triples of are taken as output, where It is the unknown relationship after screening.

[0070] In a specific embodiment, as shown in Table 2, the model ZEHD proposed in the present invention is experimented on two zero-shot relation datasets Wiki-ZSL and FewRel. Wiki-ZSL is a subset of the Wiki-KB dataset, created through distant supervision, and eliminates the "none" relation category and some relation categories with less than 300 appearances. FewRel was originally designed for few-shot relation classification, which consists of 80 relation types and 700 instances in each relation label. The dataset is split into different sets of irrelevant relations to perform zero-shot relation triple extraction. The ZEHD model randomly selects m relations as unknown relations, and the rest are used as visible relations for training the model. The testing process is repeated 5 times with 5 different random seeds to generate 5 data folds. In the validation stage, the number of unknown relations is set to 5. In the test stage, m has three different settings of {5, 10, 15}, the purpose of which is to study the performance of the ZEHD model under different numbers of unknown relations.

[0071] To evaluate the performance of the ZEHD model, the ZEHD model is experimentally evaluated in two different environments: a sentence containing a single triple and a sentence containing multiple triples. All experimental results are averaged over 5 data folds.

[0072] Table 2 Statistics of FewRel and Wiki-ZSL datasets

[0073]

[0074]

[0075] For the extraction of a single triple, the present invention uses accuracy (Acc) as the evaluation criterion, that is, the percentage of correctly predicted results in the total samples. For the extraction of multiple triples, the present invention uses precision (Pre), recall (Rec) and F1 score (F1) as the evaluation criteria, and the specific calculation formula is as follows:

[0076]

[0077] Where TP represents the positive sample predicted by the model as the positive class, FP represents the negative sample predicted by the model as the positive class, and FN represents the positive sample predicted by the model as the negative class. At the same time, in order to retrieve more accurate results in the multiple triplet setting, ZEHD sets a threshold for the calculated score. If the score does not exceed the set threshold, the triplet is regarded as the correct extraction result. By experimenting with the model on the m=10 validation set of the two datasets, the present invention sets the score threshold to 1.6.

[0078] The performance of our model is compared with four existing triple extraction methods:

[0079] (1) TableSequence is a joint triple extraction model that uses two independent encoders to perform relation extraction and named entity recognition simultaneously. Since TableSequence is designed for supervised learning, it cannot directly handle the ZeroRTE task. According to its authors, it performs supervised training on synthetic data generated by the relation generator.

[0080] (2) RelationPrompt is the first model to propose ZeroRTE, which includes a relation generator and a triple extractor. The goal of the relation generator is to generate synthetic data of unseen relations to train the triple extractor. In addition, the purpose of the triple extractor is to create serializations and decode serializations into triples. It also has an additional model form, denoted as "NoGen", which directly utilizes the triple extractor for extraction without using synthetic data.

[0081] (3) Based on the T5 model, ZETT converts ZeroRTE into a template filling task and uses the generated natural language templates to achieve task objectives consistent with the pre-training objectives.

[0082] Table 3 lists the experimental results of the proposed model ZEHD on FewRel and Wiki-ZSL. First, under the setting of single triple extraction, the experimental results of ZEHD on FewRel and Wiki-ZSL are significantly improved compared with ZETT. Specifically, ZEHD on the FewRel dataset and m = {5, 10, 15}, its Acc is improved by 2.76%, 1.02% and 0.10% respectively. ZEHD on the Wiki-ZSL dataset and the same unknown number of relations, its Acc is also improved by 1.50%, 1.15% and 1.08% respectively.

[0083] In addition, under the setting of multiple triple extractions, the experimental results of ZEHD are also significantly better than ZETT. On the FewRel and Wiki-ZSL datasets with m = {5, 10, 15}, the F1 scores of the ZEHD model are improved by 4.79%, 1.95%, 0.74% and 3.93%, 4.41%, 3.64% respectively. Although ZEHD does not always achieve the highest values ​​in both Pre and Rec, ZEHD tends to maintain a relative balance between the two indicators, resulting in a model with a higher F1 score. In contrast, TableSequence or RelationPrompt (NoGen) has very unbalanced values ​​between Pre and Rec, resulting in a lower F1 score. In addition, in the Wiki-ZSL dataset, the experimental results of the present invention are slightly inferior to RelationPrompt when m = 10, because the dynamic data created by RelationPrompt when m = 10 has a higher average quality. In summary, the model of the present invention still has higher stability and superiority than RelationPrompt.

[0084] When in FewRel and the number of unknown relations m=10, the Acc and F1 scores of the model of the present invention on single triple extraction and multiple triple extraction are improved by 1.02% and 1.95% respectively compared with ZETT. The above analysis proves the effectiveness of the hybrid cross entropy loss function for the model of the present invention, and the improvement of the loss function enables the model to generate more accurate text under basically similar decoding strategies. In addition, the improvement of Pre is greater than the improvement of Rec in the comparison of single evaluation indicators. For example, in the case of unknown number of relations m=10 in Wiki-ZSL, the Pre indicator of the model of the present invention increased by 5.41% compared with ZETT, while the Rec indicator increased by 1.47%. This phenomenon proves the effectiveness of the discriminant training stage in the model of the present invention, which enables the model to more accurately rank the quality of predicted candidate triples.

[0085] Table 3 Comparison of ZEHD with different methods on the Wiki-ZSL dataset and FewRel dataset

[0086]

[0087] The effectiveness of each part of the model is verified by ablation experiments. In the ablation experiment, the model of the present invention is selected to be experimentally verified on the FewRel and Wiki-ZSL datasets under the condition of m=10. Specifically, the present invention verifies the necessity of setting a mixed cross entropy loss function in the model. From the experimental results in Table 4, it can be seen that ZEHD loses 0.12% and 0.08% of Acc in the single triple setting on Wiki-ZSL and FewRel, and loses 0.27% and 1.24% of F1 scores in the multi-triplet setting. This phenomenon shows that the improved mixed cross entropy loss function in the present invention can indeed effectively improve the quality of relation triple generation. Finally, ZEHD removes the discriminative training stage of the model and only retains the generative training stage of the model. It can be seen that in the setting of single triple extraction, the ZEHD model reduces the Acc by 0.41% and 0.33%. In the setting of multiple triple extraction, the ZEHD model loses 0.73% and 0.55% F1 score on Wiki-ZSL and FewRel, respectively. The above results prove that the discriminative training phase proposed in ZEHD is effective in accurately ranking candidate triplets.

[0088] Table 4 Results of ablation experiments

[0089]

[0090] One of the challenges of relation triple extraction is to accurately detect entity spans from input sentences. Since most relation triplets are not symmetric, i.e. (h, r, t) = (t, r, h), it becomes critical to identify which entity is the head entity or the tail entity in triple extraction. Therefore, by observing whether the model proposed in the present invention can accurately predict the head and tail entities when the original sentence and the true relation label are input, it is evaluated whether ZEHD can effectively perform entity extraction. At the same time, the present invention also demonstrates entity recognition results to verify the ability of ZEHD to recognize entities alone. Entity recognition refers to whether the model can correctly extract entities without considering the position of the head or tail. In terms of experimental settings, ZEHD is still fine-tuned on the seen relationship set and then tested on the unseen relationship set, and finally the accuracy (Acc) is used as the evaluation criterion.

[0091] Table 5 shows the results of entity extraction and entity recognition. Experimental results show that the proposed model ZEHD outperforms ZETT at different numbers of unseen relations on Wiki-ZSL and FewRel, where the Acc in entity extraction is improved by 5.2%, 2.9%, 3.7% and 7.3%, 3.1%, 2.5%, respectively. In addition, the Acc in entity recognition can be improved by 3.6%, 2.4%, 2.9% and 4.3%, 3.1%, 2.6%, respectively. The above phenomenon proves that the hybrid cross entropy loss function of ZEHD can effectively improve the quality of triple generation, and also verifies the shortcomings of the high-noise synthetic data of RelationPrompt and the task objectives of ZETT. In addition, compared with ZETT, ZEHD improves the Acc in entity extraction task by up to 7.3% and the Acc in entity recognition task by up to 4.3%. The above results show that ZEHD is more capable of identifying head and tail entities, and the discriminative training stage can better assign high-quality probabilities to candidate triples.

[0092] Table 5 Experimental results of ZEHD entity extraction and entity recognition

[0093]

[0094] Table 6 shows the experimental results obtained by the model ZEHD proposed in the present invention when using different cross entropy loss functions. It can be seen from the results that compared with independent forward cross entropy or reverse cross entropy, ZEHD obtains the best experimental results when using mixed cross entropy loss. Specifically, in the case of single triple extraction, the evaluation index Acc is improved by 0.72% and 1.90%, respectively, and in the case of multiple triple extraction, the evaluation index F1 score is improved by 0.33% and 7.98%, respectively. The above experimental results show that the independent application of forward or reverse cross entropy function in the generative model will affect the model distribution fitting of ZEHD, while the application of mixed cross entropy function can not only improve the efficiency of ZEHD's model distribution fitting, but also effectively improve the quality of ZEHD's model distribution fitting.

[0095] Table 6 The impact of using different cross entropy loss functions in ZEHD

[0096]

[0097]

[0098] Table 7 describes practical examples of the model and ZETT in zero-shot triple extraction. From the first example, it can be observed that the model obtains more accurate results compared with ZETT for the same top two output triples. In addition, from the second example, it can be seen that the model of the present invention outputs more reasonable ranking combinations for multiple triples compared with ZETT. Such results can prove that the discriminative re-ranking stage of the model of the present invention has a positive impact on the output of multiple triples.

[0099] Table 7 Comparative analysis of specific examples of ZEHD and ZETT in zero-shot triple extraction

[0100]

[0101] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0102] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined in the present invention may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in the present invention, but will conform to the widest scope consistent with the principles and novel features disclosed in the present invention.< / y> < / x> < / y> < / x> < / z> < / y> < / x> < / y> < / x> < / y> < / x>

Claims

1. A method for extracting triple text information, characterized in that: include: Preset a ZEHD model, wherein the ZEHD model includes a generation training phase and a discrimination training phase; In the generation training phase, the input information and the template are input into the ZEHD model, the ZEHD model is trained, and an output sequence is generated; wherein, during the ZEHD model training process, a reverse cross entropy loss function is combined to improve the text quality of the generated output sequence; In the discriminant training stage, a beam search decoding method is used to generate candidate relationship triples according to the input information, the candidate relationship triples are evaluated according to a preset scoring standard, a high-scoring candidate triple sequence and a low-scoring candidate triple sequence are generated, the loss function of the ZEHD model is optimized by the high-scoring candidate triple sequence, the ZEHD model is adjusted by the loss function, and the output sequence is sorted by the adjusted ZEHD model; The information to be input is obtained, and the information to be input is processed by the trained ZEHD model to generate and sort the sequence to be output.

2. A method for extracting triple text information according to claim 1, characterized in that: The training of the ZEHD model specifically includes: Where l is the length of the sequence s, θ is the learning parameter, represents the decoding result of position j, is the autoregressive decoding method of the reference sequence, and represents the sum of the decoding probabilities before position j, s is the output sequence, x is the input information, t(r s ) as the template, Q θ To minimize the model distribution, P is the data distribution, E x～p [logQ θ ] is to minimize the forward cross entropy.

3. A method for extracting triple text information according to claim 2, characterized in that: The combined reverse cross entropy loss function specifically includes: Among them, L gen is the mixed cross entropy function, η is a hyperparameter; when the hyperparameter η=1, L gen Equivalent to an independent forward cross entropy loss function; when the hyperparameter η = 0, L gen Equivalent to a single inverse cross entropy loss function.

4. A method for extracting triple text information according to claim 1, characterized in that: The preset scoring standard is Sentence-bleu.

5. A method for extracting triple text information according to claim 1, characterized in that: The loss function specifically includes: L dis =max(0,f(s A )-f(s B )+α(M(s A )-M(s B ))); Among them, α is the boundary weight term, l represents the length of sequence s, and s A is a high-scoring candidate triple sequence, s B is a low-scoring candidate triple sequence, t(r s ) is the template, S<l represents the sum of the decoding probabilities before position l, M is the manual scoring standard, i∈(A, B) is the index, and represents the calculation of the high-scoring candidate sequence s respectively. A and low-scoring candidate sequences s B ; A and B represent high-scoring text and low-scoring text respectively.

6. A method for extracting triple text information according to claim 5, characterized in that: The loss function of optimizing the ZEHD model through the high-scoring candidate triple sequence specifically includes: L dis =max(0,f(s A )-f(s B )+α(M(s A )-M(s B )))-f(s A )。 7. A triple text information extraction system, applying a triple text information extraction method according to any one of claims 1 to 6, characterized in that: include: Generate a training module and preset a ZEHD model, wherein the ZEHD model includes a generation training phase and a discrimination training phase; In the generation training phase, the input information and the template are input into the ZEHD model, the ZEHD model is trained, and an output sequence is generated; wherein, during the ZEHD model training process, a reverse cross entropy loss function is combined to improve the text quality of the generated output sequence; A discriminant training module, in the discriminant training stage, generates candidate relationship triples according to the input information by using a beam search decoding method, evaluates the candidate relationship triples by using a preset scoring standard, generates a high-scoring candidate triple sequence and a low-scoring candidate triple sequence, optimizes the loss function of the ZEHD model by using the high-scoring candidate triple sequence, adjusts the ZEHD model by using the loss function, and sorts the output sequence by using the adjusted ZEHD model; The output module obtains the information to be input, processes the information to be input through the trained ZEHD model, generates a sequence to be output, and sorts it.

Images