A method for automatic annotation of text relations based on similar vocabulary

Abstract

The present invention proposes a method for automatic annotation of text relations based on a similar vocabulary, determines all extracted relation names and corresponding entity types, and obtains corpus containing relation names; generates a similar vocabulary that can replace the relation name at each place where the relation name appears in the corpus, and uses this to sort out a relation type vocabulary; performs named body recognition and entity relation triple extraction on the sentence; determines whether the entity pair of the entity relation triple meets the conditions based on the result of named body recognition, and generates a candidate relation set; generates a similar vocabulary for each word of the relation phrase in the entity relation triple, and determines the candidate relation expressed by the word in combination with the relation type vocabulary, and the candidate relation expressed the most times among all words is the annotated relation, and completes automatic annotation. The present invention solves the problem that the traditional remote supervision method is difficult to construct a knowledge base and has low quality of annotated corpus, and provides a new strategy for constructing the data set required for the relation extraction model.

Description

Technical Field

[0001] The present invention relates to natural language processing technology, and in particular to an automatic text relationship annotation method based on a similar vocabulary table. Background Art

[0002] Relation extraction is one of the most important subtasks in natural language processing. Its content is to determine the relationship between a pair of entities in a text based on contextual semantics. Currently, commonly used relationship extraction methods include traditional statistical learning methods and deep learning methods. Traditional statistical methods require manual determination of text features, which is not only difficult, but also often biased and incomplete. The deep learning-based method can automatically and efficiently extract features through the model, and has become the most mainstream method at present.

[0003] The performance of the relation extraction model trained by deep learning methods depends largely on the quality of the training corpus. Currently, the commonly used methods for obtaining training corpus are manual annotation and remote supervised automatic annotation, and both methods have different disadvantages. Although the quality of the corpus annotated by manual annotation is high, it consumes a lot of time and manpower, and it is very difficult to generate a data set in a short time. The remote supervised automatic annotation method is to align the entity pairs appearing in the text to the existing knowledge base, and use the information in the knowledge base to determine the relationship between the entity pairs. Although a large amount of corpus can be obtained in a short time, there are still two problems. The first is that the knowledge base is difficult to build. Although there are some general knowledge bases such as Wikipedia's Wikidata, these knowledge bases are obviously not usable in certain specific fields, and it is not easy to build a large knowledge base in a short time. The second is that the quality of the generated corpus is low. Since the semantic expression relationship of the same entity pair in different corpora may not be the same, a large number of incorrect annotations will appear. Therefore, how to simply and efficiently obtain a large amount of training corpus is a problem that is very worthy of in-depth discussion and research.

[0004] [1]Devlin J, Chang MW, Lee K, et al. BERT: Pre-training of DeepBidirectional Transformers for Language Understanding[J].2018.

[0005] [2] Li Hao. Research on entity relationship extraction method based on distant supervision[D]. Wuhan University of Technology, 2019. Summary of the invention

[0006] The purpose of the present invention is to propose a method for automatically annotating text relations based on a similar vocabulary to solve the problems of difficulty in building a knowledge base and low quality of generated corpus.

[0007] The technical solution to achieve the purpose of the present invention is: a method for automatically annotating text relations based on a similar vocabulary, the steps of which are as follows:

[0008] Step 1: determine all extracted relationship names and corresponding entity types, and obtain the corpus containing the relationship names;

[0009] Step 2: Generate a similar vocabulary that can replace the relationship name at each place where the relationship name appears in the corpus, and use this to organize the relationship type vocabulary;

[0010] Step 3, perform named entity recognition and entity relationship triple extraction on the sentence;

[0011] Step 4, judging whether the entity pair of the entity relationship triple meets the conditions according to the result of the named entity recognition, and generating a candidate relationship set;

[0012] Step 5: Generate a similar vocabulary list for each word in the relation phrase in the entity relation triple, and combine it with the relation type vocabulary list to determine the candidate relation expressed by the word. The candidate relation with the most expressions among all words is the labeled relation, and automatic labeling is completed.

[0013] Furthermore, in step 2, a similar vocabulary that can replace the relationship name is generated at each place where the relationship name appears in the corpus, and a relationship type vocabulary is sorted out based on this. The specific method is:

[0014] For all the places where the relationship names appear in the corpus, the BERT model is used to replace the relationship names with similar words according to the context, and a similar vocabulary list of the corresponding relationship at that position is generated. The first 40 words are taken from each similar vocabulary list, and all are selected if there are no more than 40 words. For all the similar vocabulary lists generated for each relationship in the corpus, the word frequency statistics are first sorted. The words with more appearances represent closer relationships. Then, stop words are removed and the first 80 words are taken as the type vocabulary list of the relationship. If there are no more than 80 words, all are selected.

[0015] Furthermore, in step 3, the sentence is subjected to named body recognition and entity relationship triple extraction, and the specific method is as follows:

[0016] Using the named entity recognition model, we extract entities in sentences from the corpus and generate a set of entities and entity types. We also extract entity pairs and relational phrases describing relationships from sentences using open information extraction tools. Finally, we construct the set in the form of triples.

[0017] Further, in step 4, based on the result of named entity recognition, it is determined whether the entity pair of the entity relationship triplet meets the conditions, and a candidate relationship set is generated. The specific method is as follows:

[0018] For each sentence's entity set and entity relationship triplet, first determine whether the entity pair of the relationship triplet contains or is equal to the entity in the named entity recognition result. If so, change the triplet entity to the entity corresponding to the named entity recognition result. Then determine whether the entity type of the replaced entity pair is the same as the entity type of the target extracted relationship. If so, generate a candidate relationship set. If any of the above conditions is not met, the sentence labeling fails.

[0019] Further, in step 5, a similar vocabulary is generated for each word in the relation phrase in the entity relation triple, and the candidate relation expressed by the word is determined in combination with the relation type vocabulary. The candidate relation expressed the most times among all words is the labeled relation, and automatic labeling is completed. The specific method is:

[0020] For each word in the relation phrase, the BERT model is used to generate a corresponding similar vocabulary list, which is then compared with the type vocabulary list of candidate relations. The relation with the largest number of identical words is selected. If the number reaches the set threshold, the word expresses the relation type, where the threshold is set to 20. Considering all the words in the relation phrase, the relation with the largest number of expressions is the labeled relation. If the relation phrase does not have any words that express any candidate relation, the corpus is discarded.

[0021] A text relationship automatic annotation system based on a similar vocabulary list realizes automatic annotation of text relationships based on a similar vocabulary list based on the aforementioned text relationship automatic annotation method based on a similar vocabulary list.

[0022] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for automatically tagging text relations based on a similar vocabulary list is used to realize automatic tagging of text relations based on a similar vocabulary list.

[0023] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the method for automatically annotating text relations based on a similar vocabulary list is used to implement automatic annotating text relations based on a similar vocabulary list.

[0024] Compared with the prior art, the present invention has the following significant advantages: relationship labels are predefined and automatically annotated based on the generated similar vocabulary, which solves the problems of difficulty in building a knowledge base and low quality of annotated corpus in traditional remote supervision methods, and provides a new strategy for building the data set required for the relationship extraction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of the method for automatically annotating text relations based on a similar vocabulary list of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] The present invention proposes a method for automatically annotating text relations based on a similar vocabulary, the steps of which are as follows:

[0028] Step 1: Determine all extracted relationship names and corresponding entity types, and obtain the corpus containing relationship names as follows:

[0029] Determine all extracted relationship names, and determine the corresponding subject entity and object entity types based on the relationship. At the same time, based on the relationship name, obtain corpus containing the relationship name from other channels such as the Internet.

[0030] Step 2: Generate a similar vocabulary that can replace the relationship name at each location where the relationship name appears in the corpus, and use this to sort out the relationship type vocabulary as follows:

[0031] For all the places where the relationship names appear in the corpus, the BERT model is used to replace similar words for the relationship names according to the context, and a similar word list corresponding to the relationship at that position is generated. The first 40 words are taken from each similar word list. For all similar word lists generated for each relationship in the corpus, the word frequency statistics are first sorted. The more words appear, the closer they are to the relationship. Then, stop words are removed and the first 80 words are taken as the type word list of the relationship. If there are no more than 40 or 80 words, all of them are selected.

[0032] BERT is a method for pre-training language representation. It trains a general "language understanding" model on a large amount of text corpus, and this model can be used to complete various natural language processing tasks. BERT performs better than other models in various NLP tasks because it is the first unsupervised, deep bidirectional system used for pre-training. Therefore, the present invention uses this model for masked language modeling (MLM), masks a certain word in the complete sentence, and then generates words that can replace the masked words according to the context semantics, thereby generating a similar vocabulary.

[0033] Step 3: The method for performing named entity recognition and entity relationship triple extraction on the sentence is as follows:

[0034] Using the named entity recognition model, the corpus is treated as sentences, and the entities existing in the sentences are extracted to generate a set containing entities and entity types; using the open information extraction tool, the corpus is treated as sentences, and the entity pairs and relational phrases describing the relationship in the sentences are extracted, and finally the construction is completed in the form of triples.

[0035] Open information extraction refers to extracting relation tuples from plain text. The structure of the relation tuple is generally (entity 1, relation phrase, entity 2). What is extracted here is binary entity relations, and does not involve multi-relation extraction (that is, the relations of multiple entities are extracted simultaneously). Traditional relation information extraction requires first determining the type of relation, and it is difficult to predefine the complete type of relation. Compared with traditional relation information extraction, the open information extraction model does not require the relationship type to be specified in advance. It uses some words in the entity context to describe the semantic relationship between entities, thereby avoiding the definition of relationship types.

[0036] Step 4: Based on the result of named entity recognition, determine whether the entity pair of the entity relationship triple meets the conditions. If so, the method for generating the candidate relationship set is as follows:

[0037] For each sentence's entity set and entity relationship triple, first determine whether the entity pair of the relationship triple contains or is equal to the entity in the named entity recognition result. If so, change the triple entity to the entity corresponding to the named entity recognition result. Then determine whether the entity type of the replaced entity pair is the same as the entity type of the target extracted relationship. If so, generate a candidate relationship set. If any of the above conditions is not met, the sentence labeling fails.

[0038] Step 5: Generate a similar vocabulary for each word in the relation phrase in the entity relation triple, and combine it with the relation type vocabulary to determine the candidate relation expressed by the word. The candidate relation with the most expressions is the labeled relation. The method for completing automatic labeling is as follows:

[0039] For each word in the relational phrase, the BERT model is used to generate a corresponding similar vocabulary list, which is then compared with the candidate relation type vocabulary list. The relation with the largest number of identical words is selected (they can be selected in parallel, i.e., there are multiple relations to be expressed). If the number reaches the set threshold, the word expresses the relation type, where the threshold is set to 20. Comprehensively considering all the words in the relational phrase, the relation with the largest number of expressions is the labeled relation (they can be selected in parallel, i.e., there are multiple labeled relations). If the relational phrase does not have a word that expresses any of the candidate relations, the corpus is discarded.

[0040] Assume A i ,i=1,2,3,...,m represents the similarity vocabulary of the i-th word in the relational phrase, where m represents the number of words in the relational phrase, B j, where \(j = 1, 2, 3, \ldots, n\) represents the type vocabulary of the \(j\)th relationship in the candidate relationships, \(n\) represents the number of candidate relationships, and \(h\) represents the same vocabulary number threshold. Let the relationship expressed by each word in the relationship phrase be \(W\). i , where \(i = 1, 2, 3, \ldots, m\), and the number of the same vocabulary corresponding to its corresponding relationship is \(K\). i , where \(i = 1, 2, 3, \ldots, m\), then:

[0041] K i =\max(|A i \cap B j |), j = 1, 2, 3, \ldots, n ①

[0042]

[0043] If \(K i \geq h\), then the \(i\)th word expresses the relationship \(W\). i If \(K i < h\), then the \(i\)th word does not express any relationship. The relationship expressed by the most words is the labeled relationship. If none of them express a relationship, then this corpus is discarded.

[0044] Suppose the relationship phrase has four words \(a\), \(b\), \(c\), \(d\), and there are three candidate relationships \(\alpha\), \(\beta\), \(\gamma\) to be extracted. Regarding the example of the largest number of the same vocabulary: The number of the same vocabulary between the similar vocabulary list of word \(a\) and the type vocabulary lists of \(\alpha\), \(\beta\), \(\gamma\) is 2, 8, 8 respectively. Since the number of the same vocabulary between \(a\) and \(\beta\), \(\gamma\) is the same, word \(a\) expresses both the \(\beta\) relationship and the \(\gamma\) relationship. Regarding the example of the relationship with the most expression times: Word \(a\) expresses the \(\alpha\), \(\beta\) relationships, word \(b\) does not express a relationship, word \(c\) expresses the \(\alpha\) relationship, and word \(d\) expresses the \(\beta\) relationship. Finally, this sentence expresses the \(a\), \(\beta\) two relationships.

[0045] The present invention also proposes a text relationship automatic annotation system based on a similar vocabulary list, which realizes text relationship automatic annotation based on the text relationship automatic annotation method based on a similar vocabulary list.

[0046] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it realizes text relationship automatic annotation based on the text relationship automatic annotation method based on a similar vocabulary list.

[0047] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it realizes text relationship automatic annotation based on the text relationship automatic annotation method based on a similar vocabulary list.

[0048] Embodiment

[0049] In order to verify the effectiveness of the solution of the present invention, the following simulation experiment is carried out to establish a character relationship data set.

[0050] Example: In Five's earlier days, Kieran Conlon was briefly engaged to dancer Suzanne Mole, who toured with the group.

[0051] The first step is to determine some relationship types related to characters. Here we select four relationships: parent, couple, sibling, and friend, whose corresponding entity types are all characters. Then, we obtain corpus from the Internet based on these relationship names.

[0052] The second step is to generate a similar vocabulary for all the occurrences of parent, couple, sibling, and friend in the corpus, and then organize and generate a type vocabulary for each relationship.

[0053] In the third step, the named entity recognition model and open domain information extraction tool are used on the sentence to obtain entities (Sean Kieran Conlon, person), (Suzanne Mole, person); and the open domain information extraction tool is used to obtain the relation phrase triple: (Sean Kieran Conlon, was briefly engaged to, dancer Suzanne Mole).

[0054] Step 4: Determine whether the relation triple meets the conditions based on the named entity recognition result. The entity 1Sean Kieran Conlon of the relation phrase is in the named entity recognition result set, and the entity 2dancer Suzanne Mole contains the named entity recognition result Suzanne Mole, and both entity types are people, which meets the conditions. The candidate relations are parent, couple, sibling, and friend.

[0055] Step 5: Compare the similarity lists generated by the four words of the relation phrase, was, briefly, engaged, and to, with the type vocabularies of the candidate relations. Finally, the number of words in the similarity lists of was, briefly, and to that are the same as the type vocabularies of the four relations is less than 20, so they do not express any meaning. There are 21 similar words between engaged and couple, expressing a couple relationship, so the entire semantics expresses a couple relationship. The final annotation result is (Sean Kieran Conlon, couple, Suzanne Mole), and the automatic annotation is completed.

[0056] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0057] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for automatically annotating textual relations based on a similar vocabulary, characterized in that: Here are the steps: Step 1: determine all extracted relationship names and corresponding entity types, and obtain the corpus containing the relationship names; Step 2: Generate a similar vocabulary that can replace the relationship name at each place where the relationship name appears in the corpus, and use this to organize the relationship type vocabulary; Step 3, perform named entity recognition and entity relationship triple extraction on the sentence; Step 4, judging whether the entity pair of the entity relationship triple meets the conditions according to the result of the named entity recognition, and generating a candidate relationship set; Step 5: Generate a similar vocabulary list for each word in the relation phrase in the entity relation triple, and combine it with the relation type vocabulary list to determine the candidate relation expressed by the word. The candidate relation expressed the most times among all words is the labeled relation, and the automatic labeling is completed. Step 2: Generate a similar vocabulary that can replace the relationship name at each place where the relationship name appears in the corpus, and use it to organize the relationship type vocabulary. The specific method is: For all the places where the relationship names appear in the corpus, the BERT model is used to replace the relationship names with similar words according to the context, and a similar word list corresponding to the relationship at that position is generated. The first 40 words are taken from each similar word list, and all are selected if there are no more than 40 words. For all the similar word lists generated for each relationship in the corpus, the word frequency statistics are first sorted. The words with more appearances represent closer relationships with the relationship. Then, stop words are removed and the first 80 words are taken as the type word list of the relationship. If there are no more than 80 words, all are selected. Step 5: Generate a similar vocabulary list for each word in the relation phrase in the entity relation triple, and combine it with the relation type vocabulary list to determine the candidate relation expressed by the word. The candidate relation expressed the most times among all words is the labeled relation, and the automatic labeling is completed. The specific method is: For each word in the relation phrase, the BERT model is used to generate a corresponding similar vocabulary list, which is then compared with the type vocabulary list of candidate relations. The relation with the largest number of identical words is selected. If the number reaches the set threshold, the word expresses the relation type, where the threshold is set to 20. Considering all the words in the relation phrase, the relation with the largest number of expressions is the labeled relation. If the relation phrase does not have any words that express any candidate relation, the corpus is discarded.

2. The method for automatically annotating text relations based on similar vocabulary according to claim 1, characterized in that: Step 3: perform named entity recognition and entity relationship triple extraction on the sentence. The specific method is as follows: Using the named entity recognition model, the corpus is treated as sentences, and the entities existing in the sentences are extracted to generate a set containing entities and entity types; using the open information extraction tool, the corpus is treated as sentences, and the entity pairs and relational phrases describing the relationship in the sentences are extracted, and finally the construction is completed in the form of triples.

3. The method for automatically annotating textual relations based on similar vocabulary according to claim 1, characterized in that: Step 4: Based on the result of named entity recognition, determine whether the entity pair of the entity relationship triple meets the conditions and generate a candidate relationship set. The specific method is as follows: For each sentence's entity set and entity relationship triplet, first determine whether the entity pair of the relationship triplet contains or is equal to the entity in the named entity recognition result. If so, change the triplet entity to the entity corresponding to the named entity recognition result. Then determine whether the entity type of the replaced entity pair is the same as the entity type of the target extracted relationship. If so, generate a candidate relationship set. If any of the above conditions is not met, the sentence labeling fails.

4. A text relationship automatic annotation system based on similar vocabulary, characterized in that: Based on the method for automatic annotation of text relations based on a similar vocabulary list as described in any one of claims 1 to 3, automatic annotation of text relations based on a similar vocabulary list is implemented.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for automatic tagging of text relations based on a similar vocabulary list according to any one of claims 1 to 3 is used to implement automatic tagging of text relations based on a similar vocabulary list.

6. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for automatically tagging text relations based on a similar vocabulary list according to any one of claims 1 to 3 is used to implement automatic tagging of text relations based on a similar vocabulary list.

Images