Entity attribute generation method and system based on remote supervision and prompt learning

Abstract

The present invention discloses a method and system for generating entity attributes based on remote supervision and prompt learning, which relates to the field of knowledge graphs. The method includes S1 constructing a knowledge graph; S2 entity e, e's a ke , and a ve The triples composed of e and a are retrieved in the open document library D. ve , recall text set D e ; S3 will D e Correct expression of a ke d e Marked as positive text s e , the set C is a triple with s e A set of four tuples composed of: S4 obtains text x; S5 identifies e in x and extracts a of e ke ; S6 will x, e and a ke Convert to x prompt ; S7 obtains the attribute value of the entity; the system includes a knowledge graph, an open document database, a prompt learning template engine, a pre-trained language module and an entity recognition module; prompt learning is used in entity attribute extraction to effectively solve the problems existing in traditional classifiers.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graphs, and in particular to a method and system for generating entity attributes based on remote supervision and prompt learning. Background Art

[0002] Knowledge construction technology is the cornerstone of the entire knowledge graph technology. Knowledge construction includes importing knowledge from structured data and extracting knowledge from unstructured documents. There is a large amount of unstructured data in the open world of the Internet. It is very necessary to study how to build knowledge from unstructured documents. Extracting knowledge from unstructured documents requires extracting knowledge elements such as entities, attributes, relationships, and events in the documents. Among them, attribute extraction, as an important means of knowledge extraction, has attracted the attention of many researchers. Existing methods usually regard attribute extraction as a relationship extraction task. The relationship extraction task is often based on classification methods. First, each entity in the text is identified, and then the semantic representation features of the text are input into the classifier to classify the relationship between the two entities in the text. Using classification-based methods for attribute extraction mainly has the following two problems:

[0003] 1. Attribute extraction is different from relationship extraction. The attribute value to be extracted may not be an entity type in the strict sense, but a free text fragment. The classifier-based relationship extraction method can only extract the relationship between two known entities.

[0004] 2. Classification-based methods require careful design of classifiers when using pre-trained models for relationship extraction, which is very tedious. Summary of the Invention

[0005] The purpose of the present invention is to design a method and system for generating entity attributes based on remote supervision and prompt learning in order to solve the above problems.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0007] Entity attribute generation methods based on distant supervision and hint learning, including:

[0008] S1. Build a knowledge graph , where E is the set of entities in the knowledge graph, and R is the set of relations in the knowledge graph. It is a set of attributes in the knowledge graph, and a is a name containing the attribute and attribute values tuples of ;

[0009] S2. Using remote supervision methods, entities in the graph , the attribute name of entity e , and its attribute names Corresponding attribute value The triplet As a seed, through the entity e and attribute value Go to the open document library D to search and recall both entity e and attribute value Text collection ;

[0010] S3, the text set D e The attribute type can be correctly expressed in Text d e Mark as positive example , define the set For all seed triples in the knowledge graph ( ) Recalled positive text A set of four tuples composed of

[0011] S4. Obtain the text x of the entity attribute to be generated;

[0012] S5. Use the entity recognition model to identify each entity e in the text x and extract the attribute name of entity e ;

[0013] S6, pass Text x, entity e, attribute name Converted into an input form that prompts learning tasks ;

[0014] S7, will As the input of the pre-trained language model BERT, the attribute values ​​of the entities in the text x are obtained.

[0015] Entity attribute generation system based on distant supervision and hint learning, including:

[0016] Knowledge graph; the knowledge graph stores triple information such as entities, relationships, and attributes.

[0017] Open Document Database;

[0018] Prompt learning template engine; the prompt learning template engine is used to convert text input into a text format that can be used for prompt learning, and is used for prompting and training language models to fine-tune downstream entity attribute extraction tasks;

[0019] Pre-training language module: The pre-training language module pre-trains the corpus and then uses the prompt learning form text generated by the prompt learning template engine as a sample to fine-tune downstream tasks.

[0020] Entity recognition module for entity extraction from online business data.

[0021] The beneficial effect of the present invention is that this method uses prompt learning in entity attribute extraction to effectively solve the problems existing in traditional classifiers. On the one hand, traditional classifiers usually adjust model parameters based on pre-trained language models. With the development of technology, pre-trained language models are getting larger and larger, and the cost of training is also getting higher and higher. Prompt learning does not require adjustment of the parameters of the original pre-trained model, which optimizes the training method of downstream tasks. On the other hand, by using prompt learning for attribute generation, the flexibility of attribute extraction is increased. Not only can the attributes of specific entity types be extracted, but also free text fragments can be extracted from the article as attribute values. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a system structure diagram of the entity attribute generation method based on remote supervision and prompt learning of the present invention;

[0023] Figure 2 It is a flow chart of the entity attribute generation method based on remote supervision and prompt learning of the present invention. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be understood that the described embodiments are only a portion of the embodiments of the present invention, not all of them. Generally, the components of the embodiments of the present invention described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

[0025] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0026] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0027] In the description of the present invention, it should be understood that the terms "upper", "lower", "inside", "outside", "left", "right", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the inventive product is conventionally placed when in use, or are the orientations or positional relationships conventionally understood by those skilled in the art. These are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or component referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0028] Furthermore, the terms “first”, “second”, etc. are merely used for distinguishing descriptions and should not be understood as indicating or implying relative importance.

[0029] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, terms such as "disposed" and "connected" should be understood in a broad sense. For example, "connected" can mean a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can also mean internal communication between two components. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0030] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0031] Entity attribute generation methods based on distant supervision and hint learning, including:

[0032] S1. Build a knowledge graph , where E is the set of entities in the knowledge graph, and R is the set of relations in the knowledge graph. It is a set of attributes in the knowledge graph, and a is a name containing the attribute and attribute values tuples of ;

[0033] S2. Using remote supervision methods, entities in the graph , the attribute name of entity e , and its attribute names Corresponding attribute value The triplet As a seed, through the entity e and attribute value Go to the open document library D to search and recall both entity e and attribute value Text collection ;

[0034] S3, the text set D e The attribute type can be correctly expressed in Text d e Mark as positive example , define the set For all seed triples in the knowledge graph ( ) Recalled positive text A set of four tuples composed of

[0035] S4. Obtain the text x of the entity attribute to be generated;

[0036] S5. Use the entity recognition model to identify each entity e in the text x and extract the attribute name of entity e ;

[0037] S6, pass Text x, entity e, attribute name Converted into an input form that prompts learning tasks ;

[0038] S7, will As the input of the pre-trained language model BERT, the attribute values ​​of the entities in the text x are obtained; specifically:

[0039] S71, will As the input of the pre-trained language model BERT, let it predict The probability distribution of the word vocabulary that fills the middle slot [z];

[0040] S72. Take the word with the largest probability value and insert it before slot [z] to form a new prompt learning template:

[0041]

[0042] Where v represents the vocabulary in the document vocabulary, Represents the word with the highest probability; The function is to perform the above insertion operation to modify the current sample;

[0043] S73, determine whether the generated word is the end word "." of the sentence, if so, enter S74, otherwise, return to S71 with the modified current sample;

[0044] S74. Delete the termination symbol “.” in the modified current sample and use it as the attribute value of the entity.

[0045] The following steps are involved:

[0046] 1) Define the template t as:

[0047] [x]; Attribute extraction: [Entity] [e] - [Attribute name] [a] - [Answer] [z]

[0048] The template content t is a segment consisting of input slot [x] and entity slot [e], entity attribute slot , a text string composed of the answer slot [z], where the input slot [x], entity slot [e], entity attribute slot They are the input text x, entity e, and the name of the attribute to be extracted for the attribute extraction task based on prompt learning. The answer slot [z] is the value of the attribute that needs to be extracted by entity e. Placeholder for

[0049] 2) Fill the input text x into the position of template t input slot [x], and fill the entity e into template t entity slot [e]. The name of the attribute to be extracted from entity e Fill in the template t entity attribute slot ;

[0050] 3) Return the padded string as the prompt form of input x.

[0051] Entity attribute generation system based on distant supervision and hint learning, including:

[0052] Knowledge graph; the knowledge graph stores triple information such as entities, relationships, and attributes.

[0053] Open Document Database;

[0054] Prompt learning template engine; the prompt learning template engine is used to convert text input into a text format that can be used for prompt learning, and is used for prompting and training language models to fine-tune downstream entity attribute extraction tasks;

[0055] Pre-training language module: The pre-training language module pre-trains the corpus and then uses the prompt learning form text generated by the prompt learning template engine as a sample to fine-tune downstream tasks.

[0056] Entity recognition module for entity extraction from online business data.

[0057] Example

[0058] Using entity-attribute triples in the knowledge graph, we retrieve a document set D from an open document repository using a remote supervision method. Specifically, Figure 2 The knowledge graph in [1] includes person entities such as Zhang San, Li Si, and Wang Wu, as well as occupation entities such as singer. Person entities include attributes such as birthplace and date of birth. Taking the entity Wang Wu as an example, we use its birthplace attribute triple (Wang Wu, birthplace, XXX place) to recall documents related to the entity Wang Wu and the attribute value XXX place. Based on the semantic information of the recalled documents, we select documents that correctly describe Wang Wu's "birthplace" attribute, such as "Wang Wu, born on XXX day of XXX year in XXX place in China, native place of Xinhui, Guangdong." as the positive example set.

[0059] Learning template functions through hints Generate positive text with template t Specifically, the positive example text "Wang Wu, born on X-month-X-year in a certain place in China, native place: Xinhui, Guangdong." and its corresponding entity "Wang Wu" and the attribute name "birthplace" of the entity "Wang Wu" are used as inputs to the template function, which uses the prompt to learn the template:

[0060] "[x]; Attribute extraction:

Entity

Attribute Name

Answer

[0061] The prompt form for generating positive example texts:

[0062] "Wang Wu was born in a certain place in China on X year X month X day, and his native place is Xinhui, Guangdong. ; Attribute extraction:

Entity

Attribute Name

Answer

[0063] For each seed entity - attribute triple in the knowledge graph, generate its positive example prompt form as the input sample for the entity - attribute generation task of the pre - trained language model BERT, fine - tune BERT, update the model parameters of BERT, and save them.

[0064] Entity - attribute generation stage: For online business texts, first use the entity recognition model to extract the entities in them. Then, combined with the ontology structure of the knowledge graph, take the input text, the recognized entities, and the attribute names of the entities in the ontology as the input of the prompt learning template function to generate the prompt form of the input text, and send it into the previously fine - tuned BERT model to predict the probability of the vocabulary distribution at the slot [Z] position. Take the vocabulary corresponding to the maximum probability and insert it before the slot [z]. Repeat modifying the sample and predicting the result until the predicted vocabulary is the end symbol "." and then stop. After the prediction, sequentially take the predicted vocabulary results of each step and splice them as the generated entity - attribute value. Specifically, for the online input text "Xiaoming was born in a certain place in China on X year X month X day, his ancestral home is Yongchun County, Quanzhou City, Fujian Province, and he is a male pop singer and musician in a certain place in China.", first extract the entity of the person type "Xiaoming" and the entities of the occupation type "singer", "musician", etc. According to the ontology structure, the person - type entity has attributes such as "place of birth" and "date of birth". Taking the extraction of the "place of birth" attribute as an example, input the text "Xiaoming was born in a certain place in China on X year X month X day, his ancestral home is Yongchun County, Quanzhou City, Fujian Province, and he is a male pop singer and musician in a certain place in China.", the person - type entity "Xiaoming", and the task - type entity attribute "place of birth" as the input of the prompt learning template function to obtain the prompt form text of the input text:

[0065] "Xiaoming was born in a certain place in China on X year X month X day, his ancestral home is Yongchun County, Quanzhou City, Fujian Province, and he is a male pop singer and musician in a certain place in China. ; Attribute extraction:

Entity

Attribute Name

Answer

[0066] Use this prompt form text as a sample and fill the slot [z] with the [MASK] token, input it into the fine - tuned pre - trained language model BERT, predict the vocabulary probability distribution of the slot [MASK], take the word "a certain" with the largest probability distribution, and modify the previous sample to

[0067] "Xiao Ming, born on X-month-X-year in a certain place in China, his ancestral home is Yongchun County, Quanzhou City, Fujian Province, and he is a male pop singer and musician from a certain place in China. Attribute extraction: [Entity] Xiao Ming - [Attribute Name] Birthplace - [Answer] [z]"

[0068] Continue to input the BERT pre-trained model to predict the probability distribution of the word for slot [z]. This continues until the last word generated is the terminal symbol ".". The final result is as follows:

[0069] "Xiao Ming, born on X-month-X-year in a certain place in China, his ancestral home is Yongchun County, Quanzhou City, Fujian Province, and he is a male pop singer and musician from a certain place in China. Attribute extraction: [Entity] Xiao Ming - [Attribute Name] Birthplace - [Answer] A certain place in China"

[0070] The last generated result "a certain place in China" is taken as the attribute value "a certain place in China" of the "birthplace" attribute of the entity "Xiao Ming".

[0071] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. Any technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims

1. Entity attribute generation method based on remote supervision and prompt learning, characterized by: include: S1. Build a knowledge graph ,in is the set of entities in the knowledge graph, is the set of relations in the knowledge graph, is a set of attributes in the knowledge graph. is a string containing the attribute name and attribute values tuples of ; S2. Using remote supervision methods, entities in the knowledge graph ,entity Attribute name , and its attribute names Corresponding attribute value The triplet As a seed, through the entity and attribute values Go to Open Document Library To search, recall both entities and attribute values Text collection ; S3, the text set D e The attribute type can be correctly expressed in Text Marked as positive text , define the set For all seed triples in the knowledge graph ( ) Recalled positive text A set of four tuples composed of S4. Get the text of the entity attributes to be generated ; S5. Use entity recognition models to identify text For each entity e in the , extract the attribute name of entity e ; S6, pass Text ,entity , attribute name Converted into an input form that prompts learning tasks ; S7, will As input to the pre-trained language model BERT, get the text The attribute value of the entity in .

2. The entity attribute generation method based on remote supervision and prompt learning according to claim 1 is characterized in that The following steps are involved: 1) Define the template t as: [x]; attribute extraction: [entity] [e]-[attribute name] [a]-[answer] [z], The template content t is a segment consisting of input slots and entity slots , entity attribute slot , answer slot The concatenated text string, where the input slot , physical slot , entity attribute slot The input text of the attribute extraction task based on prompt learning is ,entity , and its entities The name of the attribute to be extracted Placeholder for answer slot is an entity The value of the attribute to be extracted Placeholder for 2) Input text Fill the template t input slot location, entity Fill in the template t entity slot ,entity The name of the attribute to be extracted Fill in the template t entity attribute slot ; 3) Return the padded string as the prompt form of input x.

3. The entity attribute generation method based on remote supervision and prompt learning according to claim 1 is characterized in that Included in S7: S71, will As the input of the pre-trained language model BERT, let it predict The probability distribution of the word vocabulary that fills the middle slot [z]; S72. Take the word with the largest probability value and insert it before slot [z] to form a new prompt learning template: ， in Represents the words in the document vocabulary, Represents the word with the highest probability; The function is to perform the above insertion operation to modify the current sample; S73, determine whether the generated word is the end word of the sentence ".", if so, enter S74, otherwise, use the modified current sample to return to S71; S74. Delete the termination symbol "." in the modified current sample and use it as the attribute value of the entity.

4. A system for implementing the entity attribute generation method based on remote supervision and prompt learning according to any one of claims 1 to 3, characterized in that: include: Knowledge graph; The knowledge graph stores triple information of entities, relationships, and attributes; Open Document Database; Tips for learning template engines; The prompt learning template engine is used to convert text input into a text format that can be used for prompt learning. It is used to prompt and train the language model to fine-tune the downstream entity attribute extraction task. Pre-training language module: The pre-training language module pre-trains the corpus and then uses the prompt learning form text generated by the prompt learning template engine as a sample to fine-tune downstream tasks. Entity recognition module for entity extraction from online business data.

Images