A method, terminal and storage medium for joint extraction of entity relations

By obtaining and processing character vectors in text in joint extraction of entity relationships, and combining pre-trained word vectors, the accuracy of entity and relationship extraction is improved, and the problem of failure to effectively consider all entity information in the prior art is solved.

CN112364650BInactive Publication Date: 2024-11-29SHENZHEN LUOHU PEOPLELS HOSPITAL +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202011060126.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-09-30
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The prior art fails to effectively consider the information of all entities in the text in joint extraction of entity relations, resulting in poor accuracy.

Method used

By obtaining the initial word vector of each character in the target text, and combining the pretrained word vector, the intermediate word vector and the input word vector are calculated, and inputted to the pretrained neural network to realize entity recognition and relationship extraction, and sharing characters in the text and word information.

Benefits of technology

Improve the accuracy of joint extraction of entities and relationships, allowing the two types of tasks to share text information more effectively.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112364650B_ABST
    Figure CN112364650B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, a terminal and a storage medium for joint extraction of entity relations. The method for joint extraction of entity relations provided by the present invention first processes the word vector of the target word existing in the target text with the initial word vector of each character in the target text to obtain the input word vector of each character, uses the input word vector of each character as the input of a first neural network for outputting entity recognition results to output the entity recognition results, and then obtains the input of a second neural network for outputting relationship extraction results according to the entity recognition results of the target text and the input word vector of each character in the target text, so that the two types of tasks share the information of characters and words in the target text, thereby improving the accuracy of joint extraction of entities and relationships.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of natural language technology, and in particular to an entity relationship joint extraction method, a terminal and a storage medium. Background Art

[0002] Named entity recognition and relationship extraction are basic tasks in the field of natural language processing. Natural language text is composed of characters, and the combination of characters constitutes units, such as words, sentences, paragraphs, etc. The named entity recognition (NER) task is to find each named entity mentioned in the text, such as noun phrases, names of people and places, etc. Entity relationship extraction is to discover the semantic relationship between named entities in the text. These relationships are usually binary relationships, such as child relationships, part-whole relationships, and geographic spatial relationships.

[0003] In the prior art, the joint model of named entity recognition and relation extraction does not directly consider the information of words in the text in the entity recognition task, nor does it directly consider the information of all characters in the text in the relation extraction task, and its accuracy is poor.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0005] In view of the above-mentioned defects of the prior art, a method, terminal and storage medium for joint extraction of entity relationships are provided, aiming to solve the problem in the prior art that the information of all entities is not directly considered in the relationship extraction task during joint extraction of entity relationships, resulting in poor accuracy.

[0006] A first aspect of the present invention provides a method for joint extraction of entity relationships, comprising:

[0007] Obtain a target text, convert each character in the target text into an initial word vector, and determine whether there is a target word corresponding to the pre-trained word vector in the target text;

[0008] If so, obtaining an intermediate word vector for each character in the target text according to the pre-trained word vector of the target word and the initial word vector of the character included in the target word;

[0009] Connecting the intermediate word vector of each character in the target text and the sum of the intermediate word vectors of each character in the target text in parallel to obtain input word vectors corresponding to each character;

[0010] Inputting the input word vector into a pre-trained first neural network, and obtaining an entity recognition result of the target text through the first neural network;

[0011] Obtaining input data for inputting into a pre-trained second neural network according to the entity recognition result and the input word vector, and obtaining an entity relationship extraction result of the target text through the second neural network;

[0012] The first neural network and the second neural network are trained together by a training set including multiple groups of training data, each group of training data includes sample text, entity annotation results corresponding to the sample text, and entity relationship annotation results corresponding to the sample text.

[0013] The entity relationship joint extraction method, wherein the step of obtaining the intermediate word vector of each character in the target text according to the pre-trained word vector of the target word and the initial word vector of the characters included in the target word, comprises:

[0014] For the characters included in the target word, the pre-trained word vector of the target word is input into a pre-trained third neural network, and in the third neural network, the pre-trained word vector of the target word is subjected to multi-head attention processing to obtain the weight corresponding to each character in the target word, and the pre-trained word vector of the target word is added to the initial word vector of each character according to the weight corresponding to the character in the target word to obtain the intermediate word vector of each character;

[0015] For a character that does not have a corresponding target word, using the corresponding initial word vector as the intermediate word vector;

[0016] The third neural network is trained together with the first neural network and the second neural network through the training set.

[0017] The entity relationship joint extraction method, wherein the first neural network includes a first encoder module and a multi-layer perception module; the entity recognition result of the target text obtained by the first neural network includes:

[0018] Encode each of the input word vectors by using the first encoder module to obtain an encoded word vector;

[0019] Using the encoded word vector as the input of the multi-layer perception module, and obtaining the result output by the multi-layer perception module as the entity recognition result of the target text;

[0020] The first encoder module includes multiple layers of encoders, and the input of each layer of encoders includes the outputs of all previous encoders in the first encoder module.

[0021] The entity relationship joint extraction method, wherein the step of obtaining input data for inputting into a pre-trained second neural network according to the entity recognition result and the input word vector, comprises:

[0022] Obtaining the encoded word vector output after the first encoder module encodes the input word vector;

[0023] The input word vectors of characters belonging to the same entity word in the target text are summed to obtain respective input word vectors as input data of the second neural network.

[0024] The entity relationship joint extraction method, wherein the summing of the input word vectors of the characters belonging to the same entity word in the target text comprises:

[0025] In the pre-trained fourth neural network, the weights of the characters belonging to the same entity word are obtained through a multi-head attention mechanism, and the input word vectors of the characters belonging to the same entity word are summed according to the weights of the characters;

[0026] The fourth neural network, the first neural network and the second neural network are trained together through the training set.

[0027] The entity relationship joint extraction method, wherein the second neural network includes a second encoder module and a relationship classification module; the entity relationship extraction result of the target text obtained by the second neural network includes:

[0028] Connecting each of the input word vectors in parallel with the sum of the encoded word vectors of all characters to obtain each intermediate word vector, and encoding each of the intermediate word vectors through the second encoder module to obtain an encoded word vector;

[0029] Using each of the encoded word vectors as the input of the relationship classification module, and obtaining the output of the relationship classification module as the entity relationship extraction result of the target text;

[0030] The second encoder module includes multiple layers of encoders, and the input of each layer of encoders includes the outputs of all previous encoders in the second encoder module.

[0031] The entity relationship joint extraction method, wherein before obtaining the target text, the method further includes:

[0032] In the process of training the first neural network and the second neural network, obtaining a target loss value corresponding to each group of training data in the training set;

[0033] The parameters in the first neural network and the second neural network are updated according to the target loss value to implement training of the first neural network and the second neural network.

[0034] The entity relationship joint extraction method, wherein the step of obtaining the target loss value corresponding to each set of training data in the training set includes:

[0035] For the target sample text:

[0036] Obtaining a predicted entity recognition result of the target sample text through the second neural network, and obtaining a first loss value according to the predicted entity recognition result and an entity labeling result of the target sample text;

[0037] According to the entity annotation result of the target sample text, the coded word vectors corresponding to each character in the target sample text are summed to obtain the sample word vector corresponding to the target sample text, the mean vector of the sample word vectors belonging to the same category entity is calculated, and the similarity between the mean vector of each category entity and the preset label vectors corresponding to other categories is used as the second loss value;

[0038] Obtaining a predicted entity relationship extraction result of the target sample text through the second neural network, and obtaining a third loss value according to the predicted entity relationship extraction result and the entity relationship labeling result of the target sample text;

[0039] The sum of the first loss value, the second loss value and the third loss value is obtained as the target loss value.

[0040] The second aspect of the present invention provides a terminal, comprising: a processor, a storage medium communicatively connected to the processor, the storage medium being suitable for storing multiple instructions, and the processor being suitable for calling the instructions in the storage medium to execute the steps of implementing any of the above-mentioned entity relationship joint extraction methods.

[0041] According to a third aspect of the present invention, a storage medium is provided, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any of the above-mentioned entity relationship joint extraction methods.

[0042] Beneficial effect: Compared with the prior art, the present invention provides an entity-relationship joint extraction method, a terminal and a storage medium. The entity-relationship joint extraction method provided by the present invention first processes the initial word vector of each character in the target text with the word vector of the target word existing in the target text to obtain the input word vector of each character, and uses the input word vector of each character as the input of the first neural network for outputting the entity recognition result to output the entity recognition result, and then obtains the input of the second neural network for outputting the relationship extraction result according to the entity recognition result of the target text and the input word vector of each character in the target text, so that the two types of tasks share the information of characters and words in the target text, thereby improving the accuracy of the joint extraction of entities and relationships. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 An overall flow chart of an embodiment of the entity relationship joint extraction method provided by the present invention;

[0044] Figure 2 A schematic diagram of obtaining intermediate word vectors in an embodiment of the entity relationship joint extraction method provided by the present invention;

[0045] Figure 3 A schematic diagram of a specific flow chart of an embodiment of the entity relationship joint extraction method provided by the present invention;

[0046] Figure 4 A structural principle diagram of an embodiment of a terminal provided by the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] The entity relationship joint extraction method provided by the present invention can be applied in a terminal, and the terminal can be but not limited to various personal computers, laptops, mobile phones, tablet computers, etc.

[0049] Embodiment 1

[0050] like Figure 1 As shown, the entity relationship joint extraction method provided by the present invention includes the steps of:

[0051] S100, obtaining a target text, converting each character in the target text into an initial word vector, and determining whether there is a target word corresponding to the pre-trained word vector in the target text;

[0052] S200, if it exists, obtaining an intermediate word vector for each character in the target text according to the pre-trained word vector of the target word and the initial word vector of the character included in the target word;

[0053] S300, connecting the intermediate word vector of each character in the target text and the sum of the intermediate word vectors of each character in the target text in parallel, respectively, to obtain input word vectors corresponding to each character;

[0054] S400, inputting the input word vector into a pre-trained second neural network, and obtaining an entity recognition result of the target text through the second neural network;

[0055] S500, obtaining input data for inputting into a pre-trained third neural network according to the entity recognition result and the input word vector, and obtaining an entity relationship extraction result of the target text through the third neural network.

[0056] The first neural network and the second neural network are trained together using a training set including multiple sets of training data, each set of training data including sample text, entity annotation results corresponding to the sample text, and entity relationship annotation results corresponding to the sample text. Specifically, the steps of the entity relationship joint extraction method provided in this embodiment can be implemented in an entity relationship joint extraction model, which includes a first neural network for outputting entity recognition results and a second neural network for outputting entity relationship extraction results. The model may also include other neural networks connected to the first neural network and the second neural network, and all neural networks included in the model are trained as a whole, that is, the training of the model is end-to-end, which will be described in detail later.

[0057] The following is a detailed description of each step:

[0058] S100, obtaining a target text, converting each character in the target text into an initial word vector, and determining whether there is a target word corresponding to the pre-trained word vector in the target text.

[0059] The target text is a text to be subjected to named entity recognition (abbreviated as entity recognition) and entity relationship extraction. The target text is composed of multiple characters. After obtaining the target text, the vector of each character in the target text is obtained as the initial word vector of the character, and at the same time, it is determined whether there is a target word corresponding to the pre-trained word vector in the target text.

[0060] Specifically, the pre-trained word vectors are word vectors obtained by pre-selecting multiple words for training, such as the ctb.50d.vec pre-trained word vectors. Each pre-trained word vector corresponds to a word (target word). After obtaining the target text, it is determined whether the target word exists in the target text. As Figure 2 shown, for the text "Wuhan Yangtze River Bridge", the existing target words are: "Wuhan", "Wuchang", "Mayor", "Yangtze River", "Bridge", "Yangtze River Bridge".

[0061] S200. If it exists, then according to the pre-trained word vector of the target word and the initial word vector of the characters included in the target word, the intermediate word vector of each character in the target text is obtained.

[0062] The target word includes characters. For a single character included in a target word, the intermediate word vector of the character is obtained by combining the initial word vector of the character and the word vector of the target word. As Figure 2 shown, for the target word "Wuchang", which includes three characters "Wu", "Chang", and "City", the intermediate word vector corresponding to the character "Wu" is obtained from the initial word vector of "Wu" and the pre-trained word vector of "Wuchang".

[0063] Specifically, the step of obtaining the intermediate word vector of each character in the target text according to the pre-trained word vector of the target word and the initial word vector of the characters included in the target word includes:

[0064] For the characters included in the target word, the pre-trained word vector of the target word is input into a pre-trained third neural network. In the third neural network, multi-head attention processing is performed on the pre-trained word vector of the target word to obtain the weights corresponding to each character in the target word. The pre-trained word vector of the target word and the initial word vectors of each character are added respectively according to the weights corresponding to the characters in the target word to obtain the intermediate word vector of each character.

[0065] For the characters included in the target word, it can be directly adding the initial word vector of the character and the pre-trained word vector of the corresponding target word to obtain the intermediate word vector of the character. In this embodiment, in order to improve the accuracy, multi-head attention processing is also performed on the pre-trained word vector of the target word to obtain the word vector weights corresponding to each character in the target word. When summing the intermediate word vector of the character in the target word and the corresponding pre-trained word vector, considering the weight of the character, that is, multiplying the corresponding pre-trained word vector by the weight value and then concatenating it with the intermediate word vector of the character.

[0066] The weight of each character in the target word can be obtained through a third neural network, and the third neural network is an attention network that can realize multi-head attention processing. When the intermediate word vector of each character is obtained through the third neural network, the third neural network is trained together with the first neural network and the second neural network as a whole through a training set. The training set includes multiple groups of training data, and each group of training data includes sample text, entity annotation results corresponding to the sample text, and entity relationship annotation results corresponding to the sample text.

[0067] For a character that does not have a corresponding target word, the corresponding initial word vector is used as the intermediate word vector.

[0068] like Figure 2 As shown, the process of obtaining the intermediate word vector can be regarded as converting the target text into a heterogeneous graph structure. Characters and words are two types of nodes in the graph. Words are composed of characters and are connected by edges.

[0069] S300 , connecting the intermediate word vector of each character in the target text and the sum of the intermediate word vectors of each character in the target text in parallel, respectively, to obtain input word vectors corresponding to each character.

[0070] Specifically, after obtaining the intermediate word vector corresponding to each character of the target text, the intermediate word vector corresponding to each character in the target text is summed, and the sum of each intermediate word vector is connected in parallel with the intermediate word vector of each character to obtain the input word vector corresponding to each character. The input word vector can be regarded as adding a global node to the heterogeneous graph structure corresponding to the target text, ensuring that the input word vector corresponding to each character can take into account the global information of the target text. In the subsequent processing, the input of the first neural network for outputting the entity recognition result of the target text and the second neural network for outputting the entity relationship extraction result of the target text both include the input word vector. That is to say, in this embodiment, the entity recognition task and the entity relationship extraction task share the information of characters and words in the target text, which can improve the accuracy of the task.

[0071] S400: Input the input word vector to a pre-trained first neural network, and obtain an entity recognition result of the target text through the first neural network.

[0072] Specifically, the first neural network includes a first encoder module and a multi-layer perception module; and obtaining the entity recognition result of the target text through the first neural network includes:

[0073] S410. Encode each of the input word vectors through the first encoder module to obtain encoded word vectors;

[0074] S420. Use the encoded word vectors as the input of the multi-layer perceptron module, and obtain the output of the multi-layer perceptron module as the entity recognition result of the target text.

[0075] The first encoder module includes multiple layers of encoders (Encoder). In this embodiment, the first encoder module includes 4 layers of Transformer encoders. Moreover, the structure of the first encoder module adopts a structure similar to DenseNet, that is, the input of each layer of encoder includes the outputs of all the previous encodings in the first encoder module. For example, the first encoder module includes four encoders A, B, C, and D arranged in the order of A, B, C, and D. That is, if the input of A is the input word vector, then the input of encoder C includes the outputs of encoders A and B. In a possible implementation, the input of each encoder in the first encoder module also includes the input of the first encoder, that is, each of the input word vectors. The input of each layer of encoder including the outputs of all the previous encodings in the first encoder module enables each layer of encoder to receive the information of all the previous layers of encoders, effectively improving the generalization ability of the network and achieving higher accuracy with fewer parameters.

[0076] The relationship classification module is a multi-layer perceptron (MLP). The output of the relationship classification model is the entity label of each character. The entity label of each character in the target text can reflect whether the character belongs to an entity, which characters belong to an entity, and the category of the entity. That is to say, the entity recognition result includes each entity existing in the target text and the category of each entity.

[0077] In this embodiment, the BIO method is used to label characters, that is, the label output for each character contains "B", "I", "O". "B" indicates that the character is the starting character of an entity, "I" indicates that the character is the middle character of an entity, and "O" indicates that the character is not an entity character. For example, for the entity "Yang Zhenning" with the category of person name, the corresponding correct entity labels should be: "B-PER", "I-PER", "I-PER".

[0078] S500. Obtain the input data for inputting into a pre-trained second neural network according to the entity recognition result and the input word vectors, and obtain the entity relationship extraction result of the target text through the second neural network.

[0079] The step of obtaining input data for inputting into a pre-trained second neural network according to the entity recognition result and the input word vector includes:

[0080] S510, obtaining the encoded word vector output after the first encoder module encodes the input word vector;

[0081] S520. Sum the input word vectors of the characters belonging to the same entity word in the target text to obtain each input word vector as input data of the second neural network.

[0082] For specific instructions on how the first encoder module encodes the input word vector and outputs the encoded word vector, please refer to the description of step S400.

[0083] The step of summing the input word vectors of characters belonging to the same entity word in the target text includes:

[0084] In the pre-trained fourth neural network, the weights of the characters belonging to the same entity word are obtained through a multi-head attention mechanism, and the input word vectors of the characters belonging to the same entity word are summed according to the weights of the characters.

[0085] In a possible implementation, the input word vectors of the characters belonging to the same entity word in the target text can be directly summed to obtain the input word vectors. In this embodiment, the input word vectors of the characters belonging to the same entity word are summed by a pre-trained fourth neural network. Specifically, the fourth neural network is an attention network, in which the weights of the characters belonging to the same entity word are obtained through a multi-head attention mechanism, and the input word vectors of the characters belonging to the same entity word are summed according to the weights of the characters, that is, for each character belonging to the same entity word, the corresponding input word vector is multiplied by the corresponding weight and then added to obtain the input word vector corresponding to the entity word.

[0086] In the case of obtaining the input word vector by summing up the fourth neural network, the fourth neural network is trained together with the first neural network and the second neural network through the training set. Of course, it can be understood that in the case where the third neural network also exists, the first neural network, the second neural network, the third neural network and the fourth neural network are all trained together through the training set as a whole.

[0087] The second neural network includes a second encoder module and a relationship classification module; the step of obtaining the entity relationship extraction result of the target text through the second neural network includes:

[0088] S530, connecting each of the input word vectors and the sum of the encoded word vectors of all characters in parallel to obtain each intermediate word vector, and encoding each of the intermediate word vectors through the second encoder module to obtain an encoded word vector;

[0089] S540, using each of the encoded word vectors as an input of the relationship classification module, and obtaining an output of the relationship classification module as an entity relationship extraction result of the target text;

[0090] After obtaining each of the input word vectors, each of the input word vectors is connected in parallel with the sum of the encoding word vectors to obtain each intermediate word vector, and each intermediate word vector is used as the input of the second encoder module for encoding. Similar to the first encoder, the second encoder module includes multiple layers of encoders (Encoder). In this embodiment, the second encoder module includes 4 layers of transformer encoders, and the structure of the second encoder module adopts a structure similar to DenseNet, that is, the input of each layer of encoder includes the output of all previous encodings in the second encoder module. For example, the second encoder module includes four encoders ABCD, which are arranged in the order of ABCD, that is, the input of A is the input word vector, then the input of the C encoder includes the output of the AB encoder, and in a possible implementation, the input of each encoder in the second encoder module also includes the input of the first encoder, that is, each of the input word vectors. The input of each layer of encoder includes the output of all previous encodings in the second encoder module, so that each layer of encoder can receive the information of all previous layers of encoders, effectively improving the generalization ability of the network, and can achieve higher accuracy with fewer parameters.

[0091] The relationship classification module can be a relationship network (Relation Network). After encoding and obtaining the encoded word vectors, each of the encoded word vectors is used as the input of the relationship classification module, and the relationship classification module outputs the entity relationship extraction result of the target text.

[0092] Before obtaining the target text, the first neural network and the second neural network need to be trained, and the training process is introduced below. Specifically, before obtaining the target text, the method further includes:

[0093] In the process of training the first neural network and the second neural network, obtaining a target loss value corresponding to each group of training data in the training set;

[0094] The parameters in the first neural network and the second neural network are updated according to the target loss value to implement training of the first neural network and the second neural network.

[0095] It can be understood that when other neural networks are used in the entity relationship joint extraction method provided in this embodiment, the parameters of other neural networks, such as the third neural network and the fourth neural network described above, are also updated according to the target loss value to achieve overall training.

[0096] Specifically, obtaining the target loss value corresponding to each group of training data in the training set includes:

[0097] For the target sample text:

[0098] S001. Obtaining a predicted entity recognition result of the target sample text through the second neural network, and obtaining a first loss value according to the predicted entity recognition result and an entity labeling result of the target sample text;

[0099] S002, summing the coded word vectors corresponding to the characters in the target sample text according to the entity annotation result of the target sample text, obtaining the sample word vector corresponding to the target sample text, calculating the mean vector of the sample word vectors belonging to the same category entity, and taking the similarity between the mean vector of each category entity and the preset label vectors corresponding to other categories as the second loss value;

[0100] S003, obtaining a predicted entity relationship extraction result of the target sample text through the second neural network, and obtaining a third loss value according to the predicted entity relationship extraction result and the entity relationship annotation result of the target sample text;

[0101] S004. Obtain the sum of the first loss value, the second loss value and the third loss value as the target loss value.

[0102] For the target sample text, according to the steps of the entity relationship joint extraction method provided in this embodiment, the input word vector of the target sample text is obtained as the input of the second neural network, the predicted entity recognition result output by the second neural network is obtained, and the first loss value is calculated through the cross entropy loss function according to the predicted entity recognition result and the entity annotation result of the target sample text. Then, according to the entity annotation result of the target sample text, the coded word vectors corresponding to each character in the target sample text are summed, that is, the coded word vectors of the characters belonging to the same entity are summed, and the sample word vectors corresponding to each entity in the target sample text are obtained. In the process of obtaining the sample word vector, the coded word vectors of the characters can be directly summed, or they can be summed after multi-head attention processing through the fourth neural network. After obtaining the sample word vectors, the mean vector of the sample word vectors corresponding to the entity words belonging to the same entity category is calculated, and the similarity between the mean vector of each category entity and the preset label vectors corresponding to other categories is calculated. For example, there are 4 entities ABCD, AB belongs to the first category, C belongs to the second category, and D belongs to the third category. Then, the similarity between the mean vector of AB and the preset label vectors of the second category and the third category is calculated, and the similarity between the sample word vector of C and the preset label vectors of the first category and the third category is calculated, and the similarity between the sample word vector of D and the preset label vectors of the first category and the second category is calculated. The sum of all similarities is used as the second loss value. The preset label vector includes vectors of multiple entity categories that are preset. The pairwise similarities between word vectors of different categories and threshold label vectors are calculated as part of the loss function. By minimizing the loss function, the similarity between word vectors of different categories is low, and the similarity between the same category is high, thereby improving the accuracy of entity recognition. After obtaining the entity recognition result of the target sample text, each of the sample word vectors is connected in parallel with the sum of the encoded word vectors of each character in the target sample text, and each sample intermediate word vector is obtained. After the intermediate word vectors are obtained, the predicted entity relationship extraction result of the target sample text is output through the second encoding module and the relationship classification module of the second neural network, and the third loss value is obtained according to the predicted entity relationship extraction result and the entity relationship labeling result of the target sample text. Finally, the sum of the first loss value, the second loss value and the third loss value is used as the target loss value for training.

[0103] Each sample text in the training set is processed as the target sample text once. After multiple trainings, the target loss value converges and the training is completed.

[0104] The specific flow chart of the entity relationship joint extraction method provided in this embodiment can be as follows Figure 3 shown.

[0105] In summary, this embodiment provides a method for joint extraction of entities and relationships. First, the word vector of the target word existing in the target text is processed with the initial word vector of each character in the target text to obtain the input word vector of each character. The input word vector of each character is used as the input of the first neural network for outputting the entity recognition result to output the entity recognition result. Then, the input of the second neural network for outputting the relationship extraction result is obtained based on the entity recognition result of the target text and the input word vector of each character in the target text, so that the two types of tasks share the information of characters and words in the target text, thereby improving the accuracy of the joint extraction of entities and relationships.

[0106] It should be understood that, although the various steps in the flowcharts given in the accompanying drawings of the present invention are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of the steps in the present invention is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps of the present invention may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0107] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0108] Embodiment 2

[0109] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Figure 4 The terminal includes a memory 10 and a processor 20. The memory 10 stores a computer program. When the processor 10 executes the computer program, the steps of the entity relationship joint extraction method provided by the present invention can be executed, as described in the first embodiment.

[0110] Embodiment 3

[0111] The present invention also provides a storage medium storing one or more programs, which can be executed by one or more processors to implement the steps of the entity relationship joint extraction method described in the above embodiment.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for joint extraction of entity relations, characterized in that: include: Obtain a target text, convert each character in the target text into an initial word vector, and determine whether there is a target word corresponding to the pre-trained word vector in the target text; If so, obtaining an intermediate word vector for each character in the target text according to the pre-trained word vector of the target word and the initial word vector of the character included in the target word; Connecting the intermediate word vector of each character in the target text and the sum of the intermediate word vectors of each character in the target text in parallel to obtain input word vectors corresponding to each character; Inputting the input word vector into a pre-trained first neural network, and obtaining an entity recognition result of the target text through the first neural network; Obtaining input data for inputting into a pre-trained second neural network according to the entity recognition result and the input word vector, and obtaining an entity relationship extraction result of the target text through the second neural network; The first neural network and the second neural network are trained together by using a training set including multiple sets of training data, each set of training data including sample text, entity annotation results corresponding to the sample text, and entity relationship annotation results corresponding to the sample text; The step of obtaining an intermediate word vector for each character in the target text according to the pre-trained word vector of the target word and the initial word vector of the character included in the target word comprises: For the characters included in the target word, the pre-trained word vector of the target word is input into a pre-trained third neural network, and in the third neural network, the pre-trained word vector of the target word is subjected to multi-head attention processing to obtain the weight corresponding to each character in the target word, and the pre-trained word vector of the target word is added to the initial word vector of each character according to the weight corresponding to the character in the target word to obtain the intermediate word vector of each character; For a character that does not have a corresponding target word, using the corresponding initial word vector as the intermediate word vector; The third neural network is trained together with the first neural network and the second neural network through the training set.

2. The entity relationship joint extraction method according to claim 1 is characterized in that: The first neural network includes a first encoder module and a multi-layer perception module; the obtaining the entity recognition result of the target text through the first neural network includes: Encode each of the input word vectors by using the first encoder module to obtain an encoded word vector; Using the encoded word vector as the input of the multi-layer perception module, and obtaining the result output by the multi-layer perception module as the entity recognition result of the target text; The first encoder module includes multiple layers of encoders, and the input of each layer of encoders includes the outputs of all previous encoders in the first encoder module.

3. The entity relationship joint extraction method according to claim 2 is characterized in that: The step of obtaining input data for inputting into a pre-trained second neural network according to the entity recognition result and the input word vector includes: Obtaining the encoded word vector output after the first encoder module encodes the input word vector; The input word vectors of characters belonging to the same entity word in the target text are summed to obtain respective input word vectors as input data of the second neural network.

4. The entity relationship joint extraction method according to claim 3 is characterized in that: The step of summing the input word vectors of characters belonging to the same entity word in the target text includes: In the pre-trained fourth neural network, the weights of the characters belonging to the same entity word are obtained through a multi-head attention mechanism, and the input word vectors of the characters belonging to the same entity word are summed according to the weights of the characters; The fourth neural network, the first neural network and the second neural network are trained together through the training set.

5. The entity relationship joint extraction method according to claim 3 is characterized in that: The second neural network includes a second encoder module and a relationship classification module; the step of obtaining the entity relationship extraction result of the target text through the second neural network includes: Connecting each of the input word vectors in parallel with the sum of the encoded word vectors of all characters to obtain each intermediate word vector, and encoding each of the intermediate word vectors through the second encoder module to obtain an encoded word vector; Using each of the encoded word vectors as an input of the relationship classification module, and obtaining the output of the relationship classification module as an entity relationship extraction result of the target text; The second encoder module includes multiple layers of encoders, and the input of each layer of encoders includes the outputs of all previous encoders in the second encoder module.

6. The entity relationship joint extraction method according to claim 5, characterized in that: Before obtaining the target text, the method further includes: In the process of training the first neural network and the second neural network, obtaining a target loss value corresponding to each group of training data in the training set; The parameters in the first neural network and the second neural network are updated according to the target loss value to implement training of the first neural network and the second neural network.

7. The entity relationship joint extraction method according to claim 6, characterized in that: The obtaining of the target loss value corresponding to each set of training data in the training set includes: For the target sample text: Obtaining a predicted entity recognition result of the target sample text through the second neural network, and obtaining a first loss value according to the predicted entity recognition result and an entity labeling result of the target sample text; According to the entity annotation result of the target sample text, the coded word vectors corresponding to each character in the target sample text are summed to obtain the sample word vector corresponding to the target sample text, the mean vector of the sample word vectors belonging to the same category entity is calculated, and the similarity between the mean vector of each category entity and the preset label vectors corresponding to other categories is used as the second loss value; Obtaining a predicted entity relationship extraction result of the target sample text through the second neural network, and obtaining a third loss value according to the predicted entity relationship extraction result and the entity relationship labeling result of the target sample text; The sum of the first loss value, the second loss value and the third loss value is obtained as the target loss value.

8. A terminal, characterized in that: The terminal includes: a processor, a storage medium communicatively connected to the processor, the storage medium being suitable for storing a plurality of instructions, and the processor being suitable for calling the instructions in the storage medium to execute the steps of the entity relationship joint extraction method described in any one of claims 1-7 above.

9. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the entity relationship joint extraction method as described in any one of claims 1-7.

Citation Information

Patent Citations

  • Method, device and equipment for extracting information

    CN107783960A

  • Named entity identification method, named entity recognition device, storage medium and electronic equipment

    CN110334357A