Entity and Relationship Joint Extraction Method and Device, Storage Medium, and Terminal

Abstract

The present invention discloses a method and apparatus for joint extraction of entities and relationships, a storage medium, and a terminal. The method uses a trained joint extraction model, which includes a word vector representation construction module for obtaining the final word vector representation of text data; a candidate entity representation construction module for obtaining a set of final candidate entity representations; an entity classification module for obtaining the posterior information of the entity type to which each final candidate entity representation in the set of final candidate entity representations belongs; a relationship representation construction module for based on the relationship representation vectors corresponding to each pair of entity pairs; a relationship classification module for obtaining the posterior information of the relationship type to which each pair of entity pairs belongs. The present invention fully excavates explicit semantic information and implicit semantic information, helps the model to deeply understand entity semantics and establish relationships between entities, considers global semantic information and local context length information in relationship extraction, and improves the accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of information processing, and in particular to a method and device for jointly extracting entities and relationships, a storage medium, and a terminal. Background Art

[0002] For natural language understanding, relying solely on data annotation and computing power investment cannot achieve fine and deep semantic understanding. In order to better promote the development of natural language understanding, introducing prior knowledge to guide natural language understanding has become the current development trend. Knowledge is a way of storing text in a structured manner. Currently, this structured knowledge has been widely applied in scenarios such as search engines, question-and-answer systems, and machine translation. However, due to the huge scale of knowledge and the high cost of manual annotation, in order to obtain knowledge information as timely and accurate as possible, researchers have proposed the tasks of named entity recognition and relation extraction.

[0003] Semantic role labeling can establish the dependency relationship between the predicate and arguments of a sentence, and this semantic structure information can provide rich semantics for text representation. However, currently, there is a lack of work on using semantic role labeling information for the task of jointly extracting entities and relationships. If a word or phrase is labeled as a semantic role, it is more likely to be labeled as an entity. For example, the semantic role label "ArgM-LOC" contains location information, which can provide auxiliary information for the extraction of entities of the "Location" type. At the same time, explicit semantic information can cover the semantic relationships between words, which is very helpful for relation extraction.

[0004] In addition to ignoring the introduction of explicit semantic information, many existing models do not sufficiently explore implicit semantic information. In most existing models, the representation vectors of text are shared in entity recognition and relation extraction. However, named entity recognition focuses on mining the semantic information of entities, while relation extraction focuses on mining the semantic information of the local context between entities in an entity pair. Therefore, in order to fully mine implicit semantic information, a more reasonable implicit semantic enhancement method needs to be designed based on the task of jointly extracting entities and relationships.

[0005] In summary, the existing methods for jointly extracting entities and relationships at the present stage still do not sufficiently utilize and mine semantic information, resulting in inaccurate and insufficient named entity recognition and relation extraction. Summary of the Invention

[0006] The technical problem to be solved by the present invention is that the existing methods for jointly extracting entities and relationships ignore the introduction of explicit semantic information and do not sufficiently explore implicit semantic information, resulting in inaccurate and insufficient entity recognition and relation extraction.

[0007] To solve the above technical problems, the present invention provides a method for jointly extracting entities and relationships, including:

[0008] Obtain a text data set, where the text data set includes multiple text data;

[0009] Based on the trained joint extraction model, perform joint extraction on each text data in the text data set to obtain entities in each text data and the relationships between the entities;

[0010] Among them, the joint extraction model includes a word vector representation construction module, a candidate entity representation construction module, an entity classification module, a relationship representation construction module, and a relationship classification module;

[0011] The word vector representation construction module is used to obtain the first word vector representation and the entity global semantic representation of the text data based on a pre-trained model, obtain the second word vector representation of the text data based on semantic role annotation, and splice the first word vector representation and the second word vector representation to obtain the final word vector representation of the text data;

[0012] The candidate entity representation construction module is used to obtain enhanced entity implicit semantic information based on the final word vector representation, obtain a preliminary candidate entity set including any span based on the enhanced entity implicit semantic information, perform a max pooling operation on all the preliminary candidate entities in the preliminary candidate entity set to obtain an aggregated candidate entity set, and obtain a final candidate entity representation set based on the aggregated candidate entities in the aggregated candidate entity set, the length of the preliminary candidate entity sequence to which the aggregated candidate entity belongs, and the entity global semantic representation;

[0013] The entity classification module is used to obtain the posterior information of the entity type to which each final candidate entity representation in the final candidate entity representation set belongs through a classification function based on the final candidate entity representation set;

[0014] The relationship representation construction module is used to combine pairwise the aggregated candidate entity representations whose corresponding final candidate entity representations belong to non-empty entities in the aggregated candidate entity set to form an entity pair set, obtain the global semantic representation and the local semantic representation of each pair of entity pairs, and obtain the corresponding relationship representation vector based on the global semantic representation, the local semantic representation, the local sequence length, and the entity pair ontology of each pair of entity pairs;

[0015] The relationship classification module is used to obtain the posterior information of the relationship type to which each pair of entity pairs belongs through a classification function based on the relationship representation vector of each pair of entity pairs;

[0016] Among them, the local sequence length is the sequence length of the local semantic representation.

[0017] Preferably, the word vector representation construction module is used to obtain the first word vector representation and the entity global semantic representation of the text data based on a pre-trained model, obtain the second word vector representation of the text data based on semantic role labeling, and splice the first word vector representation and the second word vector representation to obtain the final word vector representation of the text data, including:

[0018] Input the text data into the pre-trained model to obtain a sub-word vector representation sequence and an entity global semantic representation, and perform convolution and max pooling on the sub-word vector representation sequence to obtain the first word vector representation;

[0019] Input the text data into a semantic role labeling tool to obtain multiple semantic role labeling sequences, and aggregate all the semantic role labeling sequences through a fully connected network to obtain the second word vector representation;

[0020] Splice and fuse the first word vector representation and the second word vector representation to obtain the final word vector representation of the text data.

[0021] Preferably, the candidate entity representation construction module obtains enhanced entity implicit semantic information based on the final word vector representation, including:

[0022] Input the final word vector representation into a first bidirectional long short-term memory network to obtain enhanced entity implicit semantic information.

[0023] Preferably, the candidate entity representation construction module obtains the final candidate entity representation set based on the aggregated candidate entities in the aggregated candidate entity set, the length of the preliminary candidate entity sequence to which the aggregated candidate entity belongs, and the entity global semantic representation, including:

[0024] Splice each aggregated candidate entity in the aggregated candidate entity set, the length of the sequence of the preliminary candidate entity to which the aggregated candidate entity belongs, and the entity global semantic representation respectively to obtain the final candidate entity representation corresponding to each aggregated candidate entity, and form the final candidate entity representation set based on all the final candidate entity representations.

[0025] Preferably, the classification function is softmax.

[0026] Preferably, the relationship representation construction module obtains the global semantic representation and the local semantic representation of each pair of entity pairs, and obtains the corresponding relationship representation vector based on the global semantic representation, the local semantic representation, the local sequence length of each pair of entity pairs, and the entity pair ontology, including:

[0027] Input the final word vector representation into a second bidirectional long short-term memory network to obtain enhanced global context implicit semantic information, obtain local semantic representations of all entity pairs in the entity pair set based on the enhanced global context implicit semantic information, and use the last hidden state representation of the second bidirectional long short-term memory network as the global semantic representation of all entity pairs in the entity pair set;

[0028] Concatenate the global semantic representation, local semantic representation, local sequence length, and entity pair ontology of each entity pair to obtain the relationship representation vector corresponding to each entity pair.

[0029] Preferably, during the training of the joint extraction model to obtain the trained joint extraction model, its loss function is:

[0030] L = L e + L r

[0031] where, L e represents the cross-entropy loss function between the posterior information of the entity types to which all the final candidate entity representations corresponding to the text data belong and the true entity labels, and L r represents the cross-entropy loss function between the posterior information of the relationship types to which all entity pairs in the text data belong and the true relationship labels.

[0032] To solve the above technical problems, the present invention also provides an entity and relationship joint extraction device, which is characterized by including a data acquisition module and an entity and relationship extraction module;

[0033] The data acquisition module is used to acquire a text data set, and the text data set includes multiple text data;

[0034] The relationship extraction module is used to perform joint extraction on each text data in the text data set based on the trained joint extraction model to obtain entities and relationships between entities in each text data;

[0035] where, the joint extraction model includes a word vector representation construction module, a candidate entity representation construction module, an entity classification module, a relationship representation construction module, and a relationship classification module;

[0036] The word vector representation construction module is used to obtain the first word vector representation and entity global semantic representation of the text data based on a pre-trained model, obtain the second word vector representation of the text data based on semantic role annotation, and concatenate the first word vector representation and the second word vector representation to obtain the final word vector representation of the text data;

[0037] The candidate entities represent building blocks for obtaining enhanced implicit semantic information of entities based on the final word vector representation, obtaining a preliminary candidate entity set containing any span based on the enhanced implicit semantic information of entities, performing a max pooling operation on all the preliminary candidate entities in the preliminary candidate entity set to obtain an aggregated candidate entity set, and obtaining a final candidate entity representation set based on the aggregated candidate entities in the aggregated candidate entity set, the lengths of the preliminary candidate entity sequences to which the aggregated candidate entities belong, and the global semantic representation of the entities;

[0038] The entity classification module is configured to obtain posterior information of the entity type to which each final candidate entity representation in the final candidate entity representation set belongs through a classification function based on the final candidate entity representation set;

[0039] The relationship representation construction module is configured to combine pairwise the aggregated candidate entity representations whose corresponding final candidate entity representations belong to non-empty entities in the aggregated candidate entity set to form a set of entity pairs, obtain the global semantic representation and the local semantic representation of each pair of entity pairs, and obtain a corresponding relationship representation vector based on the global semantic representation, the local semantic representation, the local sequence length, and the entity pair ontology of each pair of entity pairs;

[0040] The relationship classification module is configured to obtain posterior information of the relationship type to which each pair of entity pairs belongs through a classification function based on the relationship representation vector of each pair of entity pairs;

[0041] Wherein, the local sequence length is the sequence length of the local semantic representation.

[0042] To solve the above technical problems, the present invention also provides a storage medium on which a computer program is stored, and when the program is executed by a processor, it implements the entity and relationship joint extraction method.

[0043] To solve the above technical problems, the present invention also provides a terminal, including: a processor and a memory, and the memory is communicatively connected to the processor;

[0044] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory so that the terminal executes the entity and relationship joint extraction method as described above.

[0045] Compared with the prior art, one or more embodiments of the above solutions may have the following advantages or beneficial effects:

[0046] Applying the entity and relation joint extraction method provided by the embodiments of the present invention, during the process of obtaining the word vector representation, explicit semantic information is introduced with the help of a semantic role labeling tool, and in entity recognition and relation extraction, a separate encoding method is adopted to specifically enhance implicit semantic features; the full mining of explicit semantic information and implicit semantic information helps the model to deeply understand entity semantics and establish relationships between entities. In relation extraction, this method considers global semantic information and local context length information, further improving the accuracy of the entity and relation joint extraction model.

[0047] Other features and advantages of the present invention will be described in the following specification, and will be partly obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in the specification, claims, and drawings. Brief Description of the Drawings

[0048] The drawings are used to provide further understanding of the present invention, and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. In the drawings:

[0049] Figure 1 A flowchart showing the entity and relation joint extraction method according to Embodiment 1 of the present invention;

[0050] Figure 2 A schematic diagram showing the process of the entity and relation joint extraction method according to Embodiment 1 of the present invention;

[0051] Figure 3 A schematic diagram showing the PropBank-style semantic role labeling according to Embodiment 1 of the present invention;

[0052] Figure 4 A schematic diagram showing the structure of the entity and relation joint extraction device according to Embodiment 2 of the present invention;

[0053] Figure 5 A schematic diagram showing the structure of the terminal according to Embodiment 4 of the present invention. Detailed Embodiments

[0054] The following will specifically describe the embodiments of the present invention in conjunction with the drawings and embodiments, so as to fully understand how the present invention uses technical means to solve technical problems and achieve the implementation process of technical effects. It should be noted that as long as there is no conflict, each embodiment in the present invention and each feature in each embodiment can be combined with each other, and the technical solutions formed are within the protection scope of the present invention.

[0055] Named entity recognition is to identify entities with specific meanings in text, such as person names, place names, organizations, etc. Relation extraction is to predict the relationships between entities. Finally, entity pairs and relationships together form triples, constituting structured knowledge information. Before the emergence of entity and relation joint extraction methods, most researchers implemented named entity recognition and relation extraction in a pipeline-based manner, but such methods ignored the association between the two tasks and were prone to cascading errors.

[0056] Recently, pre-trained models such as BERT, Transformer-XL, and RoBERT have received great attention. These models are usually pre-trained on large-scale document data and are transferred to target tasks with relatively less supervised training data. In many natural language understanding tasks, works based on pre-trained models have achieved the best performance, such as question answering, context sentiment detection, and entity and relation joint extraction. Despite the success of these pre-trained language models, existing entity and relation joint extraction methods only focus on the text representations provided by pre-trained models, while ignoring the introduction of explicit semantic information and the enhancement of implicit semantic information.

[0057] Example 1

[0058] To solve the technical problems existing in the prior art, an embodiment of the present invention provides an entity and relation joint extraction method.

[0059] Figure 1 The flowchart of the entity and relation joint extraction method according to Embodiment 1 of the present invention is shown; Figure 2 The process diagram of the entity and relation joint extraction method according to Embodiment 1 of the present invention is shown; Refer to Figure 1 and [[ID=1⑧]] Figure 2 As shown, the entity and relation joint extraction method according to the embodiment of the present invention includes the following steps.

[0060] Step S101, obtain a text data set.

[0061] Specifically, the text data set includes multiple text data. The text data for which entity and relation joint extraction is required are usually sentences, that is, the entity and relation joint extraction method is actually a method for extracting each entity in a sentence and the relationships between each entity. It should be noted that the text data can be Chinese data, or English or other language data.

[0062] Step S102, based on the trained joint extraction model, perform joint extraction on each text data in the text data set to obtain the entities in each text data and the relationships between the entities.

[0063] Specifically, by inputting the text data in the text dataset into the trained joint extraction model, all entities in each text data and the relationships between all entities can be obtained.

[0064] Furthermore, the joint extraction model includes a word vector representation construction module, a candidate entity representation construction module, an entity classification module, a relationship representation construction module, and a relationship classification module.

[0065] Among them, the word vector representation construction module is mainly used to obtain the final word vector representation of the text data. Specifically, the first word vector representation and the entity global semantic representation of the text data need to be obtained based on a pre-trained model first. Preferably, the pre-trained model is a BERT pre-trained model; and the pre-trained model needs to be pre-trained according to a large amount of model training text data. The trained pre-trained model has a powerful ability to capture language features and can provide a relatively high-quality word vector representation for entity and relationship joint extraction. And the BERT pre-trained model uses the byte pair encoding method to solve the encoding problem of out-of-vocabulary words. Therefore, some words will be split into sub-word sequences. For example, "treehouse" will be split into "tree" and "house". Therefore, a word may correspond to multiple word vectors. Further, the text sequence X=(x1, x2,..., x n ) of the text data is passed into the BERT pre-trained model to obtain the sub-word vector representation sequence and the entity global semantic representation CLS.

[0066] For the convenience of corresponding to semantic role labels, in this embodiment, the sub-word vector representation sequence is processed by one-dimensional convolution and max pooling to obtain the first word vector representation. Further, the convolution process of a certain word x i in the sub-word vector representation sequence is as follows: Assume that the length of the word x i is l, and its sub-word sequence representation is (s1, s2,..., s l ); the word x i passes through one-dimensional convolution to obtain e i =W1[BERT(s i ), BERT(s i+1 ),..., BERT(s i+k-1 )]+b1, where k is the convolution kernel size, and both W1 and b1 are trainable vectors. Then, the first word vector representation obtained through max pooling is where ReLU is a common activation function.

[0067] In this embodiment, a semantic role labeling tool is used to obtain explicit semantic information. Specifically, a PropBank-style annotator is used for annotation. The PropBank-style annotator takes a single sentence as a unit and analyzes the local semantic structure related to each predicate in the sentence. Semantic structure information is highly relevant to named entity recognition and relation extraction tasks. Information such as time and location can help the model better extract entities. Agents and patients can, to a certain extent, assist in the judgment of entity relationships. For specific examples of PropBank-style semantic role labeling, please refer to Figure 3 as shown. Semantic role labeling is centered around the predicate in a sentence and assigns labels to the words in the sentence. Since different predicates are concerned, different semantic label sequences are obtained.

[0068] In this embodiment, multiple semantic role labeling sequences are selected for each sentence, and the number range of semantic role labeling sequences is 3 - 6. The text sequence X=(x1,x2,...,x n ) of specific text data is passed into the semantic role labeling tool and then obtained as a word vector sequence representation: Since the semantic role labeling tool is centered around the predicate in the sentence and annotates the semantic relationship between the word and the predicate. Therefore, different predicates are selected, and different sequence annotations are obtained. To extract explicit semantic information as fully as possible, five semantic role labeling sequences can be selected for each sentence, which are respectively denoted as: T1, T2, T3, T4, T5. Then, the five semantic role labeling sequences are aggregated through a fully connected network to obtain the second word vector representation T s =W2(T1,T2,...,T5)+b2, where W2 and b2 are trainable vectors.

[0069] Finally, the first word vector representation and the second word vector representation are concatenated and fused to obtain the final word vector representation X w =[X b :T s .

[0070] Named entity recognition highly depends on the context where the entity is located. Because the context information is different, words with the same letter composition may have vastly different meanings. Therefore, in addition to adding explicit semantic information in the word vector representation, this paper also uses a bidirectional long short-term memory network in the named entity recognition stage to enhance the implicit context semantic information of the text, and then obtains an enhanced entity representation of implicit semantic information.

[0071] The candidate entity representation construction module is mainly used to obtain the final set of candidate entity representations. Specifically, the final word vector representation needs to be input into the first bidirectional long short-term memory network to obtain enhanced implicit semantic information of the entity. The bidirectional long short-term memory network alleviates the problems of gradient disappearance and gradient explosion to a certain extent compared with the ordinary recurrent neural network, and compared with the long short-term memory network, it has the characteristic of capturing bidirectional sequence information. Therefore, the final word vector representation needs to pass through the bidirectional long short-term memory network first to obtain enhanced implicit semantic information of the entity: X t = Bi-LSTM t (X w ), where Bi-LSTM represents the bidirectional long short-term memory network.

[0072] Then, based on the enhanced implicit semantic information of the entity, a preliminary candidate entity set containing any span is obtained. Further, the span is set to any possible value respectively, and for each span, all preliminary candidate entities in the corresponding span state need to be obtained from the enhanced implicit semantic information of the entity, and then all the preliminary candidate entities corresponding to all spans are combined into a preliminary candidate entity set. After that, the maximum pooling operation is performed on all the preliminary candidate entities in the preliminary candidate entity set to obtain the aggregated candidate entity set. The preliminary candidate entity with a sequence length of f can be expressed as The aggregated candidate entity obtained after performing the maximum pooling operation on this preliminary candidate entity can be expressed as e t = Maxpooling(E t ).

[0073] Since the sequence length of the candidate entity has a certain impact on entity type classification, and the entity global semantic representation CLS obtained by BERT encoding also contains rich context information, both of the above are used as one of the influencing factors for candidate entity classification in this embodiment. Further, after obtaining the aggregated candidate entity set, based on the aggregated candidate entities in the aggregated candidate entity set, the sequence length of the preliminary candidate entity sequence to which the aggregated candidate entity belongs, and the entity global semantic representation, the final set of candidate entity representations is obtained. Further, since each aggregated candidate entity has a corresponding preliminary candidate entity, and each preliminary candidate entity has a corresponding span, that is, a sequence length, each aggregated candidate entity has a corresponding sequence length of the preliminary candidate entity sequence to which the aggregated candidate entity belongs. For each aggregated candidate entity in the aggregated candidate entity set, it needs to be concatenated based on the aggregated candidate entity itself, the sequence length of the preliminary candidate entity sequence to which the aggregated candidate entity belongs, and the entity global semantic representation CLS to obtain the corresponding final candidate entity representation of the aggregated candidate entity. Based on all the final candidate entity representations obtained from all the aggregated candidate entities in the aggregated candidate entity set, the final set of candidate entity representations is formed.

[0074] The entity classification module is mainly used to obtain the posterior information of the entity type to which each final candidate entity representation in the final candidate entity representation set belongs through a classification function. Preferably, the classification function is Softmax. The final candidate entity representations in the final candidate entity representation set are respectively input into the classification function to obtain the posterior information of the entity types to which all the final candidate entity representations belong. The posterior information of the entity type to which the final candidate entity representation belongs can be expressed as: where represents the representation vector of the aggregated candidate entity belonging to the preliminary candidate entity sequence with length f, and W3 and b3 are trainable parameter vectors.

[0075] The relationship representation construction module is mainly used to obtain the relationship representation vector of the entity pair. Specifically, based on the posterior information of the entity type to which each final candidate entity representation belongs, it can be known whether the final candidate entity is a non-empty entity. Accordingly, the aggregated candidate entity representations whose entity classifications of all the final candidate entity representations in the aggregated candidate entity set are non-empty are selected, and then the aggregated candidate entity representations whose entity classifications of all the final candidate entity representations are non-empty are combined pairwise to form entity pairs, and all the obtained entity pairs form an entity pair set.

[0076] In this embodiment, the local semantic representation and the global semantic representation between entity pairs are introduced to predict the relationship between entity pairs; at the same time, since the entity pair distance also affects the judgment of the entity pair relationship, the closer the internal interval of the entity pair, the greater the possibility of forming certain relationships. Therefore, when predicting the relationship between entity pairs, the local sequence length is also introduced.

[0077] Specifically, the final word vector representation is input into the second bidirectional long short-term memory network to obtain enhanced global context implicit semantic information Then, based on the enhanced global context implicit semantic information, the local semantic representations of all entity pairs in the entity pair set are obtained; the local context of the entity pair is the sequence from the end of the first entity to the beginning of the second entity in the enhanced global context implicit semantic information, and then the local context representation of the entity pair is aggregated through a max-pooling operation to obtain the local semantic representation, where, a end represents the subscript at the end of the first entity, and b start represents the subscript at the beginning of the second entity. Then, the last hidden state representation of the second bidirectional long short-term memory network is used as the global semantic representation of all entity pairs in the entity pair set. Among them, the second bidirectional long short-term memory network is a bidirectional long short-term memory network Bi-LSTM.

[0078] Finally, the global semantic representation, local semantic representation, local sequence length of each entity pair, and the ontology of the entity pair are concatenated to obtain the relationship representation vector corresponding to each entity pair. The local sequence length is the sequence length between two entities in an entity pair.

[0079] The relationship classification module is mainly used to obtain the posterior information of the relationship type to which each entity pair belongs based on the relationship representation vector of each entity pair through a classification function. Preferably, the classification function is Softmax. Further, the relationship representation vectors of each entity pair are respectively input into the classification function to obtain the posterior information of the relationship type to which each entity pair belongs. For example, the posterior information of the relationship type to which the entity pair ab belongs can be expressed as: and where W4, W5, b4, and b5 are respectively trainable parameter vectors, and the determination of the relationship between entity pair ab is that if and any value in does not reach the threshold α, it is considered that there is no relationship between entity a and entity b. Further and any value in not reaching the threshold α is accurately understood as: is [a1, a2, a3…an], where a1, a2, a3…an respectively represent the possibilities that the relationship between entity pair ab belongs to the first type, the second type, the third type of relationship, etc. Similarly is expressed as [b1, b2, b3…bn], where b1, b2, b3…bn respectively represent the possibilities that the relationship between entity pair ba belongs to the first type, the second type, the third type of relationship, etc. Select the largest category in as the type to which the relationship between entity pair ab belongs, and select the largest category in as the type to which the relationship between entity pair ba belongs, and judge whether the type to which the relationship between ab belongs (i.e., the maximum value of the category in is the same as the type to which the relationship between ba belongs (i.e., the maximum value of the category in both do not reach the threshold α. If so, it means that there is no relationship between entity a and entity b. The threshold α can be set based on the actual situation and will not be restricted too much here.

[0080] It should be noted that whether it is in the joint extraction model during the training process or in the process of obtaining entities and relationships between entities after the training of the joint extraction model, the word vector representation construction module, the candidate entity representation construction module, the entity classification module, the relationship representation construction module, and the relationship classification module all work in the above working manner.

[0081] The process of training the joint extraction model to obtain the trained joint extraction model includes:

[0082] First, a training text dataset needs to be obtained. The training text dataset contains multiple training text data, and each training text data includes text data and the corresponding true label. The true label corresponding to the text data includes entity labels in the text data and relationship labels between entities.

[0083] Then, the training text dataset is divided into multiple groups to obtain multiple groups of training text data groups. Then, the temporary joint extraction model is trained based on the M-th group of training text data groups, and during the training, each parameter in the temporary joint extraction model is adjusted based on the loss value calculated by the loss function to obtain a parameter-updated joint extraction model. Then, it is judged whether M is equal to the preset threshold. If so, it is judged whether the current parameter-updated joint extraction model meets the condition that the loss value reaches the convergence state. If it meets, the current parameter-updated joint extraction model is used as the trained joint extraction model. If M is not equal to the preset threshold, M is incremented by 1, and the temporary joint extraction model needs to be retrained based on the M-th group of training text data groups. If the current parameter-updated joint extraction model does not meet the condition that the loss value reaches the convergence state, M is initialized to 1, and the temporary joint extraction model needs to be retrained based on the M-th group of training text data groups. Repeat the above process until the parameter-updated joint extraction model meets the condition that the loss value reaches the convergence state.

[0084] It should be noted that the preset threshold is the total number of groups of training text data groups in the training text dataset, and the initial value of M is 1; when training is first performed based on the training text data group, the temporary joint extraction model is the initially created joint extraction model.

[0085] Since each parameter in the joint extraction model needs to be adjusted based on the loss value calculated by the loss function during each training, the training process based on multiple groups of training text data groups realizes the process of optimizing each parameter in the joint extraction model multiple times; repeat multiple rounds of the above process of training the joint extraction model based on each training text data group in turn until the loss value no longer continuously decreases and reaches the convergence state, that is, the currently obtained parameter-updated joint extraction model already meets the condition that the loss value reaches the convergence state, and the trained joint extraction model can be obtained.

[0086] It should be noted that each unit in the trained joint extraction model is obtained by repeatedly adjusting various parameters after training each unit in the joint extraction model with the object attribute recognition data training dataset. Therefore, when introducing the working principle of each unit in the joint extraction model above, the working principle of each unit in the trained joint extraction model is also introduced.

[0087] The loss function in the above training process can be expressed as:

[0088] L = L e +Lr

[0089] Among them, L e represents the cross-entropy loss function of the posterior information of the entity types to which all the final candidate entity representations corresponding to the text data belong and the true labels of the entities, and L r represents the cross-entropy loss function of the posterior information of the relationship types of all entity pairs in the text data and the true labels of the relationships.

[0090] In the entity and relationship joint extraction method provided by the embodiments of the present invention, during the process of obtaining the word vector representation, explicit semantic information is introduced with the help of a semantic role annotation tool, and in entity recognition and relationship extraction, a separate coding method is adopted to specifically enhance implicit semantic features; the full mining of explicit semantic information and implicit semantic information helps the model to deeply understand entity semantics and establish relationships between entities. In relationship extraction, this method considers global semantic information and local context length information, further improving the accuracy of the entity and relationship joint extraction model.

[0091] Example 2

[0092] To solve the technical problems existing in the prior art, the embodiments of the present invention provide an entity and relationship joint extraction device.

[0093] Figure 4 Fig. shows the structural schematic diagram of the entity and relationship joint extraction device in the second embodiment of the present invention; refer to Figure 4 As shown, the entity and relationship joint extraction device in the embodiments of the present invention includes a data acquisition module and an entity and relationship extraction module.

[0094] The data acquisition module is used to acquire a text data set, and the text data set includes multiple text data;

[0095] The relationship extraction module is used to perform joint extraction on each text data in the text data set based on the trained joint extraction model to obtain the entities in each text data and the relationships between the entities;

[0096] Among them, the joint extraction model includes a word vector representation construction module, a candidate entity representation construction module, an entity classification module, a relationship representation construction module, and a relationship classification module;

[0097] The word vector representation construction module is used to obtain the first word vector representation and the entity global semantic representation of the text data based on a pre-trained model, obtain the second word vector representation of the text data based on semantic role annotation, and splice the first word vector representation and the second word vector representation to obtain the final word vector representation of the text data;

[0098] The candidate entity representation construction module is used to obtain enhanced implicit semantic information of entities based on the final word vector representation, obtain a preliminary candidate entity set including any span based on the enhanced implicit semantic information of entities, perform a max-pooling operation on all preliminary candidate entities in the preliminary candidate entity set respectively to obtain an aggregated candidate entity set, and obtain a final candidate entity representation set based on the aggregated candidate entities in the aggregated candidate entity set, the length of the preliminary candidate entity sequence to which the aggregated candidate entity belongs, and the global semantic representation of the entity;

[0099] The entity classification module is used to obtain the posterior information of the entity type to which each final candidate entity representation in the final candidate entity representation set belongs through a classification function based on the final candidate entity representation set;

[0100] The relationship representation construction module is used to combine pairwise the aggregated candidate entity representations whose corresponding final candidate entity representations belong to non-empty entities in the aggregated candidate entity set to form an entity pair set, obtain the global semantic representation and local semantic representation of each pair of entity pairs, and obtain the corresponding relationship representation vector based on the global semantic representation, local semantic representation, local sequence length of each pair of entity pairs, and the entity pair ontology;

[0101] The relationship classification module is used to obtain the posterior information of the relationship type to which each pair of entity pairs belongs through a classification function based on the relationship representation vector of each pair of entity pairs;

[0102] Wherein, the local sequence length is the sequence length of the local semantic representation.

[0103] In the entity and relationship joint extraction device provided by the embodiments of the present invention, during the process of obtaining the word vector representation, explicit semantic information is introduced with the help of a semantic role annotation tool, and in entity recognition and relationship extraction, a separate encoding method is adopted to enhance implicit semantic features specifically; the full mining of explicit semantic information and implicit semantic information helps the model to deeply understand entity semantics and establish relationships between entities. In relationship extraction, the device considers global semantic information and local context length information, further improving the accuracy of the entity and relationship joint extraction model.

[0104] Example 3

[0105] To solve the above technical problems existing in the prior art, the embodiments of the present invention also provide a storage medium, which stores a computer program, and when the computer program is executed by a processor, all steps in the entity and relationship joint extraction method in Embodiment 1 can be implemented.

[0106] The specific steps of the entity and relationship joint extraction method and the beneficial effects obtained by using the readable storage medium provided by the embodiments of the present invention are the same as those in Embodiment 1, and will not be elaborated herein.

[0107] It should be noted that the storage medium includes various media such as ROM, RAM, magnetic disk or optical disc that can store program codes.

[0108] Example 4

[0109] To solve the above technical problems existing in the prior art, an embodiment of the present invention further provides a terminal.

[0110] Figure 5 FIG. shows a schematic diagram of the terminal structure according to the fourth embodiment of the present invention. Referring to Figure 5 , the terminal of this embodiment includes a processor and a memory connected to each other; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that all steps in the entity and relationship joint extraction method in Embodiment 1 can be realized when the terminal is executed.

[0111] The specific steps of the entity and relationship joint extraction method and the beneficial effects obtained by using the terminal provided by the embodiment of the present invention are the same as those in Embodiment 1, and will not be elaborated herein.

[0112] It should be noted that the memory may include a random access memory (Random Access Memory, abbreviated as RAM), and may also include a non-volatile memory, such as at least one disk memory. Similarly, the processor may also be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc.; it may also be a digital signal processor (Digital Signal Processing, abbreviated as DSP), an application specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), a field programmable gate array (Field Programmable Gate Array, abbreviated as FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0113] Although the disclosed embodiments of the present invention are as above, the above content is only an embodiment adopted for the convenience of understanding the present invention, and is not intended to limit the present invention. Any person skilled in the art within the technical field to which the present invention pertains may make any modifications and changes in the form of implementation and details without departing from the spirit and scope disclosed by the present invention. However, the protection scope of the present invention shall still be subject to the scope defined by the appended claims.

Claims

1. A method for joint extraction of entities and relations, comprising: Acquire a text data set, wherein the text data set includes a plurality of text data; Based on the trained joint extraction model, performing joint extraction on each of the text data in the text dataset to obtain entities in each of the text data and relationships between entities; The joint extraction model includes a word vector representation building module, a candidate entity representation building module, an entity classification module, a relationship representation building module and a relationship classification module; The word vector representation construction module is used to obtain a first word vector representation and an entity global semantic representation of the text data based on a pre-trained model, obtain a second word vector representation of the text data based on semantic role labeling, and splice the first word vector representation and the second word vector representation to obtain a final word vector representation of the text data; The candidate entity representation construction module is used to obtain enhanced entity implicit semantic information based on the final word vector representation, obtain a preliminary candidate entity set containing any span based on the enhanced entity implicit semantic information, perform a maximum pooling operation on all the preliminary candidate entities in the preliminary candidate entity set to obtain an aggregated candidate entity set, and obtain a final candidate entity representation set based on the aggregated candidate entities in the aggregated candidate entity set, the length of the preliminary candidate entity sequence to which the aggregated candidate entities belong, and the global semantic representation of the entity; The entity classification module is configured to obtain, based on the final candidate entity representation set, a posteriori information of the entity type to which each final candidate entity representation in the final candidate entity representation set belongs through a classification function; The relationship representation construction module is used to combine the aggregated candidate entity representations of the entities corresponding to the final candidate entity representations in the aggregated candidate entity set, which are classified as non-empty, to form an entity pair set, obtain the global semantic representation and local semantic representation of each pair of entity pairs, and obtain the corresponding relationship representation vector based on the global semantic representation, local semantic representation, local sequence length and entity pair ontology of each pair of entity pairs; The relationship classification module is used to obtain the posterior information of the relationship type of each entity pair through a classification function based on the relationship representation vector of each entity pair; The local sequence length is the sequence length of the local semantic representation.

2. The method according to claim 1, characterized in that The word vector representation construction module is used to obtain a first word vector representation and an entity global semantic representation of text data based on a pre-trained model, obtain a second word vector representation of the text data based on semantic role labeling, and splice the first word vector representation and the second word vector representation to obtain a final word vector representation of the text data, including: Input the text data into a pre-trained model to obtain a subword vector representation sequence and an entity global semantic representation, and subject the subword vector representation sequence to convolution and maximum pooling to obtain a first word vector representation; Inputting the text data into a semantic role labeling tool to obtain multiple semantic role labeling sequences, and aggregating all the semantic role labeling sequences through a fully connected network to obtain a second word vector representation; The first word vector representation and the second word vector representation are concatenated and fused to obtain a final word vector representation of the text data.

3. The method according to claim 1, characterized in that The candidate entity representation construction module acquires enhanced entity implicit semantic information based on the final word vector representation, including: The final word vector representation is input into a first bidirectional long short-term memory network to obtain enhanced entity implicit semantic information.

4. The method according to claim 3, characterized in that The candidate entity representation construction module obtains a final candidate entity representation set based on the aggregated candidate entity in the aggregated candidate entity set, the length of the preliminary candidate entity sequence to which the aggregated candidate entity belongs, and the entity global semantic representation, including: Each aggregated candidate entity in the aggregated candidate entity set, the sequence length of the preliminary candidate entity to which the aggregated candidate entity belongs, and the entity global semantic representation are spliced separately to obtain the final candidate entity representation corresponding to each of the aggregated candidate entities, and a final candidate entity representation set is formed based on all the final candidate entity representations.

5. The method according to claim 1, wherein The classification function is softmax.

6. The method according to claim 1, characterized in that The relationship representation construction module obtains the global semantic representation and local semantic representation of each entity pair, and obtains the corresponding relationship representation vector based on the global semantic representation, local semantic representation, local sequence length and entity pair ontology of each entity pair, including: Inputting the final word vector representation into a second bidirectional long short-term memory network to obtain enhanced global contextual implicit semantic information, obtaining local semantic representations of all entity pairs in the entity pair set based on the enhanced global contextual implicit semantic information, and using the last hidden state representation of the second bidirectional long short-term memory network as the global semantic representation of all entity pairs in the entity pair set; The global semantic representation, local semantic representation, local sequence length and entity pair ontology of each entity pair are concatenated to obtain the relationship representation vector corresponding to each entity pair.

7. The method according to claim 1, characterized in that When training the joint extraction model to obtain the trained joint extraction model, its loss function is: L=L e +L r Among them, L e represents the cross entropy loss function between the posterior information of the entity type to which all the final candidate entities corresponding to the text data belong and the entity's true label, L r The cross entropy loss function represents the posterior information of the relationship type of all entity pairs in the text data and the true label of the relationship.

8. A device for joint extraction of entities and relationships, characterized in that: Includes data acquisition module and entity and relationship extraction module; The data acquisition module is used to acquire a text data set, wherein the text data set includes a plurality of text data; The relationship extraction module is used to perform joint extraction on each of the text data in the text dataset based on the trained joint extraction model to obtain entities in each of the text data and relationships between entities; The joint extraction model includes a word vector representation building module, a candidate entity representation building module, an entity classification module, a relationship representation building module and a relationship classification module; The word vector representation construction module is used to obtain a first word vector representation and an entity global semantic representation of the text data based on a pre-trained model, obtain a second word vector representation of the text data based on semantic role labeling, and splice the first word vector representation and the second word vector representation to obtain a final word vector representation of the text data; The candidate entity representation construction module is used to obtain enhanced entity implicit semantic information based on the final word vector representation, obtain a preliminary candidate entity set containing any span based on the enhanced entity implicit semantic information, perform a maximum pooling operation on all the preliminary candidate entities in the preliminary candidate entity set to obtain an aggregated candidate entity set, and obtain a final candidate entity representation set based on the aggregated candidate entities in the aggregated candidate entity set, the length of the preliminary candidate entity sequence to which the aggregated candidate entities belong, and the global semantic representation of the entity; The entity classification module is configured to obtain, based on the final candidate entity representation set, a posteriori information of the entity type to which each final candidate entity representation in the final candidate entity representation set belongs through a classification function; The relationship representation construction module is used to combine the aggregated candidate entity representations of the entities corresponding to the final candidate entity representations in the aggregated candidate entity set, which are classified as non-empty, to form an entity pair set, obtain the global semantic representation and local semantic representation of each pair of entity pairs, and obtain the corresponding relationship representation vector based on the global semantic representation, local semantic representation, local sequence length and entity pair ontology of each pair of entity pairs; The relationship classification module is used to obtain the posterior information of the relationship type of each entity pair through a classification function based on the relationship representation vector of each entity pair; The local sequence length is the sequence length of the local semantic representation.

9. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the entity and relationship joint extraction method according to any one of claims 1 to 7 is implemented.

10. A terminal, characterized in that: include: A processor and a memory, wherein the memory is communicatively connected to the processor; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal performs the entity and relationship joint extraction method according to any one of claims 1 to 7.

Images