Relationship-oriented entity relationship joint extraction method and system

Abstract

The present invention belongs to the technical field of natural language processing, and particularly relates to a method and system for joint extraction of entity relations based on relation orientation, encoding sentences in a target text, obtaining a sentence vector representation in the target text; for the sentence vector representation, extracting the relationship type contained in the target text using a relation extraction module; fusing the extracted relationship type with the word vector representation in the target text sentence as prior knowledge, and using an entity recognition module to recognize entities in the target text corresponding to the extracted relationship type. The present invention can reduce attention to irrelevant entities, avoid extracting redundant entities, and then respectively identify the entity pairs corresponding to the identified multiple relationship types, solve the entity overlap problem, and finally extract all entity relationship triples contained in the sentence, improve the accuracy of entity relationship recognition, and facilitate application in actual scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a relationship-oriented entity relationship joint extraction method and system. Background Art

[0002] With the advent of the big data era, a large amount of data is generated all the time on the Internet. Most of these data exist in unstructured form. How to transform massive unstructured data into structured information is currently a hot topic. Information extraction technology came into being in this context. Information extraction technology refers to the technology of automatically extracting information such as events, entities and relationships from natural language texts and outputting them in a structured form. Entity relationship extraction, as a subtask of information extraction, aims to identify entities in texts and the relationships between entities, and express them in the form of triples (head entity, relationship, tail entity). Entity relationship triples are the basic building blocks of knowledge graphs. Entity relationship extraction is not only of great significance in knowledge graphs, but also has important research value in applications such as intelligent search, automatic question answering, and character portraits.

[0003] Early rule-based and traditional machine learning methods required manual definition of many rules and features, required a lot of expert knowledge, had poor domain transfer capabilities, and could not cope with large-scale entity relationship extraction. With the development of deep learning technology, neural networks can automatically extract features, saving a lot of labor costs and having good domain transfer capabilities. They can effectively cope with large-scale entity relationship extraction, and more and more researchers are using deep learning technology for entity relationship extraction. In the early days, deep learning-based entity relationship extraction adopted a pipeline extraction framework, which decomposed the task into two subtasks: entity recognition and relationship classification. First, all entities in the text were identified, and then the relationship types between entity pairs were classified. Although entity recognition and relationship classification can independently and freely select models and have strong flexibility, this type of method has some disadvantages: error propagation, in the case of inaccurate entity recognition, will affect the subsequent relationship classification task; ignore the intrinsic connection between entity recognition and relationship classification tasks; generate redundant entity pairs. Since the entities of the entities need to be paired and then their relationship types are classified, N entities will generate N2 entity pairs, a large number of which are unrelated. These redundant entity pairs will affect the relationship classifier. Summary of the invention

[0004] To this end, the present invention provides a relationship-oriented entity relationship joint extraction method and system, which first extracts the relationship types contained in the text, and then identifies the entities corresponding to the relationship types, reduces attention to irrelevant entities, avoids extracting redundant entities, solves the problem of entity overlap, makes entity relationship extraction more accurate, and is convenient for application in actual scenarios.

[0005] According to the design scheme provided by the present invention, a relationship-oriented entity relationship joint extraction method includes the following contents:

[0006] Encode the sentences in the target text and obtain the sentence vector representation in the target text;

[0007] For the sentence vector representation, the relation extraction module is used to extract the relation types contained in the target text;

[0008] The extracted relationship types are used as prior knowledge and fused with the word vector representation in the target text sentence. The entity recognition module is used to identify the entities in the target text corresponding to the extracted relationship types.

[0009] As a relationship-oriented entity relationship joint extraction method of the present invention, further, a pre-trained BERT model is used to encode sentences in the target text, the word embedding vectors of each word in the target text are obtained, and the vector representation of the sentence context is generated by capturing the sentence features.

[0010] As the relationship-oriented entity relationship joint extraction method of the present invention, further, a classification identifier is added to the beginning of a sentence in the target text, and the sentence with the added classification identifier is used as an input of the BERT model, encoded by the BERT model, and an output sentence vector representation is obtained.

[0011] As a relationship-oriented entity relationship joint extraction method of the present invention, further, in the relationship extraction model, the relationship extraction is modeled as a multi-label binary classification task by adopting the sigmoid function to identify and output various relationship types in the sentence vector representation.

[0012] As the entity relationship joint extraction method based on relationship orientation of the present invention, further, the multi-label binary classification task is expressed as: r =σ(W r ·h cls +b r ), where p r is the output relation type label, W r ∈R N×d , N is the total number of relation types, d is the dimension size of sentence vector representation, b r is the bias vector, σ represents the sigmoid function, h cls is the sentence vector.

[0013] As the entity relationship joint extraction method based on relationship orientation of the present invention, further, the relationship extraction model loss function adopts a binary cross entropy function, and the binary cross entropy function is expressed as:

[0014] Among them, y i is the actual relationship type label.

[0015] As the entity relationship joint extraction method based on relationship orientation of the present invention, further, for the extracted multiple relationship types (r1, r2, ..., r m ), m is the number of extracted relations, and the relationship type encoding vector is obtained by encoding using the table lookup method according to the index. The word vector representation in the sentence and the relationship type encoding vector are superimposed and fused to use the entity recognition module to output the entity corresponding to the extracted relationship type.

[0016] As the relationship-oriented entity relationship joint extraction method of the present invention, further, the extracted relationship type and the word vector representation in the target text sentence are superimposed and fused as the input vector of the entity recognition module, and the entity position corresponding to the relationship type is marked with a binary pointer based on the extracted relationship type, and the entity triple (head entity, relationship, tail entity) corresponding to the relationship type is obtained according to the entity position.

[0017] As a relationship-oriented entity relationship joint extraction method of the present invention, further, in marking the entity position, two identical binary classifiers are used to decode the head entity and the tail entity, one classifier marks the entity start position, and the other classifier marks the entity end position. A binary label is assigned to each word in the input vector and the probability of the word being the start and end position of the entity is detected. According to the proximity principle, the end position closest to the start position is selected to generate the entity corresponding to the relationship type.

[0018] Furthermore, the present invention also provides a relationship-oriented entity-relationship joint extraction system, comprising: a sentence encoding module, a relationship extraction module and an entity recognition module, wherein:

[0019] The sentence encoding module is used to encode the sentences in the target text and obtain the sentence vector representation in the target text;

[0020] A relation extraction module is used to extract the relation types contained in the target text using the relation extraction module based on the sentence vector representation;

[0021] The entity recognition module is used to fuse the extracted relationship type as prior knowledge with the word vector representation in the target text sentence, and use the entity recognition module to identify the entities in the target text corresponding to the extracted relationship type.

[0022] Beneficial effects of the present invention:

[0023] The present invention first extracts the relationship types contained in the text through the relationship extraction module, and then integrates the pre-extracted relationship types into the entity recognition module, reducing the focus on irrelevant entities and avoiding the extraction of redundant entities. Then, the corresponding entity pairs are identified for the multiple relationship types identified, solving the entity overlap problem, and finally extracting all entity relationship triples contained in the sentence to improve the accuracy of entity relationship recognition. The experimental results on the DuIE dataset further show that the F1 value of the entity relationship joint extraction model including the relationship extraction module and the entity recognition module can reach 78.4%, which is 1-28 percentage points higher than the F1 value of the baseline model, and has good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The figure is a schematic diagram of the process of the entity relationship joint extraction method based on relationship orientation in the embodiment;

[0025] Figure 2 This is a schematic diagram of entity overlap in the embodiment;

[0026] Figure 3 This is a schematic diagram of the entity relationship joint extraction model in the embodiment;

[0027] Figure 4 This is an example of a data set in the embodiment;

[0028] Figure 5 It is a schematic diagram of the experimental results of entity overlap in the embodiment;

[0029] Figure 6 It is a schematic diagram of the experimental results of entity redundancy in the embodiment. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and technical solutions.

[0031] Entity relationship extraction is a basic task of information extraction, which aims to extract triples from unstructured data. Most of the current joint extraction methods adopt the idea of ​​identifying entities first and then extracting relationships, which has problems of entity overlap and entity redundancy. Figure 1 As shown, a relationship-oriented entity relationship joint extraction method is provided, which includes the following contents:

[0032] S101, encode the sentences in the target text to obtain the sentence vector representation in the target text;

[0033] S102, extracting the relationship types contained in the target text using a relationship extraction module based on the sentence vector representation;

[0034] S103: The extracted relationship type is used as prior knowledge and integrated with the word vector representation in the target text sentence, and the entity recognition module is used to identify the entity in the target text corresponding to the extracted relationship type.

[0035] In order to solve the drawbacks of the pipeline method, the entity-relationship joint extraction framework is used. The two subtasks of entity recognition and relationship classification share the underlying text encoding layer, so that the two tasks promote each other during the training process and achieve the purpose of task interaction. However, during decoding, the entities are still recognized first and then the relationships are classified, which leads to the problems of error propagation and redundant entities. Furthermore, there is a joint annotation framework that uses unified entities and relationships. It regards entity relationship extraction as a sequence annotation problem, encodes entity labels and relationship labels uniformly, and decodes entities and relationships at the same time. However, since each word is only assigned one label, this method ignores the problem of entity overlap. Entity overlap refers to the situation where there may be duplication between entities in triples when there are multiple triplets in the text, including single entity overlap and entity pair overlap. Single entity overlap refers to the fact that an entity has a relationship with multiple other entities, and entity pair overlap refers to the fact that the same entity pair has multiple relationships. For example Figure 2 As shown, in SingleEntityOverlap (SEO), the single entity "Smile" has a relationship with both the entity "Director Liu" and the entity "Xiao Zhang", and in EntityPairOverlap (EPO), the entity pair (Leave, Xiao Ming) has both the "Composer" and "Singer" relationships. Furthermore, there is a method to solve the entity overlap problem by using sequence-to-sequence with a copy mechanism, but since this method can only copy a single word from a sentence during entity recognition, it is not possible to recognize multi-word entities. There is also a cascade extraction model guided by the head entity, that is, the head entity is extracted first, and then the tail entity is extracted for each relationship. This extraction scheme solves the entity overlap problem well, but it is necessary to extract the tail entity for each relationship corresponding to all the extracted head entities, and many relationships corresponding to the head entities do not have tail entities, so there is a problem of extracting redundant entities. It can be seen from the characteristics of natural language that relationships can usually be obtained through sentence contexts, without relying on entities. For example, from "《A》 is a suspenseful spy film directed by Director Zhang", through the word "directed", we can directly get the relationship of director in the sentence without relying on specific entities. In this case, the relationship types contained in the text are extracted first, and then a joint extraction model is used to identify the entities corresponding to the relationships. By integrating the extracted relationships into the entity recognition module, attention to irrelevant entities is reduced, redundant entities are avoided, the problem of entity overlap is solved, and the accuracy of entity relationship recognition is improved, which is convenient for application in actual scenarios.

[0036] As a relationship-oriented entity relationship joint extraction method in an embodiment of the present invention, further, a pre-trained BERT model is used to encode sentences in the target text, and word embedding vectors of each word in the target text are obtained, and a vector representation of the sentence context is generated by capturing sentence features. Further, a classification identifier is added to the beginning of the sentence in the target text, and the sentence with the added classification identifier is used as the input of the BERT model, encoded by the BERT model, and the output sentence vector representation is obtained. Furthermore, in the relationship extraction module, the relationship extraction is modeled as a multi-label binary classification task by using a sigmoid function to identify and output multiple relationship types in the sentence vector representation.

[0037] For a given sentence S = {w1,w2,...,w n},w i Represents the i-th word in sentence S and the predefined relationship set R. The task of entity relationship extraction is to extract all possible triples (h, r, t) in sentence S, (h, t) represents the entity pair in the sentence, h represents the head entity, t represents the tail entity, r represents the relationship type between them, r∈R.

[0038] The purpose of entity relationship extraction is to extract as many triples as possible in the sentence. There may be overlaps between entities in different triples. Different from the entity-oriented cascade extraction method, the extraction scheme can be modeled as follows:

[0039] p((h,r,t)|S)=p(r|S)p((h,t)|r,S)

[0040] The joint extraction model constructed according to this scheme decomposes entity relationship extraction into two parts: p(r|S) and p((h,t)|r,S). p(r|S) first extracts the relationship type r in sentence S, and p((h,t)|r,S) identifies the entity pair (h,t) corresponding to this relationship type based on the extracted specific relationship r and sentence S.

[0041] With the introduction of pre-trained language models, many natural language processing tasks have made breakthrough progress, especially the emergence of the BERT model, which has continuously achieved SOTA results in various natural language processing tasks. The BERT model trained on a large-scale corpus based on the self-attention mechanism contains rich semantic information and provides a good foundation for downstream tasks. In order to better capture sentence features, the pre-trained BERT model can be used to encode sentences as the basis for subsequent relation classification and entity recognition. For a sentence S, add the [CLS] identifier at the beginning of the sentence to obtain the input of the BERT model as follows:

[0042] S'={[CLS],w1,w2,...,wn}

[0043] [CLS] is the classification identifier, w i Represents the words in the sentence, n is the length of the original sentence, and the length of the sentence S' after adding the special identifier is n+1. Encode the sentence S' through the BERT model and obtain the last layer output vector of the BERT model:

[0044] H w =[h cls ,h1,h2,...,h n ]

[0045] h cls represents the vector representation of the [CLS] identifier encoding, h i Represents the word w in the sentence i The encoded vector representation. h cls ,h i The dimensions are the same, that is, h cls ∈R d ,h i ∈R d , d is the dimension size of BERT encoding.

[0046] Unlike the entity-oriented cascade extraction method, the ROJER model first detects the type of relationship contained in the sentence and uses the relationship type as prior knowledge to help entity recognition, which helps to avoid the problem of extracting redundant entities. Different sentences contain different numbers of relationships, and a sentence may contain multiple relationships. Detecting the relationship in a sentence can be regarded as a multi-label classification problem. The encoded [CLS] identifier vector h cls Aggregates the feature information of sentences and is often used to perform some classification tasks on sentences. cls The vector extracts the relations contained in the sentence. Since a sentence may contain multiple relations, the sigmoid function is used to model it as a multi-label binary classification task. When a certain relation is contained, the label of the corresponding position is assigned to 1, otherwise it is assigned to label 0.

[0047] p r =σ(W r ·h cls +b r )

[0048] W r ∈R N×d , b r is the bias vector, N is the total number of relationship types, and σ represents the sigmoid function.

[0049] For the multiple relation types (r1, r2, ..., r m), m is the number of relations identified, and its encoding is obtained by table lookup according to its index 1≤j≤m, so as to integrate sentence encoding for corresponding entity recognition. The lookup table is initialized with random parameters and is continuously updated during the training process.

[0050] This classification task is binary classification, and the sigmoid function is used, so the loss function is binary cross entropy.

[0051]

[0052] N is the number of relationship types, y i is the true relation label in the training set.

[0053] As a relation-oriented entity relation joint extraction method in an embodiment of the present invention, further, the extracted relation type and the sentence vector representation in the target text are superimposed and fused as the input vector of the entity recognition module, and the entity position corresponding to the relation type is marked with a binary pointer based on the extracted relation type, and the entity triple (head entity, relation, tail entity) corresponding to the relation type is obtained according to the entity position. Further, in marking the entity position, two identical binary classifiers are used to decode the head entity and the tail entity, one classifier marks the entity start position, and the other classifier marks the entity end position, a binary label is assigned to each word in the input vector and the probability of the word being the entity start and end position is detected, and according to the proximity principle, the end position closest to the start position is selected to generate the entity corresponding to the relation type.

[0054] The entity recognition module under a given relationship aims to extract entity pairs corresponding to a given relationship in a sentence. The input vector of entity recognition is composed of the relationship type code identified by the relationship classification and the word code in the sentence, so as to integrate the acquired prior knowledge of the relationship type into the entity recognition, reduce the attention to entities irrelevant to this relationship type, avoid extracting redundant entities, and improve the effect of entity recognition.

[0055]

[0056] h i represents the word vector after BERT encoding, Represents the relationship type encoding vector obtained by table lookup. i ∈R d , l is the dimension size of the relationship type encoding.

[0057] The decoding method for the head entity and the tail entity uses two identical binary classifiers respectively. One classifier is used to identify the start position of the entity, denoted as start, and the other classifier is used to mark the end position of the entity, denoted as end. By assigning a binary label to each word, the start and end positions of the entity are detected respectively. If a word is the start or end position of the entity, it is marked as 1, otherwise it is 0.

[0058]

[0059]

[0060]

[0061]

[0062] Indicates the probability of the i-th word in the sentence being the start and end position of the head entity, Indicates the probability of the i-th word in the sentence being the start and end position of the tail entity. If the probability is greater than the set threshold, the corresponding position is marked as 1, otherwise it is marked as 0. The dimensions are the same, both are R n ×(d+l) , is the bias vector. When a sentence contains multiple identical relations, the nearest principle is used to extract entities, that is, for the start position, the entity is generated by matching the end position closest to it on the right. Since the process of extracting entities of different relation types is independent of each other, the same entity or entity pair can be identified for different relation types, avoiding the problem of entity overlap.

[0063] The entity decoding method also uses the sigmoid function, and the loss function is also the binary cross entropy.

[0064]

[0065] n is the length of the sentence, Indicates the true label of the starting or ending position of the i-th word in the sentence as the head entity or the tail entity.

[0066] In order to realize the relationship-oriented entity-relationship joint extraction, this case solution can construct the entity-oriented joint extraction model ROJER architecture based on the above method, see Figure 3As shown in the figure, it includes three modules: sentence encoding, relation extraction and entity recognition. The sentence encoding module uses the BERT model to encode the sentence and obtain the sentence vector and the vector representation of each word; the relation extraction module uses the sentence vector for multi-label classification to extract the relation type contained in the sentence; the entity recognition module identifies the corresponding head entity and tail entity based on the extracted relation type. Using this joint extraction model architecture, the relationship can be extracted first, and then the corresponding entity can be identified. Compared with the entity-oriented joint extraction method, the pre-extracted relation information can be integrated into the entity recognition, so that the entity recognition module reduces the attention to the entities irrelevant to this relation type, avoids the extraction of redundant entities, and improves the effect of entity recognition. Moreover, in the process of entity recognition, the same entity or entity pair can be recognized multiple times by different relation types, thus avoiding the problem of entity overlap. When the model is jointly trained, the entity recognition and relation extraction can be interacted through the joint training of the loss function, and the internal connection between the modules can be enhanced. The total loss function includes two parts: relation extraction and entity recognition. The loss function of each module is added to obtain the joint loss function of the model. Through joint training, the parameters in the learning model are minimized:

[0067] L=L rel +L ent

[0068] Furthermore, based on the above method, an embodiment of the present invention also provides a relationship-oriented entity relationship joint extraction system, comprising: a sentence encoding module, a relationship extraction module and an entity recognition module, wherein:

[0069] The sentence encoding module is used to encode the sentences in the target text and obtain the sentence vector representation in the target text;

[0070] A relation extraction module is used to extract the relation types contained in the target text using the relation extraction module based on the sentence vector representation;

[0071] The entity recognition module is used to fuse the extracted relationship types as prior knowledge with the word vector representation in the target text sentence, and use the trained entity recognition model to identify the entities in the target text corresponding to the extracted relationship types.

[0072] In order to verify the effectiveness of this solution, the following is a further explanation based on experimental data:

[0073] The DuIE dataset is used to train and test the ROJER model architecture in this case. All sentences in the DuIE dataset are derived from Baidu Encyclopedia and Baidu News corpus, with a total of 214,739 instances and 49 relationship types, including 173,108 instances in the training set, 21,639 instances in the validation set, and 19,992 instances in the test set. Each data contains the sentence text and one or more triples annotated from the sentence. The dataset annotates 458,184 triples, covering three types of triple data: Normal, EPO, and SEO. Therefore, it can be used to verify the performance of the model architecture in this case on the entity overlap problem. Since the test set is not published, 10% of the training set is randomly selected as the validation set, the remaining 90% is used as the training set, and the original validation set is used as the test set. The split dataset and entity overlap statistics are shown in Table 1.

[0074] Table 1 Dataset statistics

[0075]

[0076] From the statistics, we can see that there are a lot of entity overlaps in the data set, and there are cases where a sentence sample has both single entity overlap and entity pair overlap. For example, in the test set, there are 347 sentence instances with both types of overlaps, and the entity relationship extraction is relatively complicated. Figure 4 In the dataset example shown, the entity "Movi" overlaps with the entity "Kuwait" as an entity pair, and also overlaps with "July 11, 1988" as a single entity.

[0077] The accuracy, recall, and F1 value commonly used in relation extraction are used as evaluation indicators. Only when the head entity, relation type, and tail entity of the extracted triple are completely consistent with the annotation, the triple is judged to be correct. The calculation formula is as follows.

[0078]

[0079]

[0080]

[0081] TP represents the number of positive samples judged as positive, FP represents the number of negative samples judged as positive, and FN represents the number of positive samples judged as negative. The samples here refer to triplets.

[0082] The model architecture can be implemented based on HuggingFace's Transformers library, and the BERT Chinese pre-trained language model used is "bert-base-chinese". The experimental environment is shown in Table 2.

[0083] Table 2 Experimental environment

[0084]

[0085] The hyperparameter settings of the model are shown in Table 3.

[0086] Table 3 Hyperparameter settings

[0087]

[0088] Some models are selected for comparison, which include pipeline models and joint extraction models. The joint extraction models include one-shot decoding, independent decoding, and cascade decoding extraction models. The basic principles of the five comparison models are as follows.

[0089] (1) Pipeline: This model is a pipeline model. The entity recognition model uses sequence labeling to identify all entities in a sentence, and the relationship classification model uses the sentence vector and entity vector encoded by BERT to classify the relationship between entities.

[0090] (2) NovelTagging: A joint decoding model that encodes entity tags and relationship tags uniformly and can decode entities and relationships at the same time, but there is an entity overlap problem.

[0091] (3) MHS (Multi-Head Selection): A multi-head selection joint extraction model that uses CRF to predict entity labels and extracts relations and entities through Multi-Head Selection to solve the problem of entity overlap.

[0092] (4) SpERT (Span-based Entity and Relation Transformer): First identify entities, then fuse entities and the context between entities to classify relations. The two tasks share the underlying encoding.

[0093] (5) CasRel (cascade binary tagging framework): An entity-oriented cascade extraction model that first extracts the head entity and then identifies the tail entity for each relationship.

[0094] The performance of each model on the dataset is shown in Table 4.

[0095] Table 4 Experimental results of DuIE dataset

[0096]

[0097] The results show that the ROJER model is better than other comparison models.

[0098] 1) The NovelTagging model uses unified encoding of entity relationships and does not require multiple decoding, so the accuracy is high, but the recall rate is low because it cannot handle entity overlap. The ROJER model architecture in this case can solve the entity overlap problem, and the recall rate is improved by 38.5% and the F1 value is improved by 28.1% compared with NovelTagging.

[0099] 2) Although the Pipeline model is a pipeline extraction method, the underlying encoding uses BERT, and the F1 value is much higher than the NovelTagging model and MHS model of static word vectors. The ROJER model also uses BERT as an encoder, and the F1 value is greatly improved, indicating that the pre-trained BERT model has better semantic expression ability than word vectors.

[0100] 3) The SpERT model, CasRel model and the ROJER model architecture in this case are all joint extraction models based on BERT, and their effects are better than the Pipeline model, which shows that the joint extraction model can alleviate the problems of error propagation and task interaction in the pipeline model.

[0101] 4) The CasRel model uses the method of extracting the head entity first and then predicting the tail entity for each relationship. It is superior to the SpERT model that first identifies the entity and then classifies the relationship in all indicators. Compared with the current advanced CasRel model, the ROJER model improves the accuracy, recall, and F1 value by 1.9%, 0.7%, and 1.2%, respectively, indicating that the model in this case uses the method of extracting the relationship first and then predicting the corresponding entity for the identified relationship, which can improve the effect of triple extraction.

[0102] Entity overlap experimental results: In order to test the model's ability to deal with entity overlap, the test data is divided into three categories: entity non-overlap (Normal), single entity overlap (SEO) and entity pair overlap (EPO), and experiments are conducted on each type of data. Figure 5 As shown in the figure, it can be seen that the NovelTagging model can only predict one entity relationship tag for each word, so the F1 value is relatively low in single entity overlap and entity pair overlap. The Pipeline model and the SpERT model both belong to the framework of first identifying all entities and then extracting the relationship between entities. For the same entity pair, only one relationship can be predicted, so the performance is poor in the case of entity pair overlap. The MHS model, CasRel model and the ROJER model architecture in this case can all solve the two entity overlap problems and perform relatively balanced in the three cases. In comparison, the model architecture in this case has an advantage in the method of identifying the corresponding entities of multiple pre-extracted relationship categories.

[0103] In order to verify the effectiveness of the model in solving the entity redundancy problem, the detailed extraction results of each model are as follows Figure 6 As shown in the figure, it can be seen that due to the large number of entity overlaps in the data set, the NovelTagging model cannot solve the entity overlap problem, so the number of extracted triplets and correct triplets are relatively small. The SpERT model and the Pipeline model both belong to the method of first identifying entities in the sentence and then extracting the relationship between entities. The SpERT model is a shared parameter joint extraction model. Compared with the Pipeline model, the number of extracted triplets is smaller, but the number of correct triplets among the extracted triplets is larger, indicating that the joint extraction model SpERT can alleviate the entity redundancy problem of pipeline models such as Pipeline. Compared with the advanced CasRel model, the ROJER model also extracts fewer triplets and has a higher accuracy rate, indicating that the ROJER model architecture in this case first extracts the relationship type in the sentence, and then integrates the prior knowledge of the relationship type into the entity recognition, reducing the entity recognition module's attention to irrelevant entities and avoiding the extraction of redundant entities.

[0104] Entity relationship extraction technology can automatically extract triples from unstructured texts and is the basis for natural language applications such as knowledge graphs. In the experiment of this case, in order to solve the problems of entity redundancy and entity overlap in the process of triple extraction, the ROJER model is used to first identify all the relationship types contained in the sentence, and then the identified relationship types are integrated into the entity recognition process. The corresponding entity recognition under the specified relationship reduces the interference of irrelevant entities and avoids the problem of entity redundancy. The start and end positions of the head and tail entities are marked respectively by pointer marking, and the entity pairs corresponding to the corresponding relationship types are identified, solving the problem of entity overlap. The F1 value on the DuIE dataset reached 78.4%, which is better than the relevant comparison model. The extraction results show that the model architecture of this case effectively solves the problems of entity redundancy and entity overlap.

[0105] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0106] Based on the above method and / or system, an embodiment of the present invention also provides a server, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0107] Based on the above method and / or system, an embodiment of the present invention further provides a computer-readable medium having a computer program stored thereon, wherein the above method is implemented when the program is executed by a processor.

[0108] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0109] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0110] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A relationship-oriented entity relationship joint extraction method, characterized in that: Contains the following: Encode the sentences in the target text and obtain the sentence vector representation in the target text; For the sentence vector representation, the relation extraction module is used to extract the relation types contained in the target text. In the relation extraction module, the sigmoid function is used to model the relation extraction as a multi-label binary classification task to identify and output various relation types in the sentence vector representation. The multi-label binary classification task is expressed as: r =σ(W r ·h cls +b r ), where p r is the output relation type label, W r ∈R N×d , N is the total number of relation types, d is the dimension size of sentence vector representation, b r is the bias vector, σ represents the sigmoid function, h cls is the sentence vector; the loss function of the relation extraction module adopts the binary cross entropy function, which is expressed as: Among them, y i is the real relationship type label; For the extracted multiple relationship types (r1, r2, ..., r m ), m is the number of extracted relations, the extracted relation types are used as prior knowledge, encoded using the table lookup method according to the index, and the relation type encoding vector is obtained. The word vector representation in the target text sentence and the relation type encoding vector are superimposed and fused, and the entity recognition module is used to identify the entities in the target text corresponding to the extracted relation types.

2. The entity relationship joint extraction method based on relationship orientation according to claim 1 is characterized in that: The pre-trained BERT model is used to encode the sentences in the target text, obtain the word embedding vectors of each word in the target text, and generate the vector representation of the sentence context by capturing the sentence features.

3. The entity relationship joint extraction method based on relationship orientation according to claim 2 is characterized in that: Add a classification identifier to the beginning of a sentence in the target text, use the sentence with the classification identifier as the input of the BERT model, encode it through the BERT model, and obtain the output sentence vector representation.

4. The entity relationship joint extraction method based on relationship orientation according to claim 1 is characterized in that: The extracted relationship types and word vector representations in the target text sentences are superimposed and fused as the input vector of the entity recognition module. Based on the extracted relationship types, binary pointers are used to mark the entity positions corresponding to the relationship types, and the entity triples corresponding to the relationship types (head entity, relationship, tail entity) are obtained according to the entity positions.

5. The entity relationship joint extraction method based on relationship orientation according to claim 4 is characterized in that: In marking entity positions, two identical binary classifiers are used to decode the head entity and the tail entity. One classifier marks the entity start position and the other classifier marks the entity end position. A binary label is assigned to each word in the input vector and the probability of the word being the start and end position of the entity is detected. Based on the proximity principle, the end position closest to the start position is selected to generate the entity corresponding to the relationship type.

6. A relationship-oriented entity relationship joint extraction system, characterized in that: Contains: sentence encoding module, relation extraction module and entity recognition module, among which, The sentence encoding module is used to encode the sentences in the target text and obtain the sentence vector representation in the target text; The relation extraction module is used to extract the relation types contained in the target text based on the sentence vector representation. The relation extraction module uses the sigmoid function to model the relation extraction as a multi-label binary classification task to identify and output multiple relation types in the sentence vector representation. The multi-label binary classification task is expressed as: r =σ(W r ·h cls +b r ), where p r is the output relation type label, W r ∈R N×d , N is the total number of relation types, d is the dimension size of sentence vector representation, b r is the bias vector, σ represents the sigmoid function, h cls is the sentence vector; the loss function of the relation extraction module adopts the binary cross entropy function, which is expressed as: Among them, y i is the real relationship type label; Entity recognition module, used for extracting multiple relationship types (r1, r2, ..., r m ), m is the number of extracted relations, the extracted relation types are used as prior knowledge, encoded using the table lookup method according to the index, and the relation type encoding vector is obtained. The word vector representation in the target text sentence and the relation type encoding vector are superimposed and fused, and the entity recognition module is used to identify the entities in the target text corresponding to the extracted relation types.

Images