A document-level relation extraction method based on balanced labels

Abstract

The application discloses a document-level relation extraction method based on balanced labels, and belongs to the field of relation extraction in natural language processing. The application realizes the method as follows: collecting document data of application fields and relations between entities concerned by the fields. Based on entity types and positions of the entities in relation semantics, balanced relation labels are obtained by permutation and combination to join a relation label set, and the relation labels existing between entities in a document data set are labeled. A document-level relation extraction model is built, a base encoding model is selected, a three-linear pooling module is added to fuse representations of subject entities, object entities and related contexts, and weights of the contexts in reasoning are improved. A loss function for training of the improved document-level relation extraction model is constructed, and confidence scores of relation labels obtained by reasoning of the document-level relation extraction model are improved. The document-level relation extraction model is trained, the best document-level relation extraction model is selected for actual application, and accuracy of document-level entity relation extraction is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to a balanced label-based document-level relation extraction method. Specifically, it relates to a method of adding balanced relation labels between entities according to entity types to improve the accuracy of relation labels extracted by a document-level relation extraction model, and belongs to the field of relation extraction in natural language processing. TECHNICAL BACKGROUND

[0002] A significant part of today's Internet data is text data. Extracting effective structured information from unstructured text data is an important task in natural language processing, i.e., information extraction. The basis for clarifying information in text is to distinguish the relationship between different entities. The relationship extraction technology aims to give the relationship between different entities in text data, and plays an important role in information extraction. The extracted relation triplets can be used for knowledge graph construction, human-computer question answering, information retrieval, and other downstream tasks, and are one of the important supporting technologies for machine understanding of text data.

[0003] To complete the relationship extraction between entities, the commonly used technology is to design, train and apply a deep neural network model for relationship extraction according to the characteristics of the text data set and the relationship label set. Traditional relationship extraction models limit the training and application of relationship extraction models to the relationship between entities in a single sentence, i.e., sentence-level relationship extraction models. Now, thanks to the improvement of computing power, relationship extraction models are applied to extract the relationship between entities expressed in a document composed of multiple sentences, which involves the design, training and application of document-level relationship extraction models.

[0004] The method for constructing and applying document-level relationship extraction includes the following steps: first, collect document data in the application field to form a document data set {D}, and collect the relationship labels between entities in the field of interest to form a relationship label set {R}. Design the architecture of the document-level relationship extraction model, then fine-tune the document-level relationship extraction model. Preprocess the document data set {D} and the relationship label set {R} as the input and supervision of the document-level relationship extraction model, so that the trained relationship extraction model can understand and distinguish the semantics of each relationship label in the relationship label set {R} and make inferences about the relationship labels between entities. Finally, use the trained document-level relationship extraction model to infer the relationship labels between entities in the document data to be processed, and give the entity relationship triplets (head entity, relationship label, tail entity).

[0005] The research and application of the document-level relation extraction method can expand the application scope of the relation extraction, and has obtained more and more attention in the research and application in the field of natural language processing. For an entity pair composed of two entities, a document-level relation extraction model needs to infer a subset {r} of relation labels existing between the entity pair from a relation label set {R}. Such application requirements require the document-level relation extraction model to distinguish the semantic differences between the relation labels in the relation label set {R}, and require the document-level relation extraction model to infer as accurate relation labels as possible.

[0006] The training purpose of the document-level relation extraction model is to obtain a multi-label, binary classification, deep neural network model capable of determining whether a relation label exists between entities, and in the relation label set {R} concerned by a user, distinguish the relation label set {r} existing between the entity pair in the document semantics from the relation label set {R-r} not existing. A common method needs to set a threshold label separating {R} and {R-r}, and use a model training loss function corresponding to the threshold label, so that the model learns to use the confidence score of the threshold label to separate the relation extraction capability of the relation label sets {R} and {R-r}.

[0007] The design of a single threshold label and the design of a training loss function in the common method make the document-level relation extraction model have the following problems in application: 1. The relation labels in the relation label set {R} concerned by the user are similar in semantics and lack relevance, 2. The number of relation labels in the relation label set {r} existing between the entity pair is unbalanced, the model is biased towards inferring and predicting that there is no relation label between the entity pair, 3. The common loss function causes the document-level relation extraction model to learn a shortcut to reduce the training loss during the training process: by suppressing the confidence score of the relation label in the relation label set {R-r} not existing, the confidence score of the relation label in the expected relation label set {r} and the confidence score of the threshold label are not obvious enough, and finally the accuracy of the document-level relation extraction application is suboptimal. SUMMARY

[0008] In view of the problems and challenges found in the application of the prior art method, the purpose of the present application is to provide a document-level relation extraction method based on balanced labels, which expands the relation label set {R} with balanced relation labels, increases the relevance between relation labels, and helps the document-level relation extraction model to understand the semantics of the relation labels; supplements the balanced relation label annotation of the training set and the validation set, reduces the imbalance of the number of relation labels in the relation label set {r} existing between the entity pair; constructs and uses a loss function suitable for balanced labels to train the document-level relation extraction model, improves the confidence score of the relation label in the existing relation label set {r} given by the document-level relation extraction model, and improves the accuracy of the document-level relation extraction.

[0009] The object of the application is achieved by the technical method described below.

[0010] The application discloses a document-level relation extraction method based on balanced labels, which collects document data in an application field and relations between entities concerned in the field. Balanced relation labels are obtained by arrangement and combination based on entity types and positions of the entities in relation semantics, and are added to a relation label set to label relation labels existing between entity pairs in a document data set as supervision signals for training of a document-level relation extraction model. The document-level relation extraction model is built, a base encoding model is selected, a three-linear pooling module is added to fuse representations of subject entities, object entities and related contexts, and weights of the contexts in reasoning are increased. A loss function for training of the improved document-level relation extraction model is constructed to improve confidence scores of relation labels obtained by reasoning of the document-level relation extraction model. Hyperparameters for training of the document-level relation extraction model are adjusted on a validation set, and the document-level relation extraction model with the best performance on a test set is selected for actual application, so that a higher accuracy of document-level entity relation extraction is obtained.

[0011] The application discloses a document-level relation extraction method based on balanced labels, which comprises the following steps:

[0012] Step 1: When the document-level relation extraction is actually put into use in the A field, first, document data in the A field is collected to form a document data set {D}, and relations between entities concerned in the A field are collected to form a relation label set {R}. The total document data set {D} is divided into a training set, a validation set and a test set. The training set is used for training of the document-level relation extraction model, the validation set is used for quick verification of fine-tuning parameters of the document-level relation extraction model, and the test set is used for measuring and comparing relation extraction effects of the document-level relation extraction model actually put into use. For each document d of the total document data set {D}, named entity recognition is performed to obtain position annotations and type information of entities in the document, and a set of all types of the entities is denoted as {C}.

[0013] The document d is composed of S d sentences, and a set of entities contained in the document d is wherein ||{e d}|| represents the number of entities in the document d. The entity e has an entity type c, and [C} represents a set of entity types. The entity e can appear in multiple "entity mentions" in different positions in the document d, and a set of "entity mentions" of the entity e is denoted as wherein ||{m e}|| represents the number of "entity mentions" of the entity e in the document d. h The entity e is referred to as a head entity because it is located in the position of a subject entity in the relation semantics. t The entity e is referred to as a tail entity because it is located in the position of an object entity in the relation semantics.

[0014] The head-tail entity pair (e h ,e t ) in the document is annotated with a set of relationship labels {R} that the A domain is interested in. For any relationship label r h ∈{R}, if (e t ,e * ) and label r h can form a relationship semantic triple (e t ,r * ,e h ), then add relationship label r * to the expected relationship label set {r} for model reasoning. If (e t ,e * ) does not exist under the semantics of the document, any relationship label r h in the set of relationship labels {R}, then the expected relationship label set {r} for model reasoning should only have the "NA" relationship label. The entity pair (e t ,e * ) of the document d after model reasoning, the confidence score of the "NA" relationship label is used to determine whether (e h ,e t ) exists relationship label r h , the confidence score of the relationship label r t greater than the "NA" relationship label is determined as the relationship label of (e * ,e * ) exists, so the "NA" relationship label is called the threshold relationship label.

[0015] The expected relationship label set of the entity pair (e h ,e t ) in the document after preliminary relationship label annotation is {r} = {r1, r2,...}. Let the entity types of the head entity e h and the tail entity e t be c h type and c t type respectively, then based on the positional relationship of c h type and c t type, c h type and c t type, c h type and c t type, the balanced relationship label b of the entity pair (e h ,e t ) can be supplemented. The semantics of the b label is c h type and c tThe relationship labels of the types appearing at the head and tail positions, respectively, and the final expected relationship label set with balanced relationship label annotations for training is {r b}={r1,r2,...,b}, b and part of the relationship labels in the original relationship label set {R} have a correlation of co-occurrence at the same time, which is beneficial to the model to understand and distinguish the semantics of the relationship labels in the relationship label set {R}. Other introduced balanced relationship labels are also defined by entity types, and the positions of the types appearing at the head and tail. The number of types in the entity type set {C} is ||{C}||, and the entities of the ||{C}|| entity types can appear at the head or tail position of the entity pair. The total number of balanced relationship labels added to the relationship label set {R} is ||{C}|| 2 , obtaining the relationship label set {R b} with balanced labels.

[0016] Supplement the balanced relationship labels in the training set and the validation set of the Chinese document, so that the expected relationship label set {r} of each entity pair contains at least balanced relationship labels, reducing the imbalance of the expected relationship label set {r} between the entity pairs (e h ,e t ), and improving the correlation between the relationship labels in the document data set.

[0017] Step 2: Insert the start special token “[CLS]” and the special token “[SEP]” that separates the sentences at the beginning and end of the S d sentences of the document d. Highlight each entity mention m e,j of the entity e with a pair of asterisks “*”. Then use the tokenizer corresponding to the base encoding model to tokenize the document, obtaining the id list of the token of the document in the tokenizer token table [id1, id2,...]. Determine the start and end positions of the entity mention according to the id of the “*” token, and then obtain the mapping table with “key entity e and value position table of each entity mention before the * token”.

[0018] A mapping table is established, with the key entity e and the value position table of each entity mention before the “*” token, obtaining {e:[m e,1 ,m e,2 ,...} The mapping table is a preprocessing for entity representation calculation, which reduces the coupling degree between the entity representation calculation module and the tokenization and encoding module in the document-level relationship extraction model. A mapping table is established, with the key entity pair (e h ,e t ), and the value of the mapping table is defined as the expected relationship label set {r b} with balanced relationship label annotations for the training set and the validation set, obtaining {(e h ,et ):[r1,r2,...,b]}, for the test set, the value of this mapping table is defined as the expected relation label set {r}, resulting in {(e h ,e t This mapping table is a preprocessing step for entity pair representation computation, loss function calculation for document-level relation extraction model training, and model evaluation on the test set, accelerating the computation of entity pair representation and loss function modules in the document-level relation extraction model.

[0019] Step 3: Using the list of word IDs [id1, id2, ...] obtained in Step 2 as input, encode the document using a pre-trained language model according to Equation (1):

[0020] H,A = PLM([id1,id2,...]) (1)

[0021] Where PLM stands for Pretrained Language Model, representing the pretrained language encoding model used to build the document-level relation extraction model; H represents the matrix composed of the vector representations of each word in the document's word ID list; and so on. t is an abbreviation for the first letter of the word (token), N t is the number of terms in the document, and h is the dimension of the vector representation. It is a matrix composed of the attention vectors of each word output by the transformer layer of the last layer of the pre-trained language model PLM. The attention vector of each word is obtained by average pooling the attention vectors of different attention heads.

[0022] Using the mapping table {e:[m] obtained in step 2, where the key is entity e and the value is a table of the positions of each entity mentions before the asterisk (*), e,1 ,m e,2 ,...]}, retrieve the location table of entity mentions for entity e [m e,1 ,m e,2 [,...]. Extract the entity mention location table [m] from the document term vector representation matrix H. e,1 ,m e,2 The vector representations of the lexical units at each position are then pooled to fuse the vector representations of entity mentions at different positions in the document, resulting in the head entity e. h Combined tail entity e t The vector representation of H is denoted as H. h and H t The vector representation H of the head and tail entities. h and H t After the location information fusion module of the document-level relation extraction model, the entity vector representation Z with location information is obtained.h and Z t .

[0023] head entity e h and tail entity e t vector representation H h and H t is calculated by LogSumExp pooling technique, let the set of “entity mentions” of entity e be where ||{m e}|| is the number of “entity mentions” of entity e in document d, then the expression of vector representation of entity e is:

[0024]

[0025] where h is the dimension of the wordpiece vector output by the pre-trained language encoding model, where is the vector representation of the jth entity mention of entity e, thus obtaining the vector representation H h and H t of head entity e h and tail entity e t .

[0026] The calculation expression of the position information fusion module of the entity vector representation Z h and Z t with position information is:

[0027] Z e = tanh(WH e +b) (3)

[0028] where Z e is the entity vector representation with position information, h is the dimension of the wordpiece vector representation output by the pre-trained language encoding model, matrix W and vector b are the mathematical representation of the position information fusion module, and are the learnable parameters of the position information fusion module. The position information fusion module converts the vector representation H e of entity e obtained by formula (2) into the entity vector representation Z e with position information.

[0029] Step 4: In order to let the document-level relation extraction model reason the relationship label of entity pair (e h , e t ), it is also necessary to add the context vector representation that can support the reasoning. Using the matrix A composed of the attention vector of each wordpiece obtained in step 3, the attention vectors of the head and tail entities to the document context wordpieces A h and A t are obtained. The attention vectors Ah and A t First, perform the Hadamard product, then perform vector normalization to obtain the entity attention vector A with respect to document context terms. (h,t) Then A (h,t) Performing the Hadamard product of the vector representation matrix H of the document's terms yields the vector representation C of the context of common interest to the head and tail entities. (h,t) .

[0030] First, we obtain the attention vector A of entity e with respect to the document context from the matrix A composed of the attention vectors of each word. e The calculation expression is:

[0031]

[0032] in h is the dimension of the word vectors output by the pre-trained language encoding model, where Let the attention vector of the j-th entity mention of entity e for the document context lexical units be obtained, thus obtaining the head entity e. h Combined tail entity e t The attention vector A for the document context h and A t .

[0033] Entity attention vector A about document context terms (h,t) The calculation expression is:

[0034]

[0035] in The operator for Hadamard product, Norm represents vector normalization.

[0036] The vector representation C of the context that the head and tail entities are both interested in. (h,t) The calculation expression is:

[0037] C (h,t) =H T ·A (h,t) (6)

[0038] in • is the inner product operator.

[0039] By using a grouped trilinear pooling module, the head entity vector representation Z is fused. h Tail entity vector representation Z t and the contextual information C associated with the entity pair (h,t) This yields the vector representation H of the entity pairs. (h,t) The vector representation H of entity pairs (h,t)The confidence score for each relation label is obtained through the classification module of the document-level relation extraction model.

[0040] The entity vector representation Z with location information obtained according to formula (3) h and Z t The vector representation C of the context of common interest to the head and tail entities obtained by formula (6) (h,t) These three vectors Z h Z t and C (h,t) Before entering the trilinear pooling module, a dimension transformation module is required, and the operation formula is:

[0041] v * =v.view(g,h / g).unsqueeze((d1,d2)) (7)

[0042] in Representing Z h Z t and C (h,t) For vectors, the "view" and "unsqueeze" methods respectively transform the vector shape into a matrix and expand the matrix dimensions to support matrix multiplication using linear pooling. `g` is used to reduce the grouping hyperparameter of the linear pooling module parameters. `h` is the dimension of the word vector output by the pre-trained language encoding model. `d1` and `d2` are the two positions where the dimensions need to be expanded. Z... h Expanding the two dimensions (2,3) yields Z t Expanding the two dimensions (1,3) yields C (h,t) Expanding the two dimensions (1,2) yields

[0043] Vector representation H of entity pairs (h,t) The calculation method is First perform the Hadamard product, then adjust the matrix to a one-dimensional vector:

[0044]

[0045] The vector representation of entity pairs g is a hyperparameter for reducing the grouping of parameters in the linear pooling module, and h is the dimension of the word vectors output by the pre-trained language encoding model.

[0046] The number of classification heads in the classification module of the document-level relation extraction model corresponds to the relation label set {R} with balanced labels. b The number of relation labels in the matrix. The model's classification module obtains the relation labels r. * The expression for calculating the confidence score is:

[0047] P(r * |e h ,e t )=W r H (h,t) +b r (9)

[0048] where P is the confidence score of the neural network model classifying relation label r * , is a learnable parameter that reduces the dimension of the vector representation H (h,t) of the entity pair, is a learnable parameter of the classification module.

[0049] Step 5: After obtaining the confidence score of the relation label inferred by the model, calculate the loss function value of the model training. Train the document-level relation extraction model on the training set for multiple rounds until the training loss no longer decreases. Adjust the training parameters on the validation set and save the document-level relation extraction model parameters with the highest accuracy on the test set for actual document-level relation extraction.

[0050] For the calculation of the loss of the label in the expected relation label set {r b}, the Ranking Loss loss function algorithm is used. For the calculation of the loss of the label not in the expected relation label set {R b -r b}, the pooling Ranking Loss algorithm is used to calculate the training loss to avoid the model learning to reduce the training loss: by suppressing the confidence score of the relation label in the relation label set {R h -r t} that does not exist for the entity pair (e b ,e b ) to reduce the training loss. Make the confidence score of the relation label in the relation label set {r h} that exists for the entity pair (e t ,e b ) and the confidence score of the threshold label not obvious.

[0051] In the relation label r b in the relation label set {r i}={r1,r2,...,b} that exists for the entity pair (e i ,e h ) under the document semantics, let the confidence score of the model predicted relation label r t be denoted as and for the entity pair (e - ,et The non-existent relation label set {r -}={R b -r b Relationship tags in} The confidence score on is denoted as The confidence score of the threshold relation label that separates positive and negative relations is denoted as f0.

[0052] The overall calculation expression for the training loss function of the document-level relation extraction model is as follows:

[0053]

[0054] The training loss function described above aims to expand the relation set {r} b Confidence score of relation labels in} The difference between the confidence score f0 of the threshold relationship label and the entity pair (e) is widened. h ,e t The non-existent relation label set {r - Relationship tags in} Confidence score The gap.

[0055] The document-level relation extraction model is trained multiple times on the training set documents until the training loss no longer decreases. The training parameters are then adjusted on the validation set. The parameters of the document-level relation extraction model with the highest accuracy on the test set are saved for actual document-level relation extraction.

[0056] Step 6: Apply the trained document-level relation extraction model obtained in Step 5 to the relation extraction field of natural language processing to perform document-level relation extraction and obtain the document-level relation extraction results.

[0057] Step 7: Solve relevant technical problems in the corresponding field based on the document-level relationship extraction results obtained in Step 6.

[0058] The related technical issues include knowledge graph construction and integration, case analysis and decision support in the legal field, intelligent literature analysis and academic research, and auxiliary diagnosis and medical research in the medical field.

[0059] The document-level relation extraction model is applied to knowledge graph construction and integration. The relation extraction model automatically identifies the relations between different entities from a large amount of text data. The relation information is a key element for constructing a knowledge graph, connecting numerous entities and their relations to form a large knowledge network, providing a basis for subsequent knowledge reasoning and query. For the same person, there may be different expressions in different news reports or literature, but through relation extraction, it is determined that the expressions point to the same entity, and the related information is integrated into the knowledge graph.

[0060] The document-level relation extraction model is applied to the legal field, and the legal relations between the parties involved in the case, such as contract relations and tort relations, are extracted from legal documents and case texts. This helps lawyers and judges better understand the core issues and legal relations of the case, improving the efficiency and accuracy of case analysis.

[0061] The document-level relation extraction model is applied to literature analysis and academic research, and potential new knowledge and innovation points are mined by extracting various entities and relations in the literature. For example, the intersection between different research directions and potential cooperation opportunities are found, providing ideas for interdisciplinary research; problems or contradictions that have not been fully focused on are extracted from previous research, inspiring new research topics.

[0062] The document-level relation extraction model is applied to the analysis of the relations between diseases and symptoms, and the relations between diseases and symptoms in patient medical records and medical literature are extracted to help doctors more accurately diagnose diseases. For example, it is found that a certain disease is usually accompanied by certain symptoms, or certain symptoms may indicate certain diseases, providing a basis for early diagnosis and treatment of diseases.

[0063] Advantages:

[0064] 1. The document-level relation extraction method based on balanced labels disclosed in the application uses balanced relation labels to expand the relation label set {R}. During the expansion process, new balanced relation labels are integrated into the original label set through specific relation semantics. The expansion method of the application cleverly increases the correlation between relation labels, making the semantic connection between each label more closely, so that the document-level relation extraction model can more accurately grasp the meaning represented by different relation labels when processing documents, thereby better performing relation extraction work.

[0065] 2、In actual document data, the relationship label set {r} between entity pairs often has an unbalanced label quantity, which affects the effect and accuracy of model training. The document-level relationship extraction method based on balanced labels disclosed by the present application supplements the balanced relationship label annotation for the documents of the training set and the validation set. Through this supplementary annotation method, the unbalanced phenomenon can be alleviated, the data is more balanced, and the stable training and accurate prediction of the document-level relationship extraction model are helpful.

[0066] 3、The document-level relationship extraction method based on balanced labels disclosed by the present application increases a three-linear pooling module. The three-linear pooling module is used to fuse the representations of the subject entity, the object entity and the related context. For the related information of the subject entity and the object entity, the module captures the feature information of each dimension, the position characteristics in the text structure, and the potential association with other elements. And for the related context, the module also shows strong integration capability. It not only simply collects the word information of the context, but also deeply mines the semantic logic, sentiment tendency and background knowledge and other aspects of the context. Through the three-linear pooling operation, these rich representations of the subject entity, the object entity and the related context are fused together. The fusion brings significant effect, and the most prominent one is to improve the weight of the context in reasoning. In the traditional model, the context information is not fully valued in the reasoning process. However, after adding the three-linear pooling module, the status of the context information in the reasoning link is improved. It plays a more critical role in the key reasoning steps such as relationship reasoning and semantic understanding of the model, which helps the document-level relationship extraction model to output more accurate and reasonable results.

[0067] 4、The document-level relationship extraction method based on balanced labels disclosed by the present application constructs a loss function specially suitable for balanced labels to train the document-level relationship extraction model. The loss function is determined through in-depth research and repeated experiments, and the loss function can positively affect the parameter adjustment of the model during training. By using the loss function, the confidence score of the model in predicting the relationship label in the existing relationship label set {r} is significantly improved. The document-level relationship extraction model can more accurately give the relationship label when facing complex document data, and improve the accuracy of document-level relationship extraction. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 is a flowchart of the document-level relationship extraction method based on balanced labels of the present application;

[0069] Figure 2 is a loss function design diagram of the document-level relationship extraction method based on balanced labels of the present application;

[0070] Figure 3 is a system block diagram of a document-level relation extraction method based on balanced labels according to the present application. DETAILED DESCRIPTION

[0071] In order to better illustrate the purposes and advantages of the present application, the embodiments of the method of the present application will be further described in detail below in combination with the drawings and examples.

[0072] Example 1

[0073] In this example, the encyclopedia document dataset is used, and after collation, the corpus statistical information is shown in Table 1:

[0074] Table 1: Experimental data of relation extraction of encyclopedia document dataset (DocRED)

[0075]

[0076] The experiment uses F1-measure (F1) and Ign_F1 (F1-measure excluding the relationship triplets that appear in the dataset) to evaluate the results, and ten-fold cross-validation is used to determine the optimal parameters of the model according to the experimental performance.

[0077] This experiment is carried out on a server, and the specific configuration of the server is: GPU 3090 24GBx2, memory 256GB, hard disk 4TB, and the operating system is CentOS GNU / Linux 7.9.200964bit.

[0078] The specific settings of the example model are shown in the accompanying Figure 3 BERT and RoBERTa are used as base encoding models in this example.

[0079] As shown in the accompanying Figure 1 , the document-level relation extraction method based on balanced labels disclosed in this embodiment has the following specific implementation steps:

[0080] Step 1: As shown in Table 1, the encyclopedia document dataset is composed of 5053 documents, of which 3053 are training set documents, 1000 are validation set documents, and 1000 are test set documents. The encyclopedia document dataset contains 96 types of entities to be extracted, and the entity types include 6 types: person type entity, organization type entity, location type entity, time type entity, number type entity and other type entity.

[0081] The entity pair (e h ,e t ) in the document is given by the expected relationship label set of the encyclopedia document dataset {r}={r1,r2,...}. Let the head entity e h , the tail entity et The entity types of c h t The entity types of c h The entity types of c t The entity types of c h The entity types of c t The entity types of c h The entity types of c t The entity types of c h The entity types of c t The entity types of c b The entity types of c 2 The entity types of c b The entity types of c

[0082] The entity types of c h The entity types of c t The entity types of c

[0083] Step 2: For the encyclopedia document dataset, including the training set, the validation set and the test set, insert the start special token “[CLS]” and the special token “[SEP]” to separate the sentences at the beginning and end of the S d entity mentions m e,j of each entity e. Then use the corresponding tokenizer of the base encoding model to tokenize the document, and get the id list of the document word in the tokenizer word table [id1, id2,...], as shown in “document d”. Figure 3 According to the id of the “*” token, the start and end positions of the entity mention are determined, and then the mapping table “key is entity e, value is the table of the position of each entity mention before the *” is obtained.

[0084] ​A mapping table is established, the key is an entity e, and the value is the position table of the number of mentions of each entity, obtaining {e: [m e,1 ,m e,2 ,...]}. The mapping table is a preprocessing of entity representation calculation, which reduces the coupling degree of the entity representation calculation module and the word segmentation and encoding module in the document-level relation extraction model. A mapping table is established, the key is an entity pair (e h ,e t ), and for the training set and the validation set, the value of the mapping table is defined as the expected relation label set {r b} with balanced relation label annotation, obtaining {(e h ,e t ): [r1, r2,..., b]}. For the test set, the value of the mapping table is defined as the expected relation label set {r}, obtaining {(e h ,e t ): [r1, r2,...]}. The mapping table is a preprocessing of entity pair representation calculation, document-level relation extraction model training loss function calculation, and model evaluation in the test set, which accelerates the calculation of the entity pair representation calculation module and the loss function module in the document-level relation extraction model.

[0085] Step 3: Take the token id list [id1, id2,...] of the document obtained in step 2 as input, and encode according to formula (1) using the pre-trained language model:

[0086] H, A = PLM ([id1, id2,...]) (1)

[0087] Where PLM represents the first letter abbreviation of the pre-trained language encoding model selected for building the document-level relation extraction model, H represents the matrix composed of the vector representation of each token of the token id list of the document, and has As shown in the "token vector representation H" in Figure 3 . t is the first letter abbreviation of token, N t is the number of tokens of the document, and h is the dimension of the vector representation. is the matrix composed of the attention vector of each token output by the last transformer layer of the pre-trained language model PLM, and the attention vector of each token is obtained by average pooling the attention vectors of different attention heads. Corresponding Figure 3 "Attention Matrix A" in

[0088] The mapping table obtained in step 2, "the key is an entity e, and the value is the position table of the number of mentions of each entity", is used to establish a mapping table {e: [m e,1 ,m e,2,...]}, retrieve the location table of entity mentions for entity e [m e,1 ,m e,2 [,...]. Extract the entity mention location table [m] from the document term vector representation matrix H. e,1 ,m e,2 The vector representations of the lexical units at each position are then pooled to fuse the vector representations of entity mentions at different positions in the document, resulting in the head entity e. h Combined tail entity e t The vector representation of H is denoted as H. h and H t The vector representation H of the head and tail entities. h and H t After the location information fusion module of the document-level relation extraction model, the entity vector representation Z with location information is obtained. h and Z t ,like Figure 3 The “head entity representation” and “tail entity representation” are shown in the figure.

[0089] Head entity e h Combined with the entity e t vector representation of H h and H t The calculation is performed using LogSumExp pooling technology. Let the set of "entity mentions" for entity e be... Where ||{m e Let || be the number of times entity e is "mentioned" in document d. Then, the expression for obtaining the vector representation of entity e is:

[0090]

[0091] in h is the dimension of the word vectors output by the pre-trained language encoding model, where The vector representation of the j-th entity mention of entity e is thus obtained, resulting in the head entity e. h Combined tail entity e t vector representation of H h and H t .

[0092] Obtain the entity vector representation Z with location information h and Z t The calculation expression for the location information fusion module is:

[0093] Z e =tanh(WH e +b) (3)

[0094] in Z eLet H be the entity vector representation with location information, where h is the dimension of the word vector representation output by the pre-trained language encoding model, and matrix W and vector b are the mathematical representations of the location information fusion module, which are the learnable parameters of the location information fusion module. The location information fusion module makes the vector representation H of entity e obtained by formula (2)... e This is transformed into an entity vector representation Z with location information. e ,like Figure 3 The “head entity representation” and “tail entity representation” are shown in the figure.

[0095] Step 4: To enable the document-level relation extraction model to extract entity pairs (e h ,e t In addition to using relational tags for inference, context vectors that can corroborate the inference are also needed. Using matrix A, composed of attention vectors for each word obtained in step 3, we obtain the attention vectors Ahead and Atail for the head and tail entities respectively with respect to the document's context words. h and A t The attention vector A of the head and tail entities. h and A t First, perform the Hadamard product, then perform vector normalization to obtain the entity attention vector A with respect to document context terms. (h,t) Then A (h,t) Performing the Hadamard product of the vector representation matrix H of the document's terms yields the vector representation C of the context of common interest to the head and tail entities. (h,t) ,like Figure 3 As shown in the “Context Representation” section.

[0096] The matrix A, composed of the attention vectors of each word, yields the attention vector A of entity e with respect to the document context. e The calculation expression is:

[0097]

[0098] in h is the dimension of the word vectors output by the pre-trained language encoding model, where The attention vector of the j-th entity mention of entity e for the document context lexical units is obtained, thus yielding the head entity e. h Combined tail entity e t The attention vector A for the document context h and A t .

[0099] Entity attention vector A about document context terms (h,t) The calculation expression is:

[0100]

[0101] in The operator for Hadamard product, Norm represents vector normalization.

[0102] The vector representation C of the context that the head and tail entities are both interested in. (h,t) The calculation expression is:

[0103] C (h,t) =H T ·A (h,t) (6)

[0104] in • is the inner product operator.

[0105] By using a grouped trilinear pooling module, the head entity vector representation Z is fused. h Tail entity vector representation Z t and the contextual information C associated with the entity pair (h,t) This yields the vector representation H of the entity pairs. (h,t) The vector representation H of entity pairs (h,t) The confidence score for each relation label is obtained through the classification module of the document-level relation extraction model.

[0106] The entity vector representation Z with location information obtained according to formula (3) h and Z t The vector representation C of the context of common interest to the head and tail entities obtained by formula (6) (h,t) These three vectors Z h Z t and C (h,t) Before entering the trilinear pooling module, a dimension transformation module is required, and the operation formula is:

[0107] v * =v.view(g,h / g).unsqueeze((d1,d2)) (7)

[0108] in Representing Z h Z t and C (h,t) For vectors, the "view" and "unsqueeze" methods respectively transform the vector shape into a matrix and expand the matrix dimensions to support matrix multiplication using linear pooling. `g` is used to reduce the grouping hyperparameter of the linear pooling module parameters. `h` is the dimension of the word vector output by the pre-trained language encoding model. `d1` and `d2` are the two positions where the dimensions need to be expanded. Z... h Expanding the two dimensions (2,3) yields Z t Expanding the two dimensions (1,3) yields C (h,t)Extending (1, 2) two dimensions

[0109] The vector representation H of entity pairs (h,t) The calculation is First, do Hadamard product, and then adjust the matrix to a one-dimensional vector:

[0110]

[0111] The vector representation H of entity pairs g is a grouping hyperparameter to reduce the parameters of the linear pooling module, and h is the dimension of the token vector output by the pre-trained language encoding model.

[0112] The number of classification heads of the classification module of the document-level relation extraction model corresponds to the number of relation labels in the relation label set {R b} with balanced labels. The classification module of the model obtains the relation label r * The calculation expression of the confidence score is:

[0113] P(r * |e h ,e t )=W r H (h,t) +b r (9)

[0114] Where P is the confidence score of the neural network model classifying the relation label r * , is a learnable parameter that reduces the dimension of the vector representation H of entity pairs, (h,t) and W is a learnable parameter of the classification module.

[0115] Step 5: Take the document d in the training set as the input of the document-level relation extraction model described in steps 3 and 4 above, and calculate the loss function value of the model training after obtaining the confidence score of the relation label inferred by the model. Train the document-level relation extraction model on the training set for multiple rounds until the training loss no longer decreases, adjust the training parameters on the validation set, save the document-level relation extraction model parameters with the highest accuracy on the test set, and use them for actual document-level relation extraction.

[0116] For the calculation of the loss of the label in the expected relation label set {r b}, the Ranking Loss loss function algorithm is used, and for the calculation of the loss of the label not in the expected relation label set {R b -r b}, the pooling Ranking Loss algorithm is used to calculate the training loss, as Figure 2As shown. A shortcut to avoid the model learning and reduce training loss: by suppressing entity pairs (e... h ,e t The non-existent relation label set {R} b -r b The confidence scores of the relation labels in the} are used to reduce the training loss. This makes the entity pairs (e)... h ,e t The set of relation labels {r} exists b The confidence scores of the relationship labels and the threshold labels in the relationship label set are not significantly different.

[0117] Entity pairs under document semantics (e h ,e t The set of relation labels {r} exists b The relation label r in {r1, r2, ..., b} i Let the model predict the relationship label r. i The confidence score is denoted as And for entity pairs (e) h ,e t The non-existent relation label set {r -}={R b -r b Relationship tags in} The confidence score on is denoted as The confidence score of the threshold relation label that separates positive and negative relations is denoted as f0.

[0118] The overall calculation expression for the training loss function of the document-level relation extraction model is as follows:

[0119]

[0120] The document-level relation extraction model is trained multiple times on the training set documents until the training loss no longer decreases. The training parameters are then adjusted on the validation set. The parameters of the document-level relation extraction model with the highest accuracy on the test set are saved for actual document-level relation extraction.

[0121] Step 6: Apply the document-level relation extraction model obtained in Step 5 to the encyclopedia document dataset for relation extraction.

[0122] This embodiment demonstrates the performance on an encyclopedia document dataset. Using a document-level relation extraction method based on balanced labels, the present invention is shown in Table 2 below, which compares the performance of the present invention with other methods on this dataset. It can be seen that the present invention is superior to other methods, and can extract relation triples more accurately, achieving better knowledge graph construction and integration.

[0123] Table 2 Comparison of the effects of different methods

[0124]

[0125] In summary, the embodiment discloses a document-level relation extraction method based on balanced labels, which is applied to a knowledge document dataset, and has the following beneficial effects:

[0126] The balanced relation labels are used to expand the relation label set. In the expansion process, new balanced relation labels are integrated into the original label set through specific relation semantics. The expansion method of the present application skillfully increases the correlation between relation labels, making the semantic connection between labels more closely, so that the document-level relation extraction model can more accurately grasp the meaning represented by different relation labels when processing documents, thereby better performing relation extraction work.

[0127] The supplementary labeling of balanced relation labels is performed on the documents of the training set and the validation set. Through this supplementary labeling method, the imbalance phenomenon can be alleviated, the data is more balanced, and the stable training and accurate prediction of the document-level relation extraction model are facilitated.

[0128] The added tri-linear pooling module increases the weight of context in reasoning. In traditional models, context information is not fully valued in the reasoning process. However, after adding this tri-linear pooling module, the status of context information in the reasoning link is improved. It plays a more critical role in key reasoning steps such as relation reasoning and semantic understanding, which helps the document-level relation extraction model to output more accurate and reasonable results.

[0129] A loss function specially suitable for balanced labels is constructed to train the document-level relation extraction model. The loss function is determined through in-depth research and repeated experiments, and the loss function can positively affect the parameter adjustment of the model during training. By using the loss function, the confidence score of the model in predicting the existing relation label set {r} is significantly improved. The document-level relation extraction model can more accurately give the relation label when facing complex document data, and the accuracy of document-level relation extraction is improved.

[0130] Finally, the knowledge graph construction and integration achieves good results, and the extracted relation information is a key element for constructing a knowledge graph. Connecting numerous entities and their relationships forms a vast knowledge network, providing a foundation for subsequent knowledge reasoning, querying, and other tasks.

[0131] The above detailed description of the specific description, the purpose, technical scheme and beneficial effects of the application are further described in detail, it should be understood that the above description is only a specific embodiment of the application, and is not used to limit the protection scope of the application, any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A document-level relation extraction method based on balanced labels, characterized in that: Comprising the following steps, Step 1: Collect the document data set {D} in the A field and the relationship label set {R} between entities in the A field before actually putting the document-level relation extraction into use in the A field; divide the total document data set {D} into a training set, a validation set and a test set; the training set is used for training the document-level relation extraction model, the validation set is used for quickly verifying the parameters of the document-level relation extraction model, and the test set is used for measuring and comparing the relation extraction effect of the finally put into use document-level relation extraction model; perform named entity recognition on each document d in the total document data set {D} to obtain the position annotation and type information of the entity in the document, and the set of all types of entities is denoted as {C}; Step 2: For the document dataset collected in step 1, including the training set, the validation set and the test set, insert the special token "[CLS]" at the beginning and the special token "[SEP]" at the end of each sentence of the document d; highlight each entity mention m of the entity e with a pair of asterisks "*" d e,j , and then use the corresponding tokenizer of the base encoding model to tokenize the document to obtain the id list [id1, id2,...] of the token of the document in the tokenizer token table; determine the start and end positions of the entity mention according to the id of the "*" token, and then obtain the mapping table "key is entity e, value is the position table of each entity mention before the "*" token";​ Step 3: Use the word id list [id1, id2,...] obtained in step 2 as input, and encode according to formula (1) using a pre-trained language model: H, A = PLM([id1, id2,...]) (1) wherein PLM represents the first letter abbreviation of a pre-trained language model (Pretrained Language Model) selected for building a document-level relation extraction model, H represents a matrix composed of vector representations of each token id list of the document, and t is the first letter abbreviation of a token, N t is the number of tokens of the document, and h is the dimension of the vector representation. is a matrix composed of attention vectors of each token output by the last transformer layer of the pre-trained language model PLM, and the attention vector of each token is obtained by average pooling of attention vectors of different attention heads. The mapping table {e: Set[m]} of "key is entity e, value is the position table of each entity mention" obtained by step 2 is taken as H [ m e,1 , e,2 ... The position table of entity mention of entity e [m e,1 , e,2 ... ] is taken out; the vector representation of the token at each position in the position table of entity mention [m e,1 , e,2 ... ] in the vector representation matrix H of the document token is taken out, and then pooling is performed to fuse the vector representations of the entity mentions at different positions of the document to obtain the vector representations of the head entity e h and the tail entity e t , denoted as H h and H t ; the vector representations H h and H t of the head and tail entities are subjected to the position information fusion module of the document-level relation extraction model to obtain the entity vector representations Z h and Z t with position information; Head entity e h and tail entity e t vector representation H h and H t is computed by LogSumExp pooling technique, given the set of "entity mentions" of entity e in document d where ||{m e}|| is the number of "entity mentions" of entity e in document d, then the expression for the vector representation of entity e is: wherein h is the dimension of the wordpiece vector output by the pre-trained language encoding model, is the vector representation of the jth entity mention of entity e, such that the head entity e h and tail entity e t have vector representations H h and H t ; Obtaining an entity vector representation Z with location information h and Z t The computational expression of the location information fusion module is: Z e = tanh(WH e +b) (3) wherein Z e is the entity vector representation with position information, h is the dimension of the token vector representation output by the pre-trained language encoding model, the matrix W and the vector b are the mathematical representations of the position information fusion module, and are the learnable parameters of the position information fusion module; the position information fusion module makes the vector representation H of the entity e obtained by formula (2) e be converted into the entity vector representation Z with position information e . Step 4: In order to let the document-level relation extraction model infer the relation label of the entity pair (e h , e t ), the context vector representation that can support the inference is also needed; using the matrix A composed of the attention vector of each word element obtained in step 3, the attention vectors of the head and tail entities for the document context word elements A h and A t are obtained; the attention vectors A h and A t of the head and tail entities are first subjected to Hadamard product, and then subjected to vector standardization operation to obtain the attention vectors A (h,t) of the entity pair for the document context word elements; then the Hadamard product of A (h,t) and the vector representation matrix H of the document word elements is performed to obtain the vector representation C (h,t) of the context jointly focused by the head and tail entities. Step 5: Take the document d in the training set as the input of the document-level relation extraction model described in steps 3 and 4, calculate the loss function value of the model training after obtaining the confidence score of the relation label inferred by the model; train the document-level relation extraction model on the training set for multiple rounds until the training loss no longer decreases, adjust the training parameters on the validation set, save the document-level relation extraction model parameters with the highest accuracy on the test set, and use them for actual document-level relation extraction; Step 6: Apply the trained document-level relation extraction model obtained in step 5 to the field of relation extraction in natural language processing to perform document-level relation extraction and obtain the document-level relation extraction result.

2. The method of claim 1, wherein the method is a balanced label-based document-level relation extraction method. The implementation method of step 1 is, Document d is composed of S d sentences, and the set of entities contained in document d is where ||{e d}|| is the number of entities in document d; entity e has an entity type c ∈ {C}, where {C} is the set of entity types; entity e can appear multiple times in different positions in document d, and the set of "entity mentions" of entity e in document d is where ||{m e}|| is the number of "entity mentions" of entity e in document d; e h is called the head entity because it is in the subject position in the relational semantics, and e t is called the tail entity because it is in the object position in the relational semantics. (e h ,e t ), the head and tail entity pair (e h ,e t ) in the document is annotated with a relation label set {R} that is concerned by the A field; for any relation label r * ∈{R}, if (e h ,e t ), the label r * can form a relation semantic triple (e h ,r * ,e t ), then the relation label r * is added to the expected relation label set {r} of model reasoning; if (e h ,e t ) does not exist any relation label r * in the relation label set {R} under the document semantics, then the expected relation label set {r} of model reasoning should only have the "NA" relation label; the entity pair (e h ,e t ) of the document d after model reasoning, the confidence score of the "NA" relation label is used to determine whether the relation label r * exists between (e h ,e t ), the relation label r * with a confidence score greater than the "NA" relation label is determined as the relation label that exists between (e h ,e t ), so the "NA" relation label is called the threshold relation label; An entity pair (e h , e t ) in a document has an expected relation label set {r} = {r1, r2,...} after preliminary relation label annotation; let the entity types of the head entity e h and the tail entity e t be c h type and c t type respectively, then the balance relation label b of the entity pair (e h , e t ) can be supplemented based on the positional relation of c h type and c t type, c h type and c t type, the semantic of b label is the relation label when c h type and c t type appear in the head and tail positions respectively, and finally the expected relation label set with balance relation label annotation used for training is {r b} = {r1, r2,..., b}, b and part of the relation labels in the original relation label set {R} will have a correlation of co-occurrence, which is conducive to the model to understand and distinguish the semantics of the relation labels in the relation label set {R}; other balance relation labels introduced are also defined by the entity types, the positions of the types appearing in the head and tail; the number of types in the entity type set {C} is ||{C}||, the entities of the ||{C}|| entity types can appear in the head or tail position of the entity pair, and the total number of balance relation labels added to the relation label set {R} is ||{C}|| 2 , obtaining the relation label set {R b} with balance label. The balance relationship label existing in the supplementary annotation training set and the validation set document is labeled, so that the expected relationship label set {r} of each entity pair at least contains the balance relationship label, and the imbalance of the number of relationship labels in the expected relationship label set {r} between the entity pairs (e h ,e t ) is reduced, and the correlation between the relationship labels in the document data set is improved.

3. The method of claim 2, wherein the method is a balanced tag-based document-level relation extraction method. The implementation method of step 2 is, A mapping table is established, the key is entity e, and the value is the position table of each entity mention number, obtaining {e: [m e,1 ,m e,2 ,...]};The mapping table is a pre-processing of entity representation calculation, which reduces the coupling degree of entity representation calculation module and segmentation coding module in document-level relation extraction model;A mapping table is established, the key is entity pair (e h ,e t ), for the training set, the value of the mapping table is defined as the expected relation label set {r b} with balanced relation label annotation, obtaining {(e h ,e t ): [r1, r2,..., b]},For the test set, the value of the mapping table is defined as the expected relation label set {r}, obtaining {(e h ,e t ): [r1, r2,...]};The mapping table is a pre-processing of entity pair representation calculation, document-level relation extraction model training loss function calculation, and model evaluation in the test set, which accelerates the calculation of entity pair representation calculation module and loss function module in document-level relation extraction model.

4. The method of claim 3, wherein: The implementation method of step 4 is, A matrix A of attention vectors for each wordpiece is first composed, obtaining an attention vector A for the entity e with respect to the context of the document e The computational expression is: wherein h is the dimension of the wordpiece vector output by the pre-trained language encoding model, is the attention vector for the jth entity mention of entity e to the document context wordpieces, resulting in a head entity e h and tail entity e t the attention vector A for the document context h and A t ; The entity pair attention vector A on the document context wordpiece (h,t) The computational expression is: A (h,t) = Norm(A h o A t ) (5) wherein ° is the operator of Hadamard product, Norm represents the vector normalization operation; Vector representation C of the context of mutual interest of the head-tail entities (h,t) The computational expression is: C (h,t) = H T • A (h,t) (6) wherein • is an inner product operator; By using a grouped trilinear pooling module, the head entity vector representation Z is fused. h Tail entity vector representation Z t and the contextual information C associated with the entity pair (h,t) This yields the vector representation H of the entity pairs. (h,t) ; Vector representation of entity pairs H (h,t) The confidence score of each relation label is obtained through the classification module of the document-level relation extraction model; The entity vector with position information Z obtained according to formula (3) h and Z t , and the vector representation C of the context of the head-tail entity common concern obtained according to formula (6) (h,t) The three vectors Z h , Z t and C (h,t) need to pass through a dimension conversion module before entering the three-linear pooling module, and the operation formula is: v * = v. view ( g, h / g ). unsqueeze ( ( d1, d2 ) ) (7) wherein represents Z h , Z t and C (h,t) Vector, "view" method and "unsqueeze" method respectively operate vector shape to become matrix and expand matrix dimension to support matrix multiplication of linear pooling, g is a grouping hyper parameter to reduce parameters of linear pooling module, h is the dimension of word piece vector output by pre-training language encoding model, d1, d2 are two positions that need to be expanded, Z h Expand (2, 3) two dimensions to get Z t Expand (1, 3) two dimensions to get C (h,t) Expand (1, 2) two dimensions to get Vector representation of entity pairs H (h,t) The computation is First do the Hadamard product, then adjust the matrix to a one-dimensional vector: where the vector representation of an entity pair g is a hyperparameter for grouping the parameters of the linear pooling module, and h is the dimension of the token vectors output by the pre-trained language encoding model. The number of classification heads of the classification module of the document-level relation extraction model corresponds to the number of relation labels in the relation label set {R} with balanced labels b The classification module of the model obtains the relation label r * The calculation expression of the confidence score is: P(r * |e h ,e t ) = W r H (h,t) + b r (9) where P is a neural network model classification relationship label r * confidence score, is a learnable parameter that reduces the dimensionality of the vector representation H (h,t) of the entity pair, is a learnable parameter of the classification module.

5. The method of claim 4, wherein: The implementation method of step 5 is, For the calculation of the label inference loss in the expected relation label set {r b}, the Ranking Loss loss function algorithm is adopted. For the calculation of the label inference loss in the label set {R b -r b} not in the expected relation label set, the training loss is calculated by using the pooling Ranking Loss algorithm to avoid the model learning to reduce the training loss by suppressing the confidence score of the relation label in the relation label set {R h ,e t} not existing for the entity pair (e b ,e b} ; so that the confidence score of the relation label in the relation label set {r h} existing for the entity pair (e t ,e b} is not significantly different from the confidence score of the threshold label. Entity pairs under document semantics (e h ,e t The set of relation labels {r} exists b The relation label r in {r1, r2, ..., b} i Let the model predict the relationship label r. i The confidence score is denoted as And for entity pairs (e) h ,e t The non-existent relation label set {r - }={R b -r b Relationship tags in} The confidence score on is denoted as The confidence score of the threshold relation label that separates positive and negative relations is denoted as f0; The calculation expression of the total document-level relation extraction model training loss function is as follows: The training loss function described above aims to expand the relation set {r} b Confidence score of relation labels in} The difference between the confidence score f0 of the threshold relationship label and the entity pair {e} is widened. h ,e t The non-existent relation label set {r - Relationship tags in} Confidence score The gap; Train the document-level relation extraction model on the training set for multiple rounds until the training loss no longer decreases, adjust the training parameters on the validation set, save the document-level relation extraction model parameters with the highest accuracy on the test set, and use them for actual document-level relation extraction.

6. The method of claim 1, 2, 3, 4, or 5, wherein: Further comprising step 7: solving the corresponding field related technical problems according to the document-level relation extraction result obtained in step 6; The related technical problems include knowledge graph construction and integration, case analysis and auxiliary decision-making in the legal field, intelligent literature analysis and academic research, auxiliary diagnosis and medical research in the medical field; Apply the document-level relation extraction model to knowledge graph construction and integration, and the relation extraction model automatically identifies the relationship between different entities from a large amount of text data; The relationship information is a key element for constructing a knowledge graph, connecting entities and their relationships to form a knowledge network, providing a foundation for subsequent knowledge reasoning and query; for the same person, there may be different expressions in different news reports or literature, but through relation extraction it is determined that the expressions point to the same entity, and the related information is integrated into the knowledge graph; The document-level relationship extraction model is applied to the legal field to extract the legal relationship between the parties involved in a case from legal documents and case texts, improving the efficiency and accuracy of case analysis. The document-level relationship extraction model is applied to literature analysis and academic research, extracting various entities and relationships in the literature to uncover potential new knowledge and innovation points. The document-level relationship extraction model is applied to the analysis of the relationship between diseases and symptoms, extracting the relationship between diseases and symptoms in patient medical records and medical literature to help doctors more accurately diagnose diseases.

Images