Relationship sensing type two-channel entity relation extraction method

Abstract

The invention discloses a relation perception type two-channel entity relation extraction method. The method comprises the following steps of S1, encoding sentences through an encoder; s2, inputting the sentence vectors into a potential relation extractor, and extracting potential relation features; s3, inputting the potential relation characteristics into a dual-channel extractor, and extracting from two channels of head entity priority and tail entity priority respectively; and S4, judging the association among the subject, the relationship and the object through the bisimulation network, aggregating the entity pairs extracted by the two channels, and performing relationship classification on the entity pairs. The problems that relation recognition lacks entity semantic support and error prediction is irreversible in a relation priority type joint extraction method can be solved.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, specifically to a relation-aware dual-channel entity relation extraction method. Background Technology

[0002] Entity-relation triple extraction is a fundamental task in the field of information extraction, aiming to identify head entities, tail entities, and their relationships from natural language text. Traditional methods employ a pipeline architecture, first identifying entities and then classifying relationships. This step-by-step strategy inevitably suffers from error accumulation. To address this, a joint extraction method has been proposed, which achieves joint extraction of entities and relationships by constructing an end-to-end framework to avoid propagating errors.

[0003] The core of joint extraction methods lies in modeling the association between entities and relations, and realizing the information interaction between the two through a unified framework. For example, by using a binary labeling strategy, relation extraction is transformed into a mapping problem of entity span, laying the technical foundation for entity and relation modeling; subsequently, semantic relations and knowledge relations are introduced to guide the model to capture the interaction between general knowledge and domain knowledge. However, the aforementioned methods still cannot avoid the problem of entity omission, as well as the problem of relation extraction when the same pair of entities has multiple relations.

[0004] Existing relation-priority joint extraction methods typically identify relations first and then extract entity pairs based on those relations. This framework has two key problems. First, in the relation extraction stage, the model needs to identify relations in the absence of entity semantics, which limits the accuracy of relation identification. Second, if the relation prediction is wrong, the entity pairs will also be missed. This error is irreversible, making it impossible to reconstruct the correct triples. Summary of the Invention

[0005] This invention aims to solve the technical problems existing in the prior art, and innovatively proposes a relation-aware dual-channel entity relation extraction method, which can solve the problems of lack of entity semantic support and irreversible error prediction in relation recognition in relation-priority joint extraction methods.

[0006] To achieve the above objectives, the present invention provides a relation-aware dual-channel entity relation extraction method, comprising the following steps:

[0007] S1: Encode the sentence using an encoder;

[0008] S2: Input the sentence vector into the latent relation extractor to extract latent relation features;

[0009] S3: Input the potential relationship features into the dual-channel extractor and extract them from the head entity priority and tail entity priority channels respectively;

[0010] S4: Use a dual-path network to determine the association between subject, relation, and object, aggregate entity pairs extracted from the dual-path network, and classify the entity pairs according to their relationships.

[0011] In the above scheme, step S1 includes:

[0012] S1-1: Input sentence The BERT pre-trained language model is used to process the input sentence. Encode;

[0013] For the input sentence and the initial sequence representation of each token. ∈ as follows:

[0014] ;

[0015] ;

[0016] in, yes A token, It is a sentence vector, it is a sentence The vector representation of each token obtained through the BERT model, where d is the vector dimension.

[0017] Existing extraction methods, such as TPLinker and CasRel, typically use a uniform initial sequence representation for head entity, tail entity, and relation extraction. We believe that different sequence representations should be used for head entity, tail entity, and relation extraction respectively to better adapt to the needs of different extraction tasks. Therefore, we perform three linear transformations on the initial sequence representation to obtain its enhanced representation.

[0018] S1-2: Representation of the initial sequence Perform weighted processing;

[0019] Define a subject linear transformation layer, an object linear transformation layer, and a relational linear transformation layer to represent the initial sequence. The input is fed into three linear transformation layers, where three different linear transformations are performed respectively;

[0020] Transformed vector representation , , as follows:

[0021] ;

[0022] ;

[0023] ;

[0024] in, Subject vector, For object vectors, For relation vectors, The trainable weight matrix of the subject linear transformation layer. Let be the trainable weight matrix of the object linear transformation layer. The weight matrix for the linear transformation layer of the relation is trainable. , , ∈ d is the vector dimension; The bias vector of the subject linear transformation layer The bias vector of the object linear transformation layer. The deviation vector of the relational linear transformation layer. , , ∈ .

[0025] In the above scheme, step S2 includes:

[0026] Relation vector The input is fed into the relation feature extraction module, which then extracts latent relation features. Used to guide entity extraction;

[0027] The process is as follows:

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] in, It is the first The output of the layer decoder from the attention layer, ; , is the decoder's self-attention mask; It is the first The output of the layer decoder cross-attention layer; Let be the embedding representation of the t-th word. ; This represents the hidden state of the initial layer. This is the hidden state of the last decoder layer. For the first The hidden state of the layer decoder; For sentence vectors, It is a relation vector; As a multi-head attention mechanism, Represents a feedforward neural network, where d represents the sequence length and d represents the dimension of the hidden layer.

[0034] In the above scheme, step S3 includes:

[0035] S3-1: Extract entity pairs from the subject extraction channel;

[0036] S3-2: Extract entity pairs from the object extraction channel.

[0037] In the above scheme: Step S3-1 includes:

[0038] S3-1-1: Input subject vector The subject extraction module extracts latent relation features. Under the guidance of the extraction head entity and its context representation ;

[0039] The operation is as follows:

[0040] ;

[0041] ;

[0042] in, Subject vector, and For a trainable weight matrix, This is a vector concatenation operation; This is a vector concatenated from the head entity features and relation features. In the latent relation vector The contextual representation of the extracted head entity under guidance; It is a bidirectional long short-term memory network. It is a feedforward neural network;

[0043] S3-1-2: Predict the start and end tags of entities in the input sentence;

[0044] For each entity in the input sentence, its start and end markers need to be predicted. The prediction process is as follows:

[0045] ;

[0046] ;

[0047] ;

[0048] ;

[0049] in, As the starting marker, The projection matrix of the initial marker, This is the end marker. The projection matrix of the end marker, where The trainable direction vector is the initial label. The trainable direction vector for the end marker. Represents the hyperbolic tangent function. This represents the sigmoid function. The deviation vector of the initial mark, The deviation vector for the end marker. For the first The probability of using a token as the starting marker for the header entity. For the first The probability of a token being used as the end marker of a header entity;

[0050] S3-1-3: Predicting tail entities using the S2O extraction module , obtain entity pairs ( );

[0051] S3-1-4: Calculate the loss function of the head entity extraction module;

[0052] The loss function is calculated as follows:

[0053] ;

[0054] ;

[0055] in, Let `pred` be the cross-entropy loss function for binary classification, and `pred` be the predicted label, where `pred` ∈ (0,1). The target is the true label, where target∈{0,1}, 1 represents the beginning or end of the predicted entity, and 0 represents the non-entity part. N represents the number of words in the sentence. This loss function is suitable for binary classification tasks, determining the start and end points of entities, and adjusting the model parameters based on the error between the predicted and true values. For the head entity extraction module loss function, This indicates that the i-th entity in the tail entity extraction is in label m. Predicted probabilities on (start, end) This indicates that the i-th entity in the tail entity extraction is in label m. The true probability at (start, end).

[0056] In the above scheme, step S3-1-3 includes:

[0057] S3-1-3-1: Predict the starting position marker of the tail entity;

[0058] The prediction process is as follows:

[0059] ;

[0060] ;

[0061] ;

[0062] ;

[0063] in, , represents the context representation under the header entity condition of the t-th token; , representing the potential relational enhancement vector of the t-th token (and (same dimensions)

[0064] Indicates to Perform dimension-wise pooling to represent the vector. This represents the fusion vector at position t. Label the starting position of the tail entity with a trainable bias vector. Unnormalized boundary score used to measure the matching strength of the t-th token as the starting position of an entity;

[0065] To perform element-wise multiplication, the tail entity vector and results Multiply, The trainable weight matrix for the initial label of the tail entity. , For max pooling operation, For the first The probability that a token will be used as the end marker of the tail entity. It is the sigmoid function;

[0066] S3-1-3-2: Predict the end position marker of the tail entity;

[0067] The prediction process is as follows:

[0068] ;

[0069] ;

[0070] , is the trainable weight matrix for the end marker of the tail entity. For the first The probability that a token will be used as the end marker of the tail entity. For the sigmoid function, A trainable bias vector is assigned to the end position of the tail entity.

[0071] In the above scheme, step S3-2 includes:

[0072] S3-2-1: Input object vector Through the object extraction module, latent relation features are obtained. Under the guidance, extract the tail entity. and its context representation ;

[0073] The operation is as follows:

[0074] ;

[0075] ;

[0076] in, For object vectors, and For a trainable weight matrix, This is a vector concatenation operation; This is a vector concatenated from tail entity features and relation features. In the latent relation vector The contextual representation of the extracted head entity under guidance; It is a bidirectional long short-term memory network. It is a feedforward neural network;

[0077] S3-2-2: Predict the start and end tags of entities in the input sentence;

[0078] For each entity in the input sentence, its start and end markers need to be predicted. The prediction process is as follows:

[0079] ;

[0080] ;

[0081] ;

[0082] ;

[0083] in, As the starting marker, The projection matrix of the initial marker, This is the end marker. The projection matrix of the end marker, where The trainable direction vector is the initial label. The trainable direction vector for the end marker. This represents the hyperbolic tangent function, used for nonlinear mapping of the features of a sequence after offline transformation. This represents the sigmoid function. The deviation vector of the initial mark, The deviation vector for the end marker. For the first The probability of using a token as the starting marker for the tail entity. For the first The probability of a token being the end marker of a tail entity; since marker prediction is a binary classification task, we set its threshold to 0.5, where a value above 0.5 is predicted as 1 and a value below 0.5 is predicted as 0.

[0084] S3-2-3: Predicting Head Entities via the O2S Extraction Module , to obtain entity pairs ( , );

[0085] S3-2-4: Calculate the loss function of the head entity extraction module;

[0086] The loss function is calculated as follows:

[0087] ;

[0088] ;

[0089] in, Let `pred` be the cross-entropy loss function for binary classification, and `pred` be the predicted label, where `pred` ∈ (0,1). The target is the true label, where target∈{0,1}, 1 represents the beginning or end of the predicted entity, and 0 represents the non-entity part. N represents the number of words in the sentence. This loss function is suitable for binary classification tasks, determining the start and end points of entities, and adjusting the model parameters based on the error between the predicted and true values.

[0090] For the head entity extraction module loss function, This indicates that the i-th entity in the tail entity extraction is in label m. Predicted probabilities on (start, end) This indicates that the i-th entity in the tail entity extraction is in label m. The true probability at (start, end).

[0091] In the above scheme, step S3-2-3 includes:

[0092] S3-2-3-1: Predict the starting position marker of the head entity;

[0093] The prediction process is as follows:

[0094] ;

[0095] ;

[0096] ;

[0097] ;

[0098] in, , represents the context representation of the tail entity condition of the t-th token; , representing the potential relational enhancement vector of the t-th token (and (same dimensions)

[0099] Indicates to Perform dimension-wise pooling to represent the vector. This represents the fusion vector at position t. The trainable bias vector serves as the starting position marker for the head entity. Unnormalized boundary score used to measure the matching strength of the t-th token as the start / end position of an entity;

[0100] To perform element-wise multiplication, the head entity vector and results Multiply; The trainable weight matrix is ​​the initial label of the head entity. , For max pooling operation, For the first The probability of a token being used as the end marker of a header entity;

[0101] S3-2-3-2: Predict the end position marker of the head entity;

[0102] The prediction process is as follows:

[0103] ;

[0104] ;

[0105] , is the trainable weight matrix for the end marker of the head entity. For the first The probability of using a token as the end marker of a header entity. A trainable bias vector labeled with the end position of the head entity. For the head entity vector, For the sigmoid function, Unnormalized boundary score used to measure the matching strength of the t-th token as the end position of an entity.

[0106] In the above scheme, step S4 includes:

[0107] The processing procedure is as follows:

[0108] ;

[0109] ;

[0110] ;

[0111] ;

[0112] in, This represents the sequence of word vectors within the span of the k-th entity. Indicates the first A sequence of word vectors within an entity span This indicates max pooling. and Let these represent the aggregated head and tail entity representation vectors, respectively. This represents the transpose of the aggregated head entity representation vector. Representing relations The bias vector, Representing relations The deviation vector, Representing relations The weight matrix, For entity pairs ( ) belongs to the relationship The original score, For the sigmoid function, This indicates a feature concatenation operation. This is the final predicted probability;

[0113] Calculate the total loss;

[0114] The total loss is calculated as follows:

[0115] ;

[0116] ;

[0117] ;

[0118] ;

[0119] in, It represents the total number of relations; For the head entity extraction module loss function, For the head entity extraction module loss function, This represents the loss incurred when extracting the tail entity based on the head entity. This represents the loss incurred when extracting head entities based on tail entities. This represents the loss of the final relation module. This indicates the total loss. This indicates that the i-th entity in the extraction of the head entity based on the tail entity is located at label m. Predicted probabilities on (start, end) This indicates that the i-th entity in the extraction of the head entity based on the tail entity is located at label m. The true probability at (start, end). This indicates that in the extraction of the i-th entity from the head entity to the tail entity, the i-th entity is located at label m. Predicted probabilities on (start, end) This indicates that in the extraction of the i-th entity from the head entity to the tail entity, the i-th entity is located at label m. The true probability at (start, end) This indicates that a relationship exists in the prediction. The probability, Indicates the existence of a relationship The true probability, This is the cross-entropy loss function for binary classification.

[0120] In summary, the beneficial effects of this invention are as follows: It introduces a latent relation guidance mechanism, which first captures latent relation features as semantic signals in the sentence-level context to guide the entity pair extraction process. These latent relation features are not final relation labels, but rather a perceptual representation of relation semantics. They retain the advantage of relation-first extraction in filtering irrelevant entities while providing prior semantic support for entity entities before relation prediction, thus effectively reducing semantic ambiguity and extraction bias in the relation identification stage. Based on this, this paper designs a dual-channel joint extraction mechanism, constructing two complementary paths (R2S and R2O) from relation to entity, centered on the latent relation. The R2S module is responsible for forward extraction from the latent relation to the head entity and then to the tail entity, while the R2O module infers the head entity from the tail entity. This bidirectional extraction framework achieves semantic complementarity and correction: on the one hand, it can capture implicit dependencies between entities from different semantic perspectives, improving the robustness of entity pair representation; on the other hand, the reverse channel can effectively retrieve entity pairs missed in a single path, alleviating the problem of missing triples caused by relation prediction errors. Experimental results show that this mechanism significantly improves the completeness and accuracy of extraction in complex sentence structures and one-to-many relationship scenarios (such as SEO-type data). Furthermore, this paper combines BiLSTM and feedforward neural networks in feature modeling to achieve deep fusion of latent relationship features and entity features. Overall, the proposed latent relationship-aware dual-channel framework achieves a balance between latent relationship feature guidance and entity reinforcement in entity relationship extraction tasks, effectively alleviating the problems of missing semantic dependencies and irreversible prediction, and providing a new interpretable approach for joint extraction tasks. Attached Figure Description

[0121] Figure 1 This is a structural diagram of the present invention.

[0122] Figure 2 It is a similarity comparison chart for texts containing only one triplet.

[0123] Figure 3 It is a similarity comparison chart containing three triples of text. Detailed Implementation

[0124] The present invention will be further described below with reference to the embodiments and accompanying drawings:

[0125] like Figures 1-3 As shown, a relationship-aware dual-channel entity relationship extraction method includes the following steps:

[0126] S1: Encode the sentence using an encoder;

[0127] S1-1: Input sentence The BERT pre-trained language model is used to process the input sentence. Encode;

[0128] For the input sentence and the initial sequence representation of each token. ∈ as follows:

[0129] ;

[0130] ;

[0131] in, yes A token, It is a sentence vector, it is a sentence The vector representation of each token obtained through the BERT model, where d is the vector dimension.

[0132] Existing extraction methods, such as TPLinker and CasRel, typically use a uniform initial sequence representation for head entity, tail entity, and relation extraction. We believe that different sequence representations should be used for head entity, tail entity, and relation extraction respectively to better adapt to the needs of different extraction tasks. Therefore, we perform three linear transformations on the initial sequence representation to obtain its enhanced representation.

[0133] S1-2: Representation of the initial sequence Perform weighted processing;

[0134] S1-2-1: Define the subject linear transformation layer, object linear transformation layer, and relation linear transformation layer to represent the initial sequence. The input is fed into three linear transformation layers, where three different linear transformations are performed respectively;

[0135] Transformed vector representation , , as follows:

[0136] ;

[0137] ;

[0138] ;

[0139] in, Subject vector, For object vectors, For relation vectors, The trainable weight matrix of the subject linear transformation layer. Let be the trainable weight matrix of the object linear transformation layer. The weight matrix for the linear transformation layer of the relation is trainable. , , ∈ d is the vector dimension; The bias vector of the subject linear transformation layer The bias vector of the object linear transformation layer. The deviation vector of the relational linear transformation layer. , , ∈ ;

[0140] we will , , These are used for head entity, tail entity, and latent relation extraction, respectively, to enhance the relevance of sequence encoding and extraction tasks.

[0141] S2: Input the sentence vector into the latent relation extractor to extract latent relation features;

[0142] To avoid entity recall failures caused by relation extraction errors in traditional relation-first methods, we propose a latent relation extraction method. First, latent relation features are extracted from sentences, and then entities are identified based on these features.

[0143] Since a sentence often contains multiple relations, relation extraction is a typical multi-classification task. Therefore, we introduce a Transformer-based non-autoregressive decoder to predict latent relations. The non-autoregressive decoder has significant parallel processing capabilities, effectively reducing error accumulation caused by previous sequence prediction errors while improving inference speed.

[0144] In addition, we introduce the BERT encoder and cross-attention mechanism to enable the model to fully capture rich contextual semantics.

[0145] Specifically, we will use the sentence vector ht and the relation vector This serves as the input to the decoder. The decoder consists of stacked self-attention layers and cross-attention layers, ultimately generating a latent relation feature vector. .

[0146] S2-1: Extract latent relation features;

[0147] Relation vector The input is fed into the relation feature extraction module, which then extracts latent relation features. Used to guide entity extraction;

[0148] To avoid entity recall failures caused by relation extraction errors in traditional relation-first methods, we propose a latent relation extraction method. First, latent relation features are extracted from sentences, and then entities are identified based on these features.

[0149] Since a sentence often contains multiple relations, relation extraction is a typical multi-classification task. Therefore, we introduce a Transformer-based non-autoregressive decoder to predict latent relations. The non-autoregressive decoder has significant parallel processing capabilities, effectively reducing error accumulation caused by previous sequence prediction errors while improving inference speed.

[0150] In addition, we introduce the BERT encoder and cross-attention mechanism to enable the model to fully capture rich contextual semantics.

[0151] Specifically, we will use sentence vectors and relation vector This serves as the input to the decoder. The decoder consists of stacked self-attention layers and cross-attention layers, ultimately generating a latent relation feature vector. For the decoder's first... The prediction process for each layer is as follows:

[0152] ;

[0153] ;

[0154] ;

[0155] ;

[0156] ;

[0157] in, It is the first The output of the layer decoder from the attention layer, ; , is the decoder's self-attention mask; It is the first The output of the layer decoder cross-attention layer; , where is the sentence vector, and is the contextual feature output by the encoder BERT. It is a relation vector; This represents the hidden state of the initial layer. This is the hidden state of the last decoder layer. For the first The hidden state of the layer decoder; It is the self-attention mask of the decoder. We represent the embedding of the t-th word (i.e., The vector after the embedding layer is defined as follows: And use it as a mask vector in the cross-attention calculation. As a multi-head attention mechanism, Represents a feedforward neural network, where d represents the sequence length and d represents the dimension of the hidden layer.

[0158] S3: Input the potential relationship features into the dual-channel extractor and extract them from the head entity priority and tail entity priority channels respectively;

[0159] S3-1: Extract entity pairs from the subject extraction channel;

[0160] S3-1-1: Input subject vector The subject extraction module extracts latent relation features. Under the guidance of the extraction head entity and its context representation ;

[0161] The subject extraction module extracts all head entities from the input sentence, guided by latent relation features. To fully integrate latent relation features and contextual information, we propose a multi-layer information fusion technique. First, we introduce a bidirectional LSTM (BiLSTM) to leverage its strong sequence modeling capabilities and effectively capture dependencies between contextual elements, ensuring the accuracy and robustness of entity extraction. To further refine the feature representation, we feed the output features of the BiLSTM into a feedforward neural network to further mine the deep semantics of the context.

[0162] The operation is as follows:

[0163] ;

[0164] ;

[0165] in, Subject vector, and For a trainable weight matrix, This is a vector concatenation operation; This is a vector concatenated from the head entity features and relation features. In the latent relation vector The contextual representation of the extracted head entity under guidance; It is a bidirectional long short-term memory network. It is a feedforward neural network;

[0166] For entity extraction, we adopted a CasRel-based binary labeling method. Each token is labeled with either 0 or 1, where 0 indicates that the token does not belong to an entity, and 1 indicates that the token is the start or end point of an entity. In this way, entity location identification is regarded as a binary classification task, and the model solves this task by training a binary classifier.

[0167] S3-1-2: Predict the start and end tags of entities in the input sentence;

[0168] For each entity in the input sentence, its start and end markers need to be predicted. The prediction process is as follows:

[0169] ;

[0170] ;

[0171] ;

[0172] ;

[0173] in, As the starting marker, The projection matrix of the initial marker, This is the end marker. The projection matrix of the end marker, where The trainable direction vector is the initial label. The trainable direction vector for the end marker. This represents the hyperbolic tangent function, used for nonlinear mapping of the features of a sequence after offline transformation. This represents the sigmoid function. The deviation vector of the initial mark, The deviation vector for the end marker. For the first The probability of a token being used as the starting marker for the header entity, and For the first The probability of a token being used as the end marker of a head entity; since marker prediction is a binary classification task, we set its threshold to 0.5, where a marker greater than 0.5 is predicted as 1 and a marker less than 0.5 is predicted as 0.

[0174] S3-1-3: Predicting tail entities using the S2O extraction module , obtain entity pairs ( );

[0175] The S2O extraction module extracts objects from subjects. It identifies tail entities based on the contextual representation of head entities. The S2O extraction module does not use a fusion approach of head entity and latent relation features because we consider head entities and tail entities to belong to different learning tasks, and therefore the feature fusion methods should also differ.

[0176] Since the fused features of the head entity and relations are obtained, which contain valuable information about the entities and relations, the tail entity extraction module fuses the three features, namely ( , , Perform tail entity extraction.

[0177] S3-1-3-1: Predict the starting position marker of the tail entity;

[0178] The prediction process is as follows:

[0179] ;

[0180] ;

[0181] ;

[0182] ;

[0183] in, , represents the context representation under the header entity condition of the t-th token; , representing the potential relational enhancement vector of the t-th token (and (same dimensions)

[0184] Indicates to Perform dimension-wise pooling to represent the vector. The fusion vector at position t (added element by element). Label the starting position of the tail entity with a trainable bias vector. Unnormalized boundary score used to measure the matching strength of the k-th token as the start / end position of an entity.

[0185] To perform element-wise multiplication, the tail entity vector and results Multiply, The trainable weight matrix for the initial label of the tail entity. , For max pooling operation, For the first The probability that a token will be used as the end marker of the tail entity. This is the sigmoid function.

[0186] S3-1-3-2: Predict the end position marker of the tail entity;

[0187] The prediction process is as follows:

[0188] ;

[0189] ;

[0190] , is the trainable weight matrix for the end marker of the tail entity. For the first The probability that a token will be used as the end marker of the tail entity. For the sigmoid function, A trainable bias vector is assigned to the end position of the tail entity.

[0191] S3-1-4: Calculate the loss function of the head entity extraction module;

[0192] Since the extraction tasks of the two channels are different, it belongs to multi-task learning, so each module has a corresponding loss function for each direction. We denote the loss function of the head entity extraction module as... The loss function is calculated as follows:

[0193] ;

[0194] ;

[0195] in, Let `pred` be the cross-entropy loss function for binary classification, and `pred` be the predicted label, where `pred` ∈ (0,1). The target is the true label, ∈ {0,1}, and N represents the number of words in the sentence. 1 represents the beginning or end of the predicted entity, and 0 represents non-entity parts. This loss function is suitable for binary classification tasks, determining the start and end points of entities, and adjusting the model parameters based on the error between the predicted and true values.

[0196] For the head entity extraction module loss function, This indicates that the i-th entity in the tail entity extraction is in label m. Predicted probabilities on (start, end) This indicates that the i-th entity in the tail entity extraction is in label m. The true probability at (start, end);

[0197] S3-2: Extract entity pairs from the object extraction channel;

[0198] S3-2-1: Input object vector Through the object extraction module, latent relation features are obtained. Under the guidance, extract the tail entity. and its context representation ;

[0199] The operation is as follows:

[0200] ;

[0201] ;

[0202] in, For object vectors, and For a trainable weight matrix, This is a vector concatenation operation; This is a vector concatenated from tail entity features and relation features. In the latent relation vector The contextual representation of the extracted head entity under guidance; It is a bidirectional long short-term memory network. It is a feedforward neural network;

[0203] S3-2-2: Predict the start and end tags of entities in the input sentence;

[0204] For each entity in the input sentence, its start and end markers need to be predicted. The prediction process is as follows:

[0205] ;

[0206] ;

[0207] ;

[0208] ;

[0209] in, As the starting marker, The projection matrix of the initial marker, This is the end marker. The projection matrix of the end marker, where The trainable direction vector is the initial label. The trainable direction vector for the end marker. This represents the hyperbolic tangent function, used for nonlinear mapping of the features of a sequence after offline transformation. This represents the sigmoid function. The deviation vector of the initial mark, The deviation vector for the end marker. For the first The probability of a token being used as the starting token of the tail entity, and For the first The probability of a token being the end marker of a tail entity; since marker prediction is a binary classification task, we set its threshold to 0.5, where a value above 0.5 is predicted as 1 and a value below 0.5 is predicted as 0.

[0210] S3-2-3: Predicting Head Entities via the O2S Extraction Module , to obtain entity pairs ( , The O2S extraction module is a module for extracting the subject from the object.

[0211] S3-2-3-1: Predict the starting position marker of the head entity;

[0212] The prediction process is as follows:

[0213] ;

[0214] ;

[0215]

[0216] ;

[0217] in, , represents the context representation of the tail entity condition of the t-th token; , representing the potential relational enhancement vector of the t-th token (and (same dimensions)

[0218] Indicates to Perform dimension-wise pooling to represent the vector. The fusion vector at position t (added element by element). The trainable bias vector serves as the starting position marker for the head entity. Unnormalized boundary score used to measure the matching strength of the t-th token as the start / end position of an entity;

[0219] To perform element-wise multiplication, the head entity vector and results Multiply; The trainable weight matrix is ​​the initial label of the head entity. , For max pooling operation, For the first The probability of a token being used as the end marker of a header entity;

[0220] S3-2-3-2: Predict the end position marker of the head entity;

[0221] The prediction process is as follows:

[0222] ;

[0223] ;

[0224] , is the trainable weight matrix for the end marker of the head entity. For the first The probability of using a token as the end marker of a header entity. A trainable bias vector labeled with the end position of the head entity. For the head entity vector, For the sigmoid function, Unnormalized boundary score used to measure the matching strength of the t-th token as the end position of an entity.

[0225] S3-2-4: Calculate the loss function for the head entity extraction module; since the extraction tasks of the two channels are different, it belongs to multi-task learning, so each module has a corresponding loss function in each direction. We denote the loss function of the tail entity extraction module as... The loss function is calculated as follows:

[0226] ;

[0227] ;

[0228] in, Let `pred` be the cross-entropy loss function for binary classification, and `pred` be the predicted label, where `pred` ∈ (0,1). The target is the true label, where target∈{0,1}, 1 represents the beginning or end of the predicted entity, and 0 represents the non-entity part. N represents the number of words in the sentence. This loss function is suitable for binary classification tasks, determining the start and end points of entities, and adjusting the model parameters based on the error between the predicted and true values.

[0229] The loss function for the tail entity extraction module. This indicates that the i-th entity in the tail entity extraction is in label m. Predicted probabilities on (start, end) This indicates that the i-th entity in the tail entity extraction is in label m. The true probability at (start, end).

[0230] Traditional relation-first extraction methods typically predict relations within sentences first, then use these relations to guide entity extraction. The problem with this approach is the irreversibility of the extraction process; once relations are determined, the recall scope for subsequent entities is limited, potentially leading to missed entity pairs. Furthermore, directly extracting relations without sufficient entity information can result in more false positives. To address this, we propose a dual-channel entity extraction strategy: first, we extract latent relation features, and then we extract entity pairs from both channels separately.

[0231] The two channels operate independently, each identifying a set of entity pairs. This strategy effectively alleviates the recall difficulties caused by the asymmetry of head and tail entities in the grammatical context of the text.

[0232] S4: Use a dual-path network to determine the association between subject, relation, and object, aggregate entity pairs extracted from the dual-path network, and classify the entity pairs according to their relationships.

[0233] Because we use a dual-channel approach, obtaining entity pairs from the S2O and O2S directions respectively, this may result in more noisy entity pairs compared to a single-channel approach, thus requiring a more efficient relation extractor. Since the final relation extraction is guided by entity pairs, we cannot directly use a non-autoregressive decoder for latent relation feature extraction. Here, we introduce a biaffine network to determine the relations between entity pairs. First, Biaffine maintains a relation matrix for each relation, thus accurately modeling the features of that relation. Second, its probabilistic computation mechanism effectively mines the interaction information between head and tail entities. The process is as follows:

[0234] ;

[0235] ;

[0236] ;

[0237] ;

[0238] in, This represents the sequence of word vectors within the span of the k-th entity. Indicates the first A sequence of word vectors within an entity span This indicates max pooling. and Let these represent the aggregated head and tail entity representation vectors, respectively. This represents the transpose of the aggregated head entity representation vector. Representing relations The bias vector, Representing relations The deviation vector, Representing relations The weight matrix, For entity pairs ( ) belongs to the relationship The original score, For the sigmoid function, This indicates a feature concatenation operation. This is the final predicted probability;

[0239] Calculate the total loss;

[0240] To train the relation extraction module, we used cross-entropy to calculate the loss, just like with the entity extraction module. (Total number of relations) Apart from and In addition, the other modules also have , , The total loss is calculated as follows:

[0241] ;

[0242] ;

[0243] ;

[0244] ;

[0245] in, It represents the total number of relations; For the head entity extraction module loss function, For the head entity extraction module loss function, This represents the loss incurred when extracting the tail entity based on the head entity. This represents the loss incurred when extracting head entities based on tail entities. This represents the loss of the final relation module. This indicates the total loss. This indicates that the i-th entity in the extraction of the head entity based on the tail entity is located at label m. Predicted probabilities on (start, end) This indicates that the i-th entity in the extraction of the head entity based on the tail entity is located at label m. The true probability at (start, end). This indicates that in the extraction of the i-th entity from the head entity to the tail entity, the i-th entity is located at label m. Predicted probabilities on (start, end) This indicates that in the extraction of the i-th entity from the head entity to the tail entity, the i-th entity is located at label m. The true probability at (start, end) This indicates that a relationship exists in the prediction. The probability, Indicates the existence of a relationship The true probability, This is the cross-entropy loss function for binary classification.

[0246] To better illustrate our technical solution, we conducted experimental evaluations on several public datasets, including WebNLG and the NYT dataset. The WebNLG dataset was initially used for natural language generation challenges, and its excellent performance has led to its application in various fields, particularly triple extraction. The NYT dataset consists of news articles from The New York Times, containing text-annotated entities and relations, and is one of the standard evaluation datasets for many relation extraction algorithms. Specific information about the datasets is shown in Table 1.

[0247] Table 1. Statistics of the NYT and WebNLG datasets

[0248]

[0249] There are other versions of the NYT and WebNLG datasets, NYT* and WebNLG*. NYT and WebNLG annotate all tokens of an entity, while NYT* and WebNLG* only annotate the last token of each entity.

[0250] To validate various complex scenarios, the scenarios in the dataset were further subdivided into two categories: EPO (Entity Pair Overlap) and SEO (Single Entity Overlap). EPO indicates multiple relationships between the same pair of entities. For example, in the sentence "Antwerp International Airport which has an elevation of 12.0 metres abovesea level is owned and operated by the Flemish government," both "Antwerp International Airport" and "Flemish government" have relationships of "is owned" and "operated." SEO indicates that the same entity has relationships with multiple different entities. For example, in the sentence "Bhajji originates from India, a country of Indians where the leader is TS Thakur," "India" has both an "originates" relationship with "Bhajji" and a "leader" relationship with "TS Thakur."

[0251] We use precision, recall, and F1 score as evaluation metrics. In triple extraction tasks, different matching criteria are typically used for different dataset types, including partial matching and exact matching. For the WebNLG and NYT datasets, exact matching is usually used, meaning that both entities and relations must be correctly labeled for a prediction to be considered correct. For NYT* and WebNLG*, partial matching is usually used, meaning that if the header label is correct when predicting entities and relations, the prediction is considered correct.

[0252] ;

[0253] ;

[0254] ;

[0255] In this model, TP (True Positive) represents the number of samples correctly predicted as positive, FP (False Positive) represents the number of samples incorrectly predicted as positive, precision reflects the proportion of truly positive samples among those predicted as positive, and FN (False Negative) represents the number of positive samples incorrectly predicted as negative. Recall represents the proportion of correctly predicted positive data out of all correctly predicted positive data. The F1 score reflects the overall performance of the model.

[0256] We use the BERT-base-case model as the embedding layer. The BERT-base-case semantic relation embedding has a dimension of 768, and the model is trained using the Adamw optimizer. We set the threshold for judging head and tail entities and relations to 0.5, the batch size to 24 on NYT and NYT*, and the batch size to 6 on WebNLG and WebNLG*. The encoder learning rate and other learning rates are set to 2e-5 / (2e-5)*5. The number of learnable embeddings in the relation non-autoregressive decoder is 12. The dropout rate is set to 0.2 to prevent overfitting.

[0257] Since our proposed method is a relation-first, label-based extraction model, we selected excellent models from similar studies in recent years for comparison. We also compared generative and tabular methods to verify the effectiveness of our proposed method. Therefore, the comparison models are divided into three categories: the first category is label-based extraction models, including CasRel, PRGC, TDEER, PRAB, BRASK, and ERFD-RTE; the second category is generative seq2seq models, including CGT; and the third category is tabular extraction models, including TPLinker. Baseline results are obtained directly from the original paper.

[0258] ERFD-RTE: Employs a relation-first extraction strategy to mitigate triple overlap and avoid computation of additional redundant relations. The model first identifies potential relations, then performs entity identification based on each relation.

[0259] BRASK: Combines semantic relations and knowledge relations for joint extraction, integrating general knowledge and domain knowledge into the model to improve extraction performance.

[0260] TDEER: Models relations as translation operations from head entity to tail entity to handle overlapping triples.

[0261] PRGC: It narrows down the scope of entity recognition by using latent relationships and solves entity overlap by using sequence labeling, while generating triples with a low-complexity global alignment method.

[0262] CasRel: Treats relations as a mapping function from head entity to tail entity, and constructs a cascaded binary annotation framework to achieve joint extraction.

[0263] TPLinker: Models the relation extraction task as a token pair linking problem and uses a handshake marking scheme to align entity boundaries, constructing a single-stage joint extraction model.

[0264] CGT: Transforms the relation extraction task into a generation task and introduces a generative architecture based on adversarial learning to improve model robustness.

[0265] SMARTe: It proposes a slot attention mechanism and constructs the extraction task as a set prediction problem.

[0266] Table 2 shows the performance of our model on different datasets. As can be seen from Table 2, our model achieves the best F1 score on the WebNLG, NYT*, and WebNLG* datasets. On the NYT* and WebNLG* datasets, our model improves the F1 score by 0.3% and 0.9% respectively compared to the state-of-the-art label-based relation-first extraction model ERFD-RTE. We believe this is because our model ensures good recall through dual-channel extraction and maintains stable accuracy through latent relation-guided entity extraction, thus achieving a better F1 score. For the NYT and WebNLG datasets, the evaluation criteria are relatively strict; a prediction is considered correct only when all tokens of an entity and relation completely match. Under these evaluation conditions, our model achieves scores of 92.6 and 90.5 on the two datasets, respectively. Our F1 score on the WebNLG dataset is state-of-the-art (SOTA) compared to the comparison model, and our F1 score on the NYT dataset is also close to SOTA, with precision exceeding BiRTEBERT. This demonstrates that our method can achieve robust extraction performance by balancing recall and precision. Our F1 score on the WebNLG dataset outperforms that of the NYT dataset. We attribute this to the fact that the NYT training set primarily consists of standard triples, while the WebNLG training set contains more complex cases, such as SEO (which accounts for 63%). This suggests that our proposed dual-channel extraction framework is more adaptable to scenarios where a single entity maps to multiple entities. Overall, our model performs better on datasets requiring the identification of complete entity-relation tokens, demonstrating the effectiveness of prioritizing relation feature extraction and employing a unique feature fusion method within the bidirectional framework.

[0267] Table 2. Experimental results on the NYT and WebNLG datasets

[0268]

[0269] To further validate the model's performance, we conducted experiments in complex scenarios, and the results are shown in Table 3. Based on the different overlap patterns of entities and relations, we categorized triplet types into three types: Normal, SEO (Single EntityOverlap), and EPO (Entity-Pair Overlap). Table 3 illustrates the model's performance on these three data types. The results show that our model outperforms other models on subsets of the NYT* and WebNLG* datasets. On the Normal and SEO subsets of WebNLG*, we outperform PRGC by 2.7% and 0.3%, respectively; on the SEO and EPO subsets of NYT*, we outperform TPLinker by 1.3% and 1.0%, respectively. Furthermore, when the number of triples in a sentence is greater than 2, our model consistently demonstrates superior performance compared to other models.

[0270] We believe this performance improvement primarily stems from the semantic complementarity of the bidirectional extraction framework. Traditional tokenized models reason in only one direction during extraction, such as from head entity to tail entity (S2O). This makes it difficult for the model to fully capture semantic relationships that are more readily apparent in the reverse direction (O2S) in certain semantic scenarios. In contrast, our dual-channel architecture runs both S2O and O2S paths simultaneously, starting from either the head or tail entity, modeling the connections between entities from different semantic perspectives, thus creating complementarity in information capture. This not only enhances adaptability to diverse semantic patterns but also naturally brings additional entity pair recall opportunities. That is, if one path misses an entity, the other path may still fill in the gap, which is particularly advantageous for extraction in scenarios with overlapping triples.

[0271] In addition, our proposed strategy of guiding entity extraction with latent relationships deeply integrates relationship features with entity features. This design effectively ensures the extraction accuracy of the model in complex extraction tasks.

[0272] Table 3 shows the F1 scores for different forms of overlap and different numbers of triples. Results are from data in the original paper; T represents the number of triples in the sentence.

[0273]

[0274] To further observe the specific impact of our proposed method on the results, we conducted ablation experiments on the WebNLG and NYT datasets, as shown in Table 4.

[0275] Table 4 shows the ablation results on the NYT and WebNLG test sets. w / o indicates that a specific module was eliminated. In our experiment, we replaced the BiLSTM module with multi-head attention.

[0276]

[0277] In the Multi-head experiment in Table 4, we replaced BiLSTM with a multi-head attention network. In the w / o Sub / Objextract experiment, we removed the Subject Extractor and Object Extractor modules and directly weighted and fused hs and rv. In the w / o Rel-Feature-exctract experiment, we removed the RelationFeature Extractor module and performed relation prediction on hr and the final entity pair.

[0278] We replaced BiLSTM with a multi-head attention network to verify the effectiveness of our proposed latent relation feature and entity fusion method. As seen, the F1 score decreased by 0.8% on the WebNLG dataset and by 0.4% on the NYT dataset. Analysis suggests this is because BiLSTM, through bidirectional sequence traversal, effectively captures local information within sentences. While multi-head attention mechanisms can flexibly model global feature associations, their positional encoding relies on external design, such as relative or absolute positional encoding, and therefore cannot directly capture the sequential relationships of consecutive words in a sequence like BiLSTM. This also demonstrates that our proposed latent relation feature and entity fusion technique is more sensitive to sentence structure, enhances the interaction between entities and relations, and thus improves entity extraction performance.

[0279] Removing the Subject / Object Extractor component resulted in a 0.4% decrease in performance on the WebNLG dataset and a 0.1% decrease on the NYT dataset. This indicates that the component not only captures contextual information during entity extraction but also deeply fuses relational and entity features through a layer-by-layer processing mechanism involving BiLSTM, linear layers, and feedforward networks, thereby more accurately modeling the semantic relationships between them. Without this fusion process, the model's ability to perceive long-distance dependencies and fine-grained semantic features is weakened, leading to an overall performance decline.

[0280] After removing the latent relation feature extraction component, the model's performance decreased by 0.1% on the NYT dataset and by a more significant 1.0% on the WebNLG dataset, demonstrating the effectiveness of the latent relation feature extraction strategy. In contrast, the performance drop on the WebNLG dataset was more pronounced, which we attribute to the more complex grammatical scenarios in the WebNLG dataset, where the latent relation feature extraction strategy becomes more effective.

[0281] To observe the adaptability of our method to different tasks, we further analyzed the model's performance in three sub-tasks: entity and relation triple extraction. We tested on the WebNLG* and NYT* datasets, and the F1 scores are shown in Table 5. The results show that on the NYT* dataset, our model outperforms ERFD-RTE by 0.1% and 0.3% in the relation (r) and entity (h,t) extraction sub-tasks, respectively. On the WebNLG* dataset, our F1 score exceeds that of 1.0% in the relation (r) extraction sub-task and exceeds that of 0.4% in the entity (h,t) extraction sub-task. We also found that the model's F1 score is higher in relation triple recognition than in entity recognition, indicating that our model is more suitable for tasks with high relation extraction requirements. On the WebNLG* dataset, the difference between our model and other models is more significant, further demonstrating that our model is more suitable for handling datasets with many predefined relations and more complex scenarios. Overall, the model outperforms other models in handling complex scenarios like WebNLG. However, our model still faces some challenges when dealing with large datasets like NYT, especially in accurately identifying entity pairs.

[0282] Table 5. F1 scores of distinct elements in a triplet

[0283]

[0284] This improvement is primarily due to the non-autoregressive decoder and bidirectional extraction strategy, which can accurately extract relationships even in complex scenarios. By prioritizing the extraction of latent relationship features, then using these features to guide entity extraction, and finally using entity pairs to guide the prediction of the final relationship, relationships are supported by entity information, thereby improving overall extraction accuracy. The dual-channel extraction method captures entity information bidirectionally, enabling the acquisition of entity information that cannot be obtained from the S2O direction in the O2S direction, while also picking up missed entity pairs, thus improving the accuracy of entity pair extraction.

[0285] In relation extraction tasks, traditional "direct relation prediction" methods typically make relation determinations before obtaining entity information. This approach has a significant problem: relation prediction lacks the support of entity features, making the model prone to semantic bias when facing complex or ambiguous contexts. For example, when multiple candidate entity pairs exist in a sentence, without the constraint of entity features, the model may make incorrect relation judgments based on local context, introducing noisy triples. To address this issue, we introduce the concept of latent relation features into the model. Unlike direct relation prediction, latent relation features are not the final relation classification result, but rather a semantic context representation of the relation extracted from the sentence. This representation retains semantic information related to the target relation but does not make a hard decision at this stage. This design offers two advantages: firstly, the semantics of a sentence are dominated by relations; prioritizing the extraction of latent relations allows for a quicker grasp of the sentence's core, ensuring that the subsequently obtained entities are closely integrated with the current relational context. This not only filters out entity pairs irrelevant to the target relation but also improves the accuracy of entity prediction. Secondly, entity extraction is guided by latent relation features, and the extracted entity pairs are then used for final relation determination. Relation prediction can simultaneously utilize sentence-level relational context and entity semantic features. This dual-support mechanism of "semantic guidance + entity reinforcement" helps reduce prediction errors caused by semantic ambiguity. We compared the similarity between latent relations and real relations, between latent relations and entity pairs, and between real relations and entity pairs, such as... Figure 2 , 3 As shown.

[0286] The experiment was conducted on the WebNLG dataset, divided into two groups. In group 1, 20 texts were randomly selected from WebNLG, each containing one triplet; in group 2, 20 texts were randomly selected from WebNLG, each containing three triples. The similarities SP and SR between latent relation features and entity pairs, and between real relations and entity pairs, were calculated respectively. The experimental method is as follows: to broaden the similarity range for better observation, we applied an exponential mapping to the cosine similarity.

[0287] ;

[0288] ;

[0289] Where P is the latent relation feature, E1 and E2 are the embedding representations of real entities, R is the embedding representation of the real relation between E1 and E2, and COSIM is the cosine similarity. For group 2, for each text containing multiple triples, we take the average of SP and SR.

[0290] In our similarity experiments, we found significant differences in relation representation across different text types: for sentences containing only a single triplet, the similarity between latent relation features and entity pairs was, to some extent, greater than the similarity between true relations and entity pairs; however, in complex sentences containing multiple triples, the similarity between true relations and entity pairs was significantly higher. This result reflects the fundamental difference in semantic expression mechanisms between latent relation features and explicit relation labels.

[0291] Specifically, latent relation features are dynamically generated by the model based on contextual semantics, directly capturing the local semantic patterns of entities. Therefore, in semantically simple texts, their representation is closer to the context space of the entity pair, exhibiting high geometric similarity. However, in complex sentences containing multiple sets of relations, the semantics of latent relations are often dispersed across multiple relational directions, and the semantic center tends to diverge. In this case, explicit relation labels, as global semantic anchors for relation categories in the training set, have stronger semantic focusing capabilities, more accurately corresponding to relevant entity pairs in multi-relational contexts, thus exhibiting higher similarity. Therefore, this study interprets the above phenomenon as follows: in simple contexts, latent relation features have a stronger fit to entity semantics; while in complex contexts, the semantic anchoring effect of true relation labels is more significant.

Claims

1. A relation-aware dual-channel entity relation extraction method, characterized in that: Includes the following steps: S1: Encode the sentence using an encoder; S2: Input the sentence vector into the latent relation extractor to extract latent relation features; S3: Input the potential relationship features into the dual-channel extractor and extract them from the head entity priority and tail entity priority channels respectively; S4: Use a dual-path network to determine the association between subject, relation, and object, aggregate entity pairs extracted from the dual-path network, and classify the entity pairs according to their relationships.

2. The relation-aware dual-channel entity relation extraction method according to claim 1, characterized in that: Step S1 includes: S1-1: Input sentence The BERT pre-trained language model is used to process the input sentence. Encode; For the input sentence and the initial sequence representation of each token. ∈ as follows: ； ； in, yes A token, It is a sentence vector, it is a sentence The vector representation of each token obtained through the BERT model, where d is the vector dimension; S1-2: Representation of the initial sequence Perform weighted processing; Define a subject linear transformation layer, an object linear transformation layer, and a relational linear transformation layer to represent the initial sequence. The input is fed into three linear transformation layers, where three different linear transformations are performed respectively; Transformed vector representation , , as follows: ； ； ； in, Subject vector, For object vectors, For relation vectors, The trainable weight matrix of the subject linear transformation layer. Let be the trainable weight matrix of the object linear transformation layer. The weight matrix for the linear transformation layer of the relation is trainable. , , ∈ d is the vector dimension; The bias vector of the subject linear transformation layer The bias vector of the object linear transformation layer. The deviation vector of the relational linear transformation layer. , , ∈ .

3. The relation-aware dual-channel entity relation extraction method according to claim 2, characterized in that: Step S2 includes: Relation vector The input is fed into the relation feature extraction module, which then extracts latent relation features. Used to guide entity extraction; The process is as follows: ； ； ； ； ； in, It is the first The output of the layer decoder from the attention layer; , is the decoder's self-attention mask; It is the first The output of the layer decoder cross-attention layer; Let be the embedding representation of the t-th word. ; This represents the hidden state of the initial layer. This is the hidden state of the last decoder layer. For the first The hidden state of the layer decoder; For sentence vectors, It is a relation vector; As a multi-head attention mechanism, Represents a feedforward neural network, where d represents the sequence length and d represents the dimension of the hidden layer.

4. The relation-aware dual-channel entity relation extraction method according to claim 1, characterized in that: Step S3 includes: S3-1: Extract entity pairs from the subject extraction channel; S3-2: Extract entity pairs from the object extraction channel.

5. The relation-aware dual-channel entity relation extraction method according to claim 4, characterized in that: Step S3-1 includes: S3-1-1: Input subject vector The subject extraction module extracts latent relation features. Under the guidance of the extraction head entity and its context representation ; The operation is as follows: ； ； in, Subject vector, and For a trainable weight matrix, This is a vector concatenation operation; This is a vector concatenated from the head entity features and relation features. In the latent relation vector The contextual representation of the extracted head entity under guidance; It is a bidirectional long short-term memory network. It is a feedforward neural network; S3-1-2: Predict the start and end tags of entities in the input sentence; For each entity in the input sentence, its start and end markers need to be predicted. The prediction process is as follows: ； ； ； ； in, As the starting marker, The projection matrix of the initial marker, This is the end marker. The projection matrix of the end marker, where The trainable direction vector is the initial label. The trainable direction vector for the end marker. Represents the hyperbolic tangent function. This represents the sigmoid function. The deviation vector of the initial mark, The deviation vector for the end marker. For the first The probability of using a token as the starting marker for the header entity. For the first The probability of a token being used as the end marker of a header entity; S3-1-3: Predicting tail entities using the S2O extraction module , obtain entity pairs ( ); S3-1-4: Calculate the loss function of the head entity extraction module; The loss function is calculated as follows: ； ； in, Let `pred` be the cross-entropy loss function for binary classification, and `pred` be the predicted label, where `pred` ∈ (0,1). The target is the true label, where target∈{0,1}, 1 represents the beginning or end of the predicted entity, and 0 represents the non-entity part; N represents the number of words in the sentence; For the head entity extraction module loss function, This indicates that the i-th entity in the tail entity extraction is in label m. Predicted probabilities on (start, end) This indicates that the i-th entity in the tail entity extraction is in label m. The true probability at (start, end).

6. The relation-aware dual-channel entity relation extraction method according to claim 5, characterized in that: Step S3-1-3 includes: S3-1-3-1: Predict the starting position marker of the tail entity; The prediction process is as follows: ； ； ； ； in, This represents the context representation of the t-th token under the condition of the header entity. This represents the potential relational enhancement vector for the t-th token; Indicates to Perform dimension-wise pooling to represent the vector. This represents the fusion vector at position t. Label the starting position of the tail entity with a trainable bias vector. Unnormalized boundary score used to measure the matching strength of the t-th token as the starting position of an entity; To perform element-wise multiplication, the tail entity vector and results Multiply, The trainable weight matrix for the initial label of the tail entity. , For max pooling operation, For the first The probability that a token will be used as the end marker of the tail entity. It is the sigmoid function; S3-1-3-2: Predict the end position marker of the tail entity; The prediction process is as follows: ； ； , is the trainable weight matrix for the end marker of the tail entity. For the first The probability that a token will be used as the end marker of the tail entity. For the sigmoid function, A trainable bias vector is assigned to the end position of the tail entity.

7. The relation-aware dual-channel entity relation extraction method according to claim 4, characterized in that: Step S3-2 includes: S3-2-1: Input object vector Through the object extraction module, latent relation features are obtained. Under the guidance, extract the tail entity. and its context representation ; The operation is as follows: ； ； in, For object vectors, and For a trainable weight matrix, This is a vector concatenation operation; This is a vector concatenated from tail entity features and relation features. In the latent relation vector The contextual representation of the extracted head entity under guidance; It is a bidirectional long short-term memory network. It is a feedforward neural network; S3-2-2: Predict the start and end tags of entities in the input sentence; For each entity in the input sentence, its start and end markers need to be predicted. The prediction process is as follows: ； ； ； ； in, As the starting marker, The projection matrix of the initial marker, This is the end marker. The projection matrix of the end marker, where The trainable direction vector is the initial label. The trainable direction vector for the end marker. Represents the hyperbolic tangent function. This represents the sigmoid function. The deviation vector of the initial mark, The deviation vector for the end marker. For the first The probability of using a token as the starting marker for the tail entity. For the first The probability that a token will be used as the end marker of a tail entity; S3-2-3: Predicting Head Entities via the O2S Extraction Module , to obtain entity pairs ( , ); S3-2-4: Calculate the loss function of the head entity extraction module; The loss function is calculated as follows: ； ； in, Let `pred` be the cross-entropy loss function for binary classification, and `pred` be the predicted label, where `pred` ∈ (0,1). The target is the true label, where target∈{0,1}, 1 represents the beginning or end of the predicted entity, and 0 represents the non-entity part; N represents the number of words in the sentence; For the head entity extraction module loss function, This indicates that the i-th entity in the tail entity extraction is in label m. Predicted probabilities on (start, end) This indicates that the i-th entity in the tail entity extraction is in label m. The true probability at (start, end).

8. The relation-aware dual-channel entity relation extraction method according to claim 7, characterized in that: Step S3-2-3 includes: S3-2-3-1: Predict the starting position marker of the head entity; The prediction process is as follows: ； ； ； ； in, This represents the contextual representation of the t-th token under the condition of the tail entity. This represents the potential relational enhancement vector for the t-th token; Indicates to Perform dimension-wise pooling to represent the vector. This represents the fusion vector at position t. The trainable bias vector serves as the starting position marker for the head entity. Unnormalized boundary score used to measure the matching strength of the t-th token as the start / end position of an entity; To perform element-wise multiplication, the head entity vector and results Multiply; The trainable weight matrix is ​​the initial label of the head entity. , For max pooling operation, For the first The probability of a token being used as the end marker of a header entity; S3-2-3-2: Predict the end position marker of the head entity; The prediction process is as follows: ； ； The trainable weight matrix for the end marker of the head entity. For the first The probability of using a token as the end marker of a header entity. A trainable bias vector labeled with the end position of the head entity. For the head entity vector, For the sigmoid function, Unnormalized boundary score used to measure the matching strength of the t-th token as the end position of an entity.

9. The relation-aware dual-channel entity relation extraction method according to claim 1, characterized in that: Step S4 includes: The processing procedure is as follows: ； ； ； ； in, This represents the sequence of word vectors within the span of the k-th entity. Indicates the first A sequence of word vectors within an entity span This indicates max pooling. and Let these represent the aggregated head and tail entity representation vectors, respectively. This represents the transpose of the aggregated head entity representation vector. Representing relations The bias vector, Representing relations The deviation vector, Representing relations The weight matrix, For entity pairs ( ) belongs to the relationship The original score, For the sigmoid function, This indicates a feature concatenation operation. This is the final predicted probability; Calculate the total loss; The total loss is calculated as follows: ； ； ； ； in, It represents the total number of relations; For the head entity extraction module loss function, For the head entity extraction module loss function, This represents the loss incurred when extracting the tail entity based on the head entity. This represents the loss incurred when extracting head entities based on tail entities. This represents the loss of the final relation module. Indicates the total loss; This indicates that the i-th entity in the extraction of the head entity based on the tail entity is located at label m. Predicted probabilities on (start, end) This indicates that the i-th entity in the extraction of the head entity based on the tail entity is located at label m. The true probability at (start, end); This indicates that in the extraction of the i-th entity from the head entity to the tail entity, the i-th entity is located at label m. Predicted probabilities on (start, end) This indicates that in the extraction of the i-th entity from the head entity to the tail entity, the i-th entity is located at label m. The true probability at (start, end) This indicates that a relationship exists in the prediction. The probability, Indicates the existence of a relationship The true probability, This is the cross-entropy loss function for binary classification.

Images