A supervised relationship extraction method and device with low noise interference

Abstract

The present invention provides a supervised relationship extraction method and device with low noise interference. The method comprises: converting the corpus to be processed into a vectorized form to obtain a coding sequence; sending the coding sequence into the second layer MogrifierBiGRU in MogrifierBiGRU-GateAttention-softmax to obtain a coding sequence incorporating context information; sending the coding sequence incorporating context information into the third layer GateAttention in MogrifierBiGRU-GateAttention-softmax to remove task-irrelevant information, and aggregating and screening results to obtain a sentence vector; sending the sentence vector into softmax to obtain a relationship extraction result. The present invention improves the existing BiGRU encoder to improve its feature extraction capability, designs a gated attention layer, and proposes two gating strategies to remove task-irrelevant words in the sentence, which can reduce noise interference while modeling context information. In addition, no language tool is required for data preprocessing, which reduces the impact of error propagation.

Description

Technical Field

[0001] The present invention relates to the technical field of relationship extraction, and in particular to a supervised relationship extraction method and device with low noise interference based on MogrifierBiGRU-GateAttention-softmax. Background Art

[0002] With the development of information technology, the amount of available text data has increased rapidly. How to efficiently process these unstructured and semi-structured data, extract key information from them, and then convert them into structured data that is easy to analyze and reason about is an important research topic in knowledge extraction in natural language processing. As one of the main tasks in knowledge extraction, relation extraction (RE) aims to extract the semantic relationship between entity pairs after obtaining entity pairs, and construct entity, relation, and entity triples.

[0003] Relation extraction is an important foundation for downstream tasks such as knowledge completion, information retrieval, knowledge question and answer, and knowledge reasoning. Therefore, it is of great practical significance to analyze the shortcomings of existing relationship extraction methods and improve the effectiveness of extraction models.

[0004] Traditional supervised relationship extraction methods can be divided into three categories: rule-based, feature vector-based, and kernel function-based. Since these methods require a lot of manual effort for rule design and feature selection, rely on language tools to extract features, and have error propagation, there is little room for improvement. The relationship extraction method based on deep learning has the advantages of low labor input cost and less reliance on language tools, and has become the mainstream at present.

[0005] Most of the existing supervised relationship extraction methods based on deep learning convert it into a classification problem. According to the feature extraction model adopted, it can be roughly divided into three categories: based on convolutional neural network (CNN), based on recurrent neural network (RNN), and based on graph neural network (GNN).

[0006] CNN-based models use the distribution assumption that "the local context of a word can express its meaning" to extract local features, but due to the limitation of the receptive field, such models cannot obtain long-distance dependencies.

[0007] Unlike CNN models, RNN-based models have natural advantages in extracting time series features. In addition, by capturing contextual information to disambiguate the meaning of keywords such as entities, the impact of polysemy on relationship classification can be alleviated. However, when the sequence is long, the early information in such models will gradually decay as the sequence length increases, which can easily lead to some long-distance dependency losses.

[0008] Different from the chain structure of CNN and RNN, the GNN-based model can model graph-structured data and extract implicit structural information through dependency analysis trees, shortest dependency paths, etc. However, such methods rely on language tools to preprocess the data.

[0009] In addition, from linguistic knowledge, we know that in a sentence, entity pairs and a few verbs usually determine the results of relation extraction, so the removal of irrelevant information in the sentence is also a research topic in relation extraction. Among the existing methods for processing irrelevant information, the method of segmenting sentences according to statistical laws and focusing on the parts that have a greater impact on relation classification and the method based on the attention mechanism usually cannot completely eliminate the interference of irrelevant information, while the method using word segmentation information or the shortest dependency path may be affected by language tool preprocessing errors and differences in grammatical structure between training sets and test sets, and there are problems such as error propagation. Summary of the invention

[0010] In order to solve the problems of weak feature extraction capability and incomplete elimination of irrelevant information faced by current relationship extraction methods, the present invention provides a supervised relationship extraction method and device with low noise interference based on MogrifierBiGRU-GateAttention-sofimax, which can effectively improve feature extraction capability and anti-noise interference capability.

[0011] On the one hand, the present invention provides a supervised relationship extraction method with low noise interference, comprising:

[0012] Step 1: Convert the corpus to be processed into a vectorized form to obtain a coding sequence;

[0013] Step 2: Send the encoded sequence to the second layer MogrifierBiGRU in MogrifierBiGRU-GateAttention-softmax to obtain an encoded sequence incorporating context information;

[0014] Step 3: Send the encoded sequence incorporating the context information into the third layer of GateAttention in MogrifierBiGRU-GateAttention-softmax, remove task-irrelevant information, aggregate the screening results, and obtain the sentence vector;

[0015] Step 4: Send the sentence vector to softmax to obtain the relationship extraction result.

[0016] Further, step 1 is specifically as follows: use the word vector table to encode the input text with a length of n, a head and tail entity length of 1, and located at time i and j respectively, to obtain the encoding sequence Among them, x k The embedding dimension at time k is d x Pre-trained word vectors, and k∈(1,2,…,n); They are the word vectors corresponding to the start and end indicators of the head and tail entities, with a dimension of d x .

[0017] Furthermore, the step 2 specifically includes:

[0018] Step 2.1: Based on the input x at the current time t t , the hidden state h at the previous moment t-1 , calculate the hidden state h at the current time t according to formula (1) to formula (8) t :

[0019] x t ′=2σ(Qh t-1 )⊙x t (1)

[0020] h t-1 ′=2σ(Rx t ′)⊙h t-1 (2)

[0021] x t ″=2σ(Qh t-1 ′)x t ′ (3)

[0022] h t-1 ″=2σ(Rx t ″)h t-1 ′ (4)

[0023] z t =σ(W z x t ″+U z h t-1 ″+b z ) (5)

[0024] r t =σ(W r x t ″+U r h t-1 ″ +b r ) (6)

[0025]

[0026]

[0027] Among them, ⊙ is the matrix dot product, σ is the sigmoid function, tanh is the activation function, Q, R, W z , W r , W h , U z , U r , U h is the weight matrix, b z , b r , b h is the bias, x t ′、h t-1 ′、x t ″、h t-1 ″ is the intermediate state, z t is the update gate, r t To reset the gate, is the current memory content;

[0028] Step 2.2: According to formula (9), concatenate the forward and reverse hidden states to obtain the encoding of the current time t in the context Then we get the context encoding of the entire text

[0029]

[0030] Furthermore, the step 3 specifically includes:

[0031] Step 3.1: Fusion the entity information through formula (10) to formula (12) to obtain the entity vector e:

[0032]

[0033]

[0034]

[0035] Where W w is the weight matrix, b w is the offset, i is the time when the entity is in the sentence, r i is the contribution index, weight i is the normalized contribution index, j is a variable;

[0036] Step 3.2: Use the entity vector e to calculate X (2) The encoding of any time k except the entity is scored to obtain its score rela k,rela k Indicates the degree of relevance between corresponding words and relations:

[0037] p=gelu(e) (13)

[0038]

[0039]

[0040] Among them, p is the entity vector after nonlinear transformation, T represents transposition, and s k is the relevance measure, n is the sentence length, and m is the variable;

[0041] Step 3.3: Through the score vector rela = [rela 1 ,…,rela n ]Filter out task-irrelevant information in the input sentence.

[0042] Furthermore, the step 3.3 is specifically as follows:

[0043] Step 3.3.1: Through the score vector rela = [rela 1 ,…,rela n ]Use threshold strategy or truncation strategy to filter task-irrelevant information in the input sentence; where:

[0044] a) The threshold strategy is:

[0045]

[0046] b) The truncation strategy is:

[0047]

[0048] Among them, max(rela, β) is the maximum value of the first β in the selected vector rela. is the correlation measure obtained under the threshold strategy or truncation strategy;

[0049] Step 3.3.2: Obtain the text code X after removing irrelevant information through formula (18) and formula (19) (3) , that is, the screening results:

[0050]

[0051]

[0052] Among them, the word corresponding to time i is the entity.

[0053] Furthermore, step 3 specifically includes:

[0054] Step 3.4: Obtain the sentence vector S through formula (20) to formula (23):

[0055] H=gelu(X (3) ) (20)

[0056] α=softmax(W α H) (21)

[0057]

[0058]

[0059] Among them, T represents transpose, X (3) represents the text encoding after removing task-irrelevant information, i.e., the screening result; H is the text encoding after nonlinear transformation, W α is the weight matrix, α is the weight vector, Candidate status.

[0060] Furthermore, the step 4 is specifically as follows: using linear transformation and softmax function to classify the sentence vectors to obtain the relationship extraction result.

[0061] Furthermore, after step 4, it also includes: using a labeled data set to train and adjust hyperparameters of a supervised relationship extraction model obtained by executing the method of steps 1 to 4; wherein the labeled data set refers to a data set obtained by performing relationship annotation on entity pairs with relationships in the training corpus.

[0062] Furthermore, when training the supervised relation extraction model, adad e The lta optimizer trains the model parameters and uses L2 regularization to prevent the model from overfitting.

[0063] On the other hand, the present invention provides a supervised relationship extraction device with low noise interference, comprising:

[0064] The embedding module is used to convert the corpus to be processed into a vectorized form to obtain a coding sequence;

[0065] A context feature encoding module is used to integrate the encoding sequence of the second layer MogrifierBiGRU in MogrifierBiGRU-GateAttention-softmax into the context information to obtain the encoding sequence integrated with the context information;

[0066] The information filtering module is used to filter the encoding sequence of context information sent to the third layer of GateAttention in MogrifierBiGRU-GateAttention-softmax, remove task-irrelevant information, and aggregate the filtering results to obtain the sentence vector;

[0067] The relation classification module is used to classify the sentence vectors sent to softmax to obtain the relation extraction results.

[0068] Beneficial effects of the present invention:

[0069] The present invention provides a supervised relationship extraction method and device with low noise interference, which improves the existing BiGRU encoder to improve its feature extraction capability, designs a gated attention layer, captures context information through MogrifierBiGRU, and effectively represents the meaning of words in context. Compared with feature extractors such as LSTM and CNN, it has a stronger information capture capability when no additional information is used. In addition, the two proposed gating strategies can remove task-irrelevant words in sentences, effectively filter task-irrelevant information, and reduce the interference of noise on relationship extraction results while modeling context information. In addition, no language tools are required for data preprocessing, which reduces the impact of error propagation. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic diagram of a flow chart of a supervised relationship extraction method with low noise interference provided by an embodiment of the present invention;

[0071] Figure 2 A schematic diagram of the structure of MogrifierBiGRU-GateAttention-softmax provided in an embodiment of the present invention;

[0072] Figure 3 A schematic diagram of the structure of a MogrifierBiGRU unit provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] Example 1

[0075] like Figure 1As shown, the embodiment of the present invention provides a supervised relationship extraction method with low noise interference, and the structure of MogrifierBiGRU-GateAttention-softmax is referenced Figure 2 As shown, the method comprises the following steps:

[0076] S101: convert the corpus to be processed into a vectorized form to obtain a coding sequence;

[0077] S102: sending the encoding sequence to the second layer MogrifierBiGRU in MogrifierBiGRU-GateAttention-softmax to obtain an encoding sequence incorporating context information;

[0078] S103: Send the encoded sequence incorporating the context information to the third layer of GateAttention in MogrifierBiGRU-GateAttention-softmax, remove task-irrelevant information, aggregate the screening results, and obtain a sentence vector;

[0079] S104: Send the sentence vector to softmax to obtain a relationship extraction result.

[0080] The embodiment of the present invention provides a supervised relationship extraction method with low noise interference, which captures context information through MogrifierBiGRU and effectively represents the meaning of words in context. Compared with feature extractors such as LSTM and CNN, the present invention has stronger information capture capability when additional information is not used.

[0081] Example 2

[0082] Based on the above embodiment, the embodiment of the present invention provides another supervised relationship extraction method with low noise interference, comprising the following steps:

[0083] S201: Construct a labeled dataset for training;

[0084] Specifically, this step includes the following sub-steps: first, obtain the training corpus; second, find all entities in the sentence; finally, perform relationship annotation on entity pairs with relationships, so as to obtain annotated data sets.

[0085] S202: A supervised relationship extraction model based on MogrifierBiGRU-GateAttention-softmax is constructed according to steps substantially the same as steps S101 to S104 described in Example 1; based on Example 1, in this embodiment, this step specifically includes the following sub-steps:

[0086] S2021: Use the word vector table to encode the input text with a length of n, a head and tail entity length of 1, and located at time i and j respectively, to obtain the encoding sequence Among them, x k The embedding dimension at time k is d x Pre-trained word vectors, and k∈(1,2,…,n); They are the word vectors corresponding to the start and end indicators of the head and tail entities, with a dimension of d x .

[0087] S2022: Figure 3 As shown, according to the input x at the current time t t , the hidden state h at the previous moment t-1 , calculate the hidden state h at the current time t according to formula (1) to formula (8) t :

[0088] x t ′=2σ(Qh t-1 )⊙x t (1)

[0089] h t-1 ′=2σ(Rx t ′)⊙h t-1 (2)

[0090] x t ″=2σ(Qh t-1 ′)x t ′ (3)

[0091] h t-1 ″=2σ(Rx t ″)h t-1 ′ (4)

[0092] z t =σ(W z x t ″+U z h t-1 ″+b x ) (5)

[0093] r t =σ(W r x t ″+U r h t-1 ″+b r ) (6)

[0094]

[0095]

[0096] Among them, ⊙ is the matrix dot product, σ is the sigmoid function, tanh is the activation function, Q, R, W z , W r , W h , U z , U r , U h is the weight matrix, b z 、b r 、b h is the bias, x t ′、h t-1 ′、x t ″、h t-1 ″ is the intermediate state, z t is the update gate, r t To reset the gate, The current memory content.

[0097] S2023: According to formula (9), the forward and reverse hidden states are concatenated to obtain the encoding of the current time t in the context Then we get the context encoding of the entire text

[0098]

[0099] S2024: Fuse the entity information through formula (10) to formula (12) to obtain the entity vector e:

[0100]

[0101]

[0102]

[0103] Where W w is the weight matrix, b w is the offset, i is the time when the entity is in the sentence, r i is the contribution index, weight i is the normalized contribution index, and j is a variable.

[0104] S2025: Use the entity vector e to calculate X (2) The encoding of any time k except the entity is scored to obtain its score rela k ,rela k Indicates the degree of relevance between corresponding words and relations:

[0105] p=gelu(e) (13)

[0106]

[0107]

[0108] Among them, p is the entity vector after nonlinear transformation, T represents transposition, and s k is the relevance measure, n is the sentence length, and m is the variable.

[0109] S2026: By scoring vector rela=[rela 1 ,…,rela n ] Filter out task-irrelevant information in the input sentence;

[0110] As an implementable method, this step specifically includes the following sub-steps:

[0111] Step A1: Through the scoring vector rela = [rela 1 ,…,rela n ]Use threshold strategy or truncation strategy to filter task-irrelevant information in the input sentence; where:

[0112] a) The threshold strategy is:

[0113]

[0114] Specifically, rela is true if and only if all words in the sentence are equally relevant to the relation. k are all 1, so β∈(0,1.13) is generally set.

[0115] b) The truncation strategy is:

[0116]

[0117] Among them, max(rela, β) is the maximum value of the first β in the selected vector rela. is the correlation measure obtained under the threshold strategy or truncation strategy;

[0118] Specifically, the size of β is used to control the strength of irrelevant information filtering.

[0119] In the threshold strategy, rela k Both are 1, so in the threshold strategy, β∈(0, 1.13) is generally set.

[0120] In the truncation strategy, β is a positive integer. To reduce operations, β∈{1, 2, ...7} is generally set in the truncation strategy.

[0121] Step A2: Obtain the text code X after removing irrelevant information through formula (18) and formula (19) (3) , that is, the screening results:

[0122]

[0123]

[0124] Among them, the word corresponding to time i is the entity.

[0125] S2027: The sentence vector S is obtained by formula (20) to formula (23):

[0126] H=gelu(X (3) ) (20)

[0127] α=softmax(W α H) (21)

[0128]

[0129]

[0130] Among them, T represents transpose, X (3) represents the text encoding after removing task-irrelevant information, i.e., the screening result; H is the text encoding after nonlinear transformation, W α is the weight matrix, α is the weight vector, Candidate status.

[0131] S2028: Use linear transformation and softmax function to classify sentence vectors and obtain relationship extraction results.

[0132] S203: Use an annotated dataset to train and adjust hyperparameters of the supervised relationship extraction model obtained by executing the method of steps S101 to S104 (or the supervised relationship extraction model constructed in step S201); wherein the annotated dataset refers to the dataset obtained in step S201 after relationship annotation of entity pairs with relationships in the training corpus.

[0133] As an implementable method, when training the supervised relationship extraction model, the adadelta optimizer is used to train the model parameters, and L2 regularization is used to prevent the model from overfitting. The model uses the negative log-likelihood function as the loss function, which is specifically shown in formula (24) and formula (25):

[0134]

[0135]

[0136] Where W y is the weight matrix, b y is the bias, t kis the true category label, y k is the predicted category label, and N is the number of relationship categories.

[0137] S204: Send the corpus to be recognized into the model trained in step S203 to obtain a relationship classification result.

[0138] Example 3

[0139] Corresponding to the above-mentioned supervised relationship extraction method with low noise interference, an embodiment of the present invention further provides a supervised relationship extraction device with low noise interference, including:

[0140] The embedding module is used to convert the corpus to be processed into a vectorized form to obtain a coding sequence;

[0141] A context feature encoding module is used to integrate the encoding sequence of the second layer MogrifierBiGRU in MogrifierBiGRU-GateAttention-softmax into the context information to obtain the encoding sequence integrated with the context information;

[0142] The information filtering module is used to filter the encoding sequence of context information sent to the third layer of GateAttention in MogrifierBiGRU-GateAttention-sofimax, remove task-irrelevant information, and aggregate the filtering results to obtain the sentence vector;

[0143] The relation classification module is used to classify the sentence vectors sent to softmax to obtain the relation extraction results.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A supervised relationship extraction method with low noise interference, characterized in that: include: Step 1: Convert the corpus to be processed into a vectorized form to obtain a coding sequence; Step 2: Send the encoded sequence to the second layer MogrifierBiGRU in MogrifierBiGRU-GateAttention-softmax to obtain an encoded sequence incorporating context information; Step 3: Send the encoded sequence incorporating the context information into the third layer of GateAttention in MogrifierBiGRU-GateAttention-softmax, remove task-irrelevant information, aggregate the screening results, and obtain the sentence vector; Step 4: Send the sentence vector to softmax to obtain the relationship extraction result; Step 1 is as follows: Use the word vector table to encode the input text with a length of n, a head and tail entity length of 1, and located at time i and j respectively, to obtain the encoding sequence Among them, x k is the pre-trained word vector with embedding dimension dx at time k, and k∈(1,2,…,n); They are the word vectors corresponding to the start and end indicators of the head and tail entities, with a dimension of d x ; The step 2 specifically includes: Step 2.1: Based on the input x at the current time t t , the hidden state h at the previous moment t-1 , calculate the hidden state h at the current time t according to formula (1) to formula (8) t : x t ′=2σ(Qh t-1 )⊙x t (1) h t-1 ′=2σ(Rx t ′)⊙h t-1 (2) x t ″=2σ(Qh t-1 ′)x t ′ (3) h t-1 ″=2σ(Rx t ″)h t-1 ′ (4) z t =σ(W z x t ″+U z h t-1 ″+b z ) (5) r t =σ(W r x t ″+U r h t-1 "+b r ) (6) Among them, ⊙ is the matrix dot product, σ is the sigmoid function, tanh is the activation function, Q, R, W z , W r , W h , U z , U r , U h is the weight matrix, b z 、b r 、b h is the bias, x t ′、h t-1 ′、x t ″、h t-1 ″ is the intermediate state, z t is the update gate, r t To reset the gate, is the current memory content; Step 2.2: According to formula (9), concatenate the forward and reverse hidden states to obtain the encoding of the current time t in the context Then we get the context encoding of the entire text The step 3 specifically includes: Step 3.1: Fusion the entity information through formula (10) to formula (12) to obtain the entity vector e: Where W w is the weight matrix, b w is the offset, i is the time when the entity is in the sentence, r i is the contribution index, weight i is the normalized contribution index, j is a variable; Step 3.2: Use the entity vector e to calculate X (2) The encoding of any time k except the entity is scored to obtain its score rela k ,rela k Indicates the degree of relevance between corresponding words and relations: p=gelu(e) (13) Among them, p is the entity vector after nonlinear transformation, T represents transposition, and s k is the relevance measure, n is the sentence length, and m is the variable; Step 3.3: Through the score vector rela = [rela1, ..., rela n ] Filter out task-irrelevant information in the input sentence; Step 3.4: Obtain the sentence vector S through formula (20) to formula (23): H=gel(X (3) ) (20) α=softmax(W α H) (21) Among them, T represents transpose, X (3) represents the text encoding after removing task-irrelevant information, i.e., the screening result; H is the text encoding after nonlinear transformation, W α is the weight matrix, α is the weight vector, Candidate status.

2. A supervised relationship extraction method with low noise interference according to claim 1, characterized in that: The step 3.3 is specifically as follows: Step 3.3.1: Use the score vector rela = [rela1, ..., rela n ]Use threshold strategy or truncation strategy to filter task-irrelevant information in the input sentence; where: a) The threshold strategy is: b) The truncation strategy is: Among them, max(rela,β) is the maximum value of the first β in the selected vector rela. is the correlation measure obtained under the threshold strategy or truncation strategy; Step 3.3.2: Obtain the text code X after removing irrelevant information through formula (18) and formula (19) (3) , that is, the screening results: Among them, the word corresponding to time i is the entity.

3. A supervised relationship extraction method with low noise interference according to claim 1, characterized in that: The step 4 is specifically as follows: using linear transformation and softmax function to classify the sentence vectors to obtain the relationship extraction result.

4. A supervised relationship extraction method with low noise interference according to claim 1, characterized in that: After step 4, the method further includes: using a labeled data set to train and adjust hyperparameters of a supervised relationship extraction model obtained by executing the method of steps 1 to 4; wherein the labeled data set refers to a data set obtained by performing relationship annotation on entity pairs having relationships in a training corpus.

5. A supervised relationship extraction method with low noise interference according to claim 4, characterized in that: When training the supervised relation extraction model, the adadelta optimizer is used to train the model parameters, and L2 regularization is used to prevent the model from overfitting.

6. A supervised relationship extraction device with low noise interference, characterized in that: include: The embedding module is used to convert the corpus to be processed into a vectorized form to obtain a coding sequence; A context feature encoding module is used to integrate the encoding sequence of the second layer MogrifierBiGRU in MogrifierBiGRU-GateAttention-softmax into the context information to obtain the encoding sequence integrated with the context information; The information filtering module is used to filter the encoding sequence of context information sent to the third layer of GateAttention in MogrifierBiGRU-GateAttention-softmax, remove task-irrelevant information, and aggregate the filtering results to obtain the sentence vector; The relation classification module is used to classify the sentence vectors sent to softmax to obtain the relation extraction results; Wherein, the embedded module is specifically used for: Use the word vector table to encode the input text with a length of n, a head and tail entity length of 1, and located at time i and j respectively, to obtain the encoding sequence Among them, x k The embedding dimension at time k is d x Pre-trained word vectors, and k∈(1,2,…,n); They are the word vectors corresponding to the start and end indicators of the head and tail entities, with a dimension of d x ; The context feature encoding module is specifically used for: According to the input x at the current time t t , the hidden state h at the previous moment t-1 , calculate the hidden state h at the current time t according to formula (1) to formula (8) t : x t ′=2σ(Qh t-1 )⊙x t (1) h t-1 ′=2σ(Rx t ′)⊙h t-1 (2) x t ″=2σ(Qh t-1 ′)x t ′ (3) h t-1 ″=2σ(Rx t ″)h t-1 ′ (4) z t =σ(W z x t ″+U z h t-1 ″+b z ) (5) r t =σ(W r x t ″+U r h t-1 ″+b r ) (6) Among them, ⊙ is the matrix dot product, σ is the sigmoid function, tanh is the activation function, Q, R, W z , W r , W h , U z , U r , U h is the weight matrix, b z 、b r 、b h is the bias, x t ′、h t-1 ′、x t ″、h t-1 ″ is the intermediate state, z t is the update gate, r t To reset the gate, is the current memory content; According to formula (9), the forward and reverse hidden states are concatenated to obtain the encoding of the current time t in the context Then we get the context encoding of the entire text The information screening module is specifically used for: The entity information is fused through formula (10) to formula (12) to obtain the entity vector e: Where W v is the weight matrix, b w is the offset, i is the time when the entity is in the sentence, r i is the contribution index, weight i is the normalized contribution index, j is a variable; Using the entity vector e to X (2) The encoding of any time k except the entity is scored to obtain its score rela k ,rela k Indicates the degree of relevance between corresponding words and relations: p=gelu(e) (13) Among them, p is the entity vector after nonlinear transformation, T represents transposition, and s k is the relevance measure, n is the sentence length, and m is the variable; By scoring vector rela = [rela1, ..., rela n ] Filter out task-irrelevant information in the input sentence; The sentence vector S is obtained by formula (20) to formula (23): H=gel(X (3) ) (20) α=softmax(W α H) (21) Among them, T represents transpose, X (3) represents the text encoding after removing task-irrelevant information, i.e., the screening result; H is the text encoding after nonlinear transformation, W α is the weight matrix, α is the weight vector, Candidate status.

Images