A document-level relation extraction method based on dual-perspective attention fusion

Abstract

This invention provides a document-level relation extraction method based on dual-angle attention fusion. The method includes: dividing the document into multiple segments using a sliding window approach; obtaining entity representations and their representation vectors within each segment using a trained BERT model; classifying and combining these representation vectors to obtain different entity vectors; each entity vector includes a head entity vector and a tail entity vector; concatenating the head and tail entity vectors to obtain entity pairs; calculating the correlation between entity pairs to obtain a first feature; obtaining the context embedding of the entity pairs using the trained BERT model; calculating the correlation between the entity pairs and the context embedding to obtain a second feature; and fusing the first and second features to obtain the extraction result. This method ensures the completeness of the extraction result and improves model performance by leveraging the latent relationships between entity pairs and contextual information.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a document-level relation extraction method based on dual-angle attention fusion. Background Technology

[0002] In the field of document-level relation extraction, existing technologies mainly focus on sentence-level relation extraction. However, in practical applications, entities and their relations often appear in long documents. Entities in these documents are widely distributed, and relation extraction requires complex multi-hop reasoning. Existing document-level relation extraction methods are mainly divided into three categories: document graph-based methods, pre-trained language model-based methods, and noise-processing-based methods.

[0003] Document graph-based methods primarily model semantic information within a document by constructing a document graph and learn the relationships between entities through graph propagation. When handling multi-hop inference, these methods often use only unary information (i.e., entity features), resulting in a lack of information about other entities during the multi-hop inference process. Methods based on pre-trained language models mainly employ BERT or its variants to model information in document-level relation extraction. When handling multi-hop inference, these methods typically use entity pairs independently, without considering global information between triples. Noise-handling methods primarily address noise issues in the dataset to improve model performance when dealing with noisy data.

[0004] However, these existing technologies generally fail to effectively utilize the potential association features between entity pairs and contextual information when processing document-level relation extraction, resulting in entity pair representations that do not fully leverage contextual and multi-hop reasoning information. Therefore, it is essential to design a document-level relation extraction method based on dual-perspective attention fusion. Summary of the Invention

[0005] The purpose of this invention is to provide a document-level relation extraction method based on dual-angle attention fusion, which ensures the integrity of the extraction results and improves the performance of the model by leveraging the potential relationships between entity pairs and contextual information.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A document-level relation extraction method based on dual-perspective attention fusion includes the following steps:

[0008] The document is divided into multiple segments by sliding the window, and the entity representations and entity representation vectors in the segments are obtained by using a trained BERT model.

[0009] The expression vectors are classified and combined to obtain different entity vectors; the entity vectors include: head entity vector and tail entity vector.

[0010] The head entity vector and the tail entity vector are concatenated to obtain entity pairs;

[0011] Calculate the association between entity pairs to obtain the first feature;

[0012] Contextual embeddings of entity pairs are obtained through a trained BERT model;

[0013] The association between entity pairs and context embeddings is calculated to obtain the second feature;

[0014] The first and second features are fused to obtain the extraction result.

[0015] Optionally, the expression vectors are classified and combined to obtain different entity vectors. The specific steps are as follows: different entity expressions of the same entity are classified using clustering sets, and the expression vectors of different entity expressions are combined to obtain different entity vectors; the expression for an entity vector is: A m =[a m1 a m2 a m3 , ..., a mn ]; where A m Let a be the m-th entity vector. mn It is the nth representation vector of the mth entity vector.

[0016] Optionally, the head entity vector and the tail entity vector are concatenated to obtain entity pairs. Specific steps include:

[0017] Select any two distinct entity vectors from all entity vectors as the head entity vector and the tail entity vector, respectively.

[0018] Calculate the vector distance between the head entity vector and the tail entity vector; the formula for calculating the vector distance is: Where D is the vector distance, A xi Let A be the i-th element of the head entity vector. yi Let be the i-th element of the tail entity vector, and n be the dimension of the entity vector.

[0019] The head entity vector, vector distance, and tail entity vector are combined to obtain entity pairs; the expression for an entity pair is: F = W1[A x A y [D]; where W1 is a learnable parameter, A x Let A be the head entity vector. y This is the tail entity vector.

[0020] Optionally, the association between entity pairs is calculated to obtain the first feature. Specific steps include:

[0021] Calculate the query, key, and value for each of the two entity pairs;

[0022] The query, key, and value of the two entity pairs are normalized separately to obtain the attention of the two entity pairs;

[0023] The first feature is obtained by weighting and combining the attention of the two entities.

[0024] Optionally, the formula for calculating the first feature is: T = μR1 + (1-μ)R2; where R1 and R2 are the attention of two entity pairs, T is the first feature, and μ is the entity pair weight coefficient;

[0025] The formula for calculating entity attention is: j = 1 or 2; where b i Let be the normalization coefficient of the entity with respect to the i-th element. Let i be the value of the i-th element of the entity pair;

[0026] The formula for calculating the entity pair weight coefficient is as follows:

[0027] The normalization coefficient is: Q F =W q *F;K F =W k *F;V F =W v *F; where Q F For entity pair queries, K F V is the key to the entity pair. F For the value of the entity pair, W q W k and W v These are the learning coefficients for the query, key, and value, respectively. Let i be the key of the i-th element of the entity pair.

[0028] Optionally, the contextual embeddings of entity pairs can be obtained through a trained BERT model. Specific steps include:

[0029] The contextual semantics of the fragments are analyzed based on the trained BERT model, and a semantic vector is added to each expression vector based on the analysis results;

[0030] The semantic vectors are concatenated based on the classification and combination of the expression vectors to obtain the context embeddings corresponding to the entity vectors.

[0031] Optionally, the association between entity pairs and context embeddings is calculated to obtain a second feature. Specific steps include:

[0032] Calculate the key and value of the context embedding;

[0033] Normalize the query of entity pairs, the key of the context embedding, and the value of the context embedding to obtain context attention;

[0034] The entity pair attention and context attention are concatenated and dimensionality reduced to obtain the second feature.

[0035] Optionally, the formula for calculating the second feature is: H = {U, R} * {W} q W k W v}; where U represents contextual attention, {} represents concatenation operation, H represents the second feature, and R represents entity pair attention;

[0036] The formula for calculating contextual attention is: Q F =W q *F;K Z =W k *Z;V Z =W v *Z; where c i The normalization coefficient of the i-th element is embedded in the context. To embed the value of the i-th element into the context, K Z V is the key embedded in the context. Z For the value embedded in the context, Z represents the key of the i-th element in the context embedding.

[0037] Optionally, the first feature and the second feature are fused to obtain the extraction result. The specific steps include:

[0038] Calculate the covariance matrices of the first feature and the second feature respectively to obtain the first covariance matrix and the second covariance matrix;

[0039] The eigenvectors of the first covariance matrix and the second covariance matrix are concatenated to obtain the first and second dimensionality-reduced features.

[0040] The first feature is multiplied digit by digit by the second dimensionality-reduced feature to obtain the first relation;

[0041] The second feature is multiplied digit by digit by the first dimensionality-reduced feature to obtain the second relation;

[0042] The extraction result is obtained by weighting and adding the first and second relations.

[0043] According to specific embodiments provided by the present invention, the following technical effects are disclosed: The document-level relation extraction method based on dual-angle attention fusion provided by the present invention includes: dividing a document into multiple segments by sliding a window, and obtaining entity representations and entity representation vectors in the segments using a trained BERT model; classifying and combining the representation vectors to obtain different entity vectors; the entity vectors include: head entity vectors and tail entity vectors; concatenating the head entity vectors and tail entity vectors to obtain entity pairs; calculating the correlation between entity pairs to obtain a first feature; obtaining the context embedding of entity pairs using a trained BERT model; calculating the correlation between entity pairs and context embeddings to obtain a second feature; and fusing the first feature and the second feature to obtain the extraction result. This method ensures the integrity of the extraction result and improves the performance of the model by leveraging the potential relationship between entity pairs and contextual information. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of the document-level relation extraction method based on dual-angle attention fusion according to an embodiment of the present invention;

[0046] Figure 2 This is a flowchart illustrating the entity pair calculation process according to an embodiment of the present invention.

[0047] Figure 3 This is a flowchart of the first feature calculation according to an embodiment of the present invention;

[0048] Figure 4 This is a flowchart illustrating the calculation of the second feature according to an embodiment of the present invention;

[0049] Figure 5 This is a flowchart illustrating the extraction result calculation process in an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] like Figure 1 As shown, this embodiment of the invention provides a document-level relation extraction method based on dual-angle attention fusion, including the following steps:

[0053] Step 100: Divide the document into multiple segments by sliding the window, and obtain the entity representations and entity representation vectors in the segments using the trained BERT model; specific steps include:

[0054] Remove unnecessary spaces, line breaks, and other non-essential characters from the document to ensure the text is clean and tidy.

[0055] The window slides according to the set sliding step and extracts a document fragment of the window size after each slide; the window size is 512 characters and the sliding step is 64 characters.

[0056] The document fragment is input into the trained BERT model to identify entity representations in the document fragment, obtain the token vector of the entity representation, and use the token vector as the representation vector of the entity representation.

[0057] Step 200: Classify and combine the expression vectors to obtain different entity vectors; the entity vectors include: head entity vector and tail entity vector; the specific steps are: classify different entity expressions of the same entity by clustering sets, and combine the expression vectors of different entity expressions to obtain different entity vectors.

[0058] Specifically, the cluster set adopts K-means clustering. The clustering process is as follows: the entity representation and the representation vector are integrated to form a data list. The number of clusters K in the data list is calculated according to the elbow rule. The vectors in the data list are clustered and combined according to the K value to obtain different cluster labels. The vectors in the cluster labels are weighted and averaged to obtain the representative vector of the cluster label. The cluster label is regarded as an entity and the representative vector is regarded as an entity vector.

[0059] Specifically, the same entity can have multiple different expressions due to regional and cultural differences. By clustering different entity expressions, the entities represented by different entity expressions can be extracted, making the document cleaner and clearer, which is conducive to subsequent analysis in conjunction with contextual semantics.

[0060] More specifically, the expression for the entity vector is: A m =[a m1 a m2 a m3, ..., a mn ]; where A m Let a be the m-th entity vector. mn It is the nth representation vector of the mth entity vector.

[0061] Step 300: Concatenate the head entity vector and the tail entity vector to obtain entity pairs; the specific steps are as follows. Figure 2 As shown, it includes:

[0062] Step 301: Select any two different entity vectors from all entity vectors as the head entity vector and the tail entity vector, respectively;

[0063] Step 302: Calculate the vector distance between the head entity vector and the tail entity vector; the formula for calculating the vector distance is:

[0064]

[0065] Where D is the vector distance, A xi Let A be the i-th element of the head entity vector. yi Let be the i-th element of the tail entity vector, and n be the dimension of the entity vector.

[0066] Step 303: Combine the head entity vector, vector distance, and tail entity vector to obtain entity pairs; the expression for an entity pair is:

[0067] F = W1[A x A y [D];

[0068] Where W1 is a learnable parameter, A x Let A be the head entity vector. y This is the tail entity vector.

[0069] Specifically, an entity pair includes two related entities and the relationship between them. The head entity is the starting point of the relationship, and the tail entity is the ending point. The relationship between two entities is represented by the vector distance between their vectors. The smaller the vector distance, the stronger the relationship between the head and tail entities.

[0070] Step 400: Calculate the association between entity pairs to obtain the first feature;

[0071] Specifically, in document-level relations, the generation of some relations requires reasoning through multiple relations. Therefore, it is necessary to construct the interrelationships between entity pairs to facilitate relation recursion and improve the tightness of document-level relation extraction. The specific steps for calculating the association between entity pairs to obtain the first feature are as follows: Figure 3 As shown, it includes:

[0072] Step 401: Calculate the query, key, and value for each entity pair; the specific calculation formula is as follows:

[0073] Q F =W q *F;K F =W k *F;V F =W v *F;

[0074] Among them, Q F For entity pair queries, K F V is the key to the entity pair. F For the value of the entity pair, W q W k and W v These are the learning coefficients for the query, key, and value, respectively. The learning coefficients are weight matrices for the query, key, and value, respectively.

[0075] Specifically, a query represents the current element in an entity pair, used to find its relevance to itself or other elements; a key represents other elements in an entity pair, used to calculate similarity with the query, thereby determining the degree of association between other elements in the entity pair and the current element; the query and the key match each other; the value is also another element in the entity pair, used to construct the output, which represents elements that may affect the current element.

[0076] Step 402: Normalize the query, key, and value of each entity pair to obtain the attention of the two entity pairs. Specifically, calculate the dot product of the query and key to obtain the entity pair attention score. Normalize the entity pair attention score using the softmax function to obtain the entity pair attention weight. Use the entity pair attention weight as a normalization coefficient and perform a weighted sum with the entity pair values ​​to finally obtain the entity pair attention. The specific calculation formula is as follows:

[0077] j = 1 or 2;

[0078]

[0079] Among them, b i Let be the normalized coefficient of the i-th element of the entity pair (the attention weight of the entity pair). Let i be the value of the i-th element of the entity pair. Let i be the key of the i-th element of the entity pair.

[0080] Step 403: Weight and combine the attention of the two entity pairs to obtain the first feature. Specifically, assign weights to each entity pair based on the magnitude of their respective attention, with the sum of the weights being 1. Multiply the attention of each entity pair by its assigned weight and then concatenate the results to obtain the correlation between the two entity pairs, which is considered the first feature. The greater the attention of an entity pair, the greater its assigned weight.

[0081] Specifically, the formula for calculating the first feature is:

[0082] T = μR1 + (1-μ)R2;

[0083] Where R1 and R2 are the attention of two entity pairs, T is the first feature, and μ is the entity pair weight coefficient (proportion).

[0084] More specifically, the formula for calculating the entity-to-weight ratio (proportion) is as follows:

[0085]

[0086] Furthermore, the recursion of relationships can be achieved by calculating the association between multiple entity pairs, thereby improving the accuracy of the entity relationship extraction process.

[0087] Step 500: Obtain the context embeddings of entity pairs using the trained BERT model; specific steps include:

[0088] The contextual semantics of the fragments are analyzed based on the trained BERT model, and a semantic vector is added to each expression vector based on the analysis results;

[0089] Specifically, semantic vectors are generated by combining the contextual semantics of entity expressions with the BERT model after extensive pre-training to enable the model to understand and analyze the contextual semantics of a document. Semantic vectors can dynamically adjust their representation according to the contextual semantics of the entity expression, thereby more accurately capturing the semantic meaning of the entity expression, and can also distinguish the different uses of synonyms and polysemous words.

[0090] More specifically, semantic vectors exist in the form of high-dimensional vectors.

[0091] The semantic vectors are concatenated based on the classification and combination of the expression vectors to obtain the context embeddings corresponding to the entity vectors.

[0092] Specifically, semantic vectors are concatenated using an autoencoder; the autoencoder includes an encoder and a decoder; the encoder compresses high-dimensional semantic vectors into low-dimensional representations, and the decoder restores the low-dimensional representations to the original data; finally, the original data of different representation vectors are concatenated and combined to obtain the context embedding.

[0093] More specifically, context embedding is a dynamic set that can better capture the polysemy and contextual changes of entities. Its high-dimensional representation allows it to capture more complex semantic information and, compared to traditional word embedding, can effectively capture the relationships between distant entities.

[0094] Step 600: Calculate the correlation between entity pairs and context embeddings to obtain the second feature; specific steps are as follows: Figure 4 As shown, it includes:

[0095] Step 601: Calculate the key and value of the entity pair query and context embedding; the specific calculation formula is as follows:

[0096] Q F =W q *F;K Z =W k *Z;V Z =W v *Z;

[0097] Among them, K Z V is the key embedded in the context. Z Z represents the context embedding value.

[0098] Step 602: Normalize the entity pair query, the context embedding key, and the context embedding value to obtain the context attention. Specifically, calculate the dot product of the entity pair query and the context embedding key to obtain the context attention score. Normalize the context attention score using the softmax function to obtain the context attention weight. Use the context attention weight as a normalization coefficient and perform a weighted sum with the context embedding value to finally obtain the context attention. The specific calculation formula is as follows:

[0099]

[0100] Among them, c i The normalized coefficient (context attention weight) of the i-th element is embedded in the context. Embed the value of the i-th element in the context. Embed the key of the i-th element in the context.

[0101] Step 603: Perform dimensionality reduction by concatenating the entity pair attention and the context attention to obtain the second feature. Specifically, concatenate the context attention and the entity pair attention to obtain a matrix; concatenate the learning coefficients of the query, key, and value to obtain another matrix; and perform a dot product of the two matrices to achieve dimensionality reduction and obtain the second feature.

[0102] Specifically, the formula for calculating the second feature is:

[0103] H = {U, R} * {W} q W k W v};

[0104] Where U represents contextual attention, {} represents concatenation operation, H represents the second feature, and R represents entity pair attention.

[0105] Step 700: Perform feature fusion on the first and second features to obtain the extraction result. Specific steps are as follows: Figure 5 As shown, it includes:

[0106] Step 701: Calculate the covariance matrices of the first feature and the second feature respectively to obtain the first covariance matrix and the second covariance matrix;

[0107] Specifically, the covariance matrix is ​​calculated using principal component analysis. The specific steps are as follows: define the dimension of the feature matrix as n*p, where n is the number of samples and p is the number of features; calculate the mean of each feature and subtract the mean from each feature value to obtain the centered matrix; calculate the covariance matrix using the centered matrix.

[0108] More specifically, the feature matrix includes a first feature matrix and a second feature matrix.

[0109] More specifically, the formula for calculating the covariance matrix is:

[0110]

[0111] Where M is the covariance matrix, X is the centering matrix, and n is the number of samples. T It is the transpose of the centered matrix.

[0112] Step 702: Concatenate the eigenvectors of the first covariance matrix and the second covariance matrix respectively to obtain the first dimensionality reduction feature and the second dimensionality reduction feature;

[0113] Specifically, the formula for calculating the dimensionality reduction features is:

[0114] j = 1 or 2;

[0115] Where N1 is the first dimensionality reduction feature, and σ is the activation function calculation. Let Np be the p-th eigenvector of the first covariance matrix, and N2 be the second dimensionality-reduced eigenvector. It is the p-th eigenvector of the second covariance matrix.

[0116] Step 703: Multiply the first feature and the second dimensionality-reduced feature bit by bit to obtain the first relation;

[0117] Specifically, the formula for calculating the first relation is:

[0118] E1 = T⊙N2;

[0119] Step 704: Multiply the second feature by the first dimensionality reduction feature bit by bit to obtain the second relation;

[0120] Specifically, the formula for calculating the second relation is:

[0121] E2 = R⊙N1;

[0122] Step 705: Add the first relation and the second relation with weights to obtain the extraction result.

[0123] Specifically, the formula for calculating the extraction results is as follows:

[0124] Y = εE1 + (1-ε)E2;

[0125]

[0126] Where ε is the weighting ratio, Y is the extraction result, and exp() is the calculation of the exponential function.

[0127] The beneficial effects of this invention are as follows:

[0128] 1) The completeness of the extraction results is guaranteed by the potential relationships between entity pairs and contextual information;

[0129] 2) Improved model performance.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0131] Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. Furthermore, those skilled in the art will recognize that, based on the ideas of this invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A document-level relation extraction method based on dual-angle attention fusion, characterized in that, Includes the following steps: The document is divided into multiple segments by sliding the window, and the entity representations and representation vectors of the entities in the segments are obtained by using a trained BERT model. The expression vectors are classified and combined to obtain different entity vectors; the entity vectors include: head entity vectors and tail entity vectors. The head entity vector and the tail entity vector are concatenated to obtain an entity pair; Calculate the association between entity pairs to obtain the first feature; The context embeddings of the entity pairs are obtained through the trained BERT model; The correlation between the entity pair and the context embedding is calculated to obtain the second feature; the formula for calculating the second feature is: ;in, For context attention, {} represents the concatenation operation. As the second feature, For entity pairs of attention; The formula for calculating contextual attention is: ; ; ; ; ;in, The normalization coefficient of the i-th element is embedded in the context. Embed the value of the i-th element in the context. For the key embedded in the context, Values ​​embedded in the context Let Z be the key of the i-th element embedded in the context, and Z be the context embedding. For entity pair queries, , and Here, represents the learning coefficients for the query, key, and value, respectively, and F represents the entity pair; The first feature and the second feature are fused to obtain the extraction result. The specific steps include: Calculate the covariance matrices of the first feature and the second feature respectively to obtain the first covariance matrix and the second covariance matrix; The eigenvectors of the first covariance matrix and the second covariance matrix are concatenated to obtain the first dimensionality reduction feature and the second dimensionality reduction feature. The first feature is multiplied bit by bit with the second dimensionality-reduced feature to obtain the first relationship; The second feature is multiplied bit by bit with the first dimensionality-reduced feature to obtain the second relationship; The extraction result is obtained by weighting and adding the first relationship and the second relationship.

2. The document-level relation extraction method based on dual-angle attention fusion according to claim 1, characterized in that, The expression vectors are classified and combined to obtain different entity vectors. Specifically, the steps are as follows: different entity expressions of the same entity are classified using clustering sets, and the expression vectors of the different entity expressions are combined to obtain the different entity vectors; the expression of the entity vector is: A m =[a m1 a m2 a m3 , ..., a mn ]; where A m Let a be the m-th entity vector. mn It is the nth representation vector of the mth entity vector.

3. The document-level relation extraction method based on dual-angle attention fusion according to claim 2, characterized in that, The head entity vector and the tail entity vector are concatenated to obtain entity pairs. Specific steps include: Select any two different entity vectors from all the entity vectors as the head entity vector and the tail entity vector, respectively. Calculate the vector distance between the head entity vector and the tail entity vector; the formula for calculating the vector distance is: Where D is the vector distance. The i-th element of the head entity vector. Let be the i-th element of the tail entity vector, and n be the dimension of the entity vector; The head entity vector, the vector distance, and the tail entity vector are combined to obtain the entity pair; the expression for the entity pair is: Where W1 is a learnable parameter, For the head entity vector, This is the tail entity vector.

4. The document-level relation extraction method based on dual-angle attention fusion according to claim 3, characterized in that, Calculate the association between entity pairs to obtain the first feature. The specific steps include: Calculate the query, key, and value for each of the two entity pairs; The query, key, and value of the two entity pairs are normalized respectively to obtain the attention of the two entity pairs; The first feature is obtained by weighting and combining the attention of the two entities.

5. The document-level relation extraction method based on dual-angle attention fusion according to claim 4, characterized in that, The formula for calculating the first feature is: ;in, and Each represents an attention pair of entities, with T being the first feature. For entity pair weighting coefficients; The formula for calculating the entity's attention is: ;in, Let be the normalization coefficient of the entity with respect to the i-th element. Let i be the value of the i-th element of the entity pair; The formula for calculating the weight coefficient of the entity pair is as follows: ; The normalization coefficient is: ; ; ; ;in, As the key of the entity pair, For the value of the entity pair, Let i be the key of the i-th element of the entity pair.

6. The document-level relation extraction method based on dual-angle attention fusion according to claim 1, characterized in that, The context embeddings of the entity pairs are obtained through the trained BERT model, and the specific steps include: The contextual semantics of the segment are analyzed based on the trained BERT model, and a semantic vector is added to each expression vector based on the analysis results; The semantic vectors are concatenated based on the classification combination of the expression vectors to obtain the context embedding corresponding to the entity vector.

7. The document-level relation extraction method based on dual-angle attention fusion according to claim 4, characterized in that, The association between the entity pair and the context embedding is calculated to obtain the second feature. Specific steps include: Calculate the key and value embedded in the context; The query of the entity pair, the key of the context embedding, and the value of the context embedding are normalized to obtain context attention; The entity attention and the context attention are concatenated and dimensionality reduced to obtain the second feature.

Images