Document-level relation extraction method based on double-angle attention fusion

Abstract

The invention provides a document-level relation extraction method based on double-angle attention fusion, and the method comprises the steps: dividing a document into a plurality of segments through a window sliding mode, and obtaining entity expressions in the segments and expression vectors of the entity expressions through a trained BERT model; classifying and combining the expression vectors to obtain different entity vectors; the entity vectors comprise a head entity vector and a tail entity vector; splicing the head entity vector and the tail entity vector to obtain an entity pair; calculating the relevance between the entity pairs to obtain a first feature; obtaining context embedding of the entity pair through the trained BERT model; calculating relevance between the entity pair and context embedding to obtain a second feature; and carrying out feature fusion on the first feature and the second feature to obtain an extraction result. According to the method, the integrity of the extraction result is ensured through the potential relationship between the entity pair and the context information, and the performance of the model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a document-level relationship extraction method based on dual-angle attention fusion. Background Art

[0002] In the field of document-level relation extraction, existing technologies mainly focus on sentence-level relation extraction. However, in practical applications, entities and their relations usually appear in longer documents. The entities in these documents are widely distributed, and relation acquisition requires complex multi-hop reasoning. Existing document-level relation extraction methods can be mainly divided into three categories: document graph-based methods, pre-trained language model-based methods, and noise processing-based methods.

[0003] Document graph-based methods mainly model the semantic information within the document by constructing a document graph, and learn the association between entities through graph propagation. These methods often only use univariate information (i.e., entity features) when dealing with multi-hop reasoning, resulting in a lack of information about other entities during the multi-hop reasoning process. Methods based on pre-trained language models mainly use BERT or its variants to model information in document-level relationship extraction. When dealing with multi-hop reasoning, these methods usually only use entity pairs independently without considering the global information between triplets. Methods based on noise processing mainly deal with the noise problem in the data set to improve the performance of the model when processing noisy data.

[0004] However, these existing technologies generally fail to effectively utilize the potential correlation features between entity pairs and context information when dealing with document-level relation extraction, resulting in the entity pair representation failing to fully utilize context information and multi-hop reasoning information. Therefore, it is necessary to design a document-level relation extraction method based on dual-angle attention fusion. Summary of the invention

[0005] The purpose of the present invention is to provide a document-level relationship extraction method based on dual-angle attention fusion, so as to ensure the integrity of the extraction results through the potential relationship between entity pairs and context information, and improve the performance of the model.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A document-level relation extraction method based on dual-angle attention fusion includes the following steps:

[0008] The document is divided into multiple segments by sliding the window, and the entity expressions and expression vectors of the entity expressions in the segments are obtained through the trained BERT model;

[0009] The expression vectors are classified and combined to obtain different entity vectors; the entity vectors include: head entity vector and tail entity vector;

[0010] Concatenate the head entity vector and the tail entity vector to obtain an entity pair;

[0011] Calculate the association between entity pairs and entity pairs to obtain the first feature;

[0012] Get the contextual embedding of entity pairs through the trained BERT model;

[0013] Calculate the correlation between entity pairs and context embeddings to obtain the second feature;

[0014] The first feature and the second feature are fused to obtain the extraction result.

[0015] Optionally, the expression vectors are classified and combined to obtain different entity vectors. The specific steps are: different entity expressions of the same entity are classified by clustering sets, and the expression vectors of different entity expressions are combined to obtain different entity vectors; the expression of the entity vector is: A m =[a m1 , a m2 , a m3 , ..., a mn ]; among them, A m is the mth entity vector, a mn is the nth expression vector of the mth entity vector.

[0016] Optionally, the head entity vector and the tail entity vector are concatenated to obtain an entity pair, and the specific steps include:

[0017] Select any two different entity vectors from all entity vectors as the head entity vector and the tail entity vector respectively;

[0018] Calculate the vector distance between the head entity vector and the tail entity vector; the calculation formula for the vector distance is: Where D is the vector distance, A xi is the i-th element of the head entity vector, A yi is the i-th element of the tail entity vector, and n is the number of dimensions of the entity vector.

[0019] The head entity vector, vector distance and tail entity vector are combined to obtain an entity pair; the expression of the entity pair is: F = W 1 [A x , A y , D]; where W 1 is a learnable parameter, A x is the head entity vector, A y is the tail entity vector.

[0020] Optionally, the association between entity pairs is calculated to obtain the first feature, and the specific steps include:

[0021] Calculate the query, key and value of two entity pairs respectively;

[0022] Normalize the query, key, and value of the two entity pairs respectively to obtain the attention of the two entity pairs;

[0023] The attention of the two entities is weighted and combined to obtain the first feature.

[0024] Optionally, the calculation formula of the first characteristic is: T = μR 1 +(1-μ)R 2 ; Among them, R 1 and R 2 are the attention of two entity pairs, T is the first feature, and μ is the weight coefficient of the entity pair;

[0025] The calculation formula of entity attention is: j = 1 or 2; where b i is the normalization coefficient of the entity pair i-th element, is the value of the ith element of the entity;

[0026] The calculation formula of entity pair weight coefficient is:

[0027] The normalization coefficient is: Q F =W q *F;K F =W k *F;V F =W v *F; where Q F is the query of entity pair, K F is the key of the entity pair, V F is the value of the entity pair, W q , W k and W v are the learning coefficients for query, key and value respectively, is the key of the i-th element of the entity pair.

[0028] Optionally, the context embedding of the entity pair is obtained through the trained BERT model. The specific steps include:

[0029] Analyze the contextual semantics of the fragment according to the trained BERT model, and add a semantic vector to each expression vector based on the analysis results;

[0030] The semantic vectors are concatenated according to the classification combination of the expression vectors to obtain the context embedding corresponding to the entity vector.

[0031] Optionally, the correlation between the entity pair and the context embedding is calculated to obtain the second feature. The specific steps include:

[0032] Compute contextual embedded keys and values;

[0033] Normalize the query, context embedding key, and context embedding value of the entity pair to get context attention;

[0034] The entity pair attention and context attention are concatenated and reduced in dimension to obtain the second feature.

[0035] Optionally, the calculation formula of the second feature is: H = {U, R} * {W q , W k , W v}; where U is the context attention, {} is the concatenation operation, H is the second feature, and R is the entity pair attention;

[0036] The calculation formula of contextual attention is: Q F =W q *F;K Z =W k *Z;V Z =W v *Z; where c i is the normalization coefficient of the i-th element of the context embedding, Embed the value of the i-th element in the context, K Z is the context embedding key, V Z is the value embedded in the context, is the key of the i-th element of the context embedding, and Z is the context embedding.

[0037] Optionally, the first feature and the second feature are fused to obtain an extraction result, and the specific steps include:

[0038] Calculate the covariance matrix of the first feature and the second feature respectively to obtain a first covariance matrix and a second covariance matrix;

[0039] Concatenate the eigenvectors of the first covariance matrix and the second covariance matrix respectively to obtain a first dimensionality reduction feature and a second dimensionality reduction feature;

[0040] Multiply the first feature by the second dimension reduction feature bit by bit to obtain a first relationship;

[0041] Multiply the second feature by the first dimension reduction feature bit by bit to obtain the second relationship;

[0042] The first relationship and the second relationship are weightedly added to obtain the extraction result.

[0043] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the document-level relationship extraction method based on dual-angle attention fusion provided by the present invention includes: dividing the document into multiple segments by window sliding, and obtaining entity expressions and expression vectors of entity expressions in the segments by a trained BERT model; classifying and combining the expression vectors to obtain different entity vectors; the entity vector includes: a head entity vector and a tail entity vector; concatenating the head entity vector and the tail entity vector to obtain an entity pair; calculating the correlation between entity pairs and entity pairs to obtain a first feature; obtaining the context embedding of entity pairs by a trained BERT model; calculating the correlation between entity pairs and context embedding to obtain a second feature; performing feature fusion of the first feature and the second feature to obtain an extraction result. The method ensures the integrity of the extraction result through the potential relationship between entity pairs and context information, and improves the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1 This is a flow chart of a document-level relationship extraction method based on dual-angle attention fusion according to an embodiment of the present invention;

[0046] Figure 2 is a flow chart of entity pair calculation in an embodiment of the present invention;

[0047] Figure 3 is a first characteristic calculation flow chart of an embodiment of the present invention;

[0048] Figure 4 is a second characteristic calculation flow chart of an embodiment of the present invention;

[0049] Figure 5 The figure is a flow chart of calculation of extraction results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] like Figure 1 As shown, the embodiment of the present invention provides a document-level relationship extraction method based on dual-angle attention fusion, comprising the following steps:

[0053] Step 100: Divide the document into multiple segments by sliding the window, and obtain the entity expressions and expression vectors of the entity expressions in the segments by using the trained BERT model; the specific steps include:

[0054] Remove extra spaces, line breaks and other unnecessary characters in the document to ensure the neatness of the text;

[0055] The window is slid according to the set sliding step, and the document fragments of the window size after each sliding are extracted; the window size is 512 characters and the sliding step is 64 characters;

[0056] The document fragment is input into the trained BERT model, the entity expression in the document fragment is identified, and the token vector of the entity expression is obtained, and the token vector is used as the expression vector of the entity expression.

[0057] Step 200: Classify and combine expression vectors to obtain different entity vectors; the entity vectors include: a head entity vector and a tail entity vector; the specific steps are: classify different entity expressions of the same entity by clustering sets, and combine expression vectors of different entity expressions to obtain different entity vectors.

[0058] Specifically, the clustering set adopts K-means clustering, and the clustering implementation process is: integrate the entity expression and the expression vector to form a data list, calculate the cluster number K of the data list according to the elbow rule, and cluster the vectors in the data list according to the K value to obtain different clustering labels, perform weighted averaging operation on the vectors in the clustering label to obtain the representative vector of the clustering label, and regard the clustering label as an entity and the representative vector as an entity vector.

[0059] Specifically, the same entity has a variety of different expressions due to regional and cultural differences. By clustering different entity expressions, the entities represented by different entity expressions are extracted, making the document neater and clearer, which is conducive to subsequent analysis in combination with contextual semantics.

[0060] More specifically, the entity vector is expressed as: m =[a m1 , a m2 , a m3, ..., a mn ]; among them, A m is the mth entity vector, a mn is the nth expression vector of the mth entity vector.

[0061] Step 300: concatenate the head entity vector and the tail entity vector to obtain an entity pair; the specific steps are as follows: Figure 2 As shown, including:

[0062] Step 301: Select any two different entity vectors from all entity vectors as the head entity vector and the tail entity vector respectively;

[0063] Step 302: Calculate the vector distance between the head entity vector and the tail entity vector; the calculation formula of the vector distance is:

[0064]

[0065] Where D is the vector distance, A xi is the i-th element of the head entity vector, A yi is the i-th element of the tail entity vector, and n is the number of dimensions of the entity vector.

[0066] Step 303: Combine the head entity vector, the vector distance and the tail entity vector to obtain an entity pair; the expression of the entity pair is:

[0067] F=W 1 [A x , A y , D];

[0068] Among them, W 1 is a learnable parameter, A x is the head entity vector, A y is the tail entity vector.

[0069] Specifically, an entity pair contains two related entities and the relationship between the two entities. The head entity is the starting point of the relationship, and the tail entity is the end point of the relationship. The relationship between the two entities is represented by the vector distance between the two entity vectors. When the vector distance is smaller, the relationship between the head entity and the tail entity is closer.

[0070] Step 400: Calculate the association between entity pairs and entity pairs to obtain a first feature;

[0071] Specifically, in document-level relationships, the generation of some relationships needs to be obtained by reasoning about multiple relationships, so it is necessary to build the mutual connection between entity pairs to facilitate the recursion of relationships and improve the closeness of document-level relationship extraction. The specific steps for calculating the correlation between entity pairs and obtaining the first feature are as follows: Figure 3 As shown, including:

[0072] Step 401: Calculate the query (Query), key (Key) and value (Value) of two entity pairs respectively; the specific calculation formula is:

[0073] Q F =W q *F;K F =W k *F;V F =W v *F;

[0074] Among them, Q F is the query of entity pair, K F is the key of the entity pair, V F is the value of the entity pair, W q , W k and W v The learning coefficients are the weight matrices of query, key and value respectively.

[0075] Specifically, the query represents the current element in the entity pair, which is used to find the correlation between itself and other elements; the key represents the other elements in the entity pair, which is used to calculate the similarity with the query, so as to determine the degree of association between the other elements in the entity pair and the current element; the query and the key match each other; the value is also the other element in the entity pair, which is used to construct the output, which represents the elements that may affect the current element.

[0076] Step 402: Normalize the query, key and value of the two entity pairs respectively to obtain the attention of the two entity pairs; the specific steps are: calculate the dot product of the query and the key to obtain the entity pair attention score, normalize the entity pair attention score through the softmax function to obtain the entity pair attention weight, and use the entity pair attention weight as the normalization coefficient and the value of the entity pair for weighted summation to finally obtain the entity pair attention. The specific calculation formula is:

[0077] j = 1 or 2;

[0078]

[0079] Among them, b i is the normalization coefficient of the ith element of the entity pair (entity pair attention weight), is the value of the entity pair i-th element, is the key of the i-th element of the entity pair.

[0080] Step 403: Weight the attention of the two entity pairs and combine them to obtain the first feature. The specific steps are: assign weights to the two entity pairs according to their respective attentions, the sum of the weights of the two entity pairs is 1, and the entity pair attention is multiplied by the weight ratio assigned to each pair and then spliced ​​to obtain the correlation between the two entity pairs and regard it as the first feature. The greater the entity pair attention, the greater the weight assigned.

[0081] Specifically, the calculation formula of the first feature is:

[0082] T=μR 1 +(1-μ)R 2 ;

[0083] Among them, R 1 and R 2 are the attention of two entity pairs, T is the first feature, and μ is the weight coefficient (ratio) of the entity pair;

[0084] More specifically, the calculation formula for the entity pair weight coefficient (ratio) is:

[0085]

[0086] Furthermore, the relationship can be recursively inferred by calculating the associations of multiple entity pairs, thereby improving the accuracy of extracting entity relationships.

[0087] Step 500: Obtain context embedding of entity pairs through the trained BERT model; the specific steps include:

[0088] Analyze the contextual semantics of the fragment according to the trained BERT model, and add a semantic vector to each expression vector based on the analysis results;

[0089] Specifically, the semantic vector is generated by combining the contextual semantics of entity expressions after the BERT model has been trained extensively in advance so that the model can understand and analyze the contextual semantics of the document. The semantic vector can dynamically adjust its representation according to the contextual semantics of the entity expression, thereby more accurately capturing the semantic meaning of the entity expression and distinguishing the different uses of synonyms and polysemous words.

[0090] More specifically, the semantic vector exists in the form of a high-dimensional vector.

[0091] The semantic vectors are concatenated according to the classification combination of the expression vectors to obtain the context embedding corresponding to the entity vector.

[0092] Specifically, semantic vectors are spliced ​​through an autoencoder; the autoencoder includes an encoder and a decoder; the high-dimensional semantic vector is compressed into a low-dimensional representation through the encoder, and then the low-dimensional representation is restored to the original data through the decoder, and finally the original data of different expression vectors are spliced ​​and combined to obtain context embedding.

[0093] More specifically, contextual embedding is a dynamic set that can better capture the ambiguity and contextual changes of entities. Its high-dimensional representation enables it to capture more complex semantic information. Compared with traditional word embeddings, it can effectively capture the relationships between distant entities.

[0094] Step 600: Calculate the relevance between the entity pair and the context embedding to obtain the second feature; the specific steps are as follows Figure 4 As shown, including:

[0095] Step 601: Calculate the query and context embedded keys and values ​​of the entity pair; the specific calculation formula is:

[0096] Q F =W q *F;K Z =W k *Z;V Z =W v *Z;

[0097] Among them, K Z is the context embedding key, V Z is the value of context embedding, and Z is the context embedding.

[0098] Step 602: Normalize the query of the entity pair, the key of the context embedding, and the value of the context embedding to obtain the context attention; the specific steps are: calculate the dot product of the query of the entity pair and the key of the context embedding to obtain the context attention score, normalize the context attention score through the softmax function to obtain the context attention weight, and use the context attention weight as the normalization coefficient and the value of the context embedding for weighted summation to finally obtain the context attention. The specific calculation formula is:

[0099]

[0100] Among them, c i is the normalization coefficient of the i-th element of the context embedding (context attention weight), embeds the value of the i-th element in the context, The key to embed the i-th element of the context.

[0101] Step 603: Concatenate the entity pair attention and the context attention to reduce the dimension and obtain the second feature. The specific steps are: concatenate the context attention and the entity pair attention to obtain a matrix; concatenate the learning coefficients of the query, key and value to obtain another matrix; perform a dot product on the two matrices to achieve the dimensionality reduction operation and obtain the second feature.

[0102] Specifically, the calculation formula of the second feature is:

[0103] H = {U, R} * {W q , W k , W v};

[0104] Among them, U is the context attention, {} is the concatenation operation, H is the second feature, and R is the entity pair attention.

[0105] Step 700: Fuse the first feature and the second feature to obtain an extraction result. Figure 5 As shown, including:

[0106] Step 701: Calculate the covariance matrix of the first feature and the second feature respectively to obtain a first covariance matrix and a second covariance matrix;

[0107] Specifically, the principal component analysis method is used to calculate the covariance matrix. The specific steps are: define the dimension of the feature matrix as n*p, where n is the number of samples and p is the number of features; calculate the mean of each feature and subtract the mean from each eigenvalue to obtain a centered matrix; calculate the covariance matrix through the centered matrix.

[0108] More specifically, the characteristic matrix includes a first characteristic matrix and a second characteristic matrix.

[0109] More specifically, the covariance matrix is ​​calculated as:

[0110]

[0111] Where M is the covariance matrix, X is the centering matrix, n is the number of samples, and X T is the transposed matrix of the centered matrix.

[0112] Step 702: concatenate the eigenvectors of the first covariance matrix and the second covariance matrix respectively to obtain a first dimensionality reduction feature and a second dimensionality reduction feature;

[0113] Specifically, the calculation formula of the dimensionality reduction feature is:

[0114] j = 1 or 2;

[0115] Among them, N 1is the first dimension reduction feature, σ is the activation function calculation, is the pth eigenvector of the first covariance matrix, N 2 is the second dimension reduction feature, is the pth eigenvector of the second covariance matrix.

[0116] Step 703: multiply the first feature and the second dimension reduction feature bit by bit to obtain a first relationship;

[0117] Specifically, the calculation formula of the first relationship is:

[0118] E 1 =T⊙N 2 ;

[0119] Step 704: multiply the second feature by the first dimension reduction feature bit by bit to obtain a second relationship;

[0120] Specifically, the calculation formula of the second relationship is:

[0121] E 2 =R⊙N 1 ;

[0122] Step 705: Perform weighted addition on the first relationship and the second relationship to obtain an extraction result.

[0123] Specifically, the calculation formula for the extraction result is:

[0124] Y=εE 1 +(1-ε)E 2 ;

[0125]

[0126] Among them, ε is the weighting ratio, Y is the extraction result, and exp() is the exponential function calculation.

[0127] The beneficial effects of the present invention are as follows:

[0128] 1) The integrity of the extraction results is ensured through the potential relationship between entity pairs and context information;

[0129] 2) Improved the performance of the model.

[0130] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0131] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A document-level relation extraction method based on dual-angle attention fusion, characterized in that: The steps include: Divide the document into multiple segments by window sliding, and obtain entity expressions in the segments and expression vectors of the entity expressions by using the trained BERT model; Classify and combine the expression vectors to obtain different entity vectors; the entity vectors include: a head entity vector and a tail entity vector; Concatenate the head entity vector and the tail entity vector to obtain an entity pair; Calculate the association between entity pairs and entity pairs to obtain the first feature; Obtaining the context embedding of the entity pair through the trained BERT model; Calculating the correlation between the entity pair and the context embedding to obtain a second feature; The first feature and the second feature are fused to obtain an extraction result.

2. The document-level relationship extraction method based on dual-angle attention fusion according to claim 1 is characterized in that: The expression vectors are classified and combined to obtain different entity vectors. The specific steps are: different entity expressions of the same entity are classified by clustering sets, and the expression vectors of the different entity expressions are combined to obtain the different entity vectors; the expression of the entity vector is: m =[a m1 , a m2 , a m3 , ..., a mn ]; among them, A m is the mth entity vector, a mn is the nth expression vector of the mth entity vector.

3. The document-level relationship extraction method based on dual-angle attention fusion according to claim 2 is characterized in that: The head entity vector and the tail entity vector are concatenated to obtain an entity pair, and the specific steps include: Selecting any two different entity vectors from all the entity vectors as the head entity vector and the tail entity vector respectively; Calculate the vector distance between the head entity vector and the tail entity vector; the calculation formula of the vector distance is: Where D is the vector distance, A xi is the i-th element of the head entity vector, A yi is the i-th element of the tail entity vector, and n is the number of dimensions of the entity vector. The head entity vector, the vector distance and the tail entity vector are combined to obtain the entity pair; the expression of the entity pair is: F = W1 [A x , A y , D]; where W1 is a learnable parameter, A x is the head entity vector, A y is the tail entity vector.

4. The document-level relationship extraction method based on dual-angle attention fusion according to claim 3 is characterized in that: Calculate the association between entity pairs and entity pairs to obtain the first feature. The specific steps include: Calculate the query, key and value of the two entity pairs respectively; Normalizing the query, key, and value of the two entity pairs respectively to obtain the attention of the two entity pairs; The attentions of the two entities are weighted and combined to obtain the first feature.

5. The document-level relationship extraction method based on dual-angle attention fusion according to claim 4 is characterized in that: The calculation formula of the first feature is: T = μR1 + (1-μ) R2; where R1 and R2 are the attention of two entity pairs, T is the first feature, and μ is the weight coefficient of the entity pair; The calculation formula of the entity's attention is: j = 1 or 2; where b i is the normalization coefficient of the entity pair i-th element, is the value of the ith element of the entity; The calculation formula of the entity pair weight coefficient is: The normalization coefficient is: Q F =W q *F;K F =W k *F;V F =W v *F; where Q F is the query of entity pair, K F is the key of the entity pair, V F is the value of the entity pair, W q , W k and W v are the learning coefficients for query, key and value respectively, is the key of the i-th element of the entity pair.

6. The document-level relationship extraction method based on dual-angle attention fusion according to claim 1 is characterized in that: The context embedding of the entity pair is obtained by using the trained BERT model, and the specific steps include: Analyze the context semantics of the fragment according to the trained BERT model, and add a semantic vector to each expression vector according to the analysis result; The semantic vectors are concatenated according to the classification combination of the expression vectors to obtain the context embedding corresponding to the entity vector.

7. The document-level relationship extraction method based on dual-angle attention fusion according to claim 4 is characterized in that: Calculating the correlation between the entity pair and the context embedding to obtain a second feature, the specific steps include: computing keys and values ​​embedded in the context; Normalizing the query of the entity pair, the key of the context embedding, and the value of the context embedding to obtain context attention; The entity attention and the context attention are concatenated and reduced in dimension to obtain the second feature.

8. The document-level relationship extraction method based on dual-angle attention fusion according to claim 7 is characterized in that: The calculation formula of the second feature is: H = {U, R} * {W q , W k , W v }; where U is the context attention, {} is the concatenation operation, H is the second feature, and R is the entity pair attention; The calculation formula of the context attention is: Q F =W q *F;K Z =W k *Z;V Z =W v *Z; where c i is the normalization coefficient of the i-th element of the context embedding, Embed the value of the i-th element in the context, K Z is the context embedding key, V Z is the value embedded in the context, is the key of the i-th element of the context embedding and Z is the context embedding.

9. The document-level relationship extraction method based on dual-angle attention fusion according to claim 1 is characterized in that: The first feature and the second feature are subjected to feature fusion to obtain an extraction result. The specific steps include: Calculating the covariance matrix of the first feature and the second feature respectively to obtain a first covariance matrix and a second covariance matrix; respectively concatenating the eigenvectors of the first covariance matrix and the second covariance matrix to obtain a first dimensionality reduction feature and a second dimensionality reduction feature; Multiplying the first feature by the second dimension reduction feature bit by bit to obtain a first relationship; Multiplying the second feature by the first dimension reduction feature bit by bit to obtain a second relationship; The first relationship and the second relationship are weightedly added to obtain the extraction result.

Images