An entity relation extraction method and device based on syntax information and an attention mechanism

Abstract

This invention relates to an entity relation extraction method based on syntactic information and attention mechanisms. It employs a vector transformation unit to convert each word in a sentence, and the relative position and syntactic relation of each word with two specified entities, into vectors. A gated encoding unit performs feature extraction and nonlinear processing on the relative position vector and syntactic relation vector to obtain entity association feature gate vectors. A sentence encoding unit performs gated feature extraction and pooling processing on the entity association feature gate vectors and the word vectors to obtain the final sentence vector. A relation extraction unit calculates the distance between the final sentence vector and the classification prototype center in the prototype network to obtain the relation between two specific entities in the sentence. The entity relation extraction method of this invention solves the problem in existing research where insufficient utilization of sentence entity position and syntactic information leads to ineffective filtering of confusing information. This method can accurately distinguish the relations between specified entities using only a small number of annotated instances.

Description

Technical Field

[0001] This invention relates to the technical field of entity relation extraction, and in particular to an entity relation extraction method and apparatus based on syntactic information and attention mechanisms. Background Technology

[0002] Entities are common concepts in text, which can be specific individuals, events, or objects, or abstract concepts or relationships. The goal of relation extraction is to extract entities from sentences and determine the types of relationships between them. Relation extraction is a crucial step in building knowledge bases and knowledge graphs. A drawback of relation extraction tasks is their over-reliance on already labeled corpora. The labeling process is time-consuming and laborious, leading to high costs and a problem of scarce corpora. To address this, some research has proposed remote supervision methods that heuristically label entities in text within a knowledge base. This approach can alleviate the corpus scarcity problem to some extent, but it suffers from significant noise and a long-tail distribution in the corpus. To resolve the issues of noisy corpora and long-tail distribution in remote supervision methods, research has proposed a novel relation extraction task: few-sample relation extraction.

[0003] Few-shot relation extraction (FSRE) is an emerging research topic in recent years, combining relation extraction with few-shot learning. The FewRel 1.0 task proposed by Han et al. addresses the few-shot learning problem in relation extraction, specifically how to improve model performance when there are very few samples on new relation types and datasets. For FSRE, many studies currently employ pre-training to enhance the model's pre-understanding ability for RE tasks. Soare et al.'s pre-training method, Matching the Blanks, assumes that similar relation types are more likely to originate from the same entity pairs, and based on this assumption, they use a large amount of unsupervised data for pre-training.

[0004] The FSRE method, based on sentence similarity calculation, is a metric-based approach in few-shot learning. It effectively reduces the features lost by the model during sentence encoding and exhibits good domain adaptability. Gao et al. released a new dataset, FewRel2.0, and proposed the BERT-PAIR method in this task. This method uses the BERT model to calculate the similarity between sentences and selects the support set classification with the closest similarity as the predicted classification of the instance. Ye et al. proposed the Multi-Level Matching Aggregation Network (MLMAN), which uses word-level and sentence-level attention mechanisms to perform multi-level matching aggregation on the query set and support set.

[0005] Sentences may contain different entities, and these entities may have different relationships. Current research lacks the use of positional and syntactic information in sentences, so it is difficult to accurately distinguish these relationships using only a small number of annotated instances. At the same time, texts are diverse and accompanied by noise interference. Current few-shot learning models have difficulty denoising these texts, which can affect the extraction of correct relationships between entities, resulting in inaccurate extraction of relationships between entities. Summary of the Invention

[0006] Based on this, the purpose of the present invention is to provide an entity relation extraction method based on syntactic information and attention mechanism.

[0007] An entity relation extraction method based on syntactic information and attention mechanism includes the following steps:

[0008] S10 converts each word of the sentence from which entity relations are to be extracted into a word vector; converts the relative position and syntactic relation of each word of the sentence with the specified entity into a relative position vector and a syntactic relation vector, and concatenates the relative position vector and the syntactic relation vector to obtain the entity association vector of each word;

[0009] S20 performs feature extraction processing on the entity association vector of each word to obtain the entity association feature vector of each word, and performs linear and nonlinear processing on the entity association feature vector to obtain the entity association feature gating vector of each word.

[0010] S30 performs multi-layer self-attention mechanism processing on the word vector and the entity association feature gating vector to obtain the sentence feature vector containing the gated self-attention representation, and sequentially performs concatenation, linear and max pooling processing on the sentence feature vector containing the gated self-attention representation to obtain the final sentence vector with redundant information removed;

[0011] S40 calculates the distance between the final sentence vector and the classification prototype center of the prototype network, and selects the value with the smallest distance from the prototype center as the classification of entity relations in the sentence to be extracted.

[0012] Further, the expression for the entity association vector mentioned in step S10 is:

[0013]

[0014] in, This represents the relative position vector between the i-th word and the specified entity. This represents the syntactic relation vector between the i-th word and the specified entity.

[0015] Furthermore, the nonlinear processing described in step S20 is performed using the sigmoid activation function, resulting in the expression for the entity association feature gating vector:

[0016] G = sigmoid(W) g h p +b g )

[0017] Among them, W g b represents the weight matrix for linear processing. g This represents the bias vector for linear processing.

[0018] Furthermore, the gated self-attention weights of each layer of the multi-layer self-attention mechanism described in step S30 satisfy the following expression:

[0019]

[0020] Among them, Q t Let K represent the query matrix at level t. t Let d represent the bond matrix of layer t. k G represents the dimension of the word vector, and G represents the entity association feature gating vector.

[0021] Furthermore, the prototype center C of the u-th category... u The following relationship must be satisfied:

[0022]

[0023] in, x u,v =sum{σ(g(e)} u,v )⊙g(x u,v ))}, g() is a linear layer, ⊙ is generated element-wise, σ() is the tanh() activation function, sum{} represents the sum of all elements of the vector, and M represents the number of instances included in the u-th category.

[0024] Compared with existing technologies, this invention improves the problem of insufficient utilization of sentence entity location and syntactic information in existing studies, which leads to the inability to effectively filter confusing information, by integrating entity location information and syntactic information. On the other hand, by adding an attention mechanism when calculating the prototype center, the prototype obtains more features related to the query instance, resulting in a more typical prototype representation, thereby solving the problem of low classification accuracy in existing studies.

[0025] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0026] Figure 1This is a schematic diagram of the entity relation extraction method based on syntactic information and attention mechanism of the present invention.

[0027] Figure 2 for Figure 1 The flowchart shown is a flowchart of the entity relationship extraction model. Detailed Implementation

[0028] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention.

[0029] Please also refer to Figure 1 and Figure 2 ,in, Figure 1 This is a schematic diagram of the entity relation extraction model based on syntactic information and attention mechanism constructed in this invention. Figure 2 yes Figure 1 The flowchart shown is a flowchart of the entity relationship extraction model.

[0030] The entity relation extraction model constructed in this invention includes a vector transformation unit, a gated encoding unit, a sentence encoding unit, and a relation extraction unit. The vector transformation unit converts each word in the sentence into a word vector, and the relative position and syntactic relation of each word with two specified entities are converted into a relative position vector and a syntactic relation vector, respectively. Next, the gated encoding unit performs feature extraction and nonlinear processing on the relative position vector and syntactic relation vector to obtain an entity association feature gate vector for each word with two specified entities. Then, the sentence encoding unit performs gated feature extraction and pooling processing on the entity association feature gate vector and the word vector to obtain the final sentence vector after removing redundant information. Finally, the relation extraction unit calculates the distance between the final sentence vector and the prototype center in the prototype network to obtain the relation between the two specific entities in the sentence.

[0031] Specifically, the vector conversion unit is used to perform step S10: converting each word of the sentence from which entity relations are to be extracted into a word vector; converting the relative position and syntactic relation of each word in the sentence with a specified entity into a relative position vector and a syntactic relation vector; and concatenating the relative position vector and the syntactic relation vector to obtain the entity association vector for each word. The gated encoding unit includes a first vector converter and a second vector converter.

[0032] Wherein, the first vector converter is used to perform step S11: convert each word {w1,...,w} of the input sentence containing entity relations to be extracted into a vector converter. n The vectors are converted into continuous vectors, and positional encodings are added to the continuous vectors of each word to obtain word vectors containing positional information. The word vector of the i-th word is represented as e wi .

[0033] The second vector converter is used to perform step S12: convert each word of the input sentence to be extracted from entity relations into a vector with the relative position and syntactic relation of the two specified entities, to obtain the relative position vector and syntactic relation vector of a single word; and concatenate the relative position vector and syntactic relation vector of a single word to obtain the entity association vector of each word.

[0034] Specifically, this step includes the following sub-steps.

[0035] S121 converts the relative distances from the i-th word in the sentence to the two specified entities into continuous vectors, obtaining the relative position vector of the i-th word, which satisfies equation (1):

[0036]

[0037] (1) In the formula Let represent the relative position vector from the i-th word to the first entity. Let represent the relative position vector from the i-th word to the second entity.

[0038] In a specific embodiment, Stanford CoreNLP's named entity recognition is used to perform entity recognition on the input sentence from which entity relationships are to be extracted, and to extract the positional information of the entities in the sentence, obtaining information about two specified entities and their start and end positions, and calculating the relative positional information between each word and the entity based on this. The information about the two specified entities and their start and end positions is represented by two pairs of integers s1 = (l1, r1) and s2 = (l2, r2), where s1 represents the first entity, s2 represents the second entity, l represents the start position information, and r represents the end position information.

[0039] S122 converts the syntactic relations of the i-th word in the sentence with the two specified entities into continuous vectors, obtaining the syntactic relation vector of the i-th word, and adds position encoding to the syntactic relation vector of the i-th word to obtain a syntactic relation vector containing position information, which satisfies equation (2):

[0040]

[0041] (2) In the formula This represents the syntactic relation vector between the i-th word (containing location information) and the first entity. This represents the syntactic relation vector between the i-th word (containing location information) and the second entity.

[0042] In a specific embodiment, dependency parsing using Stanford CoreNLP is employed to perform dependency analysis on the syntactic relations between each word and an entity in the input sentence from which entity relations are to be extracted. The specific analysis steps are as follows:

[0043] Based on the original dependency tree of the sentence, assuming that the words directly connected to the given entity are more important in expressing the real relationship, the dependency relationship connecting the specified entity and other words is preserved, and other words are discarded to derive a pruned dependency tree;

[0044] Based on a pruned dependency tree, a dependency label t is assigned to the i-th word in the sentence. i =(t i,1 ,t i,2 As a suggested syntactic relation, it corresponds to the word tag t of the first entity. i,1 For example, if the word w i If it is part of the first entity, then it is the tag t. i,1 Assigning the value 'self', if the word w i The tag t is directly connected to the first entity in the dependency tree. i,1 Assign dependency relations 'nmod', 'nsubj', etc., if word w i If it is neither connected to the first entity nor part of the first entity, then it is a tag t. i,1 Assign the value 'Other';

[0045] Based on the embedding lookup operation, the two dependency tags t of the i-th word are... i =(t i,1 ,t i,2 Transform into a continuous vector and Obtain the syntactic relation vector of the i-th word.

[0046] S123 concatenates the relative position vector of the i-th word in the sentence and the syntactic relation vector containing positional information into an association vector, thus obtaining the entity association vector of the i-th word, which satisfies equation (3):

[0047]

[0048] The gated encoding unit is used to perform step S20: performing feature extraction processing on the entity association vector of each word to obtain the entity association feature vector of each word, and performing linear and nonlinear processing on the entity association feature vector to obtain the entity association feature gating vector of each word, and storing the entity feature gating vector in the entity guide gate. The gated encoding unit includes a first feature extractor, an entity feature gating editor, and an entity guide gate.

[0049] The first feature extractor is used to perform step S21: to process the entity association vector of each input word using a multi-layer self-attention mechanism, extract the entity association feature information of each word, and obtain the entity association feature vector of each word containing self-attention representation.

[0050] In a specific embodiment, using a Transformer-based bidirectional encoder as the entity feature extractor, this step can be represented as follows: in, The entity association vector of the i-th word in the input is processed by the encoder of the Transformer to obtain the entity association feature vector of the word with self-attention representation.

[0051] The entity feature gating editor is used to execute step S22: For each input word, the entity association feature vector containing self-attention representation is sequentially processed by a feedforward neural network (linear processing) and a sigmoid activation function (non-linear processing) to obtain the entity association feature gating vector G = (g1, g2, ..., g...). n ), where g is the entity association feature gating vector of the i-th word. i Satisfying equation (4):

[0052]

[0053] (4) In the formula, W g Let b represent the weight matrix of the feedforward neural network. g This represents the bias vector of the feedforward neural network.

[0054] The entity guidance gate is used to store the entity association feature gating vector of each word in the input sentence.

[0055] The sentence encoding unit is used to perform step S30: processing the word vectors of the sentence from which entity relations are to be extracted with the entity association feature gating vector using a multi-layer self-attention mechanism to obtain a sentence feature vector containing gating self-attention representation; and then sequentially concatenating, linearly processing, and max-pooling the sentence feature vector containing gating self-attention representation to obtain the final sentence vector after removing redundant information. The sentence encoding unit includes a second feature extractor, a vector concatenator, a linear layer, and a pooling layer.

[0056] The second feature extractor is used to perform step S31: to process the word vectors containing positional information and the entity association feature gating vectors with multi-layer self-attention mechanism, extract the association feature information between the sentence and the entity, and obtain the sentence feature vector containing gating self-attention representation.

[0057] In a specific embodiment, the multi-layer self-attention mechanism has m layers, and the operation steps of the t-th layer include the following sub-steps, where t∈(1,m).

[0058] S311 will output the sentence feature vector H from the previous self-attention mechanism. t-1 As the input to the t-th layer self-attention mechanism, it is respectively compared with the weight matrix W of the t-th layer self-attention mechanism. Qt W Kt W Vt Perform a dot product operation to obtain the query matrix Q of the t-th level. t Key matrix K t Sum matrix V t The input to the first layer of the self-attention mechanism is a word vector e containing positional information. w .

[0059] S312 queries the t-th layer's query matrix Q t Bond matrix K t Perform a dot product operation to obtain the self-attention distribution s of the t-th layer. t s t Satisfying equation (5):

[0060]

[0061] (5) where d k This represents the dimension of the word vector.

[0062] S313 distributes the self-attention of the t-th layer as s t Multiply by the entity association feature gating vector G, and then perform a softmax operation to obtain the gating self-attention weight Att of the t-th layer. t Att t Satisfying equation (6):

[0063]

[0064] S314 assigns the gated self-attention weight Att to the t-th layer. t and the value matrix V of the t-th layer t Performing a dot product operation yields the sentence feature vector H of the t-th layer containing gated self-attention representation. t H t Satisfying equation (7):

[0065] H t =Att t V t (7)

[0066] Repeat steps S311 to S314 until the second feature extractor finally outputs the sentence feature vector H of the m-th layer containing gated self-attention representation. m .

[0067] The vector concatenation unit is used to perform step S32: processing the input sentence feature vector H containing gated self-attention representation. m With word vector e containing location information w The concatenation process is performed to obtain a concatenated feature vector containing gated self-attention representation.

[0068] The linear layer is used to perform step S33: processing the input concatenated feature vector containing gated self-attention representations. Linear processing is performed to obtain sentence vectors with gated self-attention representations, whose dimensions are consistent with those of word vectors.

[0069] The pooling layer is used to perform step S34: perform max pooling on the input sentence vector containing gated self-attention representation to obtain the final sentence vector s after removing redundant information. q .

[0070] The relation extraction unit is used to perform step S40: calculating the distance between the final sentence vector and the classification prototype center in the prototype network, and selecting the value with the smallest distance from the prototype center as the classification of the entity relation in the sentence to be extracted. The classification unit is a relation classifier.

[0071] In a specific embodiment, Euclidean distance is used to calculate the distance δ between the final vector of the sentence and the prototype center C; and the prototype center corresponding to the minimum distance is selected as the relation classification of the entities in the sentence from which the entity relations are to be extracted.

[0072]

[0073] Where, δ=(||s q -C 1 || 2 ,...,||s q -C u || 2 ,...,||s q -C N || 2 )

[0074] (8) In the formula, s q C represents the final vector of the sentence. 1 C u C N Let represent the 1st prototype center, the uth prototype center, and the Nth prototype center, respectively.

[0075] Furthermore, the classification labels of the prototype network include N categories, and the prototype center of each category is calculated from M instances. Specifically, the prototype center of the u-th category is obtained in the following way:

[0076] SA1 maps the words in the v-th instance of the u-th category to real-valued embeddings. The i-th word in the v-th instance The relative distance between the two entities in this instance is converted into two dimensions d. p A dimensional vector, and two d p Concatenate 3D vectors to form the position vector of the i-th word The real-value embedding and position vector of the i-th word are concatenated to form the final input vector of the i-th word. Then the final input vector e of the vth instance u,v Represented as:

[0077]

[0078] in,

[0079] (8) In the formula, d i =d w +d p ×2.

[0080] SA2 uses convolutional layers to process the final input vector of the i-th word of the v-th instance of the input by sliding a convolutional kernel of size n through a window. Feature extraction is performed to extract the associated feature information of the i-th word, thus obtaining the d-value of the i-th word. h 3D correlation feature vector Then the associated feature vector h of the vth instance u,v Represented as:

[0081]

[0082] in,

[0083] (9) In the formula, CNN() represents convolution processing. This represents the word embedding preceding the i-th word. This indicates the word embedding following the i-th word.

[0084] SA3 performs pooling on the associated feature vector of the v-th instance to obtain the final instance vector x of the v-th instance. u,v The final instance vector satisfies equation (10):

[0085]

[0086] SA4 assigns a weight α to each final instance vector of the u-th category. u,v This leads to the prototype center C of the u-th category in the prototype network. uThe prototype center C u Represented as:

[0087]

[0088] in,

[0089] x u,v =sum{σ(g(e)} u,v )⊙g(x u,v ))}

[0090] (11) In the formula, g() is a linear layer, ⊙ is generated element by element, σ() is the tanh() activation function, and sum{} represents the sum of all elements of the vector.

[0091] This invention addresses the problem in existing research where insufficient utilization of sentence entity location and syntactic information leads to ineffective filtering of confusing information. It leverages the relative position and syntactic relationships between each word and specified entity pairs to solve the problem of ineffective filtering of confusing information. This method accurately distinguishes relationships between specified entities using only a small number of annotated instances. Furthermore, introducing a prototype network into the few-sample relation extraction task effectively alleviates the limitations of relation extraction in low-resource environments. By extracting features from instances, assigning attention scores to each instance with weights, and then using a weighted summation method to create more accurate prototype centers for each classification, the accuracy of entity relation classification is improved.

[0092] Based on the same inventive concept, this application also provides an electronic device, which can be a server, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, tablet computer, netbook, etc.). The device includes one or more processors and a memory, wherein the processor is used to execute a program to implement an entity relation extraction method based on syntactic information and attention mechanisms; the memory is used to store computer programs executable by the processor.

[0093] Based on the same inventive concept, this application also provides a computer-readable storage medium corresponding to the aforementioned embodiment of the entity relation extraction method based on syntactic information and attention mechanism. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the entity relation extraction method described in any of the above embodiments.

[0094] This application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0095] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and the present invention also intends to include these modifications and variations.

Claims

1. A method for entity relation extraction based on syntax information and attention mechanism, characterized in that, Includes the following steps: S10: Convert each word of the sentence from which entity relations are to be extracted into a continuous vector, and add position encoding to the continuous vector of each word to obtain a word vector containing position information; convert the relative position and syntactic relation of each word of the sentence with the specified entity into a relative position vector and a syntactic relation vector, respectively, and concatenate the relative position vector and the syntactic relation vector to obtain the entity association vector of each word; S20: Perform feature extraction processing on the entity association vector of each word to obtain the entity association feature vector of each word, and perform linear and nonlinear processing on the entity association feature vector to obtain the entity association feature gating vector of each word. S30: The word vectors containing positional information and the entity association feature gating vectors are processed by a multi-layer self-attention mechanism to obtain a sentence feature vector containing gating self-attention representation. The sentence feature vector containing gating self-attention representation is then concatenated with the word vectors containing positional information, followed by linear and max pooling processes to obtain the final sentence vector with redundant information removed. The gating self-attention weights of each layer of the multi-layer self-attention mechanism satisfy the following expression: wherein, represents the first layer of query matrix, represents the first layer of key matrix, represents the dimension of word vector, represents the entity correlation feature gating vector; S40: Calculate the distance between the final vector of the sentence and the classification prototype center of the prototype network, and select the prototype center corresponding to the smallest distance from the prototype center as the classification of entity relations in the sentence to be extracted.

2. The method of claim 1, wherein, The expression for the entity association vector mentioned in step S10 is: wherein, represents a relative position vector between the ith word and the specified entity, represents a syntactic relation vector between the ith word and the specified entity.

3. The entity relation extraction method according to claim 1, characterized in that, The nonlinear processing described in step S20 uses the sigmoid activation function, and the expression for the entity association feature gating vector is: in, The weight matrix represents the linear processing. This represents the bias vector for linear processing. The entity association vector of the input word is processed by the encoder of the Transformer to obtain the entity association feature vector of the word with self-attention representation.

4. The entity relation extraction method according to claim 1, characterized in that, Step S40 uses Euclidean distance to calculate the final sentence vector and the prototype center. distance The distance The following relationship must be satisfied: in, Represents the final vector of the sentence. This represents the prototype center of the first category. This represents the prototype center of the Nth category.

5. The entity relation extraction method according to claim 1, characterized in that, The classification prototype center of the prototype network in step S40 is obtained in the following way: SA1: Map the words in the v-th instance of the u-th category to real-valued embeddings. ; The i-th word in the v-th instance The relative distance to the two entities in this instance is converted into two dimensions. A dimensional vector, and two Concatenate 3D vectors to form the position vector of the i-th word The real-valued embedding and position vector of the i-th word are concatenated to form the final input vector of the i-th word. This yields the final input vector for the v-th instance. ; SA2: Employs a convolutional layer that slides a convolutional kernel of size n through a window to input the final input vector of the i-th word of the v-th instance. Feature extraction is performed to extract the associated feature information of the i-th word, thus obtaining the i-th word's... 3D correlation feature vector And obtain the associated feature vector of the vth instance. ; SA3: Perform pooling on the associated feature vector of the input v-th instance to obtain the final instance vector of the v-th instance. ; SA4: Assign a weight to each final instance vector of the u-th class. The prototype center of the u-th category in the prototype network is obtained. This leads to the classification prototype center of the prototype network. .

6. The entity relation extraction method according to claim 5, characterized in that, The prototype center of the u-th category The following relationship must be satisfied: in, , , It's a linear layer; ⊙ represents element-wise multiplication. It is the tanh() activation function. Let represent the sum of all elements in the vector, and M represent the number of instances included in the u-th category.

7. An entity relation extraction device based on syntactic information and attention mechanism, comprising: Vector transformation unit: used to convert each word of the sentence from which entity relations are to be extracted into a continuous vector, and add position encoding to the continuous vector of each word to obtain a word vector containing position information; convert the relative position and syntactic relation of each word of the sentence with the specified entity into a relative position vector and a syntactic relation vector respectively, and concatenate the relative position vector and the syntactic relation vector to obtain the entity association vector of each word; Gated coding unit: used to perform feature extraction processing on the entity association vector of each word to obtain the entity association feature vector of each word, and to perform linear and nonlinear processing on the entity association feature vector to obtain the entity association feature gating vector of each word; Sentence encoding unit: This unit processes the word vectors containing positional information and the entity association feature gating vectors using a multi-layer self-attention mechanism to obtain a sentence feature vector containing gated self-attention representation. The sentence feature vector containing gated self-attention representation is then concatenated with the word vectors containing positional information, followed by linear and max-pooling operations to obtain the final sentence vector after removing redundant information. The gated self-attention weights of each layer of the multi-layer self-attention mechanism satisfy the following expression: in, Indicates the first The query matrix of the layer, Indicates the first The key matrix of the layer, The dimension of the word vector. Represents the entity association feature gating vector; Relation extraction unit: used to calculate the distance between the final vector of the sentence and the classification prototype center of the prototype network, and select the prototype center corresponding to the smallest distance from the prototype center as the classification of the entity relations in the sentence to be extracted.

8. The entity relation extraction device according to claim 7, characterized in that, The gated coding unit includes a first feature extractor and an entity feature gate editor; The first feature extractor is used to process the entity association vector of each input word through a multi-layer self-attention mechanism, extract the entity association feature information of each word, and obtain the entity association feature vector of each word containing self-attention representation; The entity feature gating editor is used to sequentially process the input entity association feature vector containing self-attention representation using a feedforward neural network for linear processing and a sigmoid activation function for nonlinear processing, to obtain the entity association feature gating vector.

9. The entity relationship extraction device according to claim 7 or 8, characterized in that, The sentence encoding unit includes a second feature extractor, which is used to process word vectors containing positional information and entity association feature gating vectors using a multi-layer self-attention mechanism to extract the association feature information between the sentence and the entity, and obtain a sentence feature vector containing gating self-attention representation.

Images