Relation extraction method, system, device and storage medium based on attention mechanism

Abstract

The present invention provides a relation extraction method, system, device and storage medium based on an attention mechanism, the method comprising: preprocessing a text to be processed to form a first sentence; identifying the first sentence, and semantically completing the first sentence to form a second sentence; extracting a feature vector in the second sentence, and inputting the feature vector into a relation extraction model to obtain a relation label type output by the relation extraction model, wherein the relation extraction model is trained based on semantically completed sentences and normal sentences in a training corpus. The present invention preprocesses the text to be processed, then semantically completes it, and obtains the relation label type through a relation extraction model; realizes the relation extraction of semantically incomplete sentences, improves the extraction efficiency and the extraction accuracy, and avoids the problem of time-consuming and expensive manual annotation of these sentences.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a relationship extraction method, system, device and storage medium based on an attention mechanism. Background Art

[0002] Information extraction is mainly to extract specific factual information, namely entities, from text, while entity relationship extraction, as a subtask of information extraction, is mainly aimed at extracting structured relationship information from unstructured text. Traditional relationship extraction methods are usually based on supervised learning, semi-supervised learning and unsupervised learning. The relationship extraction method based on supervised learning requires a fully manually annotated corpus as a training set to train the relationship extraction model, which requires a lot of manpower and time, and has poor prediction ability for new entity relationships that are not in the training set. The relationship extraction method based on semi-supervised learning uses a partially annotated corpus to extract entity relationships through iterative training. Although this method reduces the cost of manual annotation to a certain extent, it still requires some annotated data. The relationship extraction method based on unsupervised learning does not require a manually annotated corpus, and automatically classifies the relationships through clustering methods. This method usually obtains suboptimal results.

[0003] Distant supervision combines the advantages of semi-supervised and unsupervised methods. It automatically annotates entity relationships by aligning entities in unlabeled corpus and knowledge base, which greatly improves the performance of entity relationship extraction when extracting entity information from the knowledge base and reduces labor costs. It then solves the noise problem of distant supervision methods by calculating the similarity between the relationship words between entity pairs in the domain ontology and the dependency words between entity pairs in the unlabeled text.

[0004] Due to the lack of subject, predicate or object, there are short sentences with incomplete semantics, such as electronic medical records in the medical field, domain knowledge image construction, sentiment analysis, retrieval or prediction. For example, a normal sentence is represented as: chest X-ray shows right pleural lesions and aortic node calcification, where the entities are: chest X-ray, right pleural lesions, aortic node calcification, and the relationship is: (chest X-ray, showing, right pleural lesions), (chest X-ray, showing, aortic node calcification). The short sentence with missing semantics is represented as: Considered to be bronchitis, the entity is: bronchitis. It can be seen that the short sentence with missing semantics cannot reflect the entity relationship, so that the accuracy of its relationship extraction is poor. Summary of the invention

[0005] The present invention provides a relationship extraction method, system, device and storage medium based on an attention mechanism, which are used to solve the problem of poor accuracy in relationship extraction of semantically incomplete short sentences such as those lacking a subject, predicate or object under remote supervision, thereby improving the efficiency of relationship extraction.

[0006] The present invention provides a relationship extraction method based on an attention mechanism, comprising: preprocessing a text to be processed to form a first sentence; identifying the first sentence, and semantically completing the first sentence to form a second sentence; extracting a feature vector in the second sentence, and inputting the feature vector into a relationship extraction model to obtain a relationship label type output by the relationship extraction model, wherein the relationship extraction model is trained based on semantically completed sentences and normal sentences in a training corpus.

[0007] According to the attention mechanism-based relationship extraction method provided by the present invention, the text to be processed is preprocessed to form a first sentence, including: combining the unlabeled corpus in the text to be processed with the knowledge base, and aligning the unlabeled corpus with the entities in the knowledge base to automatically label the entity relationships; dividing the first sentence into Chinese character strings according to the entities, punctuation marks, numbers and space characters contained in the combined knowledge base, and removing stop words to form the first sentence.

[0008] According to the relationship extraction method based on the attention mechanism provided by the present invention, a first sentence is identified, and the first sentence is semantically completed to form a second sentence, including: identifying the syntactic structure of the first sentence and the first entity in the first sentence by parsing; learning the contextual features of the first entity in the first sentence to obtain the missing semantic information of the first sentence; and semantically completing the first sentence according to the missing semantic information to form a second sentence.

[0009] According to the relationship extraction method based on the attention mechanism provided by the present invention, the first sentence is semantically completed according to the type of missing semantic information to form a second sentence, including: if the missing semantic information is the subject or object is missing, then find the entity containing the missing semantic information of the first sentence as the target entity, and concatenate the target entity and the first sentence to form the second sentence; if the missing semantic information is the subject and predicate or the predicate and object are missing, then find the normal sentence containing the missing semantic information of the first sentence, and concatenate the first sentence and the normal sentence to form the second sentence.

[0010] According to the relationship extraction method based on the attention mechanism provided by the present invention, an entity containing the missing semantic information of the first sentence is found as a target entity, and the target entity and the first sentence are concatenated to form a second sentence, including: extracting an entity different from the first entity type from the context sentence of the normal sentence where the first entity is located in the knowledge base as a candidate entity; extracting a trigger word closest to the first entity from the context sentence based on the label type as a target trigger word, and using a word attention mechanism to extract a feasible relationship type between the target trigger word and the first entity; extracting a candidate entity with the highest similarity from the candidate entities as a target entity based on a remote supervision method and the feasible relationship type; and concatenating the target entity with the first sentence to form a second sentence;

[0011] Find a normal sentence containing the missing semantic information of the first sentence, and splice the first sentence and the normal sentence into one sentence, including: extracting the first entity contained in the first sentence; finding the context feature closest to the first entity in the context sentence, calculating the similarity between the context feature and the first entity, and selecting the normal sentence containing the context feature with the highest similarity as the target normal sentence; splicing the first sentence and the target normal sentence end to end to form a second sentence.

[0012] According to the relationship extraction method based on the attention mechanism provided by the present invention, after identifying the first sentence by parsing the syntactic structure of the first sentence, it also includes: labeling the first entity using a part-of-speech tag mechanism; after semantically completing the first sentence according to the missing semantic information, it also includes: proofreading the entity tag labeled with the first entity.

[0013] According to the relationship extraction method based on the attention mechanism provided by the present invention, the relationship extraction model includes a convolution layer, a segmented maximum pooling layer, a sentence attention layer and a classification layer, wherein: the convolution layer is used to perform a convolution operation on an input feature vector; the segmented maximum pooling layer divides the feature vector after the convolution operation into three parts according to the position of the entity pair in the second sentence, performs segmented pooling on each part and calculates its maximum value respectively, and combines the maximum values ​​of the three parts for output; the sentence attention layer derives sentence features according to the output of the segmented maximum pooling layer and the semantic relationship between the words and entity pairs in the second sentence; the classification layer is used to input the sentence features, and perform calculations based on the sentence features to output the relationship label type.

[0014] According to the relationship extraction method based on the attention mechanism provided by the present invention, the feature vector includes entity features, word features and position features, and the extraction of the feature vector in the second sentence includes: extracting the entity feature by adding the self features and type features of the second entity in the second sentence; using the word2vec model to convert the non-entity words in the second sentence into word vectors to extract word features; and extracting the position feature by calculating the relative distance between each word vector in the second sentence and any two second entity features.

[0015] According to the relationship extraction method based on the attention mechanism provided by the present invention, training the relationship extraction model includes: constructing a training corpus based on a remote supervision method, inputting the sentences in the training corpus as training sentences into the relationship extraction model, and the training corpus includes normal sentences and / or semantic completion sentences; the relationship extraction model is trained according to the input training sentences to output a predicted relationship label type; a loss function is obtained according to the predicted relationship label type and the actual relationship label type of the training sentence, and if the loss function remains stable, the training ends; otherwise, the relationship extraction model continues to be trained.

[0016] The present invention also provides a relationship extraction system based on an attention mechanism, which is implemented based on any of the above-mentioned relationship extraction methods based on the attention mechanism, and the system includes: a data preprocessing module, which is used to preprocess the text to be processed to form a first sentence; a short sentence processing module, which is used to identify the first sentence and semantically complete it to form a second sentence; a feature extraction module, which is used to extract a feature vector relationship extraction model module in the second sentence, and output a label type according to the feature vector extracted by the feature extraction module.

[0017] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned relationship extraction methods based on the attention mechanism are implemented.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described relationship extraction methods based on the attention mechanism.

[0019] The present invention provides a relationship extraction method and system based on an attention mechanism, which pre-processes a text to be processed to form a first sentence, so as to automatically mark entity relationships by aligning unlabeled corpus with entities in a knowledge base, thereby improving the marking efficiency; then the first sentence is recognized and semantically completed to form a second sentence, so as to facilitate subsequent relationship extraction, thereby avoiding the problem of erroneous relationship extraction caused by directly performing relationship extraction on short sentences with incomplete semantics and the time-consuming and labor-intensive problem caused by manually marking these sentences; then the feature vector in the second sentence is extracted and input into a trained relationship extraction model to obtain a relationship label type, thereby realizing the relationship extraction of semantically incomplete sentences, improving the extraction efficiency and the extraction accuracy, and reducing the influence of noise in the relationship extraction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 is a flowchart of a relationship extraction method based on an attention mechanism provided by an embodiment of the present invention;

[0022] Figure 2 It is a schematic diagram of the flow chart of the short sentence semantic completion stage provided by an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of the process flow of the relationship extraction stage provided by an embodiment of the present invention;

[0024] Figure 4 is a structural diagram of a relationship extraction system based on an attention mechanism provided by an embodiment of the present invention;

[0025] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention.

[0026] Reference numerals:

[0027] 1: Data preprocessing module; 2: Short sentence processing module; 3: Feature extraction module;

[0028] 4: Relation extraction model module; 51: Processor; 52: Communication interface;

[0029] 53: memory; 54: communication bus. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Figure 1 The following is a flowchart of a relationship extraction method based on the attention mechanism. Figure 1 As shown, including:

[0032] S01: pre-processing the text to be processed to form the first sentence;

[0033] S02: recognizing the first sentence, and semantically completing the first sentence to form a second sentence;

[0034] S03: Extract the feature vector in the second sentence, and input the feature vector into the relation extraction model to obtain the relation label type output by the relation extraction model, wherein the relation extraction model is trained based on semantic completion sentences and normal sentences in the training corpus.

[0035] It should be noted that the S0N in this specification does not represent the order of the relationship extraction method. Figure 2-Figure 3 Describe each step in detail.

[0036] Step S01, preprocessing the text to be processed to form a first sentence.

[0037] In this embodiment, the text to be processed is preprocessed to form a first sentence, including: combining the unlabeled corpus in the text to be processed with the knowledge base, and aligning the unlabeled corpus with the entities in the knowledge base to automatically label the entity relationship; dividing the text to be processed into Chinese character strings according to the entities, punctuation marks, numbers and space characters contained in the combined knowledge base, and removing stop words to form the first sentence S1 = {W1 ... e i …e o …W m}, the first sentence includes the first entity, where {W1…W m} is a non-entity word in the first sentence S1, e i and e o is the first entity in S1. It should be noted that the text to be processed may be an electronic medical record.

[0038] Step S02: Identify the first sentence, and perform semantic completion on the first sentence to form a second sentence.

[0039] In this embodiment, a first sentence is identified and semantically completed to form a second sentence, including: identifying the syntactic structure of the first sentence and a first entity in the first sentence by parsing; learning contextual features of the first entity in the first sentence to derive semantic information missing from the first sentence; and semantically completing the first sentence according to the missing semantic information to form a second sentence.

[0040] Specifically, the first sentence is identified by parsing the syntactic structure of the first sentence, including: using the syntactic parsing tool HIT LTP to analyze the syntactic structure of the first sentence. In addition, the skip-gram model is used to extract the dependency path features between the first entities in the first sentence, so as to enhance the expression of the word vector, and then the word2vec model is used to convert the non-entity words in the first sentence S1 into d-dimensional word vectors to extract word features, and each word vector in the word feature is annotated. In this embodiment, the part-of-speech tag mechanism POS can be used to annotate the word vector. For example, the first sentence S1 = {W1…e i …e o …W m}, the word2vec model is used to convert the non-entity words in the first sentence S1 into word vectors to extract word features, and form the first feature sentence s1 = {w1…g i …g o …w m},w i is the word feature in the first feature sentence S1, g i and g o is the first entity feature in the first feature sentence s1. In addition, since the text to be processed is an electronic medical record, the annotated label types may include diseases, symptoms, and the like.

[0041] Due to the writing habits of Chinese, the missing semantic information in the first sentence can usually be extracted from its adjacent context features. Therefore, the missing semantic information of the first sentence can be obtained by learning the context features of each first entity in the first sentence. In this embodiment, a long short-term memory network (BILSTM) can be used to learn the context features of each first entity in the first sentence. For example, the first feature sentence s1 = {w1…g i …g o …w m}, and the hidden units corresponding to the first feature sentence s1 are {h1,h2,…,h m}, where the dimension of the hidden unit is consistent with the word vector, and the context feature of the first entity in the first feature sentence learned by combining the forward and reverse long short-term memory networks is expressed as:

[0042]

[0043] in: represents the hidden unit of the forward long short-term memory network, represents the hidden unit of the reverse long short-term memory network, and t represents the word vector w i The state at time t is Vector concatenation operation.

[0044] Then, based on the context features, missing semantic information is obtained, which is mainly of two types: (1) missing subject or object, (2) missing subject and predicate, or missing predicate and object. It should be noted that missing subject or object means that the first sentence lacks a main component, which usually includes one or more entities and the above entities are of the same entity type, and each of the other two main components is not limited to one. For example, the short sentence may lack a subject but have two predicates or objects. Missing subject and predicate, or missing predicate and object means that the first sentence contains only one main component, which usually only contains one or more entities and are of the same entity type.

[0045] After obtaining the missing semantic information, the first sentence is semantically completed according to the missing semantic information. If the missing semantic information is that the subject or object is missing, an entity containing the missing semantic information of the first sentence is found as a target entity, and the target entity and the first sentence are concatenated to form a second sentence. If the missing semantic information is that the subject and predicate or the predicate and object are missing, a normal sentence containing the missing semantic information of the first sentence is found, and the first sentence and the normal sentence are concatenated to form a second sentence.

[0046] When the subject or object of the first sentence is missing, an entity containing the missing semantic information of the first sentence is found as the target entity, and the target entity is concatenated with the first sentence to form a second sentence, including:

[0047] First, entities of different types from the first entity are extracted from the context sentences as candidate entities. For example, the context sentences H1 and H2 contain entities {e1, e2, e3, e4} and have corresponding entity types {a, b, c, d}. The first sentence feature contains the first entity e n , the corresponding entity type c, then the candidate entity set T = {e1, e2, e4}. Secondly, based on the POS tag, the trigger word closest to the first entity is extracted from the context sentence as the target trigger word, and the word attention mechanism is used to extract the feasible relationship type between the target trigger word and the first entity.

[0048] Extracting the trigger word closest to the first entity from the context sentence based on the POS tag as the target trigger word includes: extracting the trigger word from the context sentence as the candidate trigger word based on the tag type of the first entity. In this embodiment, the predicate can be extracted as the candidate trigger word based on the POS tag, that is, {v1,v2,…,vn}=v; then calculate the relative distance between each candidate trigger word and the first entity, and select the candidate trigger word with the closest distance as the target trigger word Among them, w i Represents the word vector feature, w i vpos Indicates the part-of-speech feature of the predicate, w i p It indicates the relative distance between the candidate trigger word and the first entity. It should be noted that there is only one relative distance due to the lack of the stem part.

[0049] It should be noted that before using the word attention mechanism to extract the feasible relationship type between the trigger word and the first entity, it is necessary to first represent each relationship instance in the knowledge base as a triple. For example, {W1,…,W m}∈K, m={e1,r,e2}, where K represents the knowledge base, {W1,…W m} represents the instance contained in K, e1 and e2 represent the first entity respectively, e1 and e2 are the first entity pair, r i represents the relationship between e1 and e2, {r1,r2,…,r n}=r, r represents the relationship type.

[0050] Since the target trigger word and the first entity can coexist in multiple semantic environments, there may be multiple different types of relationships between the target trigger word and the first entity. Therefore, a word attention mechanism is used to extract the possible relationship types between the target trigger word and the first entity, namely, the feasibility relationship types. It should be noted that the feasibility relationship type can be a fine-grained relationship type and / or a coarse-grained relationship type between any two entities. Since the description information of the fine-grained relationship type between any two entities is usually composed of multiple words, in this embodiment, the feasibility relationship type only includes the coarse-grained relationship type. Using the word attention mechanism to extract the feasibility relationship type between the target trigger word and the first entity includes: first, according to the target trigger word v i The attention weight u is obtained by the relationship type r in the context sentence i , and then according to the attention weight u i Calculate the weight vector of each target trigger word, and finally calculate the feasibility relationship type based on the weight vector of each target trigger word. The formula is as follows:

[0051] u i =W a (tan[v i ;r])+b a

[0052]

[0053] f i =α i v i

[0054] Where: u i represents the attention weight, [v i ; r] represents the target trigger word v i Correlation with relation type r, W a and b a represents the weight matrix and bias term, α={α1,α2,…,α n} represents the weight vector of each target trigger word in the sentence, f i Indicates the feasibility relationship type.

[0055] Subsequently, based on the remote supervision method and the feasibility relationship type, the candidate entity with the highest similarity is extracted from the candidate entities as the target entity. In this embodiment, the assumption of remote supervision is: in the knowledge base, under the condition that a named entity e1 and a relationship type r are known, the entity e1 has a relationship r with e2, and the triple is expressed as e1+r≈e2. Based on the feasibility relationship type f i Select the entity with the highest similarity to e2 from the candidate entities as the target entity. For example, if g n +f i ≈g m , based on the modified distant supervision assumption and the feasibility relationship type, (g n +f i )=(g p +f i ), then calculate the candidate entity and g m The similarity of g is selected as the target entity. n , g m and g p They are all entity features, which are calculated by adding the corresponding entity’s own features and type features, that is, g n =e n +e ntyple , g m =e m +e mtyple ,g p =e p +e ptyple .

[0056] It should be noted that since the accuracy of the annotations of each first entity in the first sentence is not completely correct, incorrect first entity labels may cause noise problems. In addition, in the subsequent process of extracting the target entity, the accuracy of the target entity also depends to a large extent on the label information of the first entity. The wrong entity type may lead to incorrect semantic information completion. Therefore, after annotating each first entity in the first sentence and before extracting the target entity, the first entity label is also proofread.

[0057] In this embodiment, an entity tag type verification method and Bayesian probability are used to further verify the accuracy of the first entity tag based on the identification of the first entity. The entity tag type verification method includes: inputting the first entity e in the first sentence p ∈e, target trigger word v i ∈v; Output the correct entity type label. Specifically, in the first sentence of the input, the first entity e p ∈e, target trigger word v i ∈v, the first entity in the first sentence is determined to be a single-type entity or a multi-type entity according to the entity recognition structure. p If it is a single-type entity, the first entity is correctly labeled and the proofreading ends;

[0058] If the first entity e p If it is a multi-type entity, then calculate the probability:

[0059]

[0060]

[0061] If the probability e ptype1 >e ptype2 , then judge e p Is the entity type marked as type1? p If the entity type is not marked as type1, change the first entity e p The label is type2.

[0062] Finally, the extracted target entity is concatenated with the first sentence to form the second sentence.

[0063] refer to Figure 2 When the subject and predicate of the first sentence are missing, or the predicate and object are missing, a normal sentence containing the missing semantic information of the first sentence is found, and the first sentence and the normal sentence are concatenated into one sentence, specifically including:

[0064] First, extract the first entity contained in the first sentence, and find the context feature closest to the first entity in the context sentence through the bidirectional long short-term memory network. For example, the first sentence feature s1 = {w1…gi …g o …w m The first entity g in o , then its adjacent above feature is g i , the adjacent following feature is g k , if g i and g k to g o The distances are different, and the sentence containing the feature with the closest distance is extracted as the target normal sentence.

[0065] Secondly, the similarity between the context feature and the first entity is calculated, and the normal sentence containing the context feature with the highest similarity is selected as the target normal sentence. i and g k to g o If the distances are the same, then calculate g o With g i , g o With g k The similarity with g o The sentences with the features with high similarity are taken as the target normal sentences. It should be noted that since we want to extract normal sentences with contextual meanings with the first sentence, rather than semantically similar entities, we pay more attention to the similarity of entity types. In addition, in the experiment, the cosine distance similarity threshold is set to 0.5-0.7, which can generally be 0.62.

[0066] Finally, the first sentence is concatenated with the target normal sentence to form the second sentence.

[0067] Step S03: extract the feature vector in the second sentence, and input the feature vector into the relation extraction model to obtain the relation label type output by the relation extraction model, wherein the relation extraction model is trained based on the semantic completion sentences and normal sentences in the training corpus, referring to Figure 3 .

[0068] In this embodiment, the feature vector includes entity features, word features and position features, and extracting the feature vector in the second sentence includes: extracting the entity feature by adding the self feature and type feature of the second entity in the second sentence, such as g p =e p +e ptyple ; Use the word2vec model to convert the non-entity words in the second sentence into d-dimensional word vectors to extract word features. For example, if the second sentence S2 = {W1…e i …e o …W m}, where {W1…W m} is a word in S2, e i and e oThe second entity in the second sentence S2 is converted into a word vector by using the word2vec model to extract word features and form a second feature sentence, denoted as s2 = {w1…g i …g o …w m}; extract the position feature by calculating the relative distance between each word vector in the second feature sentence and any two second entity features, such as the second feature sentence s2 = {w1…g i …g o …w m}, each word vector w i There are two relative distances between any two second entity features gi and go, denoted as w i p (i=1,2).

[0069] In this embodiment, a relation extraction model is trained based on a segmented convolutional neural network (PCNN model), and the extracted feature vector is input into the trained relation extraction model to output the relation label type. The relation extraction model includes a convolution layer, a segmented maximum pooling layer, a sentence attention layer, and a classification layer, wherein:

[0070] The convolution layer is used to perform convolution operations on the input feature vector. Specifically, the second statement S2 = {W1…e i …e o …W m}, and its corresponding second feature statement s2={w1…g i …g o …w m},w i is the i-th word W in the second sentence S2 i The word vector of g i is the i-th second entity e in the second sentence S2 i The second entity characteristic, w i ∈R d , R d represents the dimension of the word vector, h represents the length of the convolution kernel, and c represents the feature map after the convolution operation. i Represented as c i =f(V·w i:i+h-1 + b), C=[c1,…c i ,…,c n ] T ∈R m(n-h+1) , m represents a series of convolution kernels, m>1, where f(·) represents a nonlinear activation function. In this embodiment, the commonly used ReLU function can be used, and w i:i+h-1Indicates the vector concatenation operation from word vector i to i+h-1, and b represents the bias term. In order to prevent overfitting, a dropout strategy is added during the convolution process, and zero padding is used to maintain the validity of the second sentence.

[0071] The segmented maximum pooling layer convolves the feature map c according to the positions of any two second entity features, i.e., the second entity pairs, in the second sentence. i Divided into three parts [c i1 ,c i2 ,c i3 ], calculate and extract the maximum value p of each part ij =max(c ij ), 1≤i≤n, to obtain the most important feature vector in the second sentence, and then combine the feature vectors of the three parts as the output of the segmented maximum pooling layer, denoted as p∈R 3n , p∈(p1…p n ). The variables of each of the three parts are pooled and the maximum value is taken. The feature of this segmented maximum pooling takes into account the position information of the second entity pair, so as to better represent the entity later.

[0072] The sentence attention layer derives sentence features based on the output of the segmented maximum pooling layer and the semantic relationship between the word and entity pairs in the second sentence, including:

[0073] First, the semantic distance and position distance between the word vector in the second sentence and any two second entity features are calculated.

[0074] The semantic distance calculation method between the word vector and the second entity pair includes:

[0075]

[0076]

[0077] The method for calculating the position distance between the word vector and the second entity pair includes:

[0078]

[0079]

[0080] where g p and g q represents any two second entity features in the second sentence, w i Represents the word vector in the second sentence, Pg p and Pg qIndicates the position of the second entity pair. It should be noted that in a sentence, the closer the semantic and position distance between a word and an entity pair is, the higher its importance coefficient is. It can be seen that in this embodiment, important semantic information related to the relationship expression can be obtained by calculating the semantic distance and the position distance.

[0081] Secondly, according to the semantic distance and position distance, each word vector w is calculated separately i The corresponding weight matrix α i , α i ∈[0,1], and α i The calculation formula is as follows:

[0082]

[0083]

[0084]

[0085]

[0086] Afterwards, based on the weight matrix α of each word vector i Get the weighted word vector WE corresponding to each word vector i and the weighted position vector PE i , based on the weighted word vector WE i and the weighted position vector PE i Respectively expressed as:

[0087]

[0088]

[0089] Then based on the weighted word vector WE corresponding to all word vectors i and the weighted position vector PE i The sentence vector is calculated and expressed as follows:

[0090]

[0091] Then, the bias term b is obtained based on the output p of the maximum layer of the segmented pooling s , b s =tan h(p), where tan h represents the activation function and p is the output of the segmented maximum pooling layer.

[0092] Finally, the bias term b is extracted based on the sentence vector s' and the PCCN model s Calculate and output the sentence feature s, which is expressed as: s = W s s'+bs , where W s is the weight matrix of sentence features, and s' is the second feature sentence input into the relation extraction model.

[0093] The classification layer is used to input the sentence feature s and perform calculations based on the sentence feature s to output the relationship label type. Specifically, the sentence feature s is put into the softmax classification layer of the PCNN model to predict the relationship label for calculation to output the predicted relationship label type. The calculation is as follows:

[0094]

[0095] in, Represents the predicted relationship label type, y represents the relationship label, n represents the type of relationship type label i∈[1,n], W is the prediction weight matrix, s is the sentence feature, and b is the prediction bias item.

[0096] The present invention also provides a training method for a relation extraction model, which is applied to the above-mentioned relation extraction method based on the attention mechanism. The training method for the relation extraction model includes:

[0097] First, a training corpus is constructed based on a remote supervision method, wherein the training corpus includes normal sentences and / or semantically completed sentences. It should be noted that the semantically completed sentences are normal sentences formed after semantically completed sentences with missing semantics.

[0098] Secondly, the relation extraction model is trained according to the input training sentence to output the predicted relation label type. It should be noted that the specific implementation of inputting the training sentence into the relation extraction model can refer to the specific implementation of inputting the second sentence into the relation extraction model mentioned above, which will not be repeated here.

[0099] Finally, the loss function is obtained according to the predicted relationship label type and the real relationship label type of the training sentence. If the loss function remains stable, the training ends; otherwise, the relationship extraction model continues to be trained. It should be noted that after the relationship extraction model outputs the predicted label type, the classification layer calculates the loss function according to the real relationship label type corresponding to the training sentence and the predicted label type output by the relationship extraction model. The loss function is calculated as follows:

[0100]

[0101] Among them, n represents the type of relationship type label i∈[1,n], y i Indicates the real relationship label type, Indicates the true relationship label type y i and predicted relationship label type The cross entropy between represents the loss function.

[0102] Loss Function The smaller it is, the higher the model accuracy. When the loss function is calculated Afterwards, according to the calculated loss function Determine whether to end the training. If the loss function remains stable, the training ends; otherwise, continue to train the relationship extraction model.

[0103] In summary, the embodiment of the present invention pre-processes the text to be processed to form a first sentence, so as to automatically annotate the entity relationship by aligning the unannotated corpus with the entities in the knowledge base, thereby improving the annotation efficiency; then the first sentence is recognized and semantically completed to form a second sentence, so as to facilitate the subsequent relationship extraction, avoiding the erroneous relationship extraction caused by directly extracting the relationship of short sentences with incomplete semantics and the time-consuming and labor-intensive problems caused by manually annotating these sentences; then the feature vector in the second sentence is extracted and input into the trained relationship extraction model to obtain the relationship label type, thereby realizing the relationship extraction of semantically incomplete sentences, improving the extraction efficiency and extraction accuracy, reducing the influence of noise in the relationship extraction process, and expanding the application field of remote supervision.

[0104] Figure 4 An example of a relationship extraction system based on an attention mechanism is provided. The system is implemented based on the above-mentioned relationship extraction method based on the attention mechanism, and the system includes:

[0105] A data preprocessing module 1, used for preprocessing the text to be processed to form a first sentence;

[0106] The short sentence processing module 2 is used to identify the first sentence and semantically complete it to form a second sentence;

[0107] Feature extraction module 3, used to extract feature vectors in the second sentence;

[0108] The relation extraction model module 4 outputs the relation label type according to the feature vector extracted by the feature extraction module.

[0109] In this embodiment, the text to be processed is first preprocessed by the data preprocessing module 1, and the preprocessing includes: dividing the text to be processed into Chinese character strings according to the entities, punctuation marks, numbers and space characters contained in the unlabeled corpus and the knowledge base, and removing stop words to form a first sentence.

[0110] Then, the short sentence processing module 2 includes a recognition unit and a semantic completion unit. The recognition unit is used to parse the syntactic structure of the first sentence and learn the contextual features of the first entity in the first sentence, so as to obtain the missing semantic information of the first sentence. The semantic completion unit performs semantic completion processing on the first sentence according to the missing semantic information to form a second sentence.

[0111] It should be noted that after forming the second sentence, the second sentence and the normal sentence are constructed to form a training corpus. The feature extraction module 3 extracts a training sentence feature vector from the training corpus, and the feature vector includes entity features, word features and position features.

[0112] Subsequently, the relation extraction model module 4 includes a relation convolution layer, a segmented maximum pooling layer, a sentence attention layer and a classification layer, wherein the convolution layer is used to perform a convolution operation on the input feature vector; the segmented maximum pooling layer divides the convolved feature map into three parts according to the position of the second entity pair in the second sentence, calculates and extracts the maximum value of each part to obtain the most important feature vector in the second sentence, and then combines the feature vectors of the three parts as the output of the segmented maximum pooling layer; the sentence attention layer calculates and extracts important semantic information related to the relational expression according to the semantic and position distance between the word and entity pair in the second sentence, and obtains sentence features according to the important semantic information; the classification layer is used to input sentence features and calculate according to the sentence features to output the relational label type. It should be noted that in the training relation extraction model module stage, the classification layer is also used to calculate the loss function of the output relational label type and the true label type, and judge whether to end the training according to whether the loss function remains stable. The specific method can refer to the above method embodiment, which will not be repeated here.

[0113] In summary, the embodiment of the present invention preprocesses the text to be processed through a data preprocessing module to form a first sentence, so as to automatically mark the entity relationship by aligning the unlabeled corpus with the entities in the knowledge base, thereby improving the marking efficiency; then the short sentence processing module recognizes the first sentence and completes its semantics to form a second sentence, so as to facilitate the subsequent relationship extraction, thereby avoiding the erroneous relationship extraction caused by directly extracting the relationship of the semantically incomplete short sentences and the time-consuming and labor-intensive problems caused by manually marking these sentences; then the feature extraction module extracts feature vectors from the second sentence and the normal sentence, and inputs them into the relationship extraction model module to output the relationship label type, thereby facilitating the relationship extraction of the semantically missing short sentences, improving the extraction efficiency and the extraction accuracy, and reducing the influence of noise in the relationship extraction process.

[0114] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5As shown, the electronic device may include: a processor 51, a communication interface 52, a memory 53 and a communication bus 54, wherein the processor 51, the communication interface 52 and the memory 53 communicate with each other through the communication bus 54. The processor 51 may call the logic instructions in the memory 53 to execute the relationship extraction method based on the attention mechanism, the method comprising: preprocessing the text to be processed to form a first sentence; identifying the first sentence and semantically completing the first sentence to form a second sentence; extracting the feature vector in the second sentence and inputting the feature vector into a pre-trained relationship extraction model to obtain the relationship label type output by the relationship extraction model, wherein the relationship extraction model is trained based on the semantically completed sentences and normal sentences in the training corpus.

[0115] In addition, the logic instructions in the above-mentioned memory 53 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0116] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the relationship extraction method based on the attention mechanism provided by the above methods, and the method includes: preprocessing the text to be processed to form a first sentence; identifying the first sentence, and semantically completing the first sentence to form a second sentence; extracting a feature vector in the second sentence, and inputting the feature vector into a pre-trained relationship extraction model to obtain the relationship label type output by the relationship extraction model, wherein the relationship extraction model is trained based on semantically completed sentences and normal sentences in a training corpus.

[0117] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned attention mechanism-based relationship extraction methods, the method comprising: preprocessing the text to be processed to form a first sentence; identifying the first sentence, and semantically completing the first sentence to form a second sentence; extracting a feature vector in the second sentence, and inputting the feature vector into a pre-trained relationship extraction model to obtain a relationship label type output by the relationship extraction model, wherein the relationship extraction model is trained based on semantically completed sentences and normal sentences in a training corpus.

[0118] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A relation extraction method based on attention mechanism, characterized in that: include: Preprocessing the text to be processed to form a first sentence; Recognizing the first sentence, and performing semantic completion on the first sentence to form a second sentence; extracting a feature vector from the second sentence, and inputting the feature vector into a relation extraction model to obtain a relation label type output by the relation extraction model, wherein the relation extraction model is trained based on semantic completion sentences and normal sentences in a training corpus, The step of identifying the first sentence and performing semantic completion on the first sentence to form a second sentence includes: Identifying the syntactic structure of the first sentence and a first entity in the first sentence by parsing; Learning context features of the first entity in the first sentence to obtain semantic information missing from the first sentence; Performing semantic completion processing on the first sentence according to the missing semantic information to form a second sentence, The performing corresponding semantic completion processing on the first sentence according to the missing semantic information to form a second sentence includes: If the missing semantic information is that the subject and predicate are missing, or the predicate and object are missing, then find a normal sentence containing the missing semantic information of the first sentence, and concatenate the first sentence and the normal sentence to form a second sentence. The step of finding a normal sentence containing the missing semantic information of the first sentence and concatenating the first sentence and the normal sentence into one sentence includes: Extracting a first entity contained in the first sentence; Find the context feature in the context sentence that is closest to the first entity, calculate the similarity between the context feature and the first entity, and select the normal sentence containing the context feature with the highest similarity as the target normal sentence; The first sentence and the target normal sentence are concatenated head to tail to form a second sentence.

2. The relationship extraction method based on the attention mechanism according to claim 1, It is characterized in that in, The performing corresponding semantic completion processing on the first sentence according to the missing semantic information to form a second sentence includes: If the missing semantic information is the missing subject or object, then find the entity containing the missing semantic information of the first sentence as the target entity, and concatenate the target entity and the first sentence to form a second sentence. The step of finding an entity containing the missing semantic information of the first sentence as a target entity and concatenating the target entity with the first sentence to form a second sentence includes: Extracting entities of different types from the first entity from context sentences of normal sentences where the first entity is located in the knowledge base as candidate entities; Extracting a trigger word closest to the first entity from the context sentence based on the tag type as a target trigger word, and using a word attention mechanism to extract a feasible relationship type between the target trigger word and the first entity; Extracting the candidate entity with the highest similarity from the candidate entities as the target entity based on the remote supervision method and the feasible relationship type; The target entity is concatenated with the first sentence to form a second sentence.

3. The relationship extraction method based on the attention mechanism according to claim 1, characterized in that: After identifying the first sentence by parsing the syntactic structure of the first sentence, the method further includes: tagging the first entity using a part-of-speech tag mechanism; After semantic completion processing is performed on the first sentence according to the missing semantic information, the method further includes: proofreading the entity tag annotated with the first entity.

4. The relationship extraction method based on the attention mechanism according to claim 1, characterized in that: The relation extraction model includes a convolutional layer, a segmented maximum pooling layer, a sentence attention layer and a classification layer, wherein: The convolution layer is used to perform a convolution operation on the input feature vector; The segmented maximum pooling layer divides the feature vector after the convolution operation into three parts according to the positions of the entity pairs in the second sentence, performs segmented pooling on each part and calculates the maximum value thereof respectively, and combines the maximum values ​​of the three parts for output; A sentence attention layer, which derives sentence features based on the output of the segmented maximum pooling layer and the semantic relationship between the word and entity pairs in the second sentence; The classification layer is used to input the sentence features and perform calculations based on the sentence features to output a relationship tag type.

5. The relationship extraction method based on the attention mechanism according to claim 1, characterized in that: Training the relation extraction model includes: Building a training corpus based on a remote supervision method, and inputting sentences in the training corpus as training sentences into the relation extraction model, wherein the training corpus includes normal sentences and / or semantic completion sentences; The relation extraction model is trained according to the input training sentence to output the predicted relation label type; A loss function is obtained according to the predicted relationship label type and the actual relationship label type of the training sentence. If the loss function remains stable, the training ends; otherwise, the relationship extraction model continues to be trained.

6. A relation extraction system based on attention mechanism, characterized in that: The system is implemented based on the attention mechanism-based relationship extraction method according to any one of claims 1 to 5, and the system includes: A data preprocessing module, used for preprocessing the text to be processed to form a first sentence; A short sentence processing module, used for identifying the first sentence and completing its semantics to form a second sentence; A feature extraction module, used for extracting a feature vector from the second sentence; The relation extraction model module outputs a label type according to the feature vector extracted by the feature extraction module.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the relationship extraction method based on the attention mechanism as described in any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the attention-based relationship extraction method as described in any one of claims 1 to 5 are implemented.

Images