A remote supervised relation extraction method based on shared semantic space

By mapping entity pairs and relationship types in shared semantic space, the semantic drift and noise problems in remote supervision relationship extraction are solved, the accuracy of relationship extraction is improved, and the accurate identification of complex relationships is achieved.

CN114281941BActive Publication Date: 2024-12-03SUZHOU AEROSPACE INFORMATION RES INST
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202111513041.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-11
Publication Date
2024-12-03
Estimated Expiration
2041-12-11

AI Technical Summary

Technical Problem

The existing remote supervision relationship extraction method has serious semantic drift and high data noise when dealing with complex relationships, resulting in low accuracy of relationship extraction results and failure to effectively distinguish between noisy and noisy data for denoising.

Method used

Using a shared semantic space-based method, the entity pairs and relationship types are mapped to the same semantic space. Through the separate modeling of the noise-free and noisy data sets, the mapping matrix of entity pairs and relationship types is learned, and the similarity score is calculated to determine the Relationship type, reduces noise impact and improves accuracy.

Benefits of technology

The impact of noise on the relationship extraction results is effectively reduced, the accuracy of relationship extraction is improved, and the results are extracted through the relationship type correlation measurement constraints are restrained, and the impact of noise on clean data is avoided.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114281941B_ABST
    Figure CN114281941B_ABST
Patent Text Reader

Abstract

The present invention proposes a remote supervision relationship extraction method based on a shared semantic space, which divides the remote supervision annotated data into a noise-free data set and a noise-containing data set; splices the sentence vector of the sentence and the position vector of the entity pair as the input vector of the BiLSTM network, respectively extracts the entity pair features in the noisy data set and the noise-free data set; maps the entity pair features and the relationship type representation to the same semantic space; counts the number of entity categories shared by the two relationship types, calculates the correlation between the relationship types, models the noise-free data set and the noisy data set respectively, and learns the entity pair mapping matrix and the relationship type mapping matrix; maps the entity pair and the relationship type to the shared semantic space through the learned mapping matrix, calculates the similarity score of the entity pair and the relationship type, and the relationship type with the highest score is the relationship of the entity pair. The present invention solves the problems of serious semantic drift and large data noise when processing complex relationships.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a remote supervision relationship extraction method based on a shared semantic space. Background Art

[0002] Relation extraction is a hot research direction in the field of natural language processing. At present, the relationship extraction task generally has problems such as insufficient annotated corpus and low corpus quality. Remote supervision technology can effectively solve the corpus annotation problem. However, the corpus annotated by remote supervision has problems such as annotation errors and high data noise. Especially for complex relations, such as when a pair of entities have multiple relations, semantic drift is serious and data noise seriously affects the results of relationship extraction. In the prior art, most methods use segmented convolution to reduce the impact of data noise. For example, a sentence is divided into three parts: front, middle and back according to two entities, and then the structural information of the entity pair in the sentence is obtained by segmented pooling, thereby alleviating the impact of data noise. However, the above methods have two main problems: (1) they do not consider focusing on denoising data with complex relations; (2) they denoise all data in the data set in general, without modeling noisy and non-noisy data separately. The above two problems are important reasons for the low accuracy of remote supervision relationship extraction results. Summary of the invention

[0003] The purpose of the present invention is to propose a remote supervised relationship extraction method based on a shared semantic space to solve the problems of serious semantic drift and large data noise in traditional remote supervised relationship extraction when dealing with complex relationships.

[0004] The technical solution to achieve the purpose of the present invention is: a remote supervision relationship extraction method based on a shared semantic space, comprising the following steps:

[0005] Step 1. Dataset division

[0006] The data annotated by remote supervision is divided. If an entity pair has only one relationship in the entire dataset, the sentence containing the entity pair is divided into a noise-free dataset; if an entity pair has multiple relationships in the entire dataset, the sentence containing the entity pair is divided into a noisy dataset.

[0007] Step 2. Extract entity pair features in sentences

[0008] The sentence vector and the position vector of the entity pair are concatenated as the input vector of the BiLSTM network to extract the entity pair features in the noisy and non-noisy datasets respectively;

[0009] Step 3. Construct a shared semantic space of entity pairs and relationship types

[0010] Map the entity pair features and relationship type representations output by the BiLSTM network into the same semantic space;

[0011] Step 4. Learn the shared semantic space mapping matrix

[0012] Count the number of entity categories shared by the two relation types, calculate the correlation between relation types, and model the noise-free and noisy datasets respectively based on hypothesis 1: "The similarity between sentence entity pairs and true relation types in the noise-free dataset is greater than the similarity between them and other relation types" and hypothesis 2: "The similarity between entity pairs and candidate relation types in the noisy dataset is greater than the similarity between them and other non-candidate relation types", and learn the entity pair mapping matrix and the relation type mapping matrix;

[0013] Step 5. Relation Extraction

[0014] Extract the entity pair features in the sentence, map the entity pairs and relationship types to the shared semantic space through the learned mapping matrix, calculate the similarity scores of the entity pairs and relationship types, and the relationship type with the highest score is the relationship of the entity pair.

[0015] Furthermore, in step 1, the relationship between the data sets is expressed as:

[0016] s=s c ∪s n (1)

[0017] Among them, s c represents a noise-free dataset, s n represents a noisy dataset, and s represents the data result of remote supervision annotation.

[0018] Furthermore, in step 2, the position vector refers to the vectorized representation of the position distance between each word in the sentence and the two entities.

[0019] Furthermore, in step 3, the formula for mapping the entity pair feature vectors in the sentence into the shared semantic space is:

[0020] φ M (m i )=Um i (2)

[0021] The formula for mapping the relation type vector into the shared semantic space is:

[0022] φ Y (y k )=Vy k (3)

[0023] Among them, U and V represent the mapping matrices of sentence entity pair feature vector and relation type vector in the shared semantic space, respectively, and m i Represents the vector representation of the feature of the i-th entity pair, y k The vector representation of the kth relationship type, φ M (.) and φ Y (.) represent the mapping functions of entities to features and relationship types respectively.

[0024] Furthermore, in step 4, the specific method of learning the shared semantic space mapping matrix is:

[0025] 1) Calculate the correlation between relationship types. The formula is:

[0026] w kk` =(|ε k ∩ε k` | / |ε k |+|ε k ∩ε k` | / |ε k` |) / 2 (4)

[0027] Among them, w kk` Indicates the correlation of different relationship types, ε k and ε k` Represents the set of entity types contained in different relationship types, and the subscript k represents the kth relationship type;

[0028] 2) Model the noise-free dataset and the noisy dataset separately, and learn the entity pair mapping matrix and relationship type mapping matrix initialized in the semantic space;

[0029] In the shared semantic space, the similarity calculation formula for the entity pairs and relationship types of sentences is:

[0030] f k (m i )=m i T U T Vy k (5)

[0031] Among them, f k (m i ) represents the similarity between the i-th entity pair and the k-th relation type, U and V represent the mapping matrices of the sentence entity pair feature vector and the relation type vector in the shared semantic space, respectively, and m i Represents the vector representation of the feature of the i-th entity pair, y k The vector representation of the k-th relationship type;

[0032] According to hypothesis 1: "The similarity between sentence entity pairs and the true relation type is greater than the similarity between them and other relation types", the loss function is modeled for the noise-free dataset:

[0033]

[0034] in, S represents the softmax function, y i represents the i-th real relationship type, represents the kth other relationship type, f k (m i ) represents the similarity between the entity pair and the true relationship type, The similarity between entity pairs and other relation types, m i represents an entity pair, Represents the correlation distance between relationship types, and the formula is:

[0035]

[0036] in, is the random coefficient;

[0037] According to Assumption 2: "The similarity between entity pairs and candidate relation types in the noisy dataset is greater than the similarity between entity pairs and other non-candidate relation types", the loss function is modeled as follows:

[0038]

[0039] in, Represents the relation type with the highest similarity score between entity pair and relation type among the candidate relation types.

[0040] Furthermore, in step 5, the model representation of sentence relation extraction is:

[0041]

[0042] Among them, y k* represents the extracted relationship type, Y represents the relationship type set, and f k (m i ) represents the similarity between the i-th entity pair and the k-th relationship type, m i Represents the vector representation of the feature of the i-th entity pair, y k A vector representation of the k-th relationship type.

[0043] A remote supervision relationship extraction system based on a shared semantic space realizes remote supervision relationship extraction based on a shared semantic space based on the remote supervision relationship extraction method based on a shared semantic space.

[0044] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, remote supervision relationship extraction based on a shared semantic space is implemented based on the remote supervision relationship extraction method based on a shared semantic space.

[0045] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements remote supervision relationship extraction based on a shared semantic space based on the remote supervision relationship extraction method based on a shared semantic space.

[0046] Compared with the prior art, the present invention has the following significant advantages: 1) a shared semantic space is used for relation extraction of noisy data. This method maps entity pair features and relation types to the same semantic space, establishes the degree of association between entity pair semantic information and relation types, and effectively reduces the impact of noisy data on relation extraction results; 2) a relation type correlation measurement coefficient is used, which can constrain the extracted relation types to a certain extent, thereby improving the accuracy of relation extraction results; 3) a method of separately modeling the noise-free data set and the noisy data set avoids the impact of noisy data on the relation extraction of the noise-free data corpus. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a flow chart of a remote supervision relationship extraction method based on a shared semantic space of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] like Figure 1 As shown, a remote supervision relationship extraction method based on a shared semantic space includes:

[0050] Step 1. Divide the data results labeled by remote supervision. In the data results labeled by remote supervision, there may be a case where a pair of entities is labeled with multiple relationships. The data with only one relationship labeled for the entity pair is divided into a noise-free data set; the data with multiple relationships labeled for the entity pair is divided into a noisy data set.

[0051] s=s c ∪s n (1)

[0052] Among them, s c represents a noise-free dataset, s nrepresents a noisy dataset, and s represents the data result of remote supervision annotation.

[0053] Step 2. Extract entity pair features in the corpus. Use the BiLSTM network to extract entity pair features in sentences. The network input uses the concatenation of sentence vectors and position vectors, where the position vector refers to the vectorized representation of the position distance between each word in the sentence and two entities.

[0054] Step 3. Construct a shared semantic space for entity pairs and relation types. The entity pair feature vector and the relation type vector representation are m i and k , the formula for mapping the entity pair feature vector in the sentence into the shared semantic space is:

[0055] φ M (m i )=Um i (2)

[0056] The formula for mapping the relationship type vector into the shared semantic space is:

[0057] φ Y (y k )=Vy k (3)

[0058] Among them, U and V represent the mapping matrices of sentence entity pair feature vectors and relationship type vectors in the shared semantic space, respectively, i represents the i-th entity pair, k represents the k-th relationship type, M represents the entity pair set, and Y represents the relationship type set.

[0059] Step 4. Learn the shared semantic space mapping matrix.

[0060] 1) Calculate the correlation between relationship types. Through the open source knowledge base Freebase, count the number of entity types shared by different relationship types. The formula for calculating the correlation between relationship types is:

[0061] w kk` =(|ε k ∩ε k` | / |ε k |+|ε k ∩ε k` | / |ε k` |) / 2 (4)

[0062] Among them, is ε k and ε k` Represents the set of entity types contained in different relationship types, k represents the kth relationship type, and the more entity types two relationships share, the more similar they are.

[0063] 2) Model the noise-free dataset and the noisy dataset separately, and learn the entity pair mapping matrix and relationship type mapping matrix initialized in the semantic space.

[0064] In the shared semantic space, the similarity calculation formula for the entity pairs and relationship types of sentences is:

[0065] f k (m i )=m i T U T Vy k (5)

[0066] According to hypothesis 1: "The similarity between sentence entity pairs and the true relation type is greater than the similarity between them and other relation types", the loss function is modeled for the noise-free dataset:

[0067]

[0068] in, S represents the softmax function, y i represents the i-th real relationship type, represents the kth other relationship type, f k (m i ) represents the similarity between the entity pair and the true relationship type, The similarity between entity pairs and other relation types, m i represents an entity pair, Represents the correlation distance between relationship types, and the formula is:

[0069]

[0070] Among them, w kk` Represents the correlation between different relationships, is a random coefficient.

[0071] According to Assumption 2: "The similarity between entity pairs and candidate relation types in the noisy dataset is greater than the similarity between entity pairs and other non-candidate relation types", the loss function is modeled as follows:

[0072]

[0073] in, Represents the relation type with the highest similarity score between entity pair and relation type among the candidate relation types.

[0074] Step 5. Sentence relationship extraction. Extract entity pair features in the sentence, map entity pairs and relationship types to the semantic space through the learned mapping matrix, calculate the similarity scores of entity pairs and relationship types, and the relationship type with the highest score is the relationship of the entity pair.

[0075]

[0076] Among them, y k* represents the extracted relationship type, Y represents the relationship type set, and f k (m i ) represents the similarity between the i-th entity pair and the k-th relationship type, m i Represents the vector representation of the feature of the i-th entity pair, y k A vector representation of the k-th relationship type.

[0077] The present invention also proposes a remote supervision relationship extraction system based on a shared semantic space, and based on the remote supervision relationship extraction method based on a shared semantic space, remote supervision relationship extraction based on a shared semantic space is implemented.

[0078] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, remote supervision relationship extraction based on a shared semantic space is implemented based on the remote supervision relationship extraction method based on a shared semantic space.

[0079] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements remote supervision relationship extraction based on a shared semantic space based on the remote supervision relationship extraction method based on a shared semantic space.

[0080] Example

[0081] In order to verify the effectiveness of the scheme of the present invention, the following experiment was carried out.

[0082] Given an input sentence s = {In Bolivia, President Evo Morales is trying to use his country's gas reservoirs to force neighboring Chile to give his landlocked country a corridor to the sea.}, the entity pair in the sentence {m1, m2} = {Bolivia, Evo Morales}; the candidate relationship type set is {capital, founders, nationality}; the entity pair mapping matrix U = [0.010386 0.114786 0.090944 0.107563 0.247448 0.080917 0.089822 0.2116910.495511 0.025301 0.354232 0.041315 0.239092 0.069313 0.081385 0.045300 0.033915 0.117850 0.164983 0.513390]; Relationship type mapping matrix V = [0.268424 0.629322 0.354320 0.059896 0.501595 0.383668 0.646036 0.546118 0.008314 0.052644 0.124476 0.383355 0.157523 0.268128 0.028797 0.202146 0.013243 0.320547 0.142381 0.000518].

[0083] Step 1. Extract entity pair features in the sentence. Convert sentence s into a sentence vector [0.087197 -0.0834350.057956 0.143120 -0.000068 0.123272 0.022439 -0.323317 -0.231756 -0.009262 -0.211264 -0.021698 0.246367 0.179090 0.054117 0.077638 -0.054555 -0.0506300.072361 The distance information between each word in the sentence and the two entities m1 and m2 are pos1:[-1,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28], pos2[-4,-3,-2,-1,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]. Then the sentence vector and position vector of the sentence are concatenated and input into the BiLstm network, and the entity pair features in the sentence are obtained as [0.214860 -0.215249 0.203039 -0.085382 -0.037741 0.207704 -0.114944 -0.202977-0.163324 -0.120949 -0.193239 0.139462 0.070843 0.157681 -0.134875 0.0879910.067728 -0.053001 -0.016407 0.054860].

[0084] Step 2. Map entity pair features and relationship types to the same semantic space based on the learned entity pair mapping matrix U and relationship type mapping matrix V. The entity pair features are projected into the semantic space as [0.174329 0.1198410.063231 0.205474 0.214166 0.184942 0.425125 0.263568 0.063927 0.0468080.132366 0.579423 0.004075 0.326017 0.062847 0.291170 0.256007 0.1831600.122336 0.108601]; the relationship type is mapped into the semantic space as [0.109005 0.2937360.041128 0.280210 0.204642 0.017830 0.425420 0.007375 0.002672 0.294175 0.269443 0.158977 0.096397 0.114196 0.280676 0.093720 0.047810 0.202099 0.053905 0.017911], [0.030103 0.225203 0.269126 0.018275 0.049839 0.064108 0.057102 0.017911 0.288971 0.054805 0.018620 0.046796 0.178140 0.1117550.080138 0.1616230.123392 0.102861 0.059298 0.414017], [0.069387 0.2638680.027413 0.045209 0.017854 0.181368 0.118309 0.043782 0.173138 0.1344000.009886 0.100458 0.022064 0.072146 0.207590 0.0324870.439067 0.4681150.366526 0.082677]

[0085] Step 3. Calculate the similarity between the entity pair and the relationship type [capital founders nationality] and the result is [0.79 0.80 0.87]. Select nationality, which corresponds to the maximum similarity value of 0.87, as the relationship between the two entities in the sentence.

[0086] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A remote supervised relation extraction method based on a shared semantic space, characterized in that: The following steps are involved: Step 1. Dataset division The data annotated by remote supervision is divided. If an entity pair has only one relationship in the entire dataset, the sentence containing the entity pair is divided into a noise-free dataset; if an entity pair has multiple relationships in the entire dataset, the sentence containing the entity pair is divided into a noisy dataset. Step 2. Extract entity pair features in sentences The sentence vector and the position vector of the entity pair are concatenated as the input vector of the BiLSTM network to extract the entity pair features in the noisy and non-noisy datasets respectively; Step 3. Construct a shared semantic space of entity pairs and relationship types Map the entity pair features and relationship type representations output by the BiLSTM network into the same semantic space; Step 4. Learn the shared semantic space mapping matrix Count the number of entity categories shared by the two relation types, calculate the correlation between relation types, and model the noise-free and noisy datasets respectively based on hypothesis 1 "the similarity between sentence entity pairs and the true relation type in the noise-free dataset is greater than the similarity between them and other relation types" and hypothesis 2 "the similarity between entity pairs and candidate relation types in the noisy dataset is greater than the similarity between them and other non-candidate relation types", and learn the entity pair mapping matrix and the relation type mapping matrix; Step 5. Relation Extraction Extract entity pair features in the sentence, map entity pairs and relationship types to a shared semantic space through the learned mapping matrix, calculate the similarity scores of entity pairs and relationship types, and the relationship type with the highest score is the relationship of the entity pair; in, In step 3, the formula for mapping the entity pair feature vectors in the sentence into the shared semantic space is: φ M (m i )=Um i (2) The formula for mapping the relation type vector into the shared semantic space is: φ Y (y k )=Vy k (3) Among them, U and V represent the mapping matrices of sentence entity pair feature vector and relation type vector in the shared semantic space, respectively, and m i Represents the vector representation of the feature of the i-th entity pair, y k The vector representation of the kth relationship type, φ M (.) and φ Y (.) represent the mapping functions of entity to feature and relationship type respectively; In step 4, the specific method of learning the shared semantic space mapping matrix is: 1) Calculate the correlation between relationship types. The formula is: w kk、 =(|e k ∩ k、 | / |e k |+|e k ∩ k、 | / |e k、 |) / 2 (4) Among them, w kk、 Indicates the correlation of different relationship types, ε k and ε k、 Represents the set of entity types contained in different relationship types, and the subscript k represents the kth relationship type; 2) Model the noise-free dataset and the noisy dataset separately, and learn the entity pair mapping matrix and relationship type mapping matrix initialized in the semantic space; In the shared semantic space, the similarity calculation formula for the entity pairs and relationship types of sentences is: f k (m i )=m i T U T You k (5) Among them, f k (m i ) represents the similarity between the i-th entity pair and the k-th relation type, U and V represent the mapping matrices of the sentence entity pair feature vector and the relation type vector in the shared semantic space, respectively, and m i Represents the vector representation of the feature of the i-th entity pair, y k The vector representation of the k-th relationship type; According to hypothesis 1: "The similarity between sentence entity pairs and true relation types is greater than the similarity between them and other relation types", the loss function is: in, S represents the softmax function, y i represents the i-th real relationship type, represents the kth other relationship type, f k (m i ) represents the similarity between the entity pair and the true relationship type, The similarity between entity pairs and other relation types, m i represents an entity pair, Represents the correlation distance between relationship types, and the formula is: in, is the random coefficient; According to Assumption 2: "The similarity between entity pairs and candidate relation types in the noisy dataset is greater than the similarity between entity pairs and other non-candidate relation types", the loss function is modeled as follows: in, Represents the relation type with the highest similarity score between entity pair and relation type among the candidate relation types.

2. The remote supervision relationship extraction method based on shared semantic space according to claim 1, characterized in that: In step 1, the relationship between the data sets is expressed as: s=s c ∪s n (1) Among them, s c represents a noise-free dataset, s n represents a noisy dataset, and s represents the data result of remote supervision annotation.

3. The remote supervision relationship extraction method based on shared semantic space according to claim 1, characterized in that: In step 2, the position vector refers to the vectorized representation of the position distance between each word in the sentence and the two entities.

4. The remote supervision relationship extraction method based on shared semantic space according to claim 1, characterized in that: In step 5, the model representation of sentence relation extraction is: Among them, y k* represents the extracted relationship type, Y represents the relationship type set, and f k (m i ) represents the similarity between the i-th entity pair and the k-th relationship type, m i Represents the vector representation of the feature of the i-th entity pair, y k A vector representation of the k-th relationship type.

5. A remote supervised relation extraction system based on shared semantic space, characterized in that: Based on the remote supervision relationship extraction method based on a shared semantic space described in any one of claims 1-4, remote supervision relationship extraction based on a shared semantic space is implemented.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the remote supervision relationship extraction based on the shared semantic space is implemented based on the remote supervision relationship extraction method based on the shared semantic space described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the remote supervision relationship extraction based on a shared semantic space is implemented based on the remote supervision relationship extraction method based on a shared semantic space as described in any one of claims 1 to 4.

Citation Information

Patent Citations

  • Method and system of relation characterizing, clustering and identifying based on the semanteme of semantic space mapping

    CN104008092A

  • Knowledge graph representation learning method for integrating text semantic features based on attention mechanism

    CN110334219A

  • Remote supervision entity relationship extraction method based on denoising convolutional neural network

    CN111078895A

  • Text entity relationship extraction method and model training method

    CN111339774A

  • Remote supervision relation extraction method based on multiple tasks and multiple examples

    CN112395393A