A text entity relationship classification method based on few-shot learning

Abstract

The present invention discloses a text entity relationship classification method based on few-shot learning, comprising the following steps: 1) Using a convolutional neural network as an instance encoder to extract the semantic features of instance vectors in the dataset; 2) In the few-shot learning scenario, by designing a prototype-level attention mechanism module, weights are assigned to each instance, and the prototype of each relationship is represented in a weighted summation manner; 3) In the few-shot learning scenario, a new metric function is replaced. Through the distance-level attention mechanism module, the feature coefficients in the support set vectors are extracted by using convolutional operations, and the product of the Manhattan distance formula and the feature coefficients is used as the new metric function to calculate the distance between each relationship prototype in the support set and the query instance; 4) Using the softmax function to implement few-shot relationship classification.

Description

Technical Field

[0001] The present invention relates to a method for classifying text entity relationships based on few-shot learning, belonging to the technical field of text data recognition. Background Art

[0002] As one of the important subtasks of knowledge extraction, relation classification has received more and more extensive attention. For unstructured text data, the task of relation classification is to extract the semantic relationship between two or more entities from the text. Currently, in the problem of relation classification, most mature technologies achieve excellent experimental results by improving traditional neural network models (such as recurrent neural networks, convolutional neural networks, etc.). However, the datasets selected during the experiment are all simple short sentences with predefined categories, and the sample distribution of each relationship is also relatively uniform. In practical applications, however, challenges such as small data scale and uneven sample distribution are often faced.

[0003] The emergence of the distant supervision method provides a solution for small-scale datasets by aligning plain text with existing knowledge graphs to obtain large-scale training data. The basic assumption of the distant supervision method is that if there is a certain relationship between two entities in the knowledge graph, then the sentences containing the two entities all express this relationship. However, due to the overly strong assumption conditions, the distant supervision dataset contains a large number of mislabeled samples. At the same time, in the real scenario, there are still long-tail distribution characteristics for relationships and entity pairs, and the available samples are few.

[0004] In fact, humans can quickly learn knowledge with fewer samples and have the ability to "draw inferences from one instance", which also applies to deep learning. Researchers have proposed the few-shot learning task. By designing a combination of few-shot learning mechanisms and relation classification, the problem of fewer and unevenly distributed datasets has been effectively solved. The text entity relation classification method based on few-shot learning mainly functions to introduce two attention mechanism modules on the basis of the few-shot learning network framework to achieve relation classification. Currently, for each relation existing in the support set, few-shot relation classification usually takes the way of averaging instance vectors to obtain the relation prototype. Due to the small amount of data in the few-shot learning scenario, when an instance is far from other instances in the mapping space, it will cause a huge deviation in the averaged prototype. Once a large amount of such noisy data exists, it will have a greater impact on the final relation classification effect; at the same time, for a relation feature vector, only a part of the dimensions have obvious discrimination effects on the final classification result. Once the instance vectors extracted from the support set have the problem of feature sparsity, it will cause a large deviation in the final classification result. Therefore, aiming at these two problems that occur in relation classification in the few-shot learning scenario, how to improve the prototype representation of relation instances and how to solve the problem of feature sparsity of relation feature vectors to meet the representation requirements under uneven example distribution are important problems to be solved. Summary of the Invention

[0005] Object of the Invention: Aiming at the problems and deficiencies existing in the prior art, the present invention provides a text entity relation classification method based on few-shot learning, where few-shot learning is a training method proposed specifically for scenarios where data is scarce and difficult to meet the requirements of model training. Aiming at the problem of error in the prototype vector representation in the scenario of a scarce dataset, it is solved by using a prototype-level attention mechanism module; aiming at the problem that the metric function cannot highlight the features of important dimensions in the vector, it is solved by using a distance-level attention mechanism module and replacing the original distance formula.

[0006] Technical Solution: A text entity relation classification method based on few-shot learning includes the following steps:

[0007] Step 1: Use a CNN network as an instance encoder to encode the support set statements and query set statements in the given dataset, and convert them into low-dimensional instance vectors to obtain the entity pair features of the extracted corpus.

[0008] Step 2: In the few-shot learning scenario, originally for each relation existing in the support set, the relation prototype was usually obtained by directly averaging the instance vectors. Now, use the prototype-level attention mechanism module to assign weights to each instance to obtain a weighted prototype vector;

[0009] Step 3: Concatenate the support set instances obtained by encoding in Step 1 into a vector matrix, and extract the semantic features of important dimensions in the support set instances through the distance-level attention mechanism module, so as to obtain the weight β of the distance-level attention mechanism.

[0010] Step 4: Use the Manhattan distance formula as the new calculation formula, and multiply the weight β of the distance-level attention mechanism obtained in Step 3 by the distance formula to obtain a new distance formula as the metric function. Using this formula, the distance between the query instances in the query set and the prototypes obtained in Step 2 can be measured.

[0011] Step 5: Compare the distance between the query instance and the prototype according to the distance formula obtained in Step 4. Use the softmax function for relation classification.

[0012] In Step 1, a CNN network is used as the instance encoder to encode the support set statements and query set statements in the given dataset, which includes the following steps:

[0013] 1-1 Convert the corpus in the input dataset into a low-dimensional word vector form by using Glove word embedding and entity position embedding. Glove word embedding (WF) converts the input corpus into a co-occurrence matrix, with dimension d w represented; represent the sentence obtained after Glove word embedding as a list of vectors (x 0 , x 1 , x 2 ,..., x i ), where x i represents the i-th word embedding, and entity position embedding (PF) calculates the distance between each word embedding vector x m (m ∈ [0, i]) in the vector list to the two head and tail entities in the sentence, with dimension 2*d p represented. Finally, combine Glove word embedding and entity position embedding, represented as {e 1 ,..., e n} = {[WF 1 ; PF 1 ..., [WF n ; PF n}, thus forming a sequence of sentence embedding vectors.

[0014] 1-2 Further process the embedding vector sequence of the final sentence obtained in step 1-1 using a CNN network to extract semantic features therefrom, which can be specifically divided into a convolutional layer and a max pooling layer. Among them, the convolutional layer uses a convolutional sliding window of length m to extract features from the sentence embedding vector sequence, and processes the obtained vector sequence after feature extraction through a ReLU activation function to obtain a sentence hidden embedding; the max pooling layer processes the sentence hidden embedding obtained by the convolutional layer. Finally, an instance vector of the entire sentence is obtained.

[0015] Step 2 uses a prototype-level attention mechanism module to assign weights to each instance, thereby obtaining a weighted prototype vector, which includes the following steps:

[0016] 2-1 Use the Gaussian function as the activation function to obtain the weight γ assigned to each sample instance ij , and finally use the weight to perform a weighted sum on each relationship instance to obtain a prototype-level relationship vector representation c of the entire relationship i , which are respectively expressed as:

[0017]

[0018]

[0019]

[0020] Among them, q j represents the query set instance, M represents the types of relationships existing in the dataset, K represents the number of instances under each relationship type, and x ij represents the j-th support set instance under relationship i, and σ i represents the parameter value of the Gaussian function.

[0021] Step 3 extracts the semantic features of important dimensions in the support set instances through a distance-level attention mechanism module, thereby obtaining the weight β of the distance-level attention mechanism, which includes the following steps:

[0022] 3-1 Divide the instance sentences in the support set according to the relationship, and divide the K support set instances [x i , x i1 , x i2 ,..., x iK in each relationship r h *1 vector matrix, where K represents the number of instances under each relationship type, and d h represents the hidden layer unit.

[0023] 3-2 Pass the vector matrix through a module interwoven with three convolutional layers and three ReLU function layers to extract the semantic features with non-zero dimension values in the support set instances, and the dimension becomes 1*d h *1, thereby obtaining the weight β of the distance-level attention mechanism. The more useful the corresponding feature dimension is, the higher the corresponding β value. Among them, since some dimension values of the instance vector are 0, it is necessary to highlight the important dimensions that are not 0, so as to exert its effect in relation classification.

[0024] The metric formula established in step 4 includes the following steps:

[0025] 4-1 Select the Manhattan distance formula as the distance formula, which can not only eliminate the error in calculation, greatly improve the operation speed, but also perform distance measurement under high-dimensional data to ensure a good classification effect.

[0026] 4-2 Multiply the weight β of the distance-level attention mechanism obtained in step 3-2 by the distance formula selected in 4-1 to obtain the distance function d(x,y) as the new metric function, which is expressed as:

[0027]

[0028] where n represents the dimension, x i and y i respectively represent the values of x and y in the i-th dimension

[0029] The use of the softmax function for relationship classification established in step 5 includes the following steps:

[0030] 5-1 According to the prototype-level relationship vector representation c obtained in step 2-2 i and the distance function d(x,y) in step 4-2, calculate the distance between a certain instance in the query set and c i between represents the vector obtained after the query set instance x passes through the instance encoding layer.

[0031] 5-2 For the instances in the query set, judge which relationship in the relationship set R they specifically belong to. Specifically expressed as:

[0032]

[0033] where the conditional probability is the probability of the query set instance x under the relationship r i under.

[0034] A computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the above-mentioned text entity relationship classification method based on few-shot learning is implemented.

[0035] A computer-readable storage medium stores a computer program for executing the above-mentioned text entity relationship classification method based on few-shot learning.

[0036] Compared with the prior art, the advantages of the present invention are as follows:

[0037] (1) The relationship classification method obtains prototypes by using a prototype-level attention mechanism, which can eliminate the influence of extreme data on the overall relationship prototype representation. At the same time, a Gaussian function is used as the activation function, which is different from general activation functions and is more suitable for processing data with small differences. In addition, the Gaussian function curve fits the characteristics of the long-tail distribution of few-shot data and is more suitable for the few-shot relationship classification task;

[0038] (2) The relationship classification method obtains feature coefficients by using a distance-level attention mechanism, which can emphasize the important dimensions in the relationship vector, play its role in the final relationship classification, and solve the problem of feature sparsity;

[0039] (3) The relationship classification method uses the Manhattan distance formula as the metric formula. Different from the Euclidean distance formula used in the past, it solves the problem of the Euclidean distance failing in high-dimensional data. At the same time, it effectively eliminates the error in calculation and greatly improves the operation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is the overall framework diagram of the method of the embodiment of the present invention;

[0041] Figure 2 is the flowchart of the prototype-level attention mechanism module of the embodiment of the present invention;

[0042] Figure 3 is the framework diagram of the distance-level attention mechanism module of the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0043] The present invention will be further illustrated below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. After reading the present invention, various equivalent forms of modification of the present invention by those skilled in the art fall within the scope defined by the appended claims of this application.

[0044] As Figure 1 shown, the text entity relationship classification method based on few-shot learning includes the following steps:

[0045] Step 1 Given a dataset that includes support set statements and query set statements, use a CNN network as an instance encoder to encode the support set statements and query set statements, and convert them into low-dimensional instance vectors, so as to obtain the entity pair features of the extracted corpus. The specific process is as follows:

[0046] 1-1 Select the FewRel dataset that is widely used in the field of small-sample relation classification;

[0047] 1-2 Pre-define a set of relation types for the FewRel dataset. The following are some entity relation types set in the training set of this embodiment, and the defined entity relation types are all classified according to the types defined in the original dataset:

[0048] Number Entity relationship type Number Entity relationship type 1 P931 6 P6 2 P4552 7 P27 3 P140 8 P449 4 P1923 9 P1435 5 P150 10 P175

[0049] 1-3 Use the methods of Glove word embedding and entity position embedding to convert the input corpus into a low-dimensional word vector form. Glove word embedding converts the input corpus into a co-occurrence matrix, which is represented by dimension d w ; Represent the sentence obtained after Glove word embedding as a list of vectors (x 0 , x 1 , x 2 ,..., x i ), where x i represents the i-th word embedding. Entity position embedding is to calculate the distance between each word embedding vector and the two head and tail entities in the sentence, which is represented by dimension 2*d p . Finally, combine Glove word embedding and entity position embedding to achieve the final embedding representation of each word.

[0050] For example, for the input sentence "The name of East Midlands Airport was at one point changed to Nottingham, so as to include the name of the city that is supposedly most internationally recognizable.", the head entity is "East Midlands Airport" and the tail entity is "Nottingham". For a certain word "one" in it, its relative distance from the head entity is 5 and its relative distance from the tail entity is -4. Therefore, the position embedding representation of "one" is [5, -4]. After passing the entire sentence through the embedding layer, the word embedding of each word is represented by WF, with the dimension set to 50, and the position embedding is represented by PF, with the dimension set to 5. Finally, the embedding vector sequence of the entire sentence is represented as {e1 ,..., e n} = {[WF 1 ; PF 1 ..., [WF n ; PF n}}。

[0051] 1 - 4 Further process the sentence embedding vector sequence using a CNN network to extract semantic features from it, which can be specifically divided into a convolutional layer and a max - pooling layer. Among them, the convolutional layer uses a convolutional sliding window of length m to extract features and passes the obtained vector sequence through the ReLU activation function; the max - pooling layer processes the sentence hidden embedding obtained by the convolutional layer. Finally, an instance vector of the entire sentence is obtained.

[0052] For example, for the final sentence embedding vector sequence {e 1 ,..., e n} obtained in 1 - 3, set the sliding window length to m, and use convolutional operations to extract the semantic features therein, which is expressed as:

[0053]

[0054] Use max - pooling operations to process the sentence hidden embedding [h 1 , h 2 ,... h n obtained by the convolutional layer, which is expressed as:

[0055] [x] i = maxpooling{[h 1 i ,..., [h n i}

[0056] Step 2 Use the prototype - level attention mechanism module to obtain each relationship prototype, which specifically includes the following steps:

[0057] 2 - 1 Introduce the prototype - level attention mechanism method to solve the relationship prototype of each relationship. The specific calculation process is as Figure 2 shown. First, pass the input statement through a judgment process to determine whether it is a support set. Then combine the divided query - set instances with all support - set instances under a relationship, use the Gaussian function as the activation function, and obtain the weight γ ij assigned to each support - set instance. Finally, sum each relationship instance to obtain the prototype representation c i of the entire relationship, which are respectively expressed as:

[0058]

[0059] ​​

[0060]

[0061] For example, in this embodiment, the prototype network is selected as the framework, and the relational prototype of the support set vectors within the input framework can be obtained through the formula in 2-1.

[0062] Step 3 splices the support set instances into a vector matrix, uses the distance-level attention mechanism module to extract the important dimensional features therein, and improves the metric formula, specifically including:

[0063] 3-1 Divide the support set instances according to the relationship, and as shown in the framework of Figure 3 , first for each relationship r i the K support set instances [x i1 , x i2 ,..., x iK in it are spliced into a K*d h *1 vector matrix, then the features are extracted respectively through a convolution module composed of three convolutional layers and three ReLU layers, and finally the weight coefficient β of the vector matrix is obtained. Among them, the dimensions of the input vectors change sequentially to K*d h *32, K*d h *64, 1*d h *1.

[0064] 3-2 Multiply the Manhattan distance formula by the weight coefficient obtained in 3-1 to obtain the final metric formula, expressed as:

[0065]

[0066] Step 4 calculates the query instance with the prototype obtained by the prototype-level attention mechanism module in Step 2, uses the distance formula obtained in Step 3 to measure the distance between the query instance and the relational prototype, and finally applies the softmax function to complete the relationship classification of the query instance.

[0067] For example, for the examples in Examples 1-3, under the prototype network framework, after comparing them with all the relational prototypes in the support set, it is found that the value obtained in the calculation with the relationship P931 is the largest. Therefore, in Examples 1-3, the relationship classification between the entity pair <EastMiddle Airport,Nottingham> is P931.

[0068] According to the above embodiments, the present invention realizes a text entity relationship classification method based on few-shot learning. According to requirements, it is set as a few-shot relationship classification scenario. The present invention uses a dual attention mechanism module to improve the few-shot relationship classification task. Among them, the prototype-level attention mechanism module represents the relationship prototype by assigning different weights to relationship instances, eliminating the influence of individual extreme examples on the prototype representation; the distance-level attention mechanism is used to highlight the dimensions in the feature space that have a greater impact on relationship classification, and introduces the Manhattan formula as a new distance function to achieve higher-performance relationship classification.

[0069] Obviously, those skilled in the art should understand that each step of the above-described text entity relationship classification method based on few-shot learning in the embodiments of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order than here, or they can be separately made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to implement. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A method for text entity relationship classification based on few-shot learning, characterized in that, it includes the following steps: Step 1: Use a CNN network as an instance encoder to encode the support set statements and query set statements in a given dataset, convert the support set statements and query set statements into low-dimensional instance vectors, and obtain the entity pair features of the extracted corpus; Step 2: In the few-shot learning scenario, use the prototype-level attention mechanism module to assign weights to each instance, thereby obtaining the weighted prototype vector; Step 3: Concatenate the support set instances encoded in Step 1 into a vector matrix, and through the distance-level attention mechanism module, extract the semantic features of the important dimensions in the support set instances, thereby obtaining the weight β of the distance-level attention mechanism; Step 4: Multiply the weight β of the distance-level attention mechanism obtained in Step 3 by the Manhattan distance formula to obtain a new distance formula as a metric function; use the metric function to measure the distance between the query instances in the query set and the prototype vector obtained in Step 2; Step 5: According to the distance formula obtained in Step 4, compare the distance between the query instance and the prototype vector; use the softmax function for relationship classification; The above Step 2 uses the prototype-level attention mechanism module to assign weights to each instance, thereby obtaining the weighted prototype vector, which includes the following steps: 2-1 Using the Gaussian function as the activation function, obtain the weight γ assigned to each sample instance ij , and finally use the weights to perform a weighted sum on each relation instance to obtain the prototype-level relation vector representation c of the entire relation i , which are respectively expressed as: Among them, q j represents the query set instance, M represents the types of relationships existing in the dataset, K represents the number of instances under each relationship type, x ij represents the j-th support set instance under relationship i, and σ i represents the parameter value for the Gaussian function; The above Step 3 extracts the semantic features of the important dimensions in the support set instances through the distance-level attention mechanism module, thereby obtaining the weight β of the distance-level attention mechanism, which includes the following steps: 3-1 Divide the supported centralized instance sentences according to the relationship, and for each relationship r i Among the K support set instances [x i1 , x i2 ,..., x iK , process them through the instance encoder in step 1 and concatenate them into a K*d h *1 vector matrix, where K represents the number of instances under each relationship category, and d h represents the hidden layer unit; 3-2 Pass the vector matrix through a module interweaving three convolutional layers and three ReLU function layers to extract the semantic features with non-zero dimension values in the support set instances, and the dimension becomes 1*d h *1, so as to obtain the weight β of the distance-level attention mechanism.

2. The method for text entity relationship classification based on few-shot learning according to claim 1, characterized in that, in the above Step 1, using a CNN network as an instance encoder to encode the support set statements and query set statements in a given dataset includes the following steps: 1-1 Convert the corpus in the input dataset into a low-dimensional word vector form by using the Glove word embedding and entity position embedding methods; represent the sentence obtained after Glove word embedding as a list of vectors (x 0 , x 1 , x 2 ,..., x i ), where x i represents the i-th word embedding. The entity position embedding calculates the distance between each word embedding vector x m (m ∈ [0, i]) in the vector list and the two head and tail entities in the sentence. Combine the Glove word embedding and entity position embedding, denoted as {e 1 ,..., e n} = {[WF 1 ; PF 1 ..., [WF n ; PF n}, thus forming a sequence of sentence embedding vectors; 1-2 Use the CNN network to further process the final sentence embedding vector sequence obtained in Step 1-1, extract the semantic features therein, and divide it into a convolutional layer and a max pooling layer; Among them, the convolutional layer uses a convolutional sliding window with a length of m to extract features from the sentence embedding vector sequence, and processes the obtained vector sequence after feature extraction through the ReLU activation function to obtain the sentence hidden embedding; The max pooling layer processes the sentence hidden embedding obtained by the convolutional layer; finally, the instance vector of the entire sentence is obtained.

3. The method for text entity relationship classification based on few-shot learning according to claim 1, characterized in that, the metric formula established in the above Step 4 includes the following steps: 4-1 Select the Manhattan distance formula as the distance formula; 4-2 Multiply the weight β of the distance-level attention mechanism by the distance formula selected in 4-1 to obtain the distance function d(x,y) as a new metric function, which is expressed as: where n represents the dimension, x i and y i respectively represent the values of x and y in the i-th dimension.

4. The method for text entity relationship classification based on few-shot learning according to claim 1, characterized in that, the relationship classification using the softmax function established in the above Step 5 includes the following steps: 5-1 Represent according to the prototype-level relationship vector c i and the distance function d(x, y), calculate the distance between a certain instance in the query set and c i The distance represents the vector obtained after the query set instance x passes through the instance encoding layer; 5-2 For the instances in the query set, judge which relationship in the relationship set R they specifically belong to; specifically expressed as: where the conditional probability is the probability of the query set instance x in relation r i under the condition.

5. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the method for text entity relationship classification based on few-shot learning described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that: the computer-readable storage medium stores a computer program that executes the method for text entity relationship classification based on few-shot learning described in any one of claims 1-4.

Images