A small sample relation classification method based on a graph neural network

Abstract

The present application relates to the technical field of relation classification, and particularly relates to a small sample relation classification method based on a graph neural network, which proposes a small sample relation classification method based on graph learning for a relation classification task containing only a small amount of samples, uses a graph neural network in graph learning to construct a small sample relation classification method, uses a prototype network to further improve the effect of relation classification, simultaneously imitates the ability of humans to quickly learn new knowledge by using past learned knowledge through meta learning, reduces the dependence on a large amount of labeled data, and the model learns the generalization ability of adaptive learning between different categories through training, so that the method does not need to be changed when facing a new category for testing, and the learned generalization knowledge can adapt to the classification task of the new category.

Description

Technical Field

[0001] This invention relates to the field of relation classification technology, and specifically to a few-sample relation classification method based on graph neural networks. Background Technology

[0002] Relation classification, also known as relation extraction, has received widespread attention in recent years due to its role in constructing knowledge graphs, an important task in natural language processing. It identifies relationships between entities and connects entities in a knowledge graph accordingly. Relation classification is widely used in data mining, network analysis, and biomedical fields. Through relation extraction, long and complex sentences or paragraphs are summarized into triples of <subject, relation, object>.

[0003] Neural network methods are widely used in relation classification tasks, encompassing both supervised and unsupervised approaches. Supervised methods primarily involve testing different feature extraction techniques followed by multi-classification. These methods place high demands on relational classification data; both the quantity and quality of the data significantly impact the classification results, while also requiring the model to possess high generalization ability. Another approach is long-distance supervision, which utilizes external data to improve learning performance. However, external data can lead to model biases and negatively affect transfer learning. Relation extraction relies on large-scale labeled data, requiring tedious manual adjustments. Summary of the Invention

[0004] The purpose of this invention is to provide a few-sample relation classification method based on graph neural networks, which aims to enable a model trained with a small number of samples to correctly classify new categories during testing, thus solving the technical problem that existing relation classification technologies rely on a large number of data samples.

[0005] To achieve the above objectives, this invention provides a few-sample relationship classification method based on graph neural networks, comprising the following steps:

[0006] Step 1: Obtain the feature vector representation of sentence sample nodes from sentence samples in the support set and query set through the encoder; generate an edge label matrix based on the category label of each sentence sample;

[0007] Step 2: Update the feature vector representation of each sentence sample node in the support set and query set using the edge label matrix information, and extract the final feature representation of the node through a graph neural network;

[0008] Step 3: Aggregate the support set by category label using weighted calculation to generate support class prototypes; calculate the similarity between the feature representation of each query sample and the support class prototypes; take the one with the highest similarity as the classification result of the query sample.

[0009] The support set and the query set together constitute the training data set, wherein the support set... A query set consists of K instances of each of N relation categories, and Q instances of each of N relation categories, denoted as . in This represents one of N*K sentence instances. This indicates one of the N relation categories that the sentence corresponds to.

[0010] For each sentence instance x i By extracting the word vectors of all words in the sentence and the position vectors of two entities, and encoding them through a convolutional neural network, the feature vector X is obtained. i For each group of N*(K+Q) instances, generate an edge label matrix E of size N*(K+Q)×N*(K+Q) based on the category of all instances. The edge label matrix is ​​initialized according to whether the instances are in the same category.

[0011] The overall process of step 2 is represented as X. (l+1) =G E2N (X (l) )=ReLU(f(XE (l) The system uses an edge label matrix and an attention mechanism to interact with the support set and query set samples. Specifically, it targets the (l+1)th feature vector X in the network layer. (l+1) It can use the feature vector X from the previous layer. (l) Through the designed graph neural network G E2N Perform computational updates using the graph neural network G. E2N Specifically, it consists of a layer of attention weight calculation represented by f and a layer of ReLU activation function.

[0012] In the attention weight calculation f, the matrix is ​​first represented by the feature vectors in the process. The correlation between the support set and the query set is calculated using the feature vectors of each category in the support set. Calculate the relevance of Q with all features of the query set. Then use the correlation α to calculate the i-th supporting sample. and the j-th query sample The eigenvectors of can be expressed as follows: Connecting the total support set samples and query set samples Obtain the feature vector representation of the new layer

[0013] For each of the N categories in the support set samples, the average of all samples is used to generate the category prototype, and the prototype feature vector corresponding to the p-th category is represented as follows: In the process of calculating the similarity between the feature representation of each query sample and the supporting class prototype, the similarity of each query sample is calculated. The similarity coefficient with the prototype, i.e., the probability β of each query sample belonging to the supporting class. p : This means that the similarity between the class prototype and the query set prototype is calculated through two fully connected layers, MLP1 and MLP2, with an activation layer of ReLU function in between. Finally, the similarity is determined by β. p The maximum value in the range corresponds to the category p, which is then classified.

[0014] Specifically, the cross-entropy loss function is used to calculate the loss, which is calculated using the cross-entropy between the true value and the predicted value of the query set for each meta-task.

[0015] This invention provides a few-sample relation classification method based on graph neural networks. It proposes a graph learning-based few-sample relation classification method for relation classification tasks involving only a small number of samples. The method utilizes graph neural networks in graph learning to construct a few-sample relation classification method, and uses a prototype network to further improve the classification performance. Simultaneously, it mimics the human ability to quickly learn new knowledge using previously learned knowledge through meta-learning, reducing reliance on large amounts of labeled data. The model learns generalization ability across different categories through training, so that when facing new categories in the test, the method does not need to be modified; it can adapt to the classification task of new categories based on the learned generalization knowledge. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a few-sample relationship classification method based on graph neural networks according to the present invention.

[0018] Figure 2 This is a schematic diagram illustrating the overall description of the small-sample relational meta-task in this invention.

[0019] Figure 3 This is a schematic diagram illustrating the construction and description of the small-sample relational meta-task support set in this invention.

[0020] Figure 4 This is a schematic diagram of the structure of the small sample relationship classification method model based on graph neural networks of the present invention.

[0021] Figure 5 This is a diagram showing the effect after performing step S1 in a specific embodiment of the present invention.

[0022] Figure 6 This is a diagram showing the effect after performing step S2 in a specific embodiment of the present invention.

[0023] Figure 7 This is a diagram showing the effect after performing step S3 in a specific embodiment of the present invention. Detailed Implementation

[0024] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0025] Please see Figure 1 This invention provides a few-sample relationship classification method based on graph neural networks, comprising the following steps:

[0026] S1: The sentence samples in the support set and query set are encoded to obtain the feature vector representation of the sentence sample nodes; an edge label matrix is ​​generated according to the category label of each sentence sample;

[0027] S2: Update the feature vector representation of each sentence sample node in the support set and query set using the edge label matrix information, and extract the final feature representation of the node through a graph neural network;

[0028] S3: Aggregate the support set by category label using weighted calculation to generate support class prototypes; calculate the similarity between the feature representation of each query sample and the support class prototypes; take the one with the highest similarity as the classification result of the query sample.

[0029] For details, please refer to Figures 2 to 4 In step S1, in the relation classification task, for the N-way K-shot problem with few samples, each training dataset consists of two parts: a support set and a query set. The query set consists of K instances of each of N relation categories. It consists of Q instances of each of N relation categories. Here, one of N*K sentence instances is represented. This represents one of the N relation categories corresponding to the sentence. For each sentence instance x... iBy extracting the word vectors of all words in the sentence and the position vectors of two entities, and encoding them through a convolutional neural network, the feature vector X is obtained. i Edge labels are set to represent the relationships between instances. Initial values ​​of 1, 0, and 0.5 are set for three categories: support sets of the same type, support sets of different types, and query sets and support sets, respectively, thus completing the initialization of the edge label matrix E. For each group of N*(K+Q) instances, an edge label matrix E of size N*(K+Q)×N*(K+Q) is generated according to the category of all instances. The edge label matrix is ​​initialized based on whether the instances are in the same category.

[0030] Furthermore, in step S2, in the relation classification task, the feature vector representation of each sentence sample node in the support set and query set is updated using the edge label matrix information, and then the final feature representation of the node is extracted through a graph neural network. The overall process can be represented as X (l+1) =G E2N (X (1) )=ReLU(f(X (l) .E)). That is, for the (l+1)th feature vector X in the network layer. (l+1) It can use the feature vector X from the previous layer. (l) Through the designed graph neural network G E2N Perform computational updates using the graph neural network G. E2N Specifically, it consists of a layer of attention weight calculation represented by f and a layer of ReLU activation function.

[0031] In the attention weight calculation f, the matrix is ​​first represented by the feature vectors in the process. The correlation between the support set and the query set is calculated using the feature vectors of each category in the support set. Calculate the relevance of Q with all features of the query set. Then use the correlation α to calculate the i-th supporting sample. and the j-th query sample The eigenvectors of can be expressed as follows: Connecting the total support set samples and query set samples Obtain the feature vector representation of the new layer

[0032] In step S3, small-sample classification is performed based on the prototype method. Simultaneously, the support set samples are summarized by category to generate a prototype for each category. For each of the N categories in the support set samples, the average of all samples is calculated to generate a category prototype. The prototype feature vector corresponding to the p-th category is represented as follows: In the process of calculating the similarity between the feature representation of each query sample and the supporting class prototype, the similarity of each query sample is calculated. The similarity coefficient with the prototype, i.e., the probability β of each query sample belonging to the supporting class. p : This means that the similarity between the class prototype and the query set prototype is calculated through two fully connected layers, MLP1 and MLP2, with an activation layer of ReLU function in between. Finally, the similarity is determined by β. p The maximum value in the range corresponds to the category p, which is then classified.

[0033] Furthermore, for classification problems, the cross-entropy loss function is used to calculate the loss. The loss is calculated as the cross-entropy between the true and predicted values ​​of the query set for each meta-task, expressed as:

[0034] The present invention also provides a specific embodiment for verification and illustration:

[0035] Please refer to details. Figures 5 to 7 , Figure 5-7 The images represent the process and results of a 5-way, 5-shot small sample relationship classification experiment. Different types of nodes are represented by five different colors, from light to dark. Figure 5 This is the node representation after regular vectorization. At this point, nodes of different types are arranged randomly, indicating that the nodes were not classified after vectorization. Figure 6 This is the node representation after the graph neural network with added edge label matrices, which is the core of the method. Nodes begin to spread outwards, and nodes of the same type exhibit a relatively consistent movement trend. Figure 5 The comparative results demonstrate that using only the graph learning method of this invention for feature learning is sufficient to extract certain useful information. Figure 7 The results, obtained after implementing the prototype method, show that nodes of the same type are generally clustered within a certain area around the image, indicating that the nodes have been clearly classified. This demonstrates that the method of this invention complements the traditional prototype method, resulting in better experimental results.

[0036] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A few-sample relationship classification method based on graph neural networks, characterized in that, Includes the following steps: Step 1: Obtain the feature vector representation of sentence sample nodes from sentence samples in the support set and query set through the encoder; generate an edge label matrix based on the category label of each sentence sample; Step 2: Update the feature vector representation of each sentence sample node in the support set and query set using the edge label matrix information, and extract the final feature representation of the node through a graph neural network; The overall process of step 2 is represented by X. (l+1) =G E2N (X (l) )=ReLU(f(X (l) ·E)) uses the side label matrix and attention mechanism to enable the interaction of sample information between the support set and the query set, that is, for the (l+1)th feature vector X in the network layer (l+1) The feature vector X from the previous layer can be used. (l) Through the designed graph neural network G E2N Perform computational updates using the graph neural network G. E2N Specifically, it consists of a layer of attention weight calculation represented by f and a layer of ReLU activation function; In the attention weight calculation f, the matrix is ​​first represented by the feature vectors in the process. The correlation between the support set and the query set is calculated using the feature vectors of each category in the support set. Calculate the relevance of Q with all features of the query set. Then use the correlation α to calculate the i-th supporting sample. and the j-th query sample The eigenvectors of can be expressed as follows: Connecting the total support set samples and query set samples We obtain the feature vector representation of the new layer. Step 3: Aggregate the support set by category label using weighted calculation to generate support class prototypes; calculate the similarity between the feature representation of each query sample and the support class prototypes; take the one with the highest similarity as the classification result of the query sample.

2. The few-sample relationship classification method based on graph neural networks as described in claim 1, characterized in that, The support set and the query set constitute the training data set, wherein the support set A query set consists of K instances of each of N relation categories, and Q instances of each of N relation categories, denoted as . in This represents one of N*K sentence instances. This indicates one of the N relation categories that the sentence corresponds to.

3. The few-sample relationship classification method based on graph neural networks as described in claim 2, characterized in that, For each sentence instance x i By extracting the word vectors of all words in the sentence and the position vectors of two entities, and encoding them through a convolutional neural network, the feature vector X is obtained. i For each group of N*(K+Q) instances, generate an edge label matrix E of size N*(K+Q)×N*(K+Q) based on the category of all instances. The edge label matrix is ​​initialized according to whether the instances are in the same category.

4. The few-sample relationship classification method based on graph neural networks as described in claim 3, characterized in that, For each set of N classes, calculate the average of all samples to generate class prototypes. The prototype feature vector corresponding to the p-th class is represented as follows: In the process of calculating the similarity between the feature representation of each query sample and the supporting class prototype, the similarity of each query sample is calculated. The similarity coefficient with the prototype, i.e., the probability β of each query sample belonging to the supporting class. p : To support class prototypes and query set prototypes, similarity is calculated using two fully connected layers, MLP1 and MLP2, with an activation layer via ReLU function in between. Finally, the similarity is determined by judging β. p The maximum value in the range corresponds to the category p, which is then classified.

5. The few-sample relationship classification method based on graph neural networks as described in claim 4, characterized in that, The loss is calculated using the cross-entropy loss function, specifically by using the cross-entropy between the true and predicted values ​​of the query set for each meta-task.

Images