Low-resource text classification method combining graph neural network and prompt tuning

Abstract

The invention discloses a low-resource text classification method combining a graph neural network and prompt tuning, which can solve the problem of low-resource text classification in a two-way mode, and comprises the following steps of: importing a data set which comprises low-resource text data and a graph; multi-modal feature extraction is carried out, a double-encoder framework is used, a text is used as an input and output text embedding vector through a transformer text encoder, and meanwhile, a graph and nodes are used as an input and output node embedding vector through a GCN graph encoder; performing comparison pre-training, text and node self-supervision comparison learning, text and context comparison learning and node and context comparison learning, and obtaining and freezing optimal encoder parameters; adjusting and optimizing prompts, manually making discrete prompts aiming at zero sample classification, and using continuous prompts aiming at few sample classification; and meta learning is carried out, by means of Meta-Net, generalization of invisible classes is optimized and enhanced through condition prompt, and finally a classification result is obtained. According to the method, low-resource text data and graph data are combined, the relationship between node text data and context is considered from two modes, an accurate classification result is obtained with less computing power by using a prompt strategy, and generalization is improved by using Meta-learning.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and text classification, and in particular to multimodal fusion based on graphs and texts and text classification based on deep learning. Background Art

[0002] With the rapid growth of Internet information, text classification, as an important natural language processing task, plays an increasingly important role. However, due to problems such as uneven quality and insufficient data volume of text data, traditional text classification methods often fail to achieve satisfactory results.

[0003] Text classification methods targeting low-resource scenarios have emerged. Graph neural networks can effectively capture semantic relationships within text, thereby improving the quality of text representation. Hint tuning can further improve model performance by fine-tuning model parameters even with limited data. This approach not only achieves good classification results with limited data, but also reduces reliance on large amounts of annotated data, lowering the cost of model training.

[0004] This approach provides an efficient and effective text classification solution for both general users and professional researchers. It not only improves the accuracy and efficiency of text classification, but also helps users better understand and utilize text data, promoting the dissemination and application of text information.

[0005] From a technological development perspective, low-resource text classification methods that combine graph neural networks and prompt tuning provide powerful technical support for solving practical problems. As this method continues to improve and expand, we believe it will play an increasingly important role in the field of text classification, driving the continued development of related technologies and applications. Summary of the Invention

[0006] This paper proposes a low-resource text classification method that combines graph neural networks with prompt optimization. This method captures rich semantic relationships through graph structures, enhancing text classification. It jointly trains the graph-text model using three graph-based interactive comparison strategies. It utilizes discrete and continuous prompt optimization for both zero-shot and few-shot classification, while also introducing a meta-learning network to improve classification generalization.

[0007] In order to achieve the above object, the present invention is implemented through the following technical solutions, and the implementation steps are as follows:

[0008] Step 1: Import the dataset and use the graph-based text corpus as the pre-training dataset.

[0009] Step 2: Multimodal feature extraction, using dual encoders (Transformer+GCN) to extract features of both text and graph nodes.

[0010] Step 3: Compare pre-training and learn a bimodal embedding space by jointly training a text encoder and a graph encoder based on three types of interactions: text, nodes, and context.

[0011] Step 4: Hint tuning. For zero-shot classification, discrete hints are handcrafted. For few-shot classification, hints are initialized using a graph encoder and continuous hint tuning is performed.

[0012] Step 5: Meta-learning: Meta-Net is introduced to generate a vector tag for each input node and merge it with its context vector. Meta-Net is fine-tuned to improve the generalization performance of the classifier and obtain the final classification result.

[0013] 1. In step 1 above, import the dataset. Here, we use the Cora dataset, a commonly used literature citation network dataset. This dataset is often used for research on literature classification and information retrieval tasks. The Cora dataset consists of the following parts:

[0014] (1) Nodes: Nodes represent papers. Each node contains a paper on machine learning (with the following features: title, author, publication year, article abstract, and paper topic);

[0015] (2) Edges: Edges represent the citation relationship between papers. If paper A cites paper B, there is a directed edge from node A to node B.

[0016] The main task of the Cora dataset is to classify papers, that is, to classify the topics or categories to which they belong based on their characteristics. Each node has rich feature information, and there are reference relationships between nodes, which can form a graph structure.

[0017] 2. In step 2 above, multimodal feature extraction includes the following steps:

[0018] (1) Select a Transformer text encoder (denoted as Φ T ). For each node v i The text on the document (ie document d i ) as input and output node v i Text embedding vector

[0019] t i =Φ T (d i θ T )

[0020] (2) Select a GCN graph encoder (denoted as ΦZ ). For a given node v i Output an embedding vector

[0021] zi=Φ Z (v i θ G )

[0022] 3. In step 3 above, the comparative pre-training includes the following steps:

[0023] (1) Text-node interaction. Considering the pairing of images and their caption texts and the mapping of content and node sequences, a self-supervised pre-training strategy is selected as the first contrast strategy, which uses the bijective interaction between text and nodes on the graph to maximize the cosine similarity of n matching pairs while minimizing n 2 -n cosine similarity of unmatched pairs. First, perform row-wise L2 normalization on the embedding matrices T and Z to obtain Then a node text similarity matrix is ​​calculated to capture the pairwise cosine similarity:

[0024]

[0025] in is a trainable temperature parameter that scales similarity values.

[0026] The contrast loss is formulated based on the bijective interaction of text nodes, the cross entropy loss of rows and columns is considered, and a multi-class N-pair loss is adopted:

[0027]

[0028] (2) Interaction between text and its context. In addition to bijective text node interactions, we further exploit high-order interactions on the graph. In particular, each document has a set of neighboring documents defined by the graph topology. Considering the semantic relevance between them, a simple mean pooling is used to generate the summary embedding without losing generality.

[0029]

[0030] To improve efficiency, only a fixed number of adjacent documents are sampled to generate context. Then, let s i represents the contextual text embedding matrix of all documents. Therefore, as a second contrastive strategy, we seek to align the text embedding of each document and its corresponding contextual text embedding based on the text summary interactions derived from the graph neighborhood. That is, maximize the cosine similarity of n matching pairs of documents and their neighborhood-based summaries while minimizing n 2 -The cosine similarity of n unmatched pairs is used to obtain the second similarity matrix:

[0031]

[0032] Afterwards, the same contrastive loss is applied:

[0033]

[0034] (3) Node context interaction. Document d i The neighborhood-based context of node v can also be used as i As our third contrastive strategy, we seek to align node embeddings with their neighborhood-based contextual text embeddings. We compute the node-context similarity matrix A3 and formulate the corresponding contrastive loss

[0035]

[0036]

[0037] 4. In step 4 above, the prompt tuning includes the following steps:

[0038] (1) Discrete hints for zero-shot classification. In N-way zeroshot classification, the class with the highest similarity to a given node is predicted among N classes. The classification weights can be generated by a text encoder based on class label text without any labeled samples for the classification task. Specifically, the class y∈{1, 2, ..., N} is the output of the pre-trained text encoder, i.e.:

[0039]

[0040] Here "hint [CLASS]" is a hint template, where [CLASS] refers to the label text y of the target class (e.g., "NLP" for paper area classification), and the hint is a hand-crafted sequence of natural language tokens to indicate the relevance of the label text (e.g., "NLP paper" helps focus on the topic of the paper). In the simplest case, the hint can be an empty string, thus relying only on the label text. The discrete tokens are still converted into continuous word embeddings as input to the text encoder. Then, given a node representation z i The class distribution of is predicted to be:

[0041]

[0042] (2) Continuous prompts for few-shot classification. Discrete natural language prompts are replaced with learnable continuous prompts. Prompt tuning can automatically optimize prompts when some labeled data is available. A series of continuous embeddings are used as prompts. The continuous prompts are fed as input to the text encoder to generate the classification weights for each class:

[0043]

[0044] The continuous cue embeddings are further updated by minimizing the cross-entropy loss using the same soffmax layer as in (1), while freezing the parameters of the dual encoder. Given the small number of learnable parameters in the cue, this cue tuning process is both data and computationally efficient.

[0045] 5. In step 5 above, meta-learning includes the following steps:

[0046] Instead of learning static hints that are specific to a particular task or a fixed set of base classes, we set hints on each individual input instance (i.e., each document node in the context). At the same time, we add a thin neural network (Meta-Net) to generate an input conditional tag (vector) for each node to extend it, which is then integrated with the learnable hint vector. Intuitively, the conditional tag is similar to a node caption or document summary, which is equivalent to a description or document for each node. A two-layer multilayer perceptron with a bottleneck structure is used. Given an embedding vector z i The input node v i , Meta-Net generates a conditional token π i ,Right now:

[0047]

[0048] Then, node v i The mth context vector, denoted as h m，i , we can get

[0049] h m，i =h m +π i

[0050] Conditional token π i Should be combined with the global context vector h m have the same dimensions. Therefore, the node v i The conditional prompt is [h1,···,h M , h CLASS ], where h CLASS is the word embedding sequence of the target class label, the same as that used in the continuous prompt. Subsequently, the conditional prompt is fed as input to the text encoder to output the classification weight, which means that the weight is also conditioned on the input node. Specifically, the node v i And the classification weight of class y is:

[0051]

[0052] Therefore, the predicted probability is calculated as:

[0053]

[0054] Finally, the classification result is obtained through the output of softmax.

[0055] The beneficial effects of the present invention are:

[0056] (1) A multimodal fusion classification method is proposed, which can more accurately classify low-resource texts;

[0057] (2) By capturing rich semantic relationships through graph structures, text classification is more reliable;

[0058] (3) After pre-training the encoder using three multimodal contrastive learning strategies, the parameters are frozen and the classification is completed using prompt tuning and fine-tuning of the thin neural network, which saves computing power and makes the classification process faster;

[0059] (4) Using the meta-learning conditional prompt adjustment method, continuous prompts can be generalized to a wider range of unseen classes, and low-resource text classification has stronger generalization performance; BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Flowchart of a low-resource text classification method combining graph neural networks and hint tuning; DETAILED DESCRIPTION

[0061] The present invention is further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0062] The flowchart of the low-resource text classification method combining graph neural network and prompt tuning is as follows Figure 1 As shown, the steps include:

[0063] Step 1: Import the dataset and use the graph-based text corpus as the pre-training dataset.

[0064] Step 2: Multimodal feature extraction, using dual encoders (Transformer+GCN) to extract features of the two modalities of text and graph nodes respectively.

[0065] Step 3: Compare pre-training and learn a bimodal embedding space by jointly training a text encoder and a graph encoder based on three types of interactions: text, nodes, and context.

[0066] Step 4: Hint tuning. For zero-shot classification, discrete hints are handcrafted. For few-shot classification, hints are initialized using a graph encoder and continuous hint tuning is performed.

[0067] Step 5: Meta-learning: Meta-Net is introduced to generate a vector tag for each input node and merge it with its context vector. Meta-Net is fine-tuned to improve the generalization performance of the classifier and obtain the final classification result.

[0068] 1. In step 1 above, import the dataset. Here, we choose the Cora dataset, which is a commonly used literature citation network dataset. The Cora dataset consists of the following parts:

[0069] (1) Nodes: Nodes represent papers. Each node contains a paper on machine learning (with the following features: title, author, publication year, article abstract, and paper topic);

[0070] (2) Edges: Edges represent the citation relationship between papers. If paper A cites paper B, there is a directed edge from node A to node B.

[0071] 2. In step 2 above, multimodal feature extraction includes the following steps:

[0072] (1) Select a Transformer text encoder (denoted as Φ T ). For each node v i The text on the document (ie document d i ) as input and output node v i Text embedding vector

[0073] t i =Φ T (d i θ T )

[0074] (2) Select a GCN graph encoder (denoted as Φ Z ). For a given node v i Output an embedding vector

[0075] z i =Φ Z (v i θ G )

[0076] 3. In step 3 above, the comparative pre-training includes the following steps:

[0077] (1) Text-node interaction. Considering the pairing of images and their caption texts and the mapping of content and node sequences, a self-supervised pre-training strategy is selected as the first contrast strategy, which uses the bijective interaction between text and nodes on the graph to maximize the cosine similarity of n matching pairs while minimizing n2 -n cosine similarity of unmatched pairs. First, perform row-wise L2 normalization on the embedding matrices T and Z to obtain Then a node text similarity matrix is ​​calculated to capture the pairwise cosine similarity:

[0078]

[0079] The contrast loss is formulated based on the bijective interaction of text nodes, the cross entropy loss of rows and columns is considered, and a multi-class N-pair loss is adopted:

[0080]

[0081] (2) Interaction between text and its context. In addition to bijective text node interactions, we further exploit high-order interactions on the graph. Considering the semantic relevance between them, a simple mean pooling is used to generate summary embeddings without losing generality.

[0082]

[0083] To improve efficiency, only a fixed number of adjacent documents are sampled to generate context. Then, let s i represents the contextual text embedding matrix of all documents. Therefore, as a second contrastive strategy, we seek to align the text embedding of each document and its corresponding contextual text embedding based on the text summary interactions derived from the graph neighborhood. We thus obtain a second similarity matrix, and then apply the same contrastive loss:

[0084]

[0085]

[0086] (3) Node context interaction. Document d i The neighborhood-based context of node v can also be used as i As the third contrastive strategy, we seek to align node embeddings with their neighborhood-based contextual text embeddings. We compute the node-context similarity matrix Λ3 and formulate the corresponding contrastive loss

[0087]

[0088]

[0089] 4. In step 4 above, the prompt tuning includes the following steps:

[0090] (1) Discrete hints for zero-shot classification. In N-way zeroshot classification, the class with the highest similarity to a given node is predicted among N classes. The classification weights can be generated by a text encoder based on class label text without any labeled samples for the classification task. Specifically, the class y∈{1, 2, ..., N} is the output of the pre-trained text encoder, i.e.:

[0091]

[0092] In the simplest case, the prompt can be an empty string, thus relying only on the label text. Discrete tokens are still converted into continuous word embeddings as input to the text encoder. Then, given a node representation z i The class distribution of is predicted to be:

[0093]

[0094] (2) Continuous prompts for few-shot classification. Discrete natural language prompts are replaced with learnable continuous prompts. Prompt tuning can automatically optimize prompts when some labeled data is available. A series of continuous embeddings are used as prompts. The continuous prompts are fed as input to the text encoder to generate the classification weights for each class:

[0095]

[0096] The continuous cue embeddings are further updated by minimizing the cross-entropy loss using the same softmax layer as in (1), while freezing the parameters of the dual encoder. Given the small number of learnable parameters in the cue, this cue tuning process is both data and computationally efficient.

[0097] 5. In step 5 above, meta-learning includes the following steps:

[0098] Instead of learning static hints that are specific to a particular task or a fixed set of base classes, we set hints on each individual input instance. At the same time, we add a thin neural network to generate an input conditional token for each node to extend, which is then integrated with the learnable hint vector. A two-layer multilayer perceptron with a bottleneck structure is used. Given an embedding vector z i The input node v i , Meta-Net generates a conditional token π i ,Right now:

[0099]

[0100] Then, node v i The mth context vector, denoted as h m，i , we can get

[0101] h m，i =h m +π i

[0102] Conditional token π i Should be combined with the global context vector h m have the same dimensions. Subsequently, the conditional prompt is fed as input to the text encoder to output the classification weight, which means that the weight is also conditioned on the input node. Specifically, the node v i And the classification weight of class y is:

[0103]

[0104] Therefore, the predicted probability is calculated as:

[0105]

[0106] Finally, the classification result is obtained through the output of softmax.

Claims

1. A low-resource text classification method combining graph neural network and prompt tuning, characterized by The following steps are involved: Step 1: Import the dataset and use the graph-based text corpus as the pre-training dataset. Step 2: Multimodal feature extraction, using a dual encoder (Transformer+GCN) to extract features of both text and graph nodes. Step 3: Compare pre-training and learn a bimodal embedding space by jointly training a text encoder and a graph encoder based on three types of interactions: text, nodes, and context. Step 4: Hint tuning, for zero-shot classification, handcraft discrete hints: for few-shot classification, use the graph encoder to initialize the hints and perform continuous hint tuning. Step 5: Meta-learning: Meta-Net is introduced to generate a vector tag for each input node and merge it with its context vector. Meta-Net is fine-tuned to improve the generalization performance of the classifier and obtain the final classification result.

2. The low-resource text classification method combining graph neural network and prompt tuning according to claim 1 is characterized in that In step 1, we import a dataset. Here, we use the Cora dataset, a commonly used literature citation network dataset. This dataset is commonly used for research on literature classification and information retrieval tasks. The Cora dataset consists of the following components: (1) Nodes: Nodes represent papers. Each node contains a paper on machine learning (with the following features: title, author, publication year, article abstract, and paper topic); (2) Edges: Edges represent the citation relationship between papers. If paper A cites paper B, there is a directed edge from node A to node B. The main task of the Cora dataset is to classify papers, that is, to classify the topics or categories to which they belong based on their characteristics. Each node has rich feature information, and there are reference relationships between nodes, which can form a graph structure.

3. The low-resource text classification method combining graph neural network and prompt tuning according to claim 1 is characterized in that In the second step, the multimodal feature extraction includes the following steps: (1) Select a Transformer text encoder (denoted as Φ T ). For each node v i The text on the document (ie document d i ) as input and output node v i Text embedding vector t i =Φ T (d i ;θ T ) (2) Select a GCN graph encoder (denoted as Φ Z ). For a given node v i Output an embedding vector z i =Φ Z (v i ;θ G ) 4. The low-resource text classification method combining graph neural network and prompt tuning according to claim 1 is characterized in that In step 3, the comparative pre-training includes the following steps: (1) Text-node interaction. Considering the pairing of images and their caption texts and the mapping of content and node sequences, a self-supervised pre-training strategy is selected as the first contrast strategy, which uses the bijective interaction between text and nodes on the graph to maximize the cosine similarity of n matching pairs while minimizing n 2 -n cosine similarity of unmatched pairs. First, perform row-wise L2 normalization on the embedding matrices T and Z to obtain Then a node text similarity matrix is ​​calculated to capture the pairwise cosine similarity: in is a trainable temperature parameter that scales similarity values. The contrast loss is formulated based on the bijective interaction of text nodes, the cross entropy loss of rows and columns is considered, and a multi-class N-pair loss is adopted: (2) Interaction between text and its context. In addition to bijective text node interactions, we further exploit high-order interactions on the graph. In particular, each document has a set of neighboring documents defined by the graph topology. Considering the semantic relevance between them, a simple mean pooling is used to generate the summary embedding without losing generality. To improve efficiency, only a fixed number of adjacent documents are sampled to generate context. Then, let s i represents the contextual text embedding matrix of all documents. Therefore, as a second contrastive strategy, we seek to align the text embedding of each document and its corresponding contextual text embedding based on the text summary interactions derived from the graph neighborhood. That is, maximize the cosine similarity of n matching pairs of documents and their neighborhood-based summaries while minimizing n 2 - n The cosine similarity of the unmatched pairs is used to obtain the second similarity matrix: Afterwards, the same contrastive loss is applied: (3) Node context interaction. Document d i The neighborhood-based context of node v can also be used as i As our third contrastive strategy, we seek to align node embeddings with their neighborhood-based contextual text embeddings. We compute the node-context similarity matrix Λ3 and formulate the corresponding contrastive loss 5. The low-resource text classification method combining graph neural network and prompt tuning according to claim 1 is characterized in that In step 4, the prompt tuning includes the following steps: (1) Discrete hints for zero-shot classification. In N-way zeroshot classification, the class with the highest similarity to a given node is predicted among N classes. The classification weights can be generated by a text encoder based on class label text without any labeled samples for the classification task. Specifically, the class y∈{1, 2, ..., N} is the output of the pre-trained text encoder, i.e.: Here "hint [CLASS]" is a hint template, where [CLASS] refers to the label text y of the target class (e.g., "NLP" for paper area classification), and the hint is a hand-crafted natural language token sequence to indicate the relevance of the label text (e.g., "NLP paper" helps focus on the topic of the paper). In the simplest case, the hint can be an empty string, thus relying only on the label text. The discrete tokens are still converted into continuous word embeddings as input to the text encoder. Then, given a node representation z i The class distribution of is predicted to be: (2) Continuous prompts for few-shot classification. Discrete natural language prompts are replaced with learnable continuous prompts. Prompt tuning can automatically optimize prompts when some labeled data is available. A series of continuous embeddings are used as prompts. The continuous prompts are fed as input to the text encoder to generate the classification weights for each class: The continuous cue embeddings are further updated by minimizing the cross-entropy loss using the same softmax layer as in (1), while freezing the parameters of the dual encoder. Given the small number of learnable parameters in the cue, this cue tuning process is both data and computationally efficient.

6. The low-resource text classification method combining graph neural network and prompt tuning according to claim 1 is characterized in that In step 5, meta-learning includes the following steps: Hints are set on each individual input instance (i.e., each document node in the context) instead of learning static hints that are specific to a particular task or a fixed set of base classes. At the same time, a thin neural network (Meta-Net) is added to generate an input conditional tag (vector) for each node to extend it, which is then integrated with the learnable hint vector. Intuitively, the conditional tag is similar to a node caption or document summary, which is equivalent to a description or document for each node. Therefore, conditional hints are more general: they are optimized to describe each instance and thus more resilient to class shifts, rather than being limited to certain specific classes. Meta-Net A two-layer multilayer perceptron with a bottleneck structure is used. Given an embedding vector z i The input node v i , Meta-Net generates a conditional token π i ,Right now: Then, node v i The mth context vector, denoted as h m，i , we can get h m，i =h m +p i Conditional token π i Should be combined with the global context vector h m have the same dimensions. Therefore, the node v i The conditional prompt is [h1,···,h M , h CLASS ], where h CLASS is the word embedding sequence of the target class label, the same as that used in the continuous prompt. Subsequently, the conditional prompt is fed as input to the text encoder to output the classification weight, which means that the weight is also conditioned on the input node. Specifically, the node v i And the classification weight of class y is: Therefore, the predicted probability is calculated as: Finally, the classification result is obtained through the output of softmax.

Images