Text classification method based on graph convolution attention network

Abstract

The invention discloses a text classification method based on a graph convolution attention network. The text classification method comprises the following steps: step 1, constructing a graph G containing word nodes, document nodes and label nodes; 2, constructing a neural network combining graph convolution and graph attention, wherein the neural network is used for updating a node feature vector H ''; 3, inputting the node feature vector H'obtained in the step 2 into a full-connection neural network to obtain a text classification result; 4, using cross entropy loss as a loss function, and updating text classification model parameters; 5, training a text classification model according to the steps 1-4 on the basis of a public text classification data set; and step 6, performing text classification based on the text classification model trained in the step 5. According to the method, the node features are updated globally and locally at the same time, and the accuracy of text classification is improved.

Description

Technical Field

[0001] The present invention relates to the field of text classification technology, and in particular to a text classification method based on graph convolutional attention network. Background Art

[0002] Text classification is a key branch of natural language processing, widely used in fields such as web page classification, public opinion monitoring, and sentiment analysis. The most commonly used text classification methods are based on machine learning and deep learning. Machine learning uses algorithms and models to automatically learn from data and use this knowledge to predict and classify unknown data. These algorithms typically rely on feature engineering, converting text into numerical features and employing classification algorithms for model training and prediction. Deep learning methods typically do not require manual feature engineering. Instead, they directly input raw text into the model and train a neural network to learn the semantics and feature representations of the text, thereby achieving classification. With the advancement and development of deep learning models such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs), more and more researchers are using deep learning models for text classification. Although CNNs and RNNs are widely used in text classification, they typically treat text as an unordered sequence of words, failing to consider relationships between words and contextual information, thereby ignoring the structural information in the text.

[0003] Text data can be viewed as a graph composed of words, where each word is a node of the graph and the relationships between words can be represented as edges of the graph. Graph Convolutional Network (GCN) is a neural network model suitable for graph data. It can process the connection relationships between nodes and capture node features by performing convolution operations on nodes. However, GCN updates node features based on the global node connection relationship weights. This global update method cannot reflect the more detailed local connection relationships of nodes. In order to better capture the local relationships between nodes, Graph Attention Network (GAT) uses an attention mechanism to calculate the attention between neighboring nodes and updates node features based on this attention. It can be seen that the global node update method cannot reflect the local features of the node, and the local node update method cannot reflect the global features of the node. In other words, a single method is prone to information loss or deviation.

[0004] In view of this, this invention is proposed. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the existing technology and propose a text classification method based on graph convolutional attention network, which updates node features globally and locally to improve the accuracy of text classification.

[0006] In order to achieve the above object, the present invention also adopts the following technical solutions:

[0007] A text classification method based on graph convolutional attention network, characterized by comprising the following steps:

[0008] Step 1: Build a graph G containing word nodes, document nodes, and label nodes;

[0009] Step 2: Build a neural network that combines graph convolution and graph attention to update the node feature vector H";

[0010] Step 3: Input the node feature vector H' obtained in step 2 into the fully connected neural network to obtain the text classification result;

[0011] Step 4: Use cross entropy loss as the loss function to update the text classification model parameters;

[0012] Step 5: Based on the public text classification dataset, train the text classification model according to steps 1 to 4;

[0013] Step 6: Perform text classification based on the text classification model trained in step 5.

[0014] Furthermore, in the constructed graph G in step 1, the weights of the word-label edge, the weights of the document-label edge, the weights of the word-word edge, and the weights of the document-word edge are as follows:

[0015]

[0016] Furthermore, in step 1, the weight of the word-label edge is obtained by calculating the cosine similarity between the word and the label, and the edges with weights greater than 0.3 are retained. The calculation formula for the weight of the word-label edge is as follows:

[0017]

[0018] Among them, · represents the dot product of the vector, |||| represents the modulus of the vector, wv i Represents the word embedding vector, lv j Embedding vector representing the label.

[0019] Furthermore, in step 1, the weight of the document-label edge is obtained by calculating the attention scores of the document and the label, and the edges with weights greater than 0.2 are retained. The calculation formula of the document-label edge weight is as follows:

[0020] Att(LV,DV)=softmax(Wσ(LV·DV T ))

[0021] LV={lv1,lv2,...,lv t}∈R t×m

[0022] DV={dv1,dv2,...,dv n}∈R n×m

[0023] Among them, LV represents the embedding vector of all tags, DV represents the embedding vector of all documents, t represents the total number of tags, n represents the dimension of the document vector, m represents the dimension of a single embedding vector, W represents the weight vector of the fully connected neural network, σ() represents the relu activation function, and T represents the transpose of the vector.

[0024] Furthermore, step 2 includes the following steps:

[0025] Step 2.1: Get the edge adjacency matrix A and node embedding vector X of graph G;

[0026] Step 2.2: Obtain the global node feature vector H through the first layer of graph convolutional neural network. The form of H is as follows:

[0027]

[0028] Among them, Relu() represents the activation function Relu, W0∈R d×d represents the trainable weight vector, and D represents the degree matrix of graph G;

[0029] Step 2.3: Obtain the attention weight b of the adjacent edge through the attention mechanism of the neighbor node ij ;

[0030] Step 2.4: Adaptively fuse bij with the global weight value aij to obtain the fusion weight ab including the global connection weight and the local connection weight ij ;

[0031] Step 2.5: Obtain a new node feature vector h' through multi-head attention i , h' i ∈R d , h' i The calculation formula is as follows:

[0032]

[0033] Among them, K represents the number of heads of multi-head attention;

[0034] Step 2.6: By superimposing the second layer of graph convolutional network and merging higher-order neighborhood information, the final node feature vector H' is obtained. The calculation formula of H' is as follows:

[0035]

[0036] H'={h'1,h'2,...,h' N}

[0037] Where W2∈R d×d Represents a trainable weight vector, H'∈R N×d Represents a weight vector that can simultaneously reflect the global and local connection relationships.

[0038] Further, the step 2.3 includes the following steps:

[0039] Step 2.3.1: Obtain the attention coefficient e of all neighbor nodes through the neighbor node attention mechanism ij , e ij The calculation formula is as follows:

[0040] e ij =a(W1h i ,W1h j )

[0041] Where W1∈R d×d represents the trainable weight vector, h i and h j is a vector in the global node feature vector H, j∈M j , M j represents the set of neighbor nodes of node i, and a is a learnable parameter vector;

[0042] Step 2.3.2: Use LeakyReLU and softmax function to transform e ij Perform normalization to obtain the attention weight value b of the edge of neighbor node j connected to node i ij , b ij The calculation formula is as follows:

[0043]

[0044] Among them, T represents the transpose of the vector, and || represents the concatenation of the vectors.

[0045] Further, the step 2.4 includes the following steps:

[0046] Step 2.4.1: Get the global weight a of the edge between nodes i and j ij ,

[0047] Step 2.4.2: Obtain the fusion weight ab including the global connection weight and the local connection weight through adaptive fusion ij , ab ij The calculation formula is as follows:

[0048] ab ij =βa ij +γb ij

[0049] Among them, β and γ represent a ij 、b ij The weight of 0<β, γ<1, and β+γ=1;

[0050] Furthermore, in step 3, the fully connected neural network has 3 layers, the number of neurons in the first two layers are 512 and 256 respectively, and the number of neurons in the last layer is equal to the number of labels in the text classification dataset.

[0051] Furthermore, in step 3, H" is input into the fully connected network and the final classification result is obtained through the softmax function. The calculation formula is as follows:

[0052]

[0053] Among them, W3 represents the weight vector of the first layer, and W4 represents the weight vector of the output layer.

[0054] Furthermore, in step 4, the following cross entropy loss is used as the loss function to update the model parameters:

[0055]

[0056] Among them, |d| represents the number of documents, |l| represents the number of categories, and y i,j represents the one-hot encoding of the actual label, Indicates the prediction result.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] 1. We add a graph attention network to the middle of a two-layer graph convolutional neural network and propose a neural network that combines graph convolution and graph attention, which updates node features both globally and locally.

[0059] 2. Considering that graph structures are usually sparse, labels are added to the graph structure as global features to effectively enhance the connection between texts;

[0060] 3. It has been verified that the solution of the present invention has high accuracy;

[0061] 4. The text classification model is composed of a neural network that combines graph convolution and graph attention and a fully connected neural network, which can be widely used in multiple fields such as web page classification, public opinion monitoring, and sentiment analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 Flowchart of a text classification method based on graph convolutional attention network. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0064] Example 1:

[0065] A text classification method based on graph convolutional attention network, such as Figure 1 As shown, the following steps are included:

[0066] Step 1: Build a graph G containing word nodes, document nodes, and label nodes;

[0067] Step 2: Build a neural network that combines graph convolution and graph attention to update the node feature vector H";

[0068] Step 3: Input the node feature vector H' obtained in step 2 into the fully connected neural network to obtain the text classification result;

[0069] Step 4: Use cross entropy loss as the loss function to update the text classification model parameters;

[0070] Step 5: Based on the public text classification dataset, train the text classification model according to steps 1 to 4;

[0071] Step 6: Perform text classification based on the text classification model trained in step 5.

[0072] The text classification method based on the graph convolutional attention network of this embodiment adds the graph attention network to the middle of the two-layer graph convolutional neural network, proposes a neural network that combines graph convolution and graph attention, and updates node features both globally and locally. At the same time, considering that the graph structure is usually relatively sparse, the label is added to the graph structure as a global feature, thereby effectively enhancing the connection relationship between texts.

[0073] In this embodiment, the text classification method based on the graph convolutional attention network is implemented. The neural network combining graph convolution and graph attention and the fully connected neural network form a text classification model, which can be widely applied in multiple fields such as web page classification, public opinion monitoring, and sentiment analysis.

[0074] In this embodiment, a graph G = (V, E) is constructed, where V is the set of nodes and E is the set of edges. Traditional GCNs only contain document nodes and word nodes. Considering that labels are predefined categories or identifiers assigned to each document sample, and labels are important for text classification tasks, this embodiment adds label nodes to the graph structure. Specifically, |V| represents the number of nodes, and |V| is the sum of the number of words |w|, the number of documents |d|, and the number of labels |l|.

[0075] In an optional embodiment, in step 1, in the constructed graph G, the weights of the word-label edge, the weights of the document-label edge, the weights of the word-word edge, and the weights of the document-word edge are in the following forms:

[0076]

[0077] Among them, word represents word, label represents label, and document represents document; the weight of word-word edge and the weight of word-document edge can be completed using existing technology, which is not elaborated here. The calculation method of the weight of word-label edge and the weight of document-label edge is detailed in the following embodiments.

[0078] In an optional embodiment, in step 1, the weight of the word-label edge is obtained by calculating the cosine similarity between the word and the label, and edges with weights greater than 0.3 are retained. The calculation formula for the word-label edge weight is as follows:

[0079]

[0080] Among them, · represents the dot product of the vector, |||| represents the modulus of the vector, wv i Represents the word embedding vector, lv j Embedding vector representing the label.

[0081] Cosine similarity measures the similarity between two vectors by calculating the cosine value of the angle between them. In this optional embodiment, the word-label weight is obtained by calculating the cosine similarity between the word and the label. In addition, the greater the cosine similarity, the stronger the association between the word and the label. Therefore, in this optional embodiment, word-label edges with a cosine similarity greater than 0.3 are retained.

[0082] In an optional embodiment, in step 1, the weight of the document-label edge is obtained by calculating the attention scores of the document and the label, and the edges with weights greater than 0.2 are retained. The calculation formula of the document-label edge weight is as follows:

[0083] Att(LV,DV)=softmax(Wσ(LV·DV T ))

[0084] LV={lv1,lv2,...,lv t}∈R t×m

[0085] DV={dv1,dv2,...,dv n}∈R n×m

[0086] Among them, LV represents the embedding vector of all tags, DV represents the embedding vector of all documents, t represents the total number of tags, n represents the dimension of the document vector, m represents the dimension of a single embedding vector, W represents the weight vector of the fully connected neural network, σ() represents the relu activation function, and T represents the transpose of the vector.

[0087] In text classification, labels are words with important meanings, and documents often contain multiple words, making it difficult to directly obtain the weight of the document-label edge by calculating the cosine similarity between the label and the document; at the same time, the label is a predefined category or identifier assigned to each document sample, and documents belonging to the same label often receive more attention from the label; therefore, in this optional embodiment, the weight of the document-label edge is obtained by calculating the attention scores of the document and the label; specifically, the attention scores of LV and DV are first obtained by dot product, and then the attention scores are input into a layer of fully connected neural network, and finally the attention scores of DV and all labels are obtained through the softmax function; in addition, the larger the attention score, the more relevant the document and the label, so in this optional embodiment, the document-label edge with an attention score greater than 0.2 is retained.

[0088] In an optional embodiment, step 2 includes the following steps:

[0089] Step 2.1: Get the edge adjacency matrix A and node embedding vector X of graph G;

[0090] After constructing the graph G = (V, E), obtain the adjacency matrix A of the edges of the graph G, A∈R N×N , and calculate the degree matrix D of graph G according to matrix A, D∈R N×N , D ij The form is as follows:

[0091]

[0092] Using X d ={d1,d2,...,d x}, X t ={t1,t2,...,t y}, X l ={l1,l2,...,l z} represent the embedding vectors of document nodes, word nodes and label nodes respectively, where d i (i=1,2,...x) represents the mean of the embedding vectors of all word nodes in the i-th document node, t i (i=1,2,...y) represents the embedding vector of the i-th word node, l i (i=1,2,...z) represents the embedding vector of the i-th label node.

[0093] Based on this, we construct the embedding vector X of the node of graph G, X∈R N×d , d represents the dimension of the embedding vector X, and the embedding vector X is in the following form:

[0094] X={d1,d2,...,d x ,t1,t2,...,t y ,l1,l2,...,l z}={x1,x2,...,x N}

[0095] N=x+y+z

[0096] Step 2.2: Obtain the global node feature vector H through the first layer of graph convolutional neural network. The form of H is as follows:

[0097]

[0098]

[0099] Among them, Relu() represents the activation function Relu, W0∈R d×d represents the trainable weight vector, and D represents the degree matrix of graph G;

[0100] Step 2.3: Obtain the attention weight b of the adjacent edge through the attention mechanism of the neighbor node ij ;

[0101] Step 2.4: Adaptively fuse bij with the global weight value aij to obtain the fusion weight ab including the global connection weight and the local connection weight ij ;

[0102] Step 2.5: Obtain a new node feature vector h' through multi-head attention i , h' i ∈R d , h' i The calculation formula is as follows:

[0103]

[0104] Among them, K represents the number of heads of multi-head attention;

[0105] Step 2.6: By superimposing the second layer of graph convolutional network and merging higher-order neighborhood information, the final node feature vector H' is obtained. The calculation formula of H' is as follows:

[0106]

[0107] H'={h'1,h'2,...,h' N}

[0108] Where W2∈R d×d Represents a trainable weight vector, H'∈R N×d Represents a weight vector that can simultaneously reflect the global and local connection relationships.

[0109] In an optional embodiment, step 2.3 includes the following steps:

[0110] Step 2.3.1: Obtain the attention coefficient e of all neighbor nodes through the neighbor node attention mechanism ij , e ij The calculation formula is as follows:

[0111] e ij =a(W1h i ,W1h j )

[0112] Where W1∈R d×d represents the trainable weight vector, h i and h j is a vector in the global node feature vector H, j∈M j , M j Represents the set of neighbor nodes of node i, a is a learnable parameter vector, through the weight vector Parameterization is obtained;

[0113] Step 2.3.2: Use LeakyReLU and softmax function to transform e ij Perform normalization to obtain the attention weight value b of the edge of neighbor node j connected to node i ij , b ij The calculation formula is as follows:

[0114]

[0115] Among them, T represents the transpose of the vector, and || represents the concatenation of the vectors.

[0116] In an optional embodiment, step 2.4 includes the following steps:

[0117] Step 2.4.1: Get the global weight a of the edge between nodes i and jij ,

[0118] Step 2.4.2: Obtain the fusion weight ab including the global connection weight and the local connection weight through adaptive fusion ij , ab ij The calculation formula is as follows:

[0119] ab ij =βa ij +γb ij

[0120] Among them, β and γ represent a ij 、b ij The weight of 0<β, γ<1, and β+γ=1;

[0121] In an optional embodiment, in step 3, the fully connected neural network has three layers, the number of neurons in the first two layers are 512 and 256 respectively, and the number of neurons in the last layer is equal to the number of labels in the text classification dataset.

[0122] In an optional embodiment, in step 3, H" is input into the fully connected network and the final classification result is obtained through the softmax function. The calculation formula is as follows:

[0123]

[0124] Among them, W3 represents the weight vector of the first layer, and W4 represents the weight vector of the output layer.

[0125] In an optional embodiment, in step 4, the following cross entropy loss is used as the loss function to update the model parameters:

[0126]

[0127] Among them, |d| represents the number of documents, |l| represents the number of categories, and y i,j represents the one-hot encoding of the actual label, Indicates the prediction result.

[0128] To verify the effectiveness of the solution in this embodiment, five public text classification data sets R8 were selected. 1 、R52 2 Ohsumed 3 , 20NG 4 As well as MR5 as the data set, the introduction of the five data sets is shown in Table 1, and the statistical results of the five data sets are shown in Table 2:

[0129] Table 1-5 Introduction to the data sets

[0130]

[0131] Table 2-5 Statistical results of data sets

[0132] Dataset #Training #Test #Nodes #Documents #Words #Labels Ave-Len R8 5,485 2,189 15,370 7,674 7,688 8 65.72 R52 6,532 2,568 18,044 9,100 8,892 52 69.82 Ohsumed 3,357 4,043 21,580 7,400 14,157 23 135.82 20NG 11,314 7,532 61,623 18,846 42,757 20 221.26 MR 7,108 3,554 29,428 10,662 18,764 2 20.39

[0133] Among them, Ave-Len represents the average length of each document, and #Nodes equals #Documents plus #Words plus #Labels.

[0134] This embodiment selects two comparison methods, one is a time series-based comparison method, and the other is a graph-based comparison method, as follows:

[0135] Comparative methods based on time series:

[0136] LSTM: This model uses LSTM for text classification;

[0137] TextCNN: This model uses CNN for text classification;

[0138] TextRCNN: This model uses both RNN and CNN for text classification;

[0139] RNN-Capsule: This model uses both RNN and capsule networks for text classification;

[0140] HAN: This model uses attention mechanism for text classification;

[0141] RoBERTa: This model is a pre-trained language representation model based on the self-attention mechanism. It is based on BERT and has undergone a series of optimizations and improvements to improve its performance and robustness.

[0142] ALBERT: This model reduces the number of parameters that need to be trained by improving the parameter sharing strategy in the BERT model, thereby achieving a lightweight model.

[0143] Graph-based comparison method:

[0144] TextGCN: This model combines information about graph structure and text content. It generates a graph structure by modeling text data, and then uses a graph convolutional neural network to model and represent the text.

[0145] GAT: This model takes into account the differences in relationships between different nodes and adaptively models the importance of nodes by introducing an attention mechanism.

[0146] TensorGCN: This model is developed based on the graph convolutional network. It models high-dimensional graph data by introducing tensor operations.

[0147] HyperGAT: This model is a graph neural network model based on the graph attention mechanism. Through a multi-layer attention mechanism, it better captures the contextual information of nodes in the learned graph representation.

[0148] TextFCG: This model first constructs a single graph for all words in each text and labels the edges by fusing their various contextual relationships. Then, based on GNN and gated recurrent units, it enables the interaction between local words and global context information, enhancing the sequential representation of nodes.

[0149] CGA2TC: This model designs noise-based and centrality-based enhancement strategies on the topological structure of the text graph to perturb unimportant connections and thus highlight relatively important edges.

[0150] GTR-GA: This model uses a text enhancement method based on graph neural networks and genetic algorithms for text classification.

[0151] BERT-GAT: This model uses an attention mechanism to capture important features of nodes in the graph and implements text classification through dynamic fusion with BERT.

[0152] DGC-BERT: This model constructs two graphs to re-examine the self-attention matrix of dual-view filtering. It then uses two multi-hop graph convolutional networks to capture the key and long-range dependencies between tokens.

[0153] This embodiment selects Accuracy and micro-F1 as the evaluation indicators of the present invention, which are defined as follows:

[0154]

[0155] Among them, t represents the total number of categories, P i Indicates the number of labels for the i-th category, TP i Indicates the number of correctly predicted labels for the i-th category, FP i Indicates the number of labels that are incorrectly predicted for the i-th category, FN i Indicates the number of labels that are incorrectly predicted as the i-th category.

[0156] The Accuracy and micro-F1 results are shown in Tables 3 and 4. As can be seen from Tables 3 and 4, the classification results of this embodiment on the five public datasets are better than all the comparison methods, which shows that the model constructed by the present invention has the best text classification performance.

[0157] By analyzing the comparison models, it can be seen that the classification results of the graph-based model are better than most traditional deep learning models (i.e., LSTM, TextCNN, TextRCNN, RNN-Capsule and HAN); the classification results of the vast majority of graph-based models are inferior to RoBERTa and ALBERT; because the graph-based model can better obtain the document connection relationship, and BERT is a large-scale pre-training model, the classification results of the models based on graph structure and BERT (i.e., BERT-GAT and DGC-BERT) are better than RoBERTa and ALBERT, so the classification results of the comparison method based on both graph and BERT are better than other comparison methods; since the method of this embodiment obtains the embedding vector through BERT and simultaneously captures the global and local connection relationships between nodes, the method of this embodiment is better than both the traditional BERT-based model and the model based on both graph structure and BERT.

[0158] Table 3 - Accuracy of the method in this example and the comparative method on 5 public datasets

[0159]

[0160]

[0161] All models were run 10 times and the mean ± standard deviation was reported.

[0162] Table 4 - micro-F1 of the method of this embodiment and the comparative method on 5 public datasets

[0163]

[0164]

[0165] All models were run 10 times and the mean ± standard deviation was reported.

[0166] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A text classification method based on graph convolutional attention network, characterized by: The steps include: Step 1: Build a graph G containing word nodes, document nodes, and label nodes; Step 2: Build a neural network that combines graph convolution and graph attention to update the node feature vector H"; Step 3: Input the node feature vector H' obtained in step 2 into the fully connected neural network to obtain the text classification result; Step 4: Use cross entropy loss as the loss function to update the text classification model parameters; Step 5: Based on the public text classification dataset, train the text classification model according to steps 1 to 4; Step 6: Perform text classification based on the text classification model trained in step 5.

2. A text classification method based on graph convolutional attention network according to claim 1, characterized in that In step 1, in the constructed graph G, the weights of the word-label edge, the weights of the document-label edge, the weights of the word-word edge, and the weights of the document-word edge are in the following form:

3. A text classification method based on graph convolutional attention network according to claim 2, characterized in that: In step 1, the weight of the word-label edge is obtained by calculating the cosine similarity between the word and the label, and the edges with weights greater than 0.3 are retained. The calculation formula for the weight of the word-label edge is as follows: Among them, · represents the dot product of the vector, |||| represents the modulus of the vector, wv i Represents the word embedding vector, lv j Embedding vector representing the label.

4. A text classification method based on graph convolutional attention network according to claim 2, characterized in that In step 1, the weight of the document-label edge is obtained by calculating the attention scores of the document and the label, and the edges with weights greater than 0.2 are retained. The calculation formula of the document-label edge weight is as follows: That(LV,DV)=softmax(Wσ(LV·DV) T )) LV={lv1,lv2,...,lv t }∈R t×m DV={dv1,dv2,...,dv n }∈R n×m Among them, LV represents the embedding vector of all tags, DV represents the embedding vector of all documents, t represents the total number of tags, n represents the dimension of the document vector, m represents the dimension of a single embedding vector, W represents the weight vector of the fully connected neural network, σ() represents the relu activation function, and T represents the transpose of the vector.

5. The text classification method based on graph convolutional attention network according to claim 1 is characterized in that The step 2 comprises the following steps: Step 2.1: Get the edge adjacency matrix A and node embedding vector X of graph G; Step 2.2: Obtain the global node feature vector H through the first layer of graph convolutional neural network. The form of H is as follows: Among them, Relu() represents the activation function Relu, W0∈R d×d represents the trainable weight vector, and D represents the degree matrix of graph G; Step 2.3: Obtain the attention weight b of the adjacent edge through the attention mechanism of the neighbor node ij ; Step 2.4: Adaptively fuse bij with the global weight value aij to obtain the fusion weight ab including the global connection weight and the local connection weight ij ; Step 2.5: Obtain a new node feature vector h' through multi-head attention i , h' i ∈R d , h' i The calculation formula is as follows: Among them, K represents the number of heads of multi-head attention; Step 2.6: By superimposing the second layer of graph convolutional network and merging higher-order neighborhood information, the final node feature vector H' is obtained. The calculation formula of H' is as follows: H'={h’1,h'2,...,h' N } Where W2∈R d×d Represents a trainable weight vector, H'∈R N×d Represents a weight vector that can simultaneously reflect the global and local connection relationships.

6. A text classification method based on graph convolutional attention network according to claim 5, characterized in that: The step 2.3 includes the following steps: Step 2.3.1: Obtain the attention coefficient e of all neighbor nodes through the neighbor node attention mechanism ij , e ij The calculation formula is as follows: e ij (W1h i ,W1h j ) Where W1∈R d×d represents the trainable weight vector, h i and h j is a vector in the global node feature vector H, j∈M j , M j represents the set of neighbor nodes of node i, and a is a learnable parameter vector; Step 2.3.2: Use LeakyReLU and softmax function to transform e ij Perform normalization to obtain the attention weight value b of the edge of neighbor node j connected to node i ij , b ij The calculation formula is as follows: Among them, T represents the transpose of the vector, and || represents the concatenation of the vectors.

7. A text classification method based on graph convolutional attention network according to claim 5, characterized in that The step 2.4 includes the following steps: Step 2.4.1: Get the global weight a of the edge between nodes i and j ij , Step 2.4.2: Obtain the fusion weight ab including the global connection weight and the local connection weight through adaptive fusion ij , ab ij The calculation formula is as follows: ab ij =βa ij +γb ij Among them, β and γ represent a ij 、b ij The weight of , 0<β, γ<1, and β+γ=1.

8. A text classification method based on graph convolutional attention network according to claim 1, characterized in that In step 3, the fully connected neural network has three layers, the number of neurons in the first two layers is 512 and 256 respectively, and the number of neurons in the last layer is equal to the number of labels in the text classification dataset.

9. A text classification method based on graph convolutional attention network according to claim 8, characterized in that: In step 3, H" is input into the fully connected network and the final classification result is obtained through the softmax function. The calculation formula is as follows: Among them, W3 represents the weight vector of the first layer, and W4 represents the weight vector of the output layer.

10. A text classification method based on graph convolutional attention network according to claim 1, characterized in that: In step 4, the following cross entropy loss is used as the loss function to update the model parameters: Among them, |d| represents the number of documents, |l| represents the number of categories, and y i,j represents the one-hot encoding of the actual label, Indicates the prediction result.

Images