A short text classification method and device based on graph convolution, medium

Abstract

The application provides a short text classification method and device based on graph convolution, equipment and storage medium, the method comprises the following steps: obtaining a plurality of short texts in a training sample, and obtaining a word segmentation result, a syntax analysis result and a part-of-speech analysis result after preprocessing; a matrix of a multi-feature fusion graph is constructed based on the above results; a graph convolutional neural network is used to train nodes of the multi-feature fusion graph based on the matrix of the multi-feature fusion graph, and a trained word node feature vector is obtained; a position embedding vector is constructed, and then spliced with the word node feature vector to train a classifier model to obtain a trained classifier model; after the short text to be classified is subjected to word segmentation processing, a word vector of the short text to be classified is obtained based on the trained word node feature vector, and classification is performed. The application provides a text multi-feature graph representation method fusing part-of-speech, dependency and document information, improves the accuracy of classification, and overcomes the defects in the background art.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a short text classification method, apparatus, and storage medium based on graph convolution. Background Technology

[0002] Over a decade has passed since the advent of Web 2.0. The rapid development of the internet and the widespread adoption of mobile devices have greatly improved people's lives. With the emergence of social networks such as Twitter and Weibo, text data has experienced explosive growth. Therefore, employing text classification techniques to scientifically organize and manage massive amounts of data is crucial. The most typical example is short text data, which typically refers to text of relatively short length, generally not exceeding 160 characters, such as Weibo comments, chat messages, news topics, opinion pieces, and text messages. Due to its lack of semantic and contextual information, highly colloquial style, and varied grammatical structures, short text faces significant challenges in text processing and classification tasks.

[0003] In existing technologies, previous deep learning methods for text classification tasks have largely ignored information such as the grammatical structure and part-of-speech of text. Neural networks based on CNN and RNN can only process structurally normal sequence data. Graph neural networks can perform more complex modeling of text. However, GCN has the drawback that nodes are equally important and the range of convolution is limited. The large pre-trained language model BERT has the drawback that the model consumes a lot of memory and the classification speed is not ideal.

[0004] Therefore, how to accurately and reasonably extract features from short texts and use appropriate models to classify short texts is a key issue. Summary of the Invention

[0005] In view of one or more technical defects in the prior art, the present invention proposes the following technical solution.

[0006] A short text classification method based on graph convolution, the method comprising:

[0007] The preprocessing step involves preprocessing multiple short texts from the acquired training samples to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results. Based on the word segmentation results, syntactic analysis results, and part-of-speech analysis results, a matrix of multi-feature fusion graphs is constructed.

[0008] The node vector training step involves using a graph convolutional neural network to train the nodes of the multi-feature fusion graph based on the matrix of the multi-feature fusion graph, thereby obtaining the trained word node feature vectors.

[0009] The classifier training steps involve constructing a position embedding vector, concatenating the position embedding vector with the trained word node feature vector, and then training the classifier model to obtain the trained classifier model.

[0010] The classification step involves segmenting the short text to be classified into words, obtaining word vectors of the short text to be classified based on the trained word node feature vectors, and then using the trained classifier model to classify the word vectors of the short text to be classified.

[0011] Furthermore, the preprocessing step is performed as follows:

[0012] After preprocessing and labeling multiple short texts in the acquired training samples, the Stanford Corenlp toolkit was used to process them to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results. The syntactic analysis results are parse trees of syntactic components, and the part-of-speech analysis results only retain nouns and adjectives.

[0013] Construct a multi-feature fusion graph, wherein the total nodes in the multi-feature fusion graph include all word nodes, all short text nodes, two part-of-speech nodes, and one root node, wherein the part-of-speech nodes are nouns and adjectives, and the root node is the root node of the syntactic component analysis tree;

[0014] Calculate the PMI value between word nodes in the multi-feature fusion graph:

[0015]

[0016] in,

[0017]

[0018]

[0019] Here, N is the total number of sliding windows in the short text, N(word1,word2) is the number of sliding windows that simultaneously contain nodes word1 and word2, and N(word) is the number of sliding windows that contain node word.

[0020] Calculate the TF-IDF value between word nodes and short text nodes in the multi-feature fusion graph:

[0021] TF-IDF(i,j) = TF(i) × IDF(i,j)

[0022]

[0023]

[0024] Where P(i,j) represents the number of times word i appears in short text j, M(j) represents the total number of words in short text j, N represents the total number of short texts in the training samples, and N(i) represents the number of short texts containing word i.

[0025] Based on the syntactic constituent analysis tree and part-of-speech tagging results, an adjacency matrix A is constructed. The weights on the adjacency matrix are represented as follows:

[0026]

[0027] Normalizing the adjacency matrix A yields the matrix of the multi-feature fusion graph.

[0028]

[0029] Where D is the degree matrix of the multi-feature fusion graph.

[0030] Furthermore, the node vector training step is performed as follows:

[0031] Nodes of a multi-feature fusion graph are trained using a graph convolutional neural network with two layers of graph convolution.

[0032]

[0033] Where X is the feature matrix of the nodes in the multi-feature fusion graph, X = [X1, X2, ..., X...]. n ], where n is the number of nodes, W0 and W1 are the initial weight matrices of the graph convolutional neural network, and ReLU and Softmax are activation functions, where

[0034] ReLU(x) = max(x,0)

[0035]

[0036] Set the loss function, optimization method, and learning rate, train and update the training parameters, and after training, obtain the word node feature vector X.

[0037] Furthermore, the classifier training step is performed as follows:

[0038] Construct a position embedding vector O, O = [one-hot1, one-hot2, ..., one-hot...] n ];

[0039] The position embedding vector is concatenated with the trained word node feature vector to obtain X = [(X1+O1),(X2+O2),...,(X... n +O n The concatenated vector X is then input into the attention model.

[0040] A nonlinear transformation is performed on the input vector X to obtain Query, Key, and Value, where Query = σ Q (XW Q +b Q Key = σ K (XW K +b K Value = σ V (XW V +b V Value is the vector of word embedding X after nonlinear transformation, and d k σ is the dimension of the vector Key, W is the weight matrix, and σ is the weight matrix. Q ,σ K ,σ V It is the tanh activation function, which is executed repeatedly h times.

[0041] Calculate multi-head attention

[0042] Multi-Head(Q,K,V)=Concat(head1,…,head h W0;

[0043] in,

[0044] head i =Self-Attention(Query) i Key i Value i (i = 1, ..., h);

[0045] in,

[0046]

[0047] Add a normalization layer

[0048]

[0049] Where X is the feature matrix after multi-head self-attention operation;

[0050]

[0051] in,

[0052]

[0053] Where X_i represents the i-th dimension of X, m represents the mean of X, σ represents the standard deviation of X, α and β are learnable parameters, ∈ is a decimal set to prevent division by zero, and H is the number of heads in multi-head attention.

[0054] Use a fully connected layer and a softmax function to perform classification prediction on the output y:

[0055] FFN(y)=ReLU(yW1+e1)W2+e2

[0056] Where y is the output vector after layer normalization, and e1 and e2 are bias terms; after passing through a Softmax activation function, the output vector reflecting the probability of each category is obtained;

[0057] Set the loss function, optimization method, and learning rate, train the classifier model parameters to obtain the trained classifier model, and save the classifier model parameters.

[0058] Furthermore, the training samples consist of customer review sentences from the service industry.

[0059] The present invention also proposes a short text classification device based on graph convolution, the device comprising:

[0060] The preprocessing unit preprocesses multiple short texts from the acquired training samples to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results, and constructs a matrix of multi-feature fusion graphs based on the word segmentation results, syntactic analysis results, and part-of-speech analysis results;

[0061] The node vector training unit uses a graph convolutional neural network to train the nodes of the multi-feature fusion graph based on the matrix of the multi-feature fusion graph, and obtains the trained word node feature vectors.

[0062] The classifier training unit constructs a position embedding vector and concatenates the position embedding vector with the trained word node feature vector to train the classifier model, thus obtaining the trained classifier model.

[0063] The classification unit performs word segmentation on the short text to be classified, obtains word vectors of the short text to be classified based on the trained word node feature vectors, and uses the trained classifier model to classify the word vectors of the short text to be classified.

[0064] Furthermore, the operation of the preprocessing unit is as follows:

[0065] After preprocessing and labeling multiple short texts in the acquired training samples, the Stanford Corenlp toolkit was used to process them to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results. The syntactic analysis results are parse trees of syntactic components, and the part-of-speech analysis results only retain nouns and adjectives.

[0066] Construct a multi-feature fusion graph, wherein the total nodes in the multi-feature fusion graph include all word nodes, all short text nodes, two part-of-speech nodes, and one root node, wherein the part-of-speech nodes are nouns and adjectives, and the root node is the root node of the syntactic component analysis tree;

[0067] Calculate the PMI value between word nodes in the multi-feature fusion graph:

[0068]

[0069] in,

[0070]

[0071]

[0072] Here, N is the total number of sliding windows in the short text, N(word1,word2) is the number of sliding windows that simultaneously contain nodes word1 and word2, and N(word) is the number of sliding windows that contain node word.

[0073] Calculate the TF-IDF value between word nodes and short text nodes in the multi-feature fusion graph:

[0074] TF-IDF(i,j) = TF(i) × IDF(i,j)

[0075]

[0076]

[0077] Where P(i,j) represents the number of times word i appears in short text j, M(j) represents the total number of words in short text j, N represents the total number of short texts in the training samples, and N(i) represents the number of short texts containing word i.

[0078] Based on the syntactic constituent analysis tree and part-of-speech tagging results, an adjacency matrix A is constructed. The weights on the adjacency matrix are represented as follows:

[0079]

[0080] Normalizing the adjacency matrix A yields the matrix of the multi-feature fusion graph.

[0081]

[0082] Where D is the degree matrix of the multi-feature fusion graph.

[0083] Furthermore, the operation of the node vector training unit is as follows:

[0084] Nodes of a multi-feature fusion graph are trained using a graph convolutional neural network with two layers of graph convolution.

[0085]

[0086] Where X is the feature matrix of the nodes in the multi-feature fusion graph, X = [X1, X2, ..., X...]. n ], where n is the number of nodes, W0 and W1 are the initial weight matrices of the graph convolutional neural network, and ReLU and Softmax are activation functions, where

[0087] ReLU(x) = max(x,0)

[0088]

[0089] Set the loss function, optimization method, and learning rate, train and update the training parameters, and after training, obtain the word node feature vector X.

[0090] Furthermore, the operation of the classifier training unit is as follows:

[0091] Construct a position embedding vector O, O = [one-hot1, one-hot2, ..., one-hot...] n ];

[0092] The position embedding vector is concatenated with the trained word node feature vector to obtain X = [(X1+O1),(X2+O2),...,(X... n +O n The concatenated vector X is then input into the attention model.

[0093] A nonlinear transformation is performed on the input vector X to obtain Query, Key, and Value, where Query = σ Q (XW Q +b Q Key = σ K (XW K +b K Value = σ V (XW V +b V Value is the vector of word embedding X after nonlinear transformation, and d kσ is the dimension of the vector Key, W is the weight matrix, and σ is the weight matrix. Q ,σ K ,σ V It is the tanh activation function, which is executed repeatedly h times.

[0094] Calculate multi-head attention

[0095] Multi-Head(Q,K,V)=Concat(head1,…,head h W0;

[0096] in,

[0097] head i =Self-Attention(Query) i Key i Value i (i = 1, ..., h);

[0098] in,

[0099]

[0100] Add a normalization layer

[0101]

[0102] Where X is the feature matrix after multi-head self-attention operation;

[0103]

[0104] in,

[0105]

[0106] Where X_i represents the i-th dimension of X, m represents the mean of X, σ represents the standard deviation of X, α and β are learnable parameters, ∈ is a decimal set to prevent division by zero, and H is the number of heads in the multi-head attention.

[0107] Use a fully connected layer and a softmax function to perform classification prediction on the output y:

[0108] FFN(y)=ReLU(yW1+e1)W2+e2

[0109] Where y is the output vector after layer normalization, and e1 and e2 are bias terms; after passing through a SoftMax activation function, the output vector reflecting the probability of each category is obtained;

[0110] Set the loss function, optimization method, and learning rate, train the classifier model parameters to obtain the trained classifier model, and save the classifier model parameters.

[0111] The present invention also proposes a computer-readable storage medium storing computer program code, which, when executed by a computer, performs any of the methods described above.

[0112] The technical advantages of this invention are as follows: This invention provides a short text classification method, apparatus, device, and storage medium based on graph convolution. The method includes: a preprocessing step, where multiple short texts in the acquired training samples are preprocessed to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results; a matrix of a multi-feature fusion graph is constructed based on the word segmentation results, syntactic analysis results, and part-of-speech analysis results; a node vector training step, where a graph convolutional neural network is used to train the nodes of the multi-feature fusion graph based on the matrix of the multi-feature fusion graph to obtain trained word node feature vectors; a classifier training step, where position embedding vectors are constructed, and the position embedding vectors are concatenated with the trained word node feature vectors to train the classifier model to obtain a trained classifier model; and a classification step, where the short text to be classified is segmented, and word vectors of the short text to be classified are obtained based on the trained word node feature vectors; the trained classifier model is then used to classify the word vectors of the short text to be classified. This invention proposes a text multi-feature graph representation method that integrates part-of-speech, dependency, and document information. It uses a graph convolutional network to train the node embedding representation. Furthermore, in order to learn multi-hop information between nodes and overcome the inherent shortcomings of graph convolutional networks, the trained words are embedded into the attention model to improve the classification accuracy and overcome the deficiencies in the background technology. Attached Figure Description

[0113] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0114] Figure 1 This is a flowchart of a short text classification method based on graph convolution according to an embodiment of the present invention.

[0115] Figure 2 This is a structural diagram of a short text classification device based on graph convolution according to an embodiment of the present invention.

[0116] Figure 3 This is a schematic diagram of a multi-feature fusion map according to an embodiment of the present invention.

[0117] Figure 4 This is a schematic diagram of an attention weight visualization graph according to an embodiment of the present invention. Detailed Implementation

[0118] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0119] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0120] Figure 1 This invention illustrates a short text classification method based on graph convolution, the method comprising:

[0121] In the preprocessing step S101, multiple short texts in the acquired training samples are preprocessed to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results. A matrix of multi-feature fusion graphs is constructed based on the word segmentation results, syntactic analysis results, and part-of-speech analysis results.

[0122] In the node vector training step S102, a graph convolutional neural network is used to train the nodes of the multi-feature fusion graph based on the matrix of the multi-feature fusion graph to obtain the trained word node feature vectors.

[0123] In classifier training step S103, a position embedding vector is constructed, and the position embedding vector is concatenated with the trained word node feature vector to train the classifier model and obtain the trained classifier model.

[0124] In classification step S104, after segmenting the short text to be classified, word vectors of the short text to be classified are obtained based on the word node feature vectors trained before the short text to be classified is classified using the trained classifier model.

[0125] This invention innovatively proposes a text multi-feature graph representation method that integrates part-of-speech, dependency, and document information. It uses a graph convolutional network to train the node embedding representation. Furthermore, in order to learn multi-hop information between nodes and overcome the inherent shortcomings of graph convolutional networks, the trained words are embedded into the attention model to improve the classification accuracy. This overcomes the deficiencies in the background technology and is an important inventive point of this invention.

[0126] In one embodiment, the preprocessing step is performed as follows:

[0127] Multiple short texts from the acquired training samples were preprocessed (including but not limited to removing special symbols and garbled characters) and annotated before being processed using the Stanford Corenlp toolkit to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results. The syntactic analysis results are parse trees of syntactic components, and the part-of-speech analysis results only retain nouns and adjectives. Annotation can be done manually or using pre-trained standard tools. Generally, if the corpus (i.e., the training sample library) has a small number of sentences, manual annotation can be used, while machine annotation is used if the number of sentences is large.

[0128] Construct a multi-feature fusion graph, wherein the total nodes in the multi-feature fusion graph include all word nodes, all short text nodes, two part-of-speech nodes, and one root node, wherein the part-of-speech nodes are nouns and adjectives, and the root node is the root node of the syntactic component analysis tree;

[0129] Calculate the PMI value between word nodes in the multi-feature fusion graph:

[0130]

[0131] in,

[0132]

[0133]

[0134] Here, N is the total number of sliding windows in the short text, N(word1,word2) is the number of sliding windows that simultaneously contain nodes word1 and word2, and N(word) is the number of sliding windows that contain node word.

[0135] Calculate the TF-IDF value between word nodes and short text nodes in the multi-feature fusion graph:

[0136] TF-IDF(i,j) = TF(i) × IDF(i,j)

[0137]

[0138]

[0139] Where P(i,j) represents the number of times word i appears in short text j, M(j) represents the total number of words in short text j, N represents the total number of short texts in the training samples, and N(i) represents the number of short texts containing word i.

[0140] Based on the syntactic constituent analysis tree and part-of-speech tagging results, an adjacency matrix A is constructed. The weights on the adjacency matrix are represented as follows:

[0141]

[0142] Normalizing the adjacency matrix A yields the matrix of the multi-feature fusion graph.

[0143]

[0144] Where D is the degree matrix of the multi-feature fusion graph.

[0145] The training samples are customer review sentences from the service industry. A specific embodiment uses the Tan Songbo Hotel Review Dataset, which contains 10,000 Chinese texts divided into two sentiment polarities: positive and negative. The maximum sentence length is 274, and the average sentence length is 60, indicating they are all short texts. The total vocabulary is 30,155. The text is first preprocessed, including but not limited to removing special characters and garbled text, while retaining punctuation and stop words. Then, a multi-feature fusion graph of the text is constructed.

[0146] The training data was processed using the Stanford Core NLP toolkit to obtain word segmentation results, syntactic component analysis trees, and part-of-speech tagging results (also known as part-of-speech analysis results). The part-of-speech tagging results only retain nouns and adjectives; however, other NLP-related toolkits can also be used. Taking sentence 23 from the positive sentiment data of the Tan Songbo Hotel Review dataset as an example, the comparison of the effects before and after processing is as follows:

[0147] Original text: "The hotel facilities were somewhat outdated, but overall they were good."

[0148] After processing with Stanford CorenLP, the word segmentation and part-of-speech tagging results are as follows: [('hotel','NN'),('of','DEG'),('facilities','NN'),('although','CS'),('some','DT'),('old','VA'),(',','PU'),('but','AD'),('generally speaking','AD'),('good','VA'),('。','PU')], where 'NN' and 'VA' represent adjectives and nouns respectively. That is, in the example corpus, the nouns are 'hotel' and 'facilities', and the adjectives are 'old' and 'good'.

[0149] The dependency parsing results are as follows: [('ROOT',0,10),('nmod:assmod',3,1),('case',1,2),('nsubj',10,3),('advmod',6,4),('nsubj',6,5),('dep',10,6),('punct',10,7),('advmod',10,8),('advmod',10,9),('punct',10,11)]

[0150] The text graph representation is constructed based on the dependency parsing tree. Here, node 0 is 'ROOT', and nodes 1-11 are respectively 'hotel's' 'facilities' 'although' 'somewhat' 'old', 'but' 'generally' 'good'. A pair of numbers in the dependency parsing results indicates that there is an edge between the nodes with the corresponding numbers.

[0151] The dependency parsing tree described above is constructed as a graph, with the addition of a ROOT node, document nodes, noun part-of-speech nodes, and adjective part-of-speech nodes. The document node is P23, representing the 23rd document with positive polarity, and the ROOT node is the root node of the parsing tree. The constructed multi-feature fusion graph of the text is attached. Figure 3 As shown.

[0152] Calculate the weights of the edges between nodes in the graph. Here, the sliding window size is 3, but other values ​​can also be used. Taking the segmented sentence as an example:

[0153] The hotel facilities were somewhat outdated, but overall they were good.

[0154] Its sliding window is as follows:

[0155] [Hotel facilities]; [Although]; [Although some facilities]; [Although some facilities are old]...and so on.

[0156] Based on the calculated edge weights between nodes, the adjacency matrix A of the multi-feature fusion graph can be obtained. The adjacency matrix contains the following nodes: word node * number of words, document node * number of documents, ROOT node * 1, noun node * 1, adjective node * 1, where punctuation mark nodes can be regarded as word nodes.

[0157] Perform the above operations on all corpora and store the corresponding adjacency matrix A, dividing the corpora into training set, development set and validation set.

[0158] Normalize the adjacency matrix A to obtain here D is the degree matrix of the nodes. It is a diagonal matrix, and the elements on the diagonal are the degrees of each node.

[0159] Initialize the features of all graph nodes. Here, each graph node is represented by a random high-dimensional dense vector. Of course, other dimensions can also be chosen. The feature matrix of the graph node is represented by X.

[0160] In a real-time example, the node vector training step is performed as follows:

[0161] Nodes of a multi-feature fusion graph are trained using a graph convolutional neural network with two layers of graph convolution.

[0162]

[0163] Where X is the feature matrix of the nodes in the multi-feature fusion graph, X = [X1, X2, ..., X...]. n ], where n is the number of nodes, W0 and W1 are the initial weight matrices of the graph convolutional neural network, ReLU and Softmax are the activation functions, and X is the vector of each node. i The dimension is m, therefore X = [X1, X2, ..., X...]. n It can also be represented as in

[0164] ReLU(x) = max(x,0)

[0165]

[0166] Set the loss function, optimization method, and learning rate, and train and update the training parameters. After training, the trained word node feature vector X is obtained. The set training parameters can be represented as θ={X,W0,b0,W1,b1}.

[0167] Continuing with the hotel review example above, initialize the weight matrices W0 and W1, and initialize the bias vectors b0 and b1 to 0.

[0168] Graph nodes are trained using a graph convolutional neural network.

[0169]

[0170] The Adam optimization function was used, with an initial learning rate of 0.001. The model parameters were continuously adjusted until the optimal classification model with the F-Score was obtained, at which point training was stopped. The F-Score is defined as follows:

[0171]

[0172] After the above steps, we obtained the feature matrix X of the trained graph nodes. We then extracted and stored the word node feature representations. At this point, the word vector training was completed.

[0173] This invention employs a specific method for calculating text multi-feature maps that integrate part-of-speech, dependency, and document information. It calculates the matrix representation of the text multi-feature map and then uses a graph convolutional network to train the node embedding representation to obtain trained word vectors. These word vectors are used for subsequent short text classification, thereby improving classification accuracy. This is another important aspect of this invention.

[0174] In one embodiment, the classifier training step is performed as follows:

[0175] Construct a position embedding vector O, O = [one-hot1, one-hot2, ..., one-hot...] n ];

[0176] The position embedding vector is concatenated with the trained word node feature vector to obtain X = [(X1+O1),(X2+O2),...,(X... n +O n The concatenated vector X is then input into the attention model.

[0177] A nonlinear transformation is performed on the input vector X to obtain Query, Key, and Value, where Query = σ Q (XW Q +b Q Key = σ K (XW K +b K Value = σ V (XW V +b V Value is the vector of word embedding X after nonlinear transformation, and d k σ is the dimension of the vector Key, W is the weight matrix, and σ is the weight matrix. Q ,σ K ,σ V It is the tanh activation function, which is executed repeatedly h times.

[0178] Calculate multi-head attention

[0179] Multi-Head(Q,K,V)=Concat(head1,…,head h W0;

[0180] in,

[0181] head i =Self-Attention(Query) i Key i Value i(i = 1, ..., h);

[0182] in,

[0183]

[0184] Add a normalization layer

[0185]

[0186] Where X is the feature matrix after multi-head self-attention operation;

[0187]

[0188] in,

[0189]

[0190] Where X_i represents the i-th dimension of X, m represents the mean of X, σ represents the standard deviation of X, α and β are learnable parameters, ∈ is a decimal set to prevent division by zero, and H is the number of heads in the multi-head attention.

[0191] Use a fully connected layer and a softmax function to perform classification prediction on the output y:

[0192] FFN(y)=ReLU(yW1+e1)W2+e2

[0193] Where y is the output vector after layer normalization, and e1 and e2 are bias terms; after passing through a SoftMax activation function, the output vector reflecting the probability of each category is obtained;

[0194] By setting the loss function, optimization method, and learning rate, the classifier model parameters are trained to obtain a trained classifier model, and the classifier model parameters are saved. In this process, the training parameters are θ = {X, W}. Q W K W V ,α,β,W0,W1,e1,e2,b Q ,b K ,b V}

[0195] Although the variables in the different training processes mentioned above use the same letters, their meanings are actually different in different training processes. In actual program design, this can be achieved through local variables, which is a common technique for those skilled in the art. That is, those skilled in the art can know the applicable scope and meaning of the above variables.

[0196] Continuing with the hotel review data from before, train the classifier:

[0197] Construct a location embedding vector in one-hot format with a dimension of 1×n, where n is the longest input length of the model. Taking the segmented example "The hotel facilities are somewhat outdated, but overall they are good." as an example, its one-hot location embedding is represented as follows:

[0198] [100…000], [010…000], [001…000], and so on.

[0199] The position embedding vector and the word vector are concatenated to form the input vector for the classifier. In the example, the word vector for "hotel" is [-5.804539 2.940231…-0.647914-0.077439], and the concatenated input vector is [-5.804539 2.940231…-0.647914-0.077439 1 0 0…000]. Other words follow the same pattern, and the input vector is denoted as X.

[0200] A nonlinear transformation is performed on the input vector X to obtain Query, Key, and Value, where Query = σ. Q (XW Q +b Q Key = σ K (XW K +b K Value = σ V (XW V +b V ), d k σ is the dimension of the vector Key, W is the initialized weight matrix, and σ is the vector dimension of the vector Key. Q ,σ K ,σ V It is the tanh activation function.

[0201] The above method is performed h times, that is... Here, h = 8.

[0202] Multi-head attention is computed, and a normalization layer is added to accelerate model convergence. The output data from the fusion layer further enters the fully connected layer, and finally enters the classifier. The classifier chosen is the Softmax function, which normalizes the input data. The number of classes in the dataset must be pre-set during initialization; in this case, the hotel review dataset has both positive and negative classes. The Softmax output is the probability that a sample belongs to these two classes, with the label corresponding to the highest probability value representing the Softmax classifier's classification of the sample.

[0203] Finally, during training, the model loss is calculated by comparing the predicted sample classes with the true sample classes, and this loss is used for backpropagation and parameter updates. During testing, the model's accuracy and F1 score can also be calculated. By continuously adjusting the model parameters, the optimal classification model with the F-score is obtained.

[0204] In this invention, in order to further learn multi-hop information between nodes and overcome the inherent shortcomings of graph convolutional networks, the trained words are embedded into the input attention model, and a specific self-attention calculation method is designed, thereby improving the text classification accuracy. This is another important inventive point of this invention.

[0205] Figure 2 This invention illustrates a graph convolution-based short text classification device, which includes:

[0206] The preprocessing unit 201 preprocesses multiple short texts in the acquired training samples to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results, and constructs a matrix of multi-feature fusion graphs based on the word segmentation results, syntactic analysis results, and part-of-speech analysis results;

[0207] The node vector training unit 202 uses a graph convolutional neural network to train the nodes of the multi-feature fusion graph based on the matrix of the multi-feature fusion graph to obtain the trained word node feature vectors.

[0208] The classifier training unit 203 constructs a position embedding vector and concatenates the position embedding vector with the trained word node feature vector to train the classifier model and obtain the trained classifier model.

[0209] Classification unit 204 performs word segmentation on the short text to be classified, obtains word vectors of the short text to be classified based on the trained word node feature vectors, and uses the trained classifier model to classify the word vectors of the short text to be classified.

[0210] This invention innovatively proposes a text multi-feature graph representation method that integrates part-of-speech, dependency, and document information. It uses a graph convolutional network to train the node embedding representation. Furthermore, in order to learn multi-hop information between nodes and overcome the inherent shortcomings of graph convolutional networks, the trained words are embedded into the attention model to improve the classification accuracy. This overcomes the deficiencies in the background technology and is an important inventive point of this invention.

[0211] In one embodiment, the preprocessing unit operates as follows:

[0212] Multiple short texts from the acquired training samples were preprocessed (including but not limited to removing special symbols and garbled characters) and annotated before being processed using the Stanford Corenlp toolkit to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results. The syntactic analysis results are parse trees of syntactic components, and the part-of-speech analysis results only retain nouns and adjectives. Annotation can be done manually or using pre-trained standard tools. Generally, if the corpus (i.e., the training sample library) has a small number of sentences, manual annotation can be used, while machine annotation is used if the number of sentences is large.

[0213] Construct a multi-feature fusion graph, wherein the total nodes in the multi-feature fusion graph include all word nodes, all short text nodes, two part-of-speech nodes, and one root node, wherein the part-of-speech nodes are nouns and adjectives, and the root node is the root node of the syntactic component analysis tree;

[0214] Calculate the PMI value between word nodes in the multi-feature fusion graph:

[0215]

[0216] in,

[0217]

[0218]

[0219] Here, N is the total number of sliding windows in the short text, N(word1,word2) is the number of sliding windows that simultaneously contain nodes word1 and word2, and N(word) is the number of sliding windows that contain node word.

[0220] Calculate the TF-IDF value between word nodes and short text nodes in the multi-feature fusion graph:

[0221] TF-IDf(i,j) = TF(i) × IDF(i,j)

[0222]

[0223]

[0224] Where P(i,j) represents the number of times word i appears in short text j, M(j) represents the total number of words in short text j, N represents the total number of short texts in the training samples, and N(i) represents the number of short texts containing word i.

[0225] Based on the syntactic constituent analysis tree and part-of-speech tagging results, an adjacency matrix A is constructed. The weights on the adjacency matrix are represented as follows:

[0226]

[0227] Normalizing the adjacency matrix A yields the matrix of the multi-feature fusion graph.

[0228]

[0229] Where D is the degree matrix of the multi-feature fusion graph.

[0230] The training samples are customer review sentences from the service industry. A specific embodiment uses the Tan Songbo Hotel Review Dataset, which contains 10,000 Chinese texts divided into two sentiment polarities: positive and negative. The maximum sentence length is 274, and the average sentence length is 60, indicating they are all short texts. The total vocabulary is 30,155. The text is first preprocessed, including but not limited to removing special characters and garbled text, while retaining punctuation and stop words. Then, a multi-feature fusion graph of the text is constructed.

[0231] The training data was processed using the Stanford Core NLP toolkit to obtain word segmentation results, syntactic component analysis trees, and part-of-speech tagging results (also known as part-of-speech analysis results). The part-of-speech tagging results only retain nouns and adjectives; however, other NLP-related toolkits can also be used. Taking sentence 23 from the positive sentiment data of the Tan Songbo Hotel Review dataset as an example, the comparison of the effects before and after processing is as follows:

[0232] Original text: "The hotel facilities were somewhat outdated, but overall they were good."

[0233] After processing with Stanford CorenLP, the word segmentation and part-of-speech tagging results are as follows: [('hotel','NN'),('of','DEG'),('facilities','NN'),('although','CS'),('some','DT'),('old','VA'),(',','PU'),('but','AD'),('generally speaking','AD'),('good','VA'),('。','PU')], where 'NN' and 'VA' represent adjectives and nouns respectively. That is, in the example corpus, the nouns are 'hotel' and 'facilities', and the adjectives are 'old' and 'good'.

[0234] The dependency parsing results are as follows: [('ROOT',0,10),('nmod:assmod',3,1),('case',1,2),('nsubj',10,3),('advmod',6,4),('nsubj',6,5),('dep',10,6),('punct',10,7),('advmod',10,8),('advmod',10,9),('punct',10,11)]

[0235] The text graph representation is constructed based on the dependency parsing tree. Here, node 0 is 'ROOT', and nodes 1-11 are respectively 'hotel's' 'facilities' 'although' 'somewhat' 'old', 'but' 'generally' 'good'. A pair of numbers in the dependency parsing results indicates that there is an edge between the nodes with the corresponding numbers.

[0236] The dependency parsing tree described above is constructed as a graph, with the addition of a ROOT node, document nodes, noun part-of-speech nodes, and adjective part-of-speech nodes. The document node is P23, representing the 23rd document with positive polarity, and the ROOT node is the root node of the parsing tree. The constructed multi-feature fusion graph of the text is attached. Figure 3 As shown.

[0237] Calculate the weights of the edges between nodes in the graph. Here, the sliding window size is 3, but other values ​​can also be used. Taking the segmented sentence as an example:

[0238] The hotel facilities were somewhat outdated, but overall they were good.

[0239] Its sliding window is as follows:

[0240] [Hotel facilities]; [Although]; [Although some facilities]; [Although some facilities are old]...and so on.

[0241] Based on the calculated edge weights between nodes, the adjacency matrix A of the multi-feature fusion graph can be obtained. The adjacency matrix contains the following nodes: word node * number of words, document node * number of documents, ROOT node * 1, noun node * 1, adjective node * 1, where punctuation mark nodes can be regarded as word nodes.

[0242] Perform the above operations on all corpora and store the corresponding adjacency matrix A, dividing the corpora into training set, development set and validation set.

[0243] Normalize the adjacency matrix A to obtain here D is the degree matrix of the nodes. It is a diagonal matrix, and the elements on the diagonal are the degrees of each node.

[0244] Initialize the features of all graph nodes. Here, each graph node is represented by a random high-dimensional dense vector. Of course, other dimensions can also be chosen. The feature matrix of the graph node is represented by X.

[0245] In a real-time example, the operation of the node vector training unit is as follows:

[0246] Nodes of a multi-feature fusion graph are trained using a graph convolutional neural network with two layers of graph convolution.

[0247]

[0248] Where X is the feature matrix of the nodes in the multi-feature fusion graph, X = [X1, X2, ..., X...]. n ], where n is the number of nodes, W0 and W1 are the initial weight matrices of the graph convolutional neural network, ReLU and Softmax are the activation functions, and X is the vector of each node. i The dimension is m, therefore X = [X1, X2, ..., X...]. n It can also be represented as in

[0249] ReLU(x) = max(x,0)

[0250]

[0251] Set the loss function, optimization method, and learning rate, and train and update the training parameters. After training, the trained word node feature vector X is obtained. The set training parameters can be represented as θ={X,W0,b0,W1,b1}.

[0252] Continuing with the hotel review example above, initialize the weight matrices W0 and W1, and initialize the bias vectors b0 and b1 to 0.

[0253] Graph nodes are trained using a graph convolutional neural network.

[0254]

[0255] The Adam optimization function was used, with an initial learning rate of 0.001. The model parameters were continuously adjusted until the optimal classification model with the F-Score was obtained, at which point training was stopped. The F-Score is defined as follows:

[0256]

[0257] After the above operations, we obtained the feature matrix X of the trained graph nodes. We then extracted and stored the word node feature representations. At this point, the word vector training was completed.

[0258] This invention employs a specific method for calculating text multi-feature maps that integrate part-of-speech, dependency, and document information. It calculates the matrix representation of the text multi-feature map and then uses a graph convolutional network to train the node embedding representation to obtain trained word vectors. These word vectors are used for subsequent short text classification, thereby improving classification accuracy. This is another important aspect of this invention.

[0259] In one embodiment, the operation of the classifier training unit is as follows:

[0260] Construct a position embedding vector O, O = [one-hot1, one-hot2, ..., one-hot...] n ];

[0261] The position embedding vector is concatenated with the trained word node feature vector to obtain X = [(X1+O1),(X2+O2),...,(X... n +O n The concatenated vector X is then input into the attention model.

[0262] A nonlinear transformation is performed on the input vector X to obtain Query, Key, and Value, where Query = σ Q (XW Q +b Q Key = σ K (XW K +b K Value = σ V (XW V +b V Value is the vector of word embedding X after nonlinear transformation, and d k σ is the dimension of the vector Key, W is the weight matrix, and σ is the weight matrix. Q ,σ K ,σ V It is the tanh activation function, which is executed repeatedly h times.

[0263] Calculate multi-head attention

[0264] Multi-Head(Q,K,V)=Concat(head1,…,head h W0;

[0265] in,

[0266] head i =Self-Attention(Query) i Key i Value i(i = 1, ..., h);

[0267] in,

[0268]

[0269] Add a normalization layer

[0270]

[0271] Where X is the feature matrix after multi-head self-attention operation;

[0272]

[0273] in,

[0274]

[0275] Where X_i represents the i-th dimension of X, m represents the mean of X, σ represents the standard deviation of X, α and β are learnable parameters, ∈ is a decimal set to prevent division by zero, and H is the number of heads in the multi-head attention.

[0276] Use a fully connected layer and a softmax function to perform classification prediction on the output y:

[0277] FFN(y)=ReLU(yW1+e1)W2+e2

[0278] Where y is the output vector after layer normalization, and e1 and e2 are bias terms; after passing through a SoftMax activation function, the output vector reflecting the probability of each category is obtained;

[0279] By setting the loss function, optimization method, and learning rate, the classifier model parameters are trained to obtain a trained classifier model, and the classifier model parameters are saved. In this process, the training parameters are θ = {X, W}. Q W K W V ,α,β,W0,W1,e1,e2,b Q ,b K ,b V}

[0280] Although the variables in the different training processes mentioned above use the same letters, their meanings are actually different in different training processes. In actual program design, this can be achieved through local variables, which is a common technique for those skilled in the art. That is, those skilled in the art can know the applicable scope and meaning of the above variables.

[0281] Continuing with the hotel review data from before, train the classifier:

[0282] Construct a location embedding vector in one-hot format with a dimension of 1×n, where n is the longest input length of the model. Taking the segmented example "The hotel facilities are somewhat outdated, but overall they are good." as an example, its one-hot location embedding is represented as follows:

[0283] [100…000], [010…000], [001…000], and so on.

[0284] The position embedding vector and the word vector are concatenated to form the input vector for the classifier. In the example, the word vector for "hotel" is [-5.804539 2.940231…-0.647914-0.077439], and the concatenated input vector is [-5.804539 2.940231…-0.647914-0.077439 1 0 0…000]. Other words follow the same pattern, and the input vector is denoted as X.

[0285] A nonlinear transformation is performed on the input vector X to obtain Query, Key, and Value, where Query = σ. Q (XW Q +b Q Key = σ K (XW K +b K Value = σ V (XW V +b V ), d k σ is the dimension of the vector Key, W is the initialized weight matrix, and σ is the vector dimension of the vector Key. Q ,σ K ,σ V It is the tanh activation function.

[0286] The above method is performed h times, that is... Here, h = 8.

[0287] Multi-head attention is computed, and a normalization layer is added to accelerate model convergence. The output data from the fusion layer further enters the fully connected layer, and finally enters the classifier. The classifier chosen is the Softmax function, which normalizes the input data. The number of classes in the dataset must be pre-set during initialization; in this case, the hotel review dataset has both positive and negative classes. The Softmax output is the probability that a sample belongs to these two classes, with the label corresponding to the highest probability value representing the Softmax classifier's classification of the sample.

[0288] Finally, during training, the model loss is calculated by comparing the predicted sample classes with the true sample classes, and this loss is used for backpropagation and parameter updates. During testing, the model's accuracy and F1 score can also be calculated. By continuously adjusting the model parameters, the optimal classification model with the F-score is obtained.

[0289] In this invention, in order to further learn multi-hop information between nodes and overcome the inherent shortcomings of graph convolutional networks, the trained words are embedded into the input attention model, and a specific self-attention calculation method is designed, thereby improving the text classification accuracy. This is another important inventive point of this invention.

[0290] To verify the effectiveness of the model, the following baseline models in the field of deep learning were selected for single-label comparison experiments: TextCNN, DPCNN, RCNN, RNN-Att, TextGCN and Transformer. To ensure the validity of the comparison results, the model inputs were all randomly initialized characters or word vectors, and the optimization function and initial learning rate were kept consistent.

[0291] (1) TextCNN-char means that the data is input into the TextCNN model in units of characters, while TextCNN-word means that the data is input into the TextCNN model in units of words. Both use three sets of convolutional kernels with sizes of 2, 3, and 4 as feature extractors, with 256 convolutional channels and the max pooling function.

[0292] (2) DPCNN is a deep pyramid CNN model that learns text representations at the character, word, sentence, and document levels by repeatedly building convolutional blocks of equal length. The convolutional blocks are connected by a residual structure. DPCNN uses three sets of convolution-pooling operations with a kernel size of 3, 256 convolutional channels, and uses the max pooling function.

[0293] (3) TextRCNN inputs the initialized word vectors into the Bi-LSTM network to capture the context representation before and after, and then concatenates the two hidden layers and the word embedding vector matrix before feeding them into the pooling layer. It is a combination of CNN and RNN ideas. The Bi-LSTM network in TextRCNN has 256 hidden layer nodes, and the pooling method uses the max pooling function.

[0294] (4) The RNN-Att inputs word vectors into the Bi-LSTM network, uses the encoder to train the attention weight matrix on the intermediate output of the input sequence, and finally weights the vectors of all time sequences. The Bi-LSTM hidden layer has 128 nodes, and the attention function adopts an additive model.

[0295] (5) TextGCN converts text into word nodes and document nodes, embeds the nodes into a graph, and performs a global graph convolution operation on the graph. TextGCN uses two graph convolutions and has 128 hidden layer neurons.

[0296] (6) The Transformer is a self-attention model based on Attention. It obtains the importance of each word to other words by designing a multi-head scaling dot product attention structure. The Transformer has 5 Headers and 1024 hidden layer neurons, and uses sine and cosine functions as position information encoding.

[0297] Table 1 - Experimental Results (F1) for Each Dataset

[0298]

[0299] Table 2 - Experimental Results for Each Dataset (Acc)

[0300]

[0301]

[0302] Tables 1 and 2 indicate that the model mentioned in this application performs well overall on evaluation text sets. Experimental results show that the model performs well on evaluation datasets, but its classification performance on the Fudan News dataset is only average. This is because, compared to news texts, evaluation texts are more casual in language and contain double negations or multiple emotional tendencies within a single text. For example, the text "The guest room was spotless and very clean, but the only drawback was that the TV was small and not very clear. The staff were very patient, and it must be said that it is a very good five-star hotel." contains both positive and negative evaluations, as well as double negative sentences like "had to." Neural network models based on sequence modeling, such as TextCNN, typically perform poorly on this type of data, while graph models can learn more complex syntactic structures and parts of speech.

[0303] In addition, the FudanNews dataset is mostly in document format with a high average text length. Typically, a single corpus consists of a dozen or so sentences, making it impossible to reasonably model the documents using a syntactic analysis tree structure.

[0304] To obtain more intuitive results, the word weight coefficients are displayed graphically, as shown in the attached figure. Figure 4As shown, taking the evaluation "The front desk attitude is so-so, the room is a bit old but clean and tidy, the facilities are complete, and the price-performance ratio is good" as an example, the attention distribution learned by the model is mainly concentrated on the word "good", followed by "front desk", "so-so", and "old". This indicates that the model learns syntactic and semantic information. For some meaningless but necessary conjunctions in the multi-feature map representation, such as "but" and "although", only a small weight matrix is ​​generated after training with the Transformer model, indicating that the model eliminates noisy nodes to a certain extent.

[0305] One embodiment of the present invention proposes a short text classification device based on graph convolution. The device includes a processor and a memory. The processor is connected to the processor via a bus. The memory stores a computer program. When the processor executes the computer program in the memory, it implements the above-mentioned method. The device can be a desktop computer, server, laptop, smart terminal, etc.

[0306] One embodiment of the present invention provides a computer storage medium storing a computer program. When the computer program on the computer storage medium is executed by a processor, the above-described method is implemented. The computer storage medium may be a hard disk, DVD, CD, flash memory, or other storage device.

[0307] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0308] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the apparatus described in various embodiments or some parts of the embodiments of this application.

[0309] Finally, it should be noted that the above embodiments are for illustration only and not for limiting the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention without departing from the spirit and scope of the present invention. Any modifications or partial substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A short text classification method based on graph convolution, characterized in that, The method includes: The preprocessing step involves preprocessing and labeling multiple short texts from the acquired training samples, and then processing them using the Stanford Corenlp toolkit to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results. The syntactic analysis results are parse trees of syntactic components, and the part-of-speech analysis results retain only nouns and adjectives. Construct a multi-feature fusion graph, wherein the total nodes in the multi-feature fusion graph include all word nodes, all short text nodes, two part-of-speech nodes, and one root node, wherein the part-of-speech nodes are nouns and adjectives, and the root node is the root node of the syntactic component analysis tree; Calculate the PMI value between word nodes in the multi-feature fusion graph: in, Here, N is the total number of sliding windows in the short text, N(word1,word2) is the number of sliding windows that simultaneously contain nodes word1 and word2, and N(word) is the number of sliding windows that contain node word. Calculate the TF-IDF value between word nodes and short text nodes in the multi-feature fusion graph: TF-IDF(i,j)=TF(i)×IDF(i,j) Where P(i,j) represents the number of times word i appears in short text j, M(j) represents the total number of words in short text j, N represents the total number of short texts in the training samples, and N(i) represents the number of short texts containing word i. Based on the syntactic constituent analysis tree and part-of-speech tagging results, an adjacency matrix A is constructed. The weights on the adjacency matrix are represented as follows: Normalizing the adjacency matrix A yields the matrix of the multi-feature fusion graph. Where D is the degree matrix of the multi-feature fusion graph; The node vector training steps involve using a two-layer graph convolutional neural network to train the nodes of the multi-feature fusion graph. Where X is the feature matrix of the nodes in the multi-feature fusion graph, X = [X1, X2, ..., X...]. n ], b0 and b1 are bias vectors, n is the number of nodes, W0 and W1 are the initial weight matrices of the graph convolutional neural network, and ReLU and Softmax are activation functions, where ReLU(x) = max(x, 0) Set the loss function, optimization method, and learning rate, train and update the training parameters, and after training, obtain the word node feature vector X. The classifier training steps include constructing a position embedding vector O, where O = [one-hot1, one-hot2, ..., one-hot...]. n ]; The position embedding vector is concatenated with the trained word node feature vector to obtain X = [(X1+O1), (X2+O2), ..., (X3+O1)]. n The concatenated vector X is then input into the attention model. A nonlinear transformation is performed on the input vector X to obtain Query, Key, and Value, where Query = σ Q (XW Q +b Q Key = σ K (XW K +b K Value = σ V (XW V +b V Value is the vector of word embedding X after nonlinear transformation, dk is the dimension of vector Key, W is the weight matrix, and σ Q , σ K , σ V It is the tanh activation function, which is executed repeatedly h times. Calculate multi-head attention Multi-Head(Q,K,V)=Concat(head1,...,head h )W0; in, head i =Self-Attention(Query i ,Key i ,Value i )(i=1,...,h); in, Add a normalization layer Where X is the feature matrix after multi-head self-attention operation; in, Where X_i represents the i-th dimension of X, m represents the mean of X, σ represents the standard deviation of X, α and β are learnable parameters, E is a decimal set to prevent division by zero, and H is the number of heads in the multi-head attention. Use a fully connected layer and a softmax function to perform classification prediction on the output Y: FFN(y)=ReLU(yW1+e1)W2+e2 Where y is the output vector after layer normalization, and e1 and e2 are bias terms; after passing through a softmax activation function, the output vector reflecting the probability of each category is obtained; Set the loss function, optimization method, and learning rate, train the classifier model parameters to obtain the trained classifier model, and save the classifier model parameters; The classification step involves segmenting the short text to be classified into words, obtaining word vectors of the short text to be classified based on the trained word node feature vectors, and then using the trained classifier model to classify the word vectors of the short text to be classified.

2. The method according to claim 1, characterized in that, The training samples consist of customer review sentences from the service industry.

3. A short text classification device based on graph convolution, characterized in that, The device includes: The preprocessing unit preprocesses and annotates multiple short texts in the acquired training samples and then processes them using the Stanford Corenlp toolkit to obtain word segmentation results, syntactic analysis results, and part-of-speech analysis results. The syntactic analysis results are parse trees of syntactic components, and the part-of-speech analysis results retain only nouns and adjectives. Construct a multi-feature fusion graph, wherein the total nodes in the multi-feature fusion graph include all word nodes, all short text nodes, two part-of-speech nodes, and one root node, wherein the part-of-speech nodes are nouns and adjectives, and the root node is the root node of the syntactic component analysis tree; Calculate the PMI value between word nodes in the multi-feature fusion graph: in, Here, N is the total number of sliding windows in the short text, N(word1,word2) is the number of sliding windows that simultaneously contain nodes word1 and word2, and N(word) is the number of sliding windows that contain node word. Calculate the TF-IDF value between word nodes and short text nodes in the multi-feature fusion graph: TF-IDF(i,j)=TF(i)×IDF(i,j) Where p(i,j) represents the number of times word i appears in short text j, M(j) represents the total number of words in short text j, N represents the total number of short texts in the training samples, and N(i) represents the number of short texts containing word i. Based on the syntactic constituent analysis tree and part-of-speech tagging results, an adjacency matrix A is constructed. The weights on the adjacency matrix are represented as follows: Normalizing the adjacency matrix A yields the matrix of the multi-feature fusion graph. Where D is the degree matrix of the multi-feature fusion graph; The node vector training unit uses a two-layer graph convolutional neural network to train the nodes of the multi-feature fusion graph. Where X is the feature matrix of the nodes in the multi-feature fusion graph, X = [X1, X2, ..., X...]. n ], b0 and b1 are bias vectors, n is the number of nodes, w0 and w1 are the initial weight matrices of the graph convolutional neural network, and ReLU and softmax are activation functions, where ReLU(x) = max(x, 0) Set the loss function, optimization method, and learning rate, train and update the training parameters, and after training, obtain the word node feature vector X. The classifier training unit constructs a position embedding vector O, where O = [one-hot1, one-hot2, ..., one-hot...]. n ]; The position embedding vector is concatenated with the trained word node feature vector to obtain X = [(X1+O1), (X2+O2), ..., (X3+O1)]. n The concatenated vector X is then input into the attention model. A nonlinear transformation is performed on the input vector X to obtain Query, Key, and Value, where Query = σ Q (XW Q +b Q Key = σ K (XW K +b K Value = σ V (XW V +b V Value is the vector of word embedding X after nonlinear transformation, dk is the dimension of vector Key, W is the weight matrix, and σ Q , σ K , σ V It is the tanh activation function, which is executed repeatedly h times. Calculate multi-head attention Multi-Head(Q,K,V)=Concat(head1,...,head h )W0; in, head i =Self-Attention(Query i ,Key i ,Value i )(i=1,...,h); in, Add a normalization layer Where X is the feature matrix after multi-head self-attention operation; in, Where X_i represents the i-th dimension of X, m represents the mean of X, σ represents the standard deviation of X, α and β are learnable parameters, E is a decimal set to prevent division by zero, and H is the number of heads in the multi-head attention. Use a fully connected layer and a softmax function to perform classification prediction on the output Y: FFN(y)=ReLU(yW1+e1)W2+e2 Where y is the output vector after layer normalization, and e1 and e2 are bias terms; after passing through a SoftMax activation function, the output vector reflecting the probability of each category is obtained; Set the loss function, optimization method, and learning rate, train the classifier model parameters to obtain the trained classifier model, and save the classifier model parameters; The classification unit performs word segmentation on the short text to be classified, obtains word vectors of the short text to be classified based on the trained word node feature vectors, and uses the trained classifier model to classify the word vectors of the short text to be classified.

4. A computer storage medium storing a computer program, wherein when the computer program on the computer storage medium is executed by a processor, the method of any one of claims 1-2 is implemented.

Images