A text classification method, device, equipment and storage medium

Abstract

The application discloses a text classification method and device, equipment and a storage medium. The method comprises the following steps: obtaining text data; determining the feature vectors of the document nodes, concept nodes and word nodes corresponding to the text data based on the text data; constructing a text heterogeneous graph based on the feature vectors of the document nodes, concept nodes and word nodes; determining the weights of the edges between the nodes in the text heterogeneous graph; obtaining the text feature vector corresponding to the text data based on the text heterogeneous graph; and classifying the text feature vector by using a classification function to determine the text category. In this way, the feature vectors of the concept nodes are obtained, the prior knowledge in the text is obtained, and the concept nodes are fused when the text heterogeneous graph is constructed, so that the feature sparsity problem caused by the lack of context in the short text can be relieved to some extent, the text feature vector extracted based on the text heterogeneous graph can more accurately represent the features of the text, and the accuracy of the text classification is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more particularly to a text classification method, apparatus, device, and storage medium. Background Technology

[0002] In the era of big data, a large number of short texts have emerged online. Due to their brevity, lack of contextual information, and high levels of conversational noise, accurately extracting text features and employing appropriate classification models is a key challenge. Existing text classification methods often fail to fully capture the semantic relationships between short texts, leading to low classification accuracy. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this application aim to provide a text classification method, apparatus, device, and storage medium.

[0004] The technical solution of this application is implemented as follows:

[0005] Firstly, a text classification method is provided, which includes:

[0006] Get text data;

[0007] Based on the text data, determine the feature vectors of the document nodes, concept nodes, and word nodes corresponding to the text data;

[0008] A text heterogeneous graph is constructed based on the feature vectors of the document nodes, the feature vectors of the concept nodes, and the feature vectors of the word nodes.

[0009] Determine the weights of the edges between nodes in the heterogeneous text graph;

[0010] Based on the text heterogeneity graph, the text feature vector corresponding to the text data is obtained;

[0011] The text feature vector is classified using a classification function to determine the text category.

[0012] In the above scheme, obtaining the text feature vector corresponding to the text data based on the text heterogeneity graph includes: determining at least one type attention weight for each node based on the text heterogeneity graph; the type attention weight is a document-type attention weight, a concept-type attention weight, or a word-type attention weight; determining the inter-node attention weight between each node and its neighboring nodes based on the at least one type attention weight, the feature vector of each node, and the feature vectors of the at least one type of neighboring nodes; and determining the text feature vector based on the inter-node attention weights between all nodes and their neighboring nodes, and the feature vectors of all nodes.

[0013] In the above scheme, determining at least one type attention weight for each node based on the text heterogeneous graph includes: calculating the sum of the feature vectors of the i-th type neighboring nodes of the k-th node in the text heterogeneous graph to obtain the i-th type feature vector of the k-th node; wherein the i-th type is any one of document type, concept type, or word type; and determining the i-th type attention weight of the k-th node based on the feature vector of the k-th node, the feature vectors of the i-th type neighboring nodes, and the i-th type feature vector.

[0014] In the above scheme, the characteristic is that determining the text feature vector based on the inter-node attention weights between all nodes and their neighboring nodes, and the feature vectors of all nodes, includes: inputting the inter-node attention weights between all nodes and their neighboring nodes, and the feature vectors of all nodes, into a heterogeneous graph convolutional network to obtain the text feature vector corresponding to the text data; wherein, the heterogeneous graph convolutional network is constructed based on a multi-head attention mechanism.

[0015] In the above scheme, determining the feature vectors of document nodes, concept nodes, and word nodes corresponding to the text data based on the text data includes: calculating the word frequency-inverse document frequency (TF-IDF) vector of the text data as the feature vector of the document node; obtaining the concept set of the text data based on the concept graph; mapping the concepts in the concept set to feature vectors based on the word vector model to obtain the feature vector of the concept node; and obtaining the one-hot code vector of the word node in the text data from a preset vocabulary as the feature vector of the word node.

[0016] In the above scheme, determining the weights of edges between nodes in the heterogeneous text graph includes: obtaining at least one concept node corresponding to a document node and a correlation value between the document node and the at least one concept node based on the concept graph; determining the weights of edges between the document node and the at least one concept node based on the correlation value; determining the weights of edges between the document node and at least one word node based on the term frequency-inverse document frequency (TF-IDF) algorithm; and determining the weights of edges between word nodes based on the point mutual information between words.

[0017] In the above scheme, obtaining text data includes: obtaining raw text data; preprocessing the raw text data to obtain the text data; wherein, the preprocessing includes at least one of the following: noise removal, word segmentation, and stop word processing.

[0018] Secondly, a text classification device is provided, the device comprising:

[0019] The acquisition module is used to acquire text data;

[0020] The processing module is used to determine the feature vectors of the document nodes, concept nodes, and word nodes corresponding to the text data based on the text data.

[0021] The processing module is also used to construct a text heterogeneous graph based on the feature vectors of the document nodes, the feature vectors of the concept nodes, and the feature vectors of the word nodes;

[0022] The processing module is also used to determine the weights of the edges between nodes in the heterogeneous text graph;

[0023] The processing module is further configured to obtain the text feature vector corresponding to the text data based on the text heterogeneity graph;

[0024] The processing module is further configured to classify the text feature vector using a classification function to determine the text category.

[0025] Thirdly, a text classification device is provided, the device comprising: a processor and a memory configured to store a computer program capable of running on the processor, wherein the processor is configured to perform the steps of any of the preceding methods when running the computer program.

[0026] Fourthly, a computer storage medium is provided that stores a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the aforementioned method.

[0027] This application discloses a text classification method, apparatus, device, and storage medium. The method obtains prior knowledge in the text by acquiring the feature vectors of concept nodes. When constructing a text heterogeneous graph, concept nodes are integrated, which can alleviate the feature sparsity problem caused by the lack of context in short texts to a certain extent. This allows the text feature vectors extracted based on the text heterogeneous graph to more accurately represent the features of the text, thereby improving the accuracy of text classification. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the first process of the text classification method in the embodiments of this application;

[0029] Figure 2 This is a schematic diagram of a text heterogeneous graph set in an embodiment of this application;

[0030] Figure 3 This is a schematic diagram of the process for determining text feature vectors in an embodiment of this application;

[0031] Figure 4This is a schematic diagram of the text heterogeneous graph aggregation calculation method in the embodiments of this application;

[0032] Figure 5 This is a schematic diagram of the composition structure of the text classification device in the embodiments of this application;

[0033] Figure 6 This is a schematic diagram of the composition structure of the text classification device in the embodiments of this application. Detailed Implementation

[0034] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0035] Figure 1 This is a schematic diagram of the first process of the text classification method in this application embodiment. For example... Figure 1 As shown, text classification methods can specifically include:

[0036] Step 101: Obtain text data;

[0037] Here, text data can be understood as the text data corresponding to the text to be classified (also referred to as "document" in this embodiment). If multiple texts need to be classified, multiple text data can form a text dataset. The acquisition of text data includes: acquiring the text data of the text to be classified from the text dataset.

[0038] For example, in some embodiments, obtaining text data includes: obtaining raw text data; preprocessing the raw text data to obtain the text data; wherein the preprocessing includes at least one of the following: noise removal, word segmentation, and stop word processing.

[0039] Here, raw text data can be understood as the original data corresponding to the text to be classified. Preprocessing is used to simplify the raw text data, retaining related terms, which can improve text classification efficiency.

[0040] Step 102: Based on the text data, determine the feature vectors of the document nodes, concept nodes, and word nodes corresponding to the text data;

[0041] For example, in some embodiments, determining the feature vectors of document nodes, concept nodes, and word nodes corresponding to the text data based on the text data includes: calculating the word frequency-inverse document frequency (TF-IDF) vector of the text data as the feature vector of the document node; obtaining the concept set of the text data based on a concept graph; mapping the concepts in the concept set to feature vectors based on a word vector model to obtain the feature vector of the concept node; and obtaining the one-hot code vector of the word node in the text data from a preset vocabulary as the feature vector of the word node.

[0042] For example, in practical applications, the TF-IDF vector of the document to be classified in the text data is calculated and used as the feature vector of that document node.

[0043] For example, obtaining the concept set of the text data based on the concept graph can be done by calling the concept graph's API, inputting the text data, and obtaining the concept set corresponding to the text data. For example, inputting the text data "Eason Chan was born in Hong Kong" into the concept graph can obtain the concept set {top Chinese entertainer, singer, place, Asian city}.

[0044] For example, in practical applications, the preset vocabulary includes all words in the corpus and the one-hot vectors corresponding to each word. The one-hot vectors of word nodes in the text data can be directly obtained from the preset vocabulary and used as the feature vectors of the word nodes.

[0045] Step 103: Construct a text heterogeneous graph based on the feature vectors of the document nodes, the concept nodes, and the word nodes;

[0046] Here, prior knowledge in the text is obtained by acquiring the feature vectors of the concept nodes corresponding to the text. By constructing a text heterogeneous graph based on the feature vectors of document nodes, word nodes, and concept nodes, the feature sparsity problem caused by the lack of context in short texts can be alleviated to some extent. This allows the text feature vectors extracted based on the text heterogeneous graph to more accurately represent the features of the text, thereby improving the accuracy of text classification.

[0047] Here, when the text data contains the text of a document, the text heterogeneity graph includes all document nodes, concept nodes, and word nodes of that document, which are used to classify the document.

[0048] For example, in some embodiments, the method further includes: constructing a set of text heterogeneous graphs containing multiple texts, which can be used to classify the multiple texts.

[0049] For example, Figure 2 This is a schematic diagram of a text heterogeneous graph set in an embodiment of this application. For example... Figure 2 As shown, this text heterogeneity graph set includes text heterogeneity graphs corresponding to at least three texts. Each text heterogeneity graph includes: document nodes, concept nodes, and word nodes.

[0050] Step 104: Determine the weights of the edges between nodes in the heterogeneous text graph;

[0051] For example, in some embodiments, determining the weights of edges between nodes in the heterogeneous text graph includes: obtaining at least one concept node corresponding to a document node and a correlation value between the document node and the at least one concept node based on a concept graph; determining the weights of edges between the document node and the at least one concept node based on the correlation value; determining the weights of edges between the document node and at least one word node based on the TF-IDF algorithm; and determining the weights of edges between word nodes based on the point mutual information between words.

[0052] For example, by inputting the text data "Eason Chan was born in Hong Kong" into a concept graph, you can obtain the concept set and relevance values.<top chinese entertainer,0.6> ,<singer,0.4> ,<place,0.2> ,<asian city,0.1> The relevance value of each concept to the text data is used as the weight of the edge between the document node and the concept node. For example, the weight of the edge between the concept node "top Chinese entertainer" and the corresponding document node "Eason Chan was born in Hong Kong" is 0.6.

[0053] For example, in some embodiments, determining the weight of the edge between the document node and at least one word node based on the TF-IDF algorithm includes: calculating the TF-IDF value between the word and the document corresponding to the text data; and using the TF-IDF value as the weight of the edge between the document node and the word node.

[0054] Specifically, the TF-IDF value between a word and its corresponding document in the text data is calculated as follows: the ratio of the number of times the word appears in the document to the total number of times the word appears in the document is used as the term frequency (TF); based on... Calculate the inverse document frequency (IDF), where N is the total number of texts and n is the number of documents in the text dataset that contain the word; calculate the product of TF and IDF as the TF-IDF value.

[0055] For example, determining the weight of the edge between word nodes based on the point mutual information between words includes: obtaining the point mutual information between words in the text data from the word point mutual information database; establishing edges between words with positive point mutual information values; and using the point mutual information between words as the weight of the edge between word nodes.

[0056] Here, the word point mutual information database is determined based on the corpus and contains the point mutual information between words in all documents of the corpus. For example, to utilize global word co-occurrence information, co-occurrence statistics are collected using a fixed-size sliding window over all documents in the corpus; point mutual information (PMI) is used to measure the association between words as the weight between two word nodes. The PMI calculation formula is as follows:

[0057]

[0058]

[0059]

[0060] In the formula, PMI(i,j) represents the semantic relationship between words i and j, #W(i) is the number of sliding windows containing word i in the corpus, #W(i,j) is the number of sliding windows containing both words i and j, and #W is the total number of sliding windows in the corpus. A positive PMI indicates a high semantic relationship between words in the corpus; a negative PMI indicates a low or zero semantic relationship between words in the corpus.

[0061] Step 105: Based on the text heterogeneity graph, obtain the text feature vector corresponding to the text data;

[0062] Here, the text heterogeneous graph includes: document nodes, concept nodes, word nodes, and the weights of the edges between nodes. The text feature vector corresponding to the text data is a feature vector used to represent the text data and is used to classify the text based on this text feature vector.

[0063] For example, in some embodiments, obtaining the text feature vector corresponding to the text data based on the text heterogeneous graph includes: calculating the attention weights between nodes in the text heterogeneous graph; inputting the attention weights between nodes and the feature vectors of all nodes in the text heterogeneous graph into a heterogeneous graph convolutional neural network to obtain the text feature vector corresponding to the text data. Here, the graph convolutional neural network is used to extract the spatial features of the topological graph and can be obtained through training.

[0064] Step 106: Use a classification function to classify the text feature vector and determine the text category.

[0065] For example, in some embodiments, the classification function may be a softmax function.

[0066] For example, in some embodiments, the text classification method further includes: obtaining a training text dataset; and training a text classification model based on the training text dataset.

[0067] For example, in some embodiments, the text classification method further includes: constructing a training text heterogeneous graph set containing all texts in the training text dataset. This training text heterogeneous graph set is used to train a text classification model, which can classify texts directly from the input text dataset to obtain the text type corresponding to each text.

[0068] For example, constructing a training text heterogeneous graph set containing all texts in the training text dataset includes: constructing a short text document D = {d1, d2, ..., d...} m}, word W = {w1, w2, ..., w n Concept C = {c1, c2, ..., c} k The training text heterogeneous graph set is used as the node; where m is the total number of documents in the corpus, n is the number of unique words in the corpus (vocabulary size), and k is the total number of concepts in all documents in the training text dataset. For document nodes, their feature vectors are represented by their term frequency-inverse document frequency (TF-IDF) vectors. The One-Hot vectors of the vocabulary are used as the feature vectors of word nodes, and pre-trained word vectors are used to map concept words to feature vectors.

[0069] For example, in some embodiments, the model uses cross-entropy as the loss function during training, while L2 regularization is used to prevent overfitting. The loss function is:

[0070]

[0071] In the formula, C is the number of categories, D is the size of the training set, Z is the predicted category, y is the actual category, and λ||θ|| 2As a regularization term, gradient descent is used for model optimization.

[0072] Here, the execution entity for steps 101 to 106 can be the processor of the text classification device.

[0073] For example, the text data is the text data corresponding to short texts. Because short texts are too short and lack contextual information, ordinary text classification methods cannot obtain effective classification features from sparse text features. However, the technical solution of this application obtains prior knowledge of the text by acquiring the feature vectors of the concept nodes corresponding to the text; by constructing a text heterogeneous graph based on the feature vectors of document nodes, word nodes, and concept nodes, it can alleviate the feature sparsity problem caused by the lack of context in short texts to a certain extent, enabling the text feature vectors extracted based on this text heterogeneous graph to more accurately represent the features of the text, thereby improving the accuracy of text classification.

[0074] Based on the above embodiments, the method for obtaining the text feature vector corresponding to the text data in step 105 based on the text heterogeneity graph is further illustrated with examples. For instance, Figure 3 This is a schematic diagram illustrating the process of determining text feature vectors in an embodiment of this application. For example... Figure 3 As shown, the process for determining the text feature vector includes:

[0075] Step 301: Based on the text heterogeneity graph, determine at least one type attention weight for each node; wherein the type attention weight is a document-type attention weight, a concept-type attention weight, or a word-type attention weight;

[0076] For example, in some embodiments, determining at least one type attention weight for each node based on the text heterogeneous graph includes: calculating the sum of the feature vectors of the i-th type neighboring nodes of the k-th node in the text heterogeneous graph to obtain the i-th type feature vector of the k-th node; wherein the i-th type is any one of document type, concept type, or word type; and determining the i-th type attention weight of the k-th node based on the feature vector of the k-th node, the feature vectors of the i-th type neighboring nodes, and the i-th type feature vector.

[0077] For example, the i-th type is: document-type (type 1), concept-type (type 2), or word-type neighboring nodes (type 3). The i-th type feature vector of the k-th node is: the document-type feature vector, concept-type feature vector, or word-type feature vector of the k-th node; wherein, the document-type feature vector of the k-th node is the sum of the feature vectors of the document-type neighboring nodes of the k-th node; the concept-type feature vector of the k-th node is the sum of the feature vectors of the concept-type neighboring nodes of the k-th node; and the word-type feature vector of the k-th node is the sum of the feature vectors of the word-type neighboring nodes of the k-th node.

[0078] For example, in some embodiments, the attention weight of the k-th node of type i is determined based on the feature vector of the k-th node, the feature vector of the i-th type neighboring node, and the feature vector of type i. This can be achieved by the following formula:

[0079]

[0080] In the formula, α i This represents the attention weight of type i at node k. h represents the attention vector of type i; k h represents the eigenvector of the k-th node; i Let i represent the feature vector of type i at node k; || represents the connection operation; Leaky ReLU(·) represents the Leaky ReLU activation function; and softmax(·) represents the softmax function.

[0081] in, The attention vector of type i for the current node is obtained by the following formula:

[0082]

[0083] In the formula, V T W, U, and h are all learnable network parameters. k Let h be the eigenvector of the k-th node. k' Let be the feature vector of the i-th type neighboring node of the k-th node.

[0084] For example, in some embodiments, the method further includes: regularizing the type attention value using a softmax function.

[0085] Step 302: Based on the at least one type of attention weight, the feature vector of each node, and the feature vector of the at least one type of neighboring nodes, determine the inter-node attention weight between each node and its neighboring nodes;

[0086] For example, in some embodiments, the inter-node attention weight β between node k (also referred to in this application as the k-th node) and its neighboring node k' is... kk' It can be calculated by the following formula, where node k is the node of type i and node k' is the node of type i':

[0087] β kk' =softmax(LeakyReLU(V T ·α i' [h k ||h k' ]))

[0088] In the formula, V represents the attention vector of node k, which is a learnable network parameter that can be obtained from all neighboring nodes of node k; α i' h represents the attention weight of type i' for node k; k h represents the eigenvector of the k-th node; k' represents the feature vector of the adjacent node k'; || represents the connection operation; Leaky ReLU(·) represents the Leaky ReLU activation function; softmax(·) represents the softmax function.

[0089] For example, in some embodiments, the method further includes: regularizing the attention values ​​between nodes using a softmax function.

[0090] Step 303: Determine the text feature vector based on the inter-node attention weights between all nodes and their neighboring nodes, and the feature vectors of all nodes.

[0091] For example, in some embodiments, determining the text feature vector based on the inter-node attention weights between all nodes and their neighboring nodes, and the feature vectors of all nodes, includes: inputting the inter-node attention weights between all nodes and their neighboring nodes, and the feature vectors of all nodes, into a heterogeneous graph convolutional network to obtain the text feature vector corresponding to the text data.

[0092] For example, in some embodiments, the propagation rules of heterogeneous graph convolutional networks include:

[0093]

[0094] In the formula, H (l+1) Let β represent the text feature vector of the (L+1)th layer. i This represents the attention weight between the i-th type node and its neighboring nodes; W represents the L-th layer feature vector of the i-th type node. i (l)Let be a trainable transformation matrix, and σ(·) be the ReLU activation function. Let I represent the set of type i.

[0095] For example, in some embodiments, the heterogeneous graph convolutional network is a multi-head heterogeneous graph convolutional network built on a multi-head attention mechanism.

[0096] By calculating different types of attention weights and combining them, the attention weights between nodes are calculated. Then, a multi-head heterogeneous graph attention mechanism is added to calculate the importance of adjacent nodes of different types, which makes the text feature extraction based on text heterogeneous graphs have better performance and stronger robustness.

[0097] To better illustrate the purpose of this application, based on the embodiments described above, further examples are provided, proposing a text classification method based on heterogeneous graph attention networks, which can be applied to text classification models. Specifically, this method may include:

[0098] Step 401: Obtain the original text dataset and preprocess it to obtain the text dataset;

[0099] The text dataset contains at least one piece of text data.

[0100] Specifically, the original short text dataset is preprocessed by using scripts to correct spelling errors, clean up useless tags, and remove punctuation and special characters. Then, the nltk toolkit is used to remove high-frequency words that do not affect the semantics of the original text data, and stop word removal is performed to obtain the text dataset.

[0101] For example, in some embodiments, the method further includes: acquiring a training text dataset. The training text dataset contains text data of multiple training texts.

[0102] Step 402: Use the term frequency-inverse document frequency (TF-IDF) vector of the text to represent its feature vector, and use it as the feature vector of the document node;

[0103] Step 403: Use Microsoft Concept Graph to obtain the concept set of the short text, and use pre-trained word vectors to map the concept set into feature vectors, which serve as the feature vectors of the concept nodes;

[0104] For example, by calling the concept graph API, short text data is input into the concept graph to obtain a short text concept set. The specific steps are as follows:

[0105] Step 411: Input the original text dataset t, set the number of expanded words Topk, set the expansion algorithm to EX, the concept graph data information MSCG, and select the mode SelectMode;

[0106] Step 412: Words = splitData(t); Segment each text in the text dataset to obtain the initial feature set Words;

[0107] Step 413: Words = ReduceStopWords(Words); / * Remove stop words from the feature words * /

[0108] Step 414: Select = GetSelectWordSet(MSCG, SelectMode); / * Obtain the corresponding word set Select from the concept map based on the representation mode * /

[0109] Step 415: Sel_Words = Words ∩ Select; / * Select the feature word set Sel_Words to be expanded based on Select * /

[0110] Step 416: Word_dic = AccessAPI(Sel_Words, EX, Topk) / * For each feature word in Sel_Words, call the interface of different conceptualization expansion algorithms. The interface returns the Topk concept words related to the feature, which are then used to form the expanded dictionary Word_dic. * /

[0111] Step 417: d* = GetingExtend(Words, Word_dic); / * Based on Word_dic, extend the original feature words to obtain the conceptualized extended semantic representation d** /

[0112] Step 418: return d*.

[0113] Besides obtaining the concept set of a short text from a concept graph, one can also obtain the relevance values ​​between the concepts in the concept set and the short text. By calling the concept graph API and inputting a short text, one can obtain the concept set C = {<c1, w1>, <c2, w2>, ..., <c1, w3>}, where ci represents the concepts in the concept set, and wi represents the relevance between concept ci and the short text. For example, inputting the short text {Eason Chan was born in Hong Kong} will yield the concept set and relevance values ​​{<top chinese entertainer,0.6> ,<singer,0.4> ,<place,0.2> ,<asian city,0.1>}

[0114] Step 404: Use the One-Hot vectors of the vocabulary as feature vectors for word nodes;

[0115] Step 405: Construct a text heterogeneous graph;

[0116] Specifically, a text heterogeneity graph is constructed for each text in the text dataset; the text heterogeneity graphs of all texts in the text dataset are then integrated to obtain a text heterogeneity graph set. The text heterogeneity graph set is represented as G = (V, E), where V is the set of nodes in the graph and E is the set of edges between nodes.

[0117] Step 406: Determine the weights of edges between nodes in the text heterogeneous graph;

[0118] Specifically, the relevance values ​​between concepts and short texts obtained from the concept graph are used to establish edges between documents and related concepts, and these relevance values ​​are used as the weights of the edges between concept nodes and document nodes.

[0119] Edges are constructed between document nodes and word nodes based on word co-occurrence in the document, and the edge weights are calculated using the TF-IDF algorithm. The term frequency is the number of times a word appears in the document, and the inverse document frequency is the logarithm of the reciprocal of the number of documents containing the word.

[0120] To utilize global word co-occurrence information, a fixed-size sliding window is used to collect co-occurrence statistics across all documents in the corpus. This system uses Point Mutual Information (PMI) to measure the association between words and calculate the weight between two word nodes. The PMI calculation formula is as follows:

[0121]

[0122]

[0123]

[0124] In the formula, represents the semantic interrelationship (PMI) between words i and j, #W(i) is the number of sliding windows containing word i in the corpus, #W(i,j) is the number of sliding windows containing both words i and j, and #W is the total number of sliding windows in the corpus. A positive PMI indicates a high semantic interrelationship between words in the corpus; a negative PMI indicates a low or zero semantic interrelationship between words in the corpus. Edges are added between word pairs with positive PMI values.

[0125] Step 407: In the constructed text heterogeneous graph, the multi-head heterogeneous graph attention mechanism is used to aggregate and calculate each node, and finally aggregate the text feature vector.

[0126] Graph Convolutional Neural Networks (GCNs) can extract spatial features from topological graphs. A GCN is a multi-layer neural network that computes the embedding vector of each node by aggregating the features of its neighboring nodes. For a textual heterogeneous graph G = (V, E), where V and E are the sets of nodes and edges in the graph, respectively, and the node feature matrix X ∈ R... |V|×q The feature vector x containing all nodes v ∈R q With the adjacency matrix A' = A + I and the degree matrix added, the propagation rules of a standard graph convolutional neural network are as follows:

[0127]

[0128] Where H (l) ∈R N×D H represents the hidden features of the Lth layer. (0) =X, W (l) For a trainable transformation matrix, Let be a normalized symmetric adjacency matrix, and σ(·) be the ReLU activation function. This formula represents the aggregation of text feature vectors from layer L to calculate the text feature vectors of layer L+1.

[0129] However, standard GCN cannot be directly applied to the text heterogeneous graph in this application because the text heterogeneous graph in this application has three different types of nodes: document-type, concept-type, and word-type. Different types of nodes have different feature spaces. A common solution is to concatenate the feature vectors of different types of nodes to obtain a new, larger feature space, and fill the irrelevant dimensions of each node with 0 values. However, this method ignores some feature information, which will affect the model's performance.

[0130] To address this issue, this method employs heterogeneous graph convolution, considering the differences between various types of information. It utilizes the transformation matrices of different types of nodes to transform them into a common implicit space. The basic propagation rules of standard heterogeneous graph convolution are as follows:

[0131]

[0132] in It is A ～ A submatrix, |V| represents all nodes, |V t | represents the adjacent nodes of node type t. The above formula represents the relationship between adjacent nodes of type t. Different types of adjacent nodes utilize transformation matrices Transform its feature vector, and then aggregate neighboring nodes of different types to obtain H. (l+1) The characteristics of the nodes take into account the differences between different feature spaces and project them onto the implicit public space. initial

[0133] Specifically, for a given node in a heterogeneous graph, its neighboring nodes of different types will have different effects on it. For neighboring nodes of the same type, more useful information can be aggregated, while for nodes of different types, there will also be some effective information transfer. In order to obtain effective information between nodes and between different types, this method uses a multi-head heterogeneous graph attention mechanism.

[0134] Given a node v (equivalent to the k-th node in this application), type attention learns the weights of neighboring nodes of different types. The feature vector of type t (where t is any one of document, concept, or word type, equivalent to the i-th type in this application) is represented as:

[0135]

[0136] in This represents the adjacency matrix of node v and its neighboring node v', where v' represents the t-type neighboring node of the current node v. The above formula represents the calculation of the eigenvector h of the t-type neighboring node. v' The sum of these elements yields the eigenvector h of type t. t Then, the feature vectors of all types are compared with the feature vector of the current node h. v The attention weight α of type t is calculated using the following formula. t ,

[0137]

[0138] In the formula, α t This represents the t-type attention weights for node v. represents a t-type attention vector; || represents a connection operation; Leaky ReLU(·) represents the Leaky ReLU activation function; softmax(·) represents processing using the softmax function.

[0139] in, The attention vector of type t for the current node can be obtained by the following formula:

[0140]

[0141] In the formula, V T W and U are all learnable network parameters.

[0142] This text classification method also includes: using the softmax function to regularize all type attention values ​​to obtain type attention weights.

[0143] To capture important neighboring node information while reducing the weight of noisy nodes, the model uses node attention to compute neighboring nodes. Given a node v of type t, and its neighboring nodes v'∈N of type t'. v And use the following formula to calculate the attention weights between nodes:

[0144] β vv' =softmax(LeakyReLU(V T ·α t' [h v ||h v' ]))

[0145] Where V is the attention vector, α t' For attention weights of type t', h v h is the feature vector of the current node. v' The feature vectors of the adjacent nodes are then used, and the node attention values ​​are regularized using the softmax function.

[0146] Integrating the attention weights between nodes obtained above into the heterogeneous graph convolution yields a new heterogeneous graph attention network, whose propagation rule is as follows:

[0147]

[0148] In the formula, H (l+1) Let β represent the text feature vector of the (L+1)th layer. t This represents the attention weights calculated between the current node and its neighboring nodes, representing type t. This represents the feature vector of the Lth layer of the current node of type t. Let represent the trainable transformation matrix, and σ(·) be the ReLU activation function. Let I represent a set of type t.

[0149] For example, Figure 4 This is a schematic diagram of the text heterogeneous graph aggregation calculation method in an embodiment of this application. For example... Figure 4 As shown, Document d represents document d i The feature vector of the l-th layer, c1, c2, and c3 respectively represent the features of document d. i The adjacent concept nodes, w1, w2, w3, and w4, respectively represent the nodes adjacent to document d. i The adjacent word nodes are represented by wavy lines with arrows, indicating convolution based on inter-node attention and node feature vectors to obtain document d. i The feature vector of the (l+1)th layer

[0150] To make the model more stable, this system incorporates a multi-head attention mechanism. K attention groups are used for independent computation, each calculating its own attention weights. The results are then concatenated to obtain the features for each head, as shown in the following equation:

[0151]

[0152] In the formula, H (l+1) This represents the text feature vector of the (L+1)th layer, where || is the join operation. Let be the attention weight between the k-th node and other nodes of type t. W represents the feature vector of the Lth layer of the current node of type t. t k Let represent the trainable transformation matrix of the k-th group.

[0153] For the final layer of the network, instead of using connections, an average summation method is used to obtain the final node features:

[0154]

[0155] Calculate adjacent concept nodes c i Word node w i The attention weights are aggregated and calculated together, and a multi-head attention mechanism is used. Different arrow patterns represent independent attention calculations. The node features of the new layer are obtained by connecting or averaging each head. The multi-head heterogeneous graph attention mechanism calculates the importance of different types of neighboring nodes, making the model have better performance and stronger robustness.

[0156] Step 408: Classify the text feature vectors using the softmax function.

[0157] Specifically, after L layers of MHGAT calculation, the text feature vectors corresponding to each text in the heterogeneous text graph can be obtained. Then, the text feature vectors are classified using the softmax function to obtain the text category corresponding to each text.

[0158] For example, in some embodiments, the text classification method further includes: obtaining a training text dataset; and training a text classification model based on the training text dataset.

[0159] For example, in some embodiments, the text classification method further includes: constructing a training text heterogeneous graph set containing all texts in the training text dataset. This training text heterogeneous graph set is used to train a text classification model, which can classify texts directly from the input text dataset to obtain the text type corresponding to each text.

[0160] For example, constructing a training text heterogeneous graph set containing all texts in the training text dataset includes: constructing a short text document D = {d1, d2, ..., d...} m}, word W = {w1, w2, ..., w n Concept C = {c1, c2, ..., c} k The training text heterogeneous graph set is used as the node; where m is the total number of documents in the corpus, n is the number of unique words in the corpus (vocabulary size), and k is the total number of concepts in all documents in the training text dataset. For document nodes, their feature vectors are represented by their term frequency-inverse document frequency (TF-IDF) vectors. The One-Hot vectors of the vocabulary are used as the feature vectors of word nodes, and pre-trained word vectors are used to map concept words to feature vectors.

[0161] For example, in some embodiments, the model uses cross-entropy as the loss function during training, while L2 regularization is used to prevent overfitting. The loss function is:

[0162]

[0163] In the formula, C is the number of categories, D is the size of the training set, Z is the predicted category, y is the actual category, and λ||θ|| 2 As a regularization term, gradient descent is used for model optimization.

[0164] The technical solution of this application, by using not only the feature vectors of document nodes and word nodes, but also the feature vectors of concept nodes when constructing a text heterogeneous graph, obtains prior knowledge in the text. This can alleviate the feature sparsity problem caused by the lack of context in short texts to a certain extent, making the text feature vectors extracted based on the text heterogeneous graph more accurately represent the features of the text, thereby improving the accuracy of text classification. By calculating the attention weights between different types of nodes, type attention weights are obtained, and then combined with type attention weights, dual attention weights between nodes are calculated. Then, a multi-head heterogeneous graph attention mechanism is added to calculate the importance of adjacent nodes of different types, making the aggregation calculation of the text heterogeneous graph have better performance and stronger robustness, thereby obtaining more accurate text feature vectors and improving the accuracy of text classification results.

[0165] Figure 5 This is a schematic diagram of the composition of a text classification device in an embodiment of this application, showing an implementation device 50 for a text classification method. The device 50 specifically includes:

[0166] Module 501 is used to acquire text data;

[0167] Processing module 502 is used to determine the feature vectors of document nodes, concept nodes, and word nodes corresponding to the text data based on the text data.

[0168] The processing module 502 is further configured to construct a text heterogeneous graph based on the feature vectors of the document nodes, the feature vectors of the concept nodes, and the feature vectors of the word nodes;

[0169] The processing module 502 is further configured to determine the weights of the edges between nodes in the text heterogeneous graph;

[0170] The processing module 502 is further configured to obtain the text feature vector corresponding to the text data based on the text heterogeneity graph;

[0171] The processing module 502 is further configured to classify the text feature vector using a classification function to determine the text category.

[0172] In some embodiments, the processing module 502 is configured to determine at least one type attention weight for each node based on the text heterogeneous graph; wherein the type attention weight is a document-type attention weight, a concept-type attention weight, or a word-type attention weight; determine the inter-node attention weight between each node and its neighboring nodes based on the at least one type attention weight, the feature vector of each node, and the feature vector of the at least one type of neighboring nodes; and determine the text feature vector based on the inter-node attention weight between all nodes and their neighboring nodes, and the feature vector of all nodes.

[0173] In some embodiments, the processing module 502 is used to calculate the sum of the feature vectors of the i-th type neighboring nodes of the k-th node in the text heterogeneous graph to obtain the i-th type feature vector of the k-th node; wherein the i-th type is any one of document type, concept type or word type; based on the feature vector of the k-th node, the feature vectors of the i-th type neighboring nodes and the i-th type feature vector, the i-th type attention weight of the k-th node is determined.

[0174] In some embodiments, the processing module 502 is used to input the inter-node attention weights between all nodes and their neighboring nodes, as well as the feature vectors of all nodes, into a heterogeneous graph convolutional network to obtain the text feature vectors corresponding to the text data; wherein, the heterogeneous graph convolutional network is constructed based on a multi-head attention mechanism.

[0175] In some embodiments, the processing module 502 is used to calculate the word frequency-inverse document frequency (TF-IDF) vector of the text data as the feature vector of the document node; obtain the concept set of the text data based on the concept graph; map the concepts in the concept set to feature vectors based on the word vector model to obtain the feature vector of the concept node; and obtain the one-hot code vector of the word node in the text data from a preset vocabulary as the feature vector of the word node.

[0176] In some embodiments, the processing module 502 is configured to, based on a concept graph, obtain at least one concept node corresponding to a document node, and a correlation value between the document node and the at least one concept node; determine the weight of the edge between the document node and the at least one concept node based on the correlation value; determine the weight of the edge between the document node and at least one word node based on the term frequency-inverse document frequency (TF-IDF) algorithm; and determine the weight of the edge between word nodes based on the point mutual information between words.

[0177] In some embodiments, the acquisition module 501 is used to acquire raw text data; preprocess the raw text data to obtain the text data; wherein the preprocessing includes at least one of the following: noise removal, word segmentation, and stop word processing.

[0178] Based on the hardware implementation of each unit in the above-described text classification device, this application also provides another text classification device. Figure 6 This is a schematic diagram of the composition structure of the text classification device in an embodiment of this application. For example... Figure 6 As shown, the device 60 includes: a processor 601 and a memory 602 configured to store computer programs capable of running on the processor;

[0179] The processor 601 is configured to execute the method steps in the foregoing embodiments when running a computer program.

[0180] Of course, in practical applications, such as Figure 6 As shown, the various components in this text classification device are coupled together via a bus system 603. It is understood that the bus system 603 is used to implement communication between these components. In addition to a data bus, the bus system 603 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 6 The general designated all buses as Bus System 603.

[0181] In practical applications, the aforementioned processor can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), controller, microcontroller, and microprocessor. It is understood that, for different devices, the electronic devices used to implement the functions of the aforementioned processor can also be other types, and the embodiments of this application do not specifically limit this.

[0182] The aforementioned memory can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provides instructions and data to the processor.

[0183] In an exemplary embodiment, this application also provides a computer-readable storage medium, such as a memory including a computer program, which can be executed by the processor of a text classification device to perform the steps of the aforementioned method.

[0184] It should be understood that the terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items. The expressions “having,” “may have,” “comprising,” and “including,” or “may include” and “may contain” used herein may be used to indicate the presence of a corresponding feature (e.g., an element such as a number, function, operation, or component), but do not exclude the presence of additional features.

[0185] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and are not necessarily used to describe a specific order or sequence. For example, without departing from the scope of this invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information.

[0186] The technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0187] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatus, and devices can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0188] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0189] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0190] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A text classification method, characterized in that, The method includes: Get text data; Based on the text data, determine the feature vectors of the document nodes, concept nodes, and word nodes corresponding to the text data; A text heterogeneous graph is constructed based on the feature vectors of the document nodes, the feature vectors of the concept nodes, and the feature vectors of the word nodes. Based on the concept graph, obtain at least one concept node corresponding to a document node, and the correlation value between the document node and the at least one concept node; The weight of the edge between the document node and the at least one concept node is determined based on the relevance value; The weight of the edge between the document node and at least one word node is determined based on the term frequency-inverse document frequency (TF-IDF) algorithm. The weights of edges between word nodes are determined based on the mutual information between words. Based on the text heterogeneity graph, at least one type of attention weight is determined for each node; wherein, the type of attention weight is a document-type attention weight, a concept-type attention weight, or a word-type attention weight; Based on the at least one type of attention weight, the feature vector of each node, and the feature vector of the at least one type of neighboring node, determine the inter-node attention weight between each node and its neighboring nodes; Based on the inter-node attention weights between all nodes and their neighboring nodes, and the feature vectors of all nodes, the text feature vectors are determined. The text feature vector is classified using a classification function to determine the text category.

2. The method according to claim 1, characterized in that, The step of determining at least one type attention weight for each node based on the text heterogeneity graph includes: The sum of the feature vectors of the i-th type neighboring nodes of the k-th node in the text heterogeneous graph is calculated to obtain the i-th type feature vector of the k-th node; wherein the i-th type is any one of document type, concept type or word type; Based on the feature vector of the k-th node, the feature vector of the i-th type neighboring node, and the i-th type feature vector, the i-th type attention weight of the k-th node is determined.

3. The method according to claim 1, characterized in that, The process of determining the text feature vector based on the inter-node attention weights between all nodes and their neighboring nodes, and the feature vectors of all nodes, includes: The attention weights between all nodes and their neighbors, as well as the feature vectors of all nodes, are input into a heterogeneous graph convolutional network to obtain the text feature vectors corresponding to the text data. The heterogeneous graph convolutional network is constructed based on a multi-head attention mechanism.

4. The method according to claim 1, characterized in that, The step of determining the feature vectors of document nodes, concept nodes, and word nodes corresponding to the text data based on the text data includes: Calculate the term frequency-inverse document frequency (TF-IDF) vector of the text data and use it as the feature vector of the document node; Based on the concept map, obtain the concept set of the text data; Based on the word vector model, the concepts in the concept set are mapped to feature vectors to obtain the feature vectors of the concept nodes; The one-hot code vectors of word nodes in the text data are obtained from the preset vocabulary and used as the feature vectors of the word nodes.

5. The method according to claim 1, characterized in that, The acquisition of text data includes: Obtain the raw text data; The original text data is preprocessed to obtain the text data; The preprocessing includes at least one of the following: noise removal, word segmentation, and stop word processing.

6. A text classification device, characterized in that, The device includes: The acquisition module is used to acquire text data; The processing module is used to determine the feature vectors of the document nodes, concept nodes, and word nodes corresponding to the text data based on the text data. The processing module is also used to construct a text heterogeneous graph based on the feature vectors of the document nodes, the feature vectors of the concept nodes, and the feature vectors of the word nodes; The processing module is further configured to: obtain at least one concept node corresponding to a document node and a correlation value between the document node and the at least one concept node based on the concept graph; determine the weight of the edge between the document node and the at least one concept node based on the correlation value; determine the weight of the edge between the document node and at least one word node based on the term frequency-inverse document frequency (TF-IDF) algorithm; and determine the weight of the edge between word nodes based on the point mutual information between words. The processing module is further configured to determine at least one type of attention weight for each node based on the text heterogeneity graph; wherein the type of attention weight is a document-type attention weight, a concept-type attention weight, or a word-type attention weight; determine the inter-node attention weight between each node and its neighboring nodes based on the at least one type of attention weight, the feature vector of each node, and the feature vector of at least one type of neighboring nodes; and determine the text feature vector based on the inter-node attention weight between all nodes and their neighboring nodes, and the feature vector of all nodes. The processing module is further configured to classify the text feature vector using a classification function to determine the text category.

7. A text classification device, characterized in that, The device includes: a processor and a memory configured to store computer programs capable of running on the processor. Wherein, when the processor is configured to run the computer program, it performs the steps of the method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-5.

Images