A short text classification method and device based on a hyperbolic graph attention network

Abstract

The application discloses a short text classification method and device based on a hyperbolic graph attention network, which comprises the following steps: firstly, the text is embedded into a hyperbolic space with a negative constant curvature by utilizing the compatibility of the complex hierarchical structure of the text and the tree similarity feature of the hyperbolic space, and the powerful expression ability of the hyperbolic flow representation is fully utilized; then, a hyperbolic graph attention network is designed to fuse node features and edge features and enhance the aggregation ability of key local information in the text; finally, a label text interaction function based on the geodesic distance in the hyperbolic space is used to further guide the text feature learning, so as to improve the text classification accuracy. By aggregating the modeling ability of the hyperbolic geometry for the hierarchical structure graph data and the learning ability of the attention mechanism for the text features, the hierarchical structure of the text information is fully mined, the correlation between the label and the text feature learning is effectively utilized, and the learning of the text embedding representation is jointly strengthened, so that the text classification effect is improved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and more specifically, to a short text classification method and apparatus based on hyperbolic graph attention networks. Background Technology

[0002] Existing short text classification methods mainly fall into three categories: those based on traditional machine learning, those based on deep learning, and those based on graph neural networks. Traditional machine learning research on text classification primarily focuses on feature learning and classification algorithms.

[0003] Deep learning-based text classification research falls into two categories. One category focuses on designing end-to-end neural networks. While these networks enrich the semantic features of text and achieve excellent results, they also increase model complexity. The other category focuses on designing models based on word and document embeddings. Recent studies have shown that the success of deep learning in text classification largely depends on the effectiveness of word embeddings. Graph neural networks have shown significant advantages in graph-related tasks because they can learn arbitrary data with graph structures by extending deep neural networks, explicitly encoding node attributes and their interactions, and implicitly learning higher-order dependencies.

[0004] Traditional machine learning transforms an input vector through various translations, scaling, and nonlinear transformations to obtain the desired output. Essentially, it processes the input geometric objects (such as points, lines, or surfaces) in Euclidean space using geometric transformations. However, Euclidean geometry has many limitations in its expressive power, such as being unsuitable for embedding tree or graph data structures, thus failing to fully represent the structural information in text data. Recent research indicates that even with fewer dimensional embedding vectors, Euclidean space suffers from representation collapse, further contributing to poor text classification performance. Summary of the Invention

[0005] Therefore, a short text classification method and apparatus based on hyperbolic graph attention networks is needed. First, leveraging the compatibility between the complex hierarchical structure of text and the tree similarity features of hyperbolic space, the text is embedded into a hyperbolic space with negative constant curvature, fully utilizing the powerful expressive capabilities of hyperbolic popular representation. Then, a hyperbolic graph attention network is designed, fusing node and edge features to enhance the aggregation ability of key local information in the text. Finally, a label-text interaction function based on geodesic distance in hyperbolic space is used to further guide text feature learning, thereby improving text classification accuracy.

[0006] The technical solutions adopted by the present invention to solve the above-mentioned technical problems include:

[0007] In a first aspect, the present invention provides a short text classification method based on a hyperbolic graph attention network, the method comprising:

[0008] The text data is preprocessed, and the processed words and text are used as nodes to construct a text graph, where the nodes include word nodes and text nodes.

[0009] The initial Euclidean representation of each node in the text is transformed into an initial hyperbolic feature using an exponential mapping function, and the initial hyperbolic feature is transformed into a higher-order hyperbolic feature using a hyperbolic transformation.

[0010] An improved hyperbolic attention mechanism is used to aggregate the weights of the node itself, the weights of the neighboring nodes, and the edge weights, and the higher-order hyperbolic features are processed into hyperbolic text features.

[0011] The distance between hyperbolic text features and hyperbolic label features is calculated using a distance mapping function, and the label to which the text data belongs is obtained by using the distance prediction.

[0012] In a second aspect, the present invention also provides a short text classification device based on a hyperbolic graph attention network, which is applied to the method described in the first aspect of the present invention. The device includes a data processing module, a hyperbolic mapping and linear transformation module, a graph attention module, and a label optimization module.

[0013] The data processing module is used to preprocess the text data and construct a text graph by using the processed words and text as nodes, wherein the nodes include word nodes and text nodes.

[0014] The hyperbolic mapping and linear transformation module is used to convert the initial Euclidean representation of each node in the text into an initial hyperbolic feature using an exponential mapping function, and to convert the initial hyperbolic feature into a higher-order hyperbolic feature using a hyperbolic transformation.

[0015] The graph attention module is used to aggregate the weights of its own nodes, neighboring nodes, and edge weights using an improved hyperbolic attention mechanism, and to process the higher-order hyperbolic features into hyperbolic text features.

[0016] The label prediction module is used to calculate the distance between hyperbolic text features and hyperbolic label features using a distance mapping function, and then uses the distance prediction to obtain the label to which the text data belongs.

[0017] The beneficial effects of this invention are:

[0018] This invention employs a hyperbolic graph attention network to learn attention coefficients based on the hyperbolic representation of nodes. This enhances the extraction of important local features and fully utilizes the statistical information contained in the edge structure to generate a high-order text representation containing deep semantic information. The text representation is further optimized using a mapping function between label information and hyperbolic distance for training and prediction. The overall architecture reduces the difficulty of distinguishing confusing categories while maintaining a lower embedding vector dimension, strengthens the utilization of the text's hierarchical structure, reduces the sparsity of the text graph, and improves the accuracy of model prediction, resulting in better short text classification performance. This invention combines the modeling capabilities of hyperbolic geometry for hierarchical graph data with the learning capabilities of the attention mechanism for text features. It fully explores the hierarchical structure of text information while effectively utilizing the correlation between label and text feature learning to jointly strengthen the learned text embedding representation, thereby improving the effectiveness of text classification. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the short text classification method based on hyperbolic graph attention network described in an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of the model architecture of the short text classification method based on hyperbolic graph attention network described in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] like Figures 1-2 As shown, a short text classification method based on hyperbolic graph attention networks includes, but is not limited to, the following steps:

[0023] 101. Perform data preprocessing on the text data, and construct a text graph by using the processed words and text as nodes, wherein the nodes include word nodes and text nodes;

[0024] In one embodiment, the text data is first processed by removing duplicate and excessively short text data, as well as special symbols, garbled characters, and meaningless stop words. Word embeddings are used to obtain the initial Euclidean representation of word nodes. The mutual information of word nodes within a sliding window of width w is used to construct edges between word nodes. A relevance scoring algorithm is used to construct edges between text nodes and word nodes. A text graph is then constructed by connecting word nodes, text nodes, and corresponding edges.

[0025] Therefore, it is understood that, unless otherwise specified, the nodes in this invention can refer to either text nodes or word nodes. When a node is a text node, its corresponding neighbor nodes can only be word nodes. When a node is a word node, its corresponding neighbor nodes can be either word nodes or text nodes. This invention optimizes the learning of text node features by learning the relationships between word nodes and between text nodes, so that the resulting hyperbolic text features can reflect the characteristics of short texts, resulting in better short text classification performance.

[0026] In some embodiments, this embodiment uses the GloVe word embedding method to obtain the initial Euclidean representation of words, uses the mutual information (PMI) of word nodes within a sliding window of width w to construct edges between word nodes, and uses the BM25 algorithm to construct edges between text nodes and word nodes. The weight is 1 when nodes form a self-loop, and 0 otherwise. The weight expression is as follows:

[0027]

[0028] Among them, PMI is a commonly used word association metric:

[0029]

[0030]

[0031]

[0032] Where #W(i) represents the number of windows in the data that contain word node i, #W(i,j) is the number of windows in the dataset that contain both word node i and word node j, and #W is the total number of windows in the dataset.

[0033] The BM25 algorithm is a common formula used for relevance scoring. It calculates and sums the relevance scores of all words and documents, and the relevance score of each word is affected by TF-IDF.

[0034] TF-IDF is a mathematical statistical method. TF stands for Term Frequency, representing the number of times or frequency a word appears in a text. The more times a word appears, the more important it is. IDF stands for Inverse Document Frequency; if a word appears in multiple texts, it is considered a common word. TF-IDF is the product of TF and IDF; a higher value indicates a more important word.

[0035] 102. The initial Euclidean representation of each node in the text is transformed into a hyperbolic manifold representation using an exponential mapping, and further transformed into a higher-order hyperbolic representation using a hyperbolic transformation.

[0036] In this embodiment of the invention, hyperbolic space is a space with negative constant curvature, meaning the curvature at any point in the space is negative constant. Many models can be used to describe hyperbolic space, the most common being the Poincaré disk model; each hyperbolic space can be described by a corresponding Poincaré disk. Vector space can be seen as a natural, native description of Euclidean space, making vector computation easy. Correspondingly, gyrovector space provides a similar description for hyperbolic space. Therefore, vector or matrix computation operations in hyperbolic space can be defined in gyrovector space, and its expression reflects the manifold characteristics of hyperbolic space. Exponential and logarithmic transformations can achieve mutual mapping between the local gyrovector space and the tangent space at a point; the logarithmic transformation maps the point to the tangent space, and the exponential transformation maps it back to the gyrovector space.

[0037] In one embodiment, taking a word node as an example, in order to enable the word node features to propagate computation in hyperbolic space, they are first projected onto hyperbolic space using an exponential mapping function, i.e.:

[0038]

[0039] Among them, t i This represents the characteristic of node i in Euclidean space. Let m represent a point in hyperbolic space; This is represented as the tangent space at point m. The feature t of the pair of nodes at point m i The initial hyperbolic features after exponential mapping. c represents mapping the feature to a hyperbolic space with curvature of -1 / c, and the radius of the Poincaré sphere model used to describe the hyperbolic space is... Represents Möbius strip.

[0040] make When m = 0:

[0041]

[0042] Representing Möbiuska, This allows us to obtain the characteristics of the nodes in hyperbolic space. To achieve better representational power, a linear transformation method parameterized by the weight-sharing matrix M is used to convert the hyperbolic features into a higher-order representation. Unlike matrix-vector multiplication in Euclidean space, Möbius multiplication is required in hyperbolic space. Therefore, the higher-order hyperbolic feature of each node can be represented as:

[0043]

[0044] Among them, h i Let M represent the higher-order hyperbolic feature of node i, and M represent the weight sharing matrix. This represents Möbius multiplication. The feature t of the pair of nodes at point 0 i The initial hyperbolic features after exponential mapping. Let Mq represent the radius of the Poincaré sphere model. Möbius multiplication satisfies the matrix associative law, and if Mq... i =0, then h i As a representation of node i in hyperbolic space, it can also be seen as a hidden layer feature of the hyperbolic attention mechanism.

[0045] 103. An improved hyperbolic attention mechanism is used to aggregate the weights of the node itself, the weights of the neighboring nodes, and the edge weights, and the higher-order hyperbolic features are processed into hyperbolic text features.

[0046] In one embodiment, the purpose of using a hyperbolic attention mechanism is to differentiate between different neighboring nodes by leveraging the hyperbolic similarity between the center node and its neighboring nodes. Attention coefficients based on the hyperbolic representation of nodes are learned, and these coefficients are normalized. An improved attention weight is calculated using the node's own weight and the corresponding edge weights. The improved attention weight is then used to aggregate the higher-order hyperbolic features of each node to obtain the hyperbolic text features of each text node. This embodiment enhances the extraction capability of locally important features and obtains aggregated representations in hyperbolic space based on the attention coefficients.

[0047] In some embodiments, the improved hyperbolic attention mechanism includes using the hyperbolic similarity between higher-order hyperbolic features between nodes as a first attention coefficient; normalizing the first attention coefficient, and using the normalized first attention coefficient between nodes to sum with the Möbius of the initial hyperbolic features to calculate the improved first attention weight; and using the improved first attention weight to aggregate the higher-order hyperbolic features of each node to obtain the single-head hyperbolic text features of each text node.

[0048] The first attention coefficient of node i to node j is represented by α. ij =f(h i ,h j ), where f represents a function for calculating inter-node attention based on hyperbolic distance:

[0049]

[0050]

[0051] Here, <.,.> represents the inner product in Euclidean space. Since hyperbolic space is a metric space, unlike the inner product in Euclidean space, hyperbolic distance satisfies the triangle inequality. Therefore, self-attention can preserve transitivity between nodes. Given node i and its attention coefficient α... ii =f(h i ,h i If N = 0, then the attention coefficient between a node and itself is 0. The attention coefficient between a node and its neighbors is always greater than its self-attention coefficient. To facilitate comparison of the attention coefficients of neighboring nodes, let N = 0. i Let i represent the set of neighboring nodes. The attention function is normalized using the softmax function.

[0052]

[0053] Unlike traditional graph attention networks (GAT), the hyperbolic graph attention network of this invention uses hyperbolic text features that include their own weights and corresponding edge weights. Recorded as By fully utilizing the statistical information contained in the edge structure, a higher-order text representation can be generated:

[0054]

[0055] Among them, h i ′ represents the hyperbolic text feature of node i. This represents the summation of the Möbius method, where φ represents the hyperbolic nonlinear variation. This represents the improved attention weights between node i and node j. Represented as Möbius scalar multiplication, Represented as Möbius strip, h i h represents the higher-order hyperbolic feature of node i. j N represents the higher-order hyperbolic feature of node j. i Let i represent the set of neighboring nodes. Represented as Möbius scalar multiplication, specifically:

[0056]

[0057] When c approaches 0, the expression transforms into Euclidean scalar multiplication:

[0058]

[0059] Among them, Möbius scalar multiplication satisfies the scalar addition distribution and scalar multiplication associativity.

[0060] In a preferred embodiment of the invention, multi-head attention can improve the generalization ability of the model. Multiple single-head attention results are concatenated in Euclidean space, and the space of the concatenated attention is the Cartesian product of the spaces of each single-head attention. Therefore, the operation of multi-head attention in Euclidean space is the same as that of single-head attention. Unlike Euclidean space, the product of hyperbolic spaces is not equal to a higher-dimensional hyperbolic space. This means that multi-head attention operations in Euclidean space cannot be directly applied to hyperbolic space. Therefore, a hyperbolic space multi-head attention algorithm needs to be designed.

[0061] Based on the above embodiments, the improved hyperbolic attention mechanism in this embodiment further includes using the higher-order hyperbolic text features of the nodes to further determine the higher-order latent semantic features of the nodes in the hyperbolic space, using the hyperbolic similarity of the higher-order latent semantic features of the nodes as the second attention coefficient between the nodes; normalizing the second attention coefficient, using the normalized second attention coefficient between the nodes and the initial hyperbolic features to calculate the improved second attention weight; and using the improved second attention weight to concatenate the single-head hyperbolic text features of each text node to obtain the multi-head attention hyperbolic text features of each text node.

[0062] First, represent h with K m-dimensional hyperbolic features. i (1) h i (2) , ..., h i (K) The higher-order latent semantic features of node i are constructed by concatenating the components:

[0063]

[0064] Based on the aforementioned high-order latent semantic features, the method for calculating the multi-head hyperbolic attention coefficient is designed as follows:

[0065] α′ i,j =-f p (H i H j )

[0066] Where, α′ i,j -f represents the second attention coefficient between node i and node j. p (.,.) denotes the distance function in the hyperbolic product:

[0067]

[0068] The second attention coefficients of all neighbors of node i are normalized using the softmax function:

[0069]

[0070] The k-th single-headed hyperbolic text representation of node i can be represented as:

[0071]

[0072] Among them, h i '( k ) represents the single-headed hyperbolic text feature of node i in the k-th dimension; φ represents the hyperbolic nonlinear transformation. This represents the summation of Möbius stripes. This represents the improved second attention weight between node i and node j. ω' ij This represents the normalized second attention coefficient between node i and node j. The edge feature e between nodes i and j at node 0 is represented by the edge feature e. ij Initial hyperbolic features after exponential mapping; Represented as and h j (k) Möbius scalar multiplication, Represented as Möbius scalar multiplication, Represented as Möbius strip, h j (k) N represents the higher-order hyperbolic text feature of node j in the k-th dimension. i h represents the set of neighboring nodes of node i; i (k) This represents the higher-order hyperbolic text feature of node j in the k-th dimension.

[0073] Next, concatenating all K single-headed hyperbolic text features of node i yields:

[0074]

[0075] Among them, H' i Let || represent the multi-head hyperbolic text feature resulting from concatenating all K single-head hyperbolic text features of node i, where || represents the multi-head concatenation symbol.

[0076] The computational complexity increases with the number of nodes, and the Möbius method, which is neither commutative nor associative, can only calculate results sequentially. Using a logarithmic mapping method, mapping nodes to the tangent space for matrix operations, and then using an exponential mapping method to map them back to the hyperbolic space, can significantly reduce the computational complexity.

[0077] In hyperbolic space, for node v i ,v j The logarithmic mapping is specifically as follows:

[0078]

[0079] When v i When = 0, Therefore, the k-th single-headed hyperbolic text feature of node i can be represented as:

[0080]

[0081] The hyperbolic nonlinear transformation method used is specifically as follows:

[0082]

[0083] Where c and c' represent the curvature of the hyperbolic space of the current layer, respectively. The curvature of the hyperbolic space of the next layer is Since the two hyperbolic spaces share a tangent space at point 0, the curvature of the two layers can be smoothed during nonlinear activation. It is a nonlinear transformation function.

[0084] 104. Calculate the distance between hyperbolic text features and hyperbolic label features using a distance mapping function, and use the distance to predict the label to which the text data belongs.

[0085] Furthermore, in one embodiment, the lack of label information in the text leads to poor performance in few-sample classification. Label information not only retains deep semantic information but also contains various dependencies such as inclusion, mutual exclusion, and association. Learning text features for specific labels requires learning both the semantic information of the labels and their dependencies simultaneously.

[0086] Learning text features based on label semantic relationships primarily involves calculating the semantic similarity weight matrix between labels and text to guide the generation of semantic representations of the labeled text. In Euclidean space, researchers often use cosine similarity distance to measure the similarity between words. However, hyperbolic space, with its high cohesion and low coupling, cannot guarantee that the angle between the word vectors of similar words will be closer. Therefore, this invention requires a similarity metric based on hyperbolic geodesic distance. Simply using hyperbolic distance is computationally expensive; a distance mapping function needs to be designed, satisfying the interval [0,1]. Furthermore, the value of this function increases with the greater the correlation between words and decreases accordingly, thereby learning text features specific to a particular label and inferring the label to which the text belongs.

[0087] For single-head attention text representation models, the distance mapping function sim(h') is used. i ,y p The calculation result is used as the interaction value between text and tag semantics.

[0088]

[0089] fD (d)=0.5-arctan(8d-20) / π

[0090] Among them, f D : [0,∞]→[0,1], and satisfies monotonically decreasing, f D (0) = 1, f D (∞) = 0, d represents the hyperbolic text feature h' i With hyperbolic label semantic features y p The geodesic distance. Values ​​with a similarity below ε are recorded as 0.

[0091] For multi-head attention text representation models, the distance mapping function sim(H') is used. i ,y p The calculation result is used as the interaction value between text and tag semantics.

[0092] Taking single-head attention text representation as an example, for the l-th label, its semantically adapted text representation is as follows:

[0093]

[0094] This represents the number of nodes contained in the p-th text, where p∈[0,S] and S is the number of document samples in the dataset.

[0095] Learning text features based on label dependencies primarily involves constructing a label relationship co-occurrence matrix to guide the generation of label-text relationship representations. First, a label relationship co-occurrence matrix is ​​built based on the label set, constructing a label relationship graph. Then, the node2vector algorithm is used to maximize the occurrence probability of neighboring nodes of a label node, generating label relationship vectors and mapping them to a hyperbolic space to generate hyperbolic label relationship features r. p The distance mapping function is then used to calculate the interaction value between the text and label dependency features. For the l-th label, the text representation adapted to its label relationship is:

[0096]

[0097] The impact of label semantics and label relations on text often differs across different training corpora. A coefficient β is used to assign weights to these two types of data.

[0098] T l =βD l +(1-β)R l

[0099] After obtaining all text features, they are mapped to Euclidean space and input into the fully connected layer and output layer to calculate the probability of each label:

[0100] P l=σ(W e ψ(W f T l ))

[0101] Where σ is the sigmoid activation function, ψ is the nonlinear transformation function, and W... e W f The computational cost of the above algorithm increases significantly with the size of the label data, where the weights are the parameters. Since most documents contain only a small number of labels, negative sampling is used to reduce the computational cost. The loss function is minimized as follows:

[0102]

[0103] Among them, L + L represents the set of real labels - This represents the set of negative example labels obtained from negative sampling.

[0104] It is understood that, in the embodiments of the present invention, by utilizing the relationship between word nodes and the relationship between word nodes and text nodes, better text features are learned. Therefore, when calculating the distance between hyperbolic text features and hyperbolic label features, it is only necessary to calculate the distance between the hyperbolic text features and hyperbolic label features corresponding to the text nodes of the text data to be processed, thereby determining the label to which the text node belongs.

[0105] In some embodiments of the present invention, the present invention provides a short text classification device based on a hyperbolic graph attention network, which is applied to a short text classification method based on a hyperbolic graph attention network. The device includes a data processing module, a hyperbolic mapping and linear transformation module, a graph attention module, and a label optimization module.

[0106] The data processing module is used to preprocess the text data and construct a text graph by using the processed words and text as nodes, wherein the nodes include word nodes and text nodes.

[0107] The hyperbolic mapping and linear transformation module is used to convert the initial Euclidean representation of each node in the text into an initial hyperbolic feature using an exponential mapping function, and to convert the initial hyperbolic feature into a higher-order hyperbolic feature using a hyperbolic transformation.

[0108] The graph attention module is used to aggregate the weights of its own nodes, neighboring nodes, and edge weights using an improved hyperbolic attention mechanism, and to process the higher-order hyperbolic features into hyperbolic text features.

[0109] The label prediction module is used to calculate the distance between hyperbolic text features and hyperbolic label features using a distance mapping function, and then uses the distance prediction to obtain the label to which the text data belongs.

[0110] In some embodiments of the present invention, a terminal for a short text classification method based on a hyperbolic graph attention network is also provided, including a processor and a memory. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in the above-mentioned short text classification method based on a hyperbolic graph attention network.

[0111] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include ROM, RAM, disk, or optical disk, etc.

[0112] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for short text classification based on hyperbolic graph attention network, characterized in that, The method comprises the following steps: Text data is pre-processed, and the processed words and texts are respectively taken as nodes to construct a text graph, wherein the nodes comprise word nodes and text nodes; An exponential mapping function is used to convert an initial Euclidean representation of each node in the text into an initial hyperbolic feature, and a hyperbolic linear transformation is used to convert the initial hyperbolic feature into a high-order hyperbolic feature; An improved hyperbolic attention mechanism is used to aggregate the node weight, the neighbor node weight and the edge weight, and the high-order hyperbolic feature is processed into a hyperbolic text feature; The improved hyperbolic attention mechanism comprises the following steps: taking the hyperbolic similarity between the high-order hyperbolic features of the nodes as a first attention coefficient; performing normalization processing on the first attention coefficient, and using the normalized first attention coefficient between the nodes and the initial hyperbolic feature to calculate a first attention weight; using the first attention weight to aggregate the high-order hyperbolic features of the nodes to obtain a single-head hyperbolic text feature of each node; determining the high-order latent semantic feature of the node in the hyperbolic space by using the multiple high-order hyperbolic text features of the node, taking the hyperbolic similarity of the high-order latent semantic features of the nodes as a second attention coefficient between the nodes; performing normalization processing on the second attention coefficient, and using the normalized second attention coefficient between the nodes and the initial hyperbolic feature to calculate a second attention weight; using the second attention weight to obtain the single-head hyperbolic text feature of each node and splicing to obtain a multi-head attention hyperbolic text feature of each node; A distance mapping function is used to calculate the distance between the hyperbolic text feature and the hyperbolic label feature, and the distance is used to predict the label to which the text data belongs.

2. The short text classification method based on hyperbolic graph attention network according to claim 1, characterized in that, The data preprocessing comprises: The text data is segmented, and repeated data, short data, random codes and stop words without actual meaning in the text are deleted.

3. The short text classification method based on hyperbolic graph attention network according to claim 1, characterized in that, The text graph is constructed by: An initial Euclidean representation of the word nodes is obtained using word embeddings, edges between word nodes are constructed using mutual information of word nodes within a sliding window of width A text graph is constructed by connecting word nodes, text nodes and corresponding edges. An initial Euclidean representation of the word nodes is obtained using word embeddings, edges between word nodes are constructed using mutual information of word nodes within a sliding window of width A text graph is constructed by connecting word nodes, text nodes and corresponding edges.

4. The short text classification method based on hyperbolic graph attention network according to claim 1, characterized in that, The single-head hyperbolic text feature is represented as: wherein, denotes the cumulative summation of the Möbius addition, denotes the hyperbolic nonlinear transformation, denotes the node the first attention weight between the node and the node , denotes the normalized first attention coefficient between the node and the node , denotes the initial hyperbolic feature after exponential mapping of the edge feature between the node 0 and the node , denotes the initial hyperbolic feature after exponential mapping of the edge feature between the node 0 and the node and denotes the Möbius scalar multiplication of denotes the Möbius addition of denotes the high-order hyperbolic feature of the node , denotes the high-order hyperbolic feature of the node , denotes the neighbor node set of the node .

5. The short text classification method based on hyperbolic graph attention network according to claim 1, characterized in that, The multi-head attention hyperbolic text feature is represented as: wherein, denotes the multi-headed hyperbolic text feature after concatenation of all K single-headed hyperbolic text features of node i, denotes a multi-headed concatenation symbol, denotes the k-th single-headed hyperbolic text feature of node denotes the k-th single-headed hyperbolic text feature of node is denoted as: wherein, represents a hyperbolic nonlinear transformation, represents an accumulative summation of the Möbius addition, represents a node and a second attention weight between the node , , represents a normalized second attention coefficient between the node and the node , represents an initial hyperbolic feature after an exponential mapping on an edge feature between the node and the node , represents a Möbius scalar multiplication of and , represents a Möbius addition of represents a high-order hyperbolic text feature of the node j in the k-th dimension, represents a neighbor node set of the node , represents a high-order hyperbolic text feature of the node in the k-th dimension.​ 6. The short text classification method based on hyperbolic graph attention network according to claim 5, characterized in that, The calculation process of the Möbius scalar multiplication includes mapping the nodes and the nodes between the Möbius scalar mapping into a weight vector value; when the node is 0, using the logarithmic mapping method, mapping the node to the tangent space for matrix operation, and then using the exponential mapping method to map it back to the hyperbolic space.

7. The short text classification method based on hyperbolic graph attention network according to claim 1, characterized in that, The distance mapping function is specifically: wherein, represents a hyperbolic text feature and a distance mapping function of a hyperbolic label feature , let be a hyperbolic text feature and a geodesic distance of a hyperbolic label feature , , , and satisfy monotone decreasing, , represents a preset threshold, and a value with a similarity lower than is recorded as 0.​ 8. A hyperbolic graph attention network-based short text classification method device, applied to the hyperbolic graph attention network-based short text classification method of any one of claims 1-7, characterized in that, The device comprises a data processing module, a hyperbolic mapping and linear transformation module, a graph attention module and a label prediction module; The data processing module is used to pre-process the text data, and the processed words and texts are respectively taken as nodes to construct a text graph, wherein the nodes comprise word nodes and text nodes; The hyperbolic mapping and linear transformation module is used to convert an initial Euclidean representation of each node in the text into an initial hyperbolic feature by using an exponential mapping function, and convert the initial hyperbolic feature into a high-order hyperbolic feature by using a hyperbolic linear transformation; The graph attention module is used to aggregate the node weight, the neighbor node weight and the edge weight by using an improved hyperbolic attention mechanism, and process the high-order hyperbolic feature into a hyperbolic text feature; The improved hyperbolic attention mechanism comprises: using the hyperbolic similarity between high-order hyperbolic features between nodes as a first attention coefficient; performing normalization processing on the first attention coefficient, using the normalized first attention coefficient between nodes and the Mobius sum of the initial hyperbolic feature to calculate a first attention weight; using the first attention weight to aggregate the high-order hyperbolic features of each node to obtain a single-head hyperbolic text feature of each node; determining the high-order latent semantic feature of the node in the hyperbolic space by using the multiple high-order hyperbolic text features of the node, using the hyperbolic similarity of the high-order latent semantic feature of the node as a second attention coefficient between nodes; performing normalization processing on the second attention coefficient, using the normalized second attention coefficient between nodes and the initial hyperbolic feature to calculate a second attention weight; using the second attention weight to obtain the single-head hyperbolic text feature of each node and splicing to obtain a multi-head attention hyperbolic text feature of each node; The label prediction module is used for calculating the hyperbolic text feature and the hyperbolic label feature distance by using a distance mapping function, and predicting the label to which the text data belongs by using the distance.

Images