A topic classification method based on hyperbolic graph convolutional network and hierarchical clustering

Abstract

The application discloses a topic classification method based on a hyperbolic graph convolutional network and hierarchical clustering, and belongs to the technical field of text topic classification in natural language processing. The method comprises the following steps: preprocessing microblog text data, including data cleaning, Chinese word segmentation and removal of stop words; and calculating a feature vector by using a TF-IDF method. The preprocessed text data is mapped to a hyperbolic space, information is converted between Euclidean space and hyperbolic space by using exponential mapping and logarithmic mapping, and the hierarchical structure characteristics of the hyperbolic space are retained. The hyperbolic distance between nodes is calculated, and a hierarchical clustering algorithm is combined to capture the relationship in the hierarchical structure data. Similar or related nodes can obtain a higher weight in the information aggregation process of the hyperbolic graph convolutional network, the hierarchical structure of the text data is mined, the hierarchical structure relationship of the microblog text data is captured by training the hyperbolic graph convolutional neural network, and high-quality topic classification is realized by using the trained hyperbolic graph convolutional neural network.

Description

TECHNICAL FIELD

[0001] The application relates to a neural network text classification method, in particular to a method based on a hyperbolic graph convolution network and a hierarchical clustering method, and belongs to the technical field of text topic classification in natural language processing. BACKGROUND

[0002] With the continuous advancement of the informationization process of the country, the Internet has become an important platform for the majority of netizens to express opinions and wishes, and gradually formed a network information with considerable influence. The information network is a typical social complex network with the characteristics of a complex system, so it is necessary to learn the representation of the complex network. Since the network in the real world usually shows a non-Euclidean structure, has a hierarchical structure similar to a tree, and some also satisfy the power law distribution, etc., which leads to distortion and dimension explosion in traditional Euclidean space embedding. The content published on the microblog can be organized into different levels according to the theme or label. This topic level can help understand the microblog content, but the existing microblog text topic classification mostly uses a latent Dirichlet allocation model and a convolutional neural network model to classify the microblog text topic, which has the problems of insufficient hierarchical structure mining depth, single network topology structure information and node attribute information learning. Therefore, a microblog text topic classification method based on a hyperbolic graph convolution neural network and hierarchical clustering is proposed. SUMMARY

[0003] The application aims to provide a topic classification method based on a hyperbolic graph convolution network and hierarchical clustering. The method preprocesses the microblog text data, including data cleaning, removal of stop words and Chinese word segmentation. The TF-IDF method is used to calculate the feature vector. The cosine similarity is used to establish the adjacency matrix of the Euclidean space. Further, the data is processed in the hyperbolic space, mapped to the hyperbolic space and the hyperbolic distance between nodes is calculated. The hierarchical clustering algorithm is combined to establish the adjacency matrix of the hyperbolic space. The two matrices are fused as the input of the graph convolution. The application captures the hierarchical structure relationship of the microblog text data by training the hyperbolic graph convolution network, and realizes high-quality topic classification. The application is particularly suitable for processing the topic classification and public information monitoring of social information networks and other complex networks. The application solves the classification difficulty of the traditional method in the high-dimensional, non-Euclidean structure data space, and improves the representation and classification ability of the microblog text in the network data.

[0004] The application is realized by the following technical scheme:

[0005] The application discloses a topic classification method based on hyperbolic graph convolutional network and hierarchical clustering, which maps the preprocessed text data to hyperbolic space, converts information between Euclidean space and hyperbolic space by using exponential mapping and logarithmic mapping, and preserves the hierarchical structure characteristics of the hyperbolic space. The hyperbolic distance between nodes is calculated, and a hierarchical clustering algorithm is combined to capture the relationship in the hierarchical structure data. Similar or related nodes can obtain a higher weight in the information aggregation process of the hyperbolic graph convolutional network, and the hierarchical structure of the text data is mined. The hierarchical structure relationship of microblog text data is captured by training the hyperbolic graph convolutional network, and the trained hyperbolic graph convolutional network is used to realize high-quality topic classification.

[0006] The application discloses a microblog text topic classification method based on hyperbolic graph convolutional network and hierarchical clustering, which comprises the following steps:

[0007] Step one: collect microblog texts under a specified topic, preprocess the text data, and label the topic categories of the microblog texts to obtain a labeled microblog text dataset, and use the microblog text dataset as the input for microblog text vectorization in subsequent step two. The preprocessing includes data cleaning, Chinese word segmentation and stop word removal.

[0008] Step 1.1: collect microblog texts under a specified topic.

[0009] Step 1.2: clean the microblog text data collected in step 1.1 according to a preset standard information cleaning principle to remove irrelevant information and obtain microblog texts with standard information. The preset standard information cleaning principle includes cleaning and filtering microblog texts with a length less than a preset length threshold and microblog texts containing only expressions without text information.

[0010] Step 1.3: Chinese word segmentation is performed on the microblog texts with standard information obtained in step 1.2. Chinese text is different from English text, which is composed of words and separated by spaces. Chinese text is coherent into a sentence, so a Chinese word segmentation tool is used for Chinese word segmentation to obtain Chinese segmented microblog texts for step 1.4.

[0011] Step 1.4: According to the stop word list, meaningless information in the segmented microblog text is removed, including Chinese auxiliary words and conjunctions. Chinese auxiliary words include "of", "is", "in", and conjunctions include "and", "but". Although Chinese auxiliary words and conjunctions play a role in the grammatical structure of the sentence, they provide less information value for the topic classification task of the text. Removing these high-frequency but low-information words allows the hyperbolic graph convolutional network to focus more on the key words that have a substantial contribution to the meaning of the text, resulting in a stop word removed microblog text, reducing the amount of microblog text data processed by the subsequent hyperbolic graph convolutional network, and improving the accuracy and efficiency of the hyperbolic graph convolutional network classification.

[0012] Step 1.5: The microblog text data set after data cleaning, Chinese segmentation, and stop word removal preprocessing is labeled with topic categories to obtain the labeled microblog text data set, which is used as input for microblog text vectorization in subsequent step two.

[0013] Step two: For the microblog text in the microblog text data set obtained in step one, the feature vector of the microblog text is calculated in Euclidean space E by the TF-IDF method, and the microblog text is converted into vector form, where each dimension of the vector corresponds to the TF-IDF value of a word, i.e. the feature vector x corresponding to each microblog text is obtained. E .

[0014] Step 2.1: For the microblog text in the microblog text data set obtained in step one, in Euclidean space E, the number of times a word appears in the microblog text is normalized according to the term frequency TF normalization formula as shown in formula (1) to obtain the normalized term frequency TF of the microblog text. The numerator of the TF normalization formula is the number of times the word appears in the document, and the denominator is the total number of characters in the document.

[0015]

[0016] where n t,d is the number of times the word t appears in the document d, and the denominator is the sum of the number of times all words appear in the document d.

[0017] Step 2.2: For the microblog text in the microblog text data set obtained in step one, in Euclidean space E, the inverse document frequency IDF is calculated according to formula (2).

[0018] IDF, or inverse document frequency, is the inverse of the document frequency, used to reduce the effect of common words in all documents that have little impact on the document. The fewer documents containing the word t, the more it can be used to distinguish documents. The formula for IDF is as follows:

[0019]

[0020] where |D| is the total number of documents in the corpus. The denominator is the number of documents containing the word t.

[0021] Step 2.3: According to the word frequency TF obtained in step 2.1 and the inverse document frequency IDF obtained in step 2.2, the feature vector x of the microblog text in the Euclidean space E is calculated according to the TF-IDF feature vector formula as shown in formula (3) E The vector x E Each dimension corresponds to the TF-IDF value of a word.

[0022] The TF-IDF value is obtained by the product of the two, and the formula is:

[0023] TF-IDF(t,d,D) = TF(t,d) x IDF(t,D) # (3)

[0024] Convert the text into vector form, and each dimension corresponds to the TF-IDF value of a word, that is, obtain the feature vector x corresponding to each microblog text E , where E represents the Euclidean space.

[0025] Step three: according to the feature vector x corresponding to each microblog text obtained in step two E , the similarity between word vectors is calculated according to the formula of cosine similarity, and the adjacency matrix M for the hyperbolic graph convolution network is established.

[0026] The adjacency matrix M is established by the similarity between the obtained word vectors, so the cosine similarity is used to calculate their adjacency relationship, and the formula of the cosine similarity is:

[0027] cosine-similarity i,j = (x i · x j ) / (||x i || | |x j ||) # (4)

[0028] where x i · x j is the dot product of vectors x i and x j , and ||·|| is the length of the vector.

[0029] The adjacency matrix M is an n*n matrix representing the relationship between n microblog texts. If the cosine similarity between two microblog texts x i and x j is greater than a set threshold, the value of the adjacency matrix M at the corresponding position M i,j is 1.

[0030] Step four: use exponential mapping to get the feature vector x corresponding to each micro-blog text in step two E mapped to hyperbolic space, convert information between Euclidean space and hyperbolic space while preserving the hyperbolic structure of hyperbolic space x 0,H as input for step five.

[0031] The processed data is mapped to hyperbolic space, and the exponential mapping is used to convert information between Euclidean space and hyperbolic space while preserving the hierarchical structure of hyperbolic space.

[0032]

[0033] where x 0,E represents a feature in Euclidean space, H d,K is a d-dimensional hyperbolic sphere with constant negative curvature -1 / K (K>0), represents the origin used as a reference point in H d,K , used to perform tangent space operations.

[0034] Step five: take the hyperbolic feature x 0,H obtained in step four as input, and calculate the hyperbolic distance between nodes according to the formula for calculating the hyperbolic distance between nodes in hyperbolic space; the hierarchical clustering algorithm captures the relationship in the hierarchical structure data according to the hyperbolic distance, and obtains the adjacency matrix in the hyperbolic space as the input of the hyperbolic graph convolution network.

[0035] Step 5.1: take the hyperbolic feature x 0,H obtained in step four as input, and calculate the hyperbolic distance between nodes according to formula (6):

[0036]

[0037] where and are the hyperbolic space node representation, ||·|| 2 is the Euclidean norm.

[0038] Step 5.2: according to the hyperbolic distance between nodes obtained in step 5.1, perform hierarchical clustering in hyperbolic space to obtain the adjacency matrix in hyperbolic space as the input of the hyperbolic graph convolution network.

[0039] The basic steps of hierarchical clustering in hyperbolic space are as follows:

[0040] 1) initialization: at the beginning, each data point is considered as a cluster, so if there are N data points, the number of initial clusters is also N.

[0041] 2) calculate the distance: calculate the hyperbolic distance between all possible pairs of clusters.

[0042] 3) Merge clusters: Find the two closest clusters and merge them into a new cluster.

[0043] 4) Update distance: Update the distance matrix to reflect the distance between the merged cluster and other clusters.

[0044] 5) Repeat steps 2) through 4) until all data points are aggregated into one cluster.

[0045] In each merging step of hierarchical clustering, the distance between any two sub-clusters is calculated according to equation (7):

[0046]

[0047] in the formula Sub-clusters C a C b The sample in n a n b C respectively a C b The number of samples in the dataset. The two closest subclusters will be divided into C. b They are grouped together to form a new sub-cluster.

[0048] Hierarchical clustering is performed by calculating the distance in hyperbolic space to obtain a dendrogram. Each merging operation in the dendrogram is represented by a connecting line, and the height of the line represents the distance between the merged clusters. The data is partitioned by different hyperbolic distance scales through the resulting dendrogram to obtain the corresponding clustering results. The adjacency matrix X is constructed using the clustering information of hyperbolic space hierarchical clustering. The calculation of each element in matrix X is shown in formula (8):

[0049]

[0050] If node x i With x j In hyperbolic hierarchical clustering, if two elements belong to the same class, their position in matrix G (row i, column j) is set to 1; otherwise, it is set to 0. An adjacency matrix G = X + M is constructed, merging the adjacency matrices of Euclidean space and hyperbolic space, and used as the input to the hyperbolic graph convolutional network.

[0051] Step 6: Convert the hyperbolic feature vector x 0,HThe adjacency matrix G is input into a hyperbolic graph convolutional network (HBR), where hierarchical information is extracted. The HBR updates node information through three steps: feature transformation, neighborhood aggregation, and nonlinear activation. In the topic classification layer, the Softmax function is used to transform the output of the HBR into probability values, representing the likelihood of different topic categories, which are then used for topic classification. The topic with the highest probability is selected as the predicted classification result.

[0052] Step 6.1: Using the hyperbolic feature x obtained in Step 4 0,H Using the hyperbolic graph as the initial input, feature transformation is performed in the hyperbolic graph convolutional neural network to obtain hyperbolic features. The feature transformation formula is shown in equation (10).

[0053]

[0054] Where l is the corresponding layer number, W l With b l These are trainable parameters, and K is the curvature.

[0055] Step 6.2: Input the hyperbolic feature from Step 6.1 The adjacency matrix G obtained in step five is used to aggregate the information of the nodes. In the aggregation operation, each node aggregates its own information with the information of its neighboring nodes, thereby updating its own representation and obtaining the aggregated node information representation. The information aggregation formula for hyperbolic space is shown in equation (11).

[0056]

[0057] w i,j The calculation is shown in formula (11).

[0058]

[0059] Step 6.3: Perform nonlinear activation on the aggregated features, introducing nonlinear transformation to enhance the learning ability of the hyperbolic neural network model, improve the expressive power of the hyperbolic neural network model, and improve the convergence speed of the model. When performing nonlinear activation in hyperbolic space, the hyperbolic graph convolutional network can achieve nonlinear processing of graph data while maintaining hyperbolic geometric properties through the mapping operation between hyperbolic space and tangent space. The nonlinear activation formula in hyperbolic space is shown in Equation (12).

[0060]

[0061] Step 6.4: By utilizing the Softmax function, the topic classification layer converts the x-axis output by the hyperbolic graph convolutional network into a single image. l+1'transformed into probability values in the range [0, 1] representing the likelihood of different topic categories for performing topic classification. The topic with the highest probability is selected as the predicted classification result.

[0062]

[0063] where X is the hyperbolic vector output by the hyperbolic neural network step 6.3, W a is a parameter matrix, and b a is a bias term.

[0064] Step seven: train the hyperbolic graph convolutional network constructed in step six in combination with a loss function to obtain a trained hyperbolic graph convolutional network.

[0065] The loss function uses a cross-entropy function, as shown in equation (15).

[0066]

[0067] where c is the total number of categories, represents the category of the topic, N is the total number of texts, y i,c is an indicator function that is 1 when sample i belongs to category c, and 0 otherwise; is the probability that the model predicts that sample i belongs to category c.

[0068] Step eight: input the microblog text to be classified into the trained hyperbolic graph convolutional network to obtain the topic classification result of the microblog text to be classified, i.e., to implement topic classification of microblog text based on the hyperbolic graph convolutional network and hierarchical clustering.

[0069] It also includes step nine: applying the topic classification result of the microblog text obtained in step eight to the field of natural language processing and machine learning to solve related technical problems.

[0070] The related technical problems include intelligent text mining and services, social network information mining, and information monitoring.

[0071] The trained hyperbolic graph convolutional network is applied to network information monitoring, topic classification of microblog text is performed, information related to predetermined events and topics is monitored, real-time monitoring and early warning of information are achieved. According to the topic classification result, different coping strategies are adopted for different information topics, negative information is handled, and the spread of positive information is strengthened, improving the accuracy and efficiency of the information monitoring system, enhancing the effectiveness of information monitoring work, and maintaining positive network information and social stability.

[0072] Beneficial effects:

[0073] 1. The application discloses a topic classification method based on hyperbolic graph convolutional network and hierarchical clustering, which establishes an adjacency matrix of Euclidean space through cosine similarity, maps data to hyperbolic space and calculates hyperbolic distance between nodes, combines a hierarchical clustering algorithm, establishes an adjacency matrix of hyperbolic space, and fuses the two matrices as inputs of graph convolution, so that the method can effectively capture the hierarchical structure relationship of microblog text data and realize high-accuracy topic classification.

[0074] 2. The application discloses a topic classification method based on hyperbolic graph convolutional network and hierarchical clustering, which uses a hyperbolic graph convolutional network to extract features of data, mines hierarchical features of data, is suitable for processing topic classification of complex networks such as social information networks, solves the classification difficulty of traditional methods in high-dimensional, non-Euclidean structure data space, and improves the representation and classification ability of microblog text in network data.

[0075] 3. The application discloses a topic classification method based on hyperbolic graph convolutional network and hierarchical clustering, which combines the beneficial effects 1 and 2, maps the preprocessed text data to hyperbolic space, converts information between Euclidean space and hyperbolic space by using exponential mapping and logarithmic mapping, and retains the hierarchical structure features of hyperbolic space. The hyperbolic distance between nodes is calculated and combined with a hierarchical clustering algorithm to capture the relationship in hierarchical structure data. Similar or related nodes can obtain higher weights in the information aggregation process of the hyperbolic graph convolutional network, mine the hierarchical structure of text data, capture the hierarchical structure relationship of microblog text data, and realize high-quality topic classification.

[0076] 4. The application discloses a topic classification method based on hyperbolic graph convolutional network and hierarchical clustering, which collects microblog text under a specified topic, pre-processes text data, labels the topic categories of the microblog text, obtains labeled microblog text data set, and uses the microblog text data set as input of microblog text vectorization. The preprocessing includes data cleaning, Chinese word segmentation and stop word removal. Through preprocessing, the hyperbolic graph convolutional network can focus more on keywords that have substantial contribution to the meaning of text, the microblog text without stop words is obtained, the amount of microblog text data processed by the subsequent hyperbolic graph convolutional network is reduced, and the accuracy and classification efficiency of the hyperbolic graph convolutional network classification are improved.

[0077] 5. The application discloses a topic classification method based on hyperbolic graph convolutional network and hierarchical clustering, which applies the topic classification result of microblog text to the field of natural language processing and machine learning, and solves related technical problems. For example, intelligent text mining and service, social network information mining, information monitoring and other technical problems. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1A flowchart of a topic classification method based on a hyperbolic graph convolution network and hierarchical clustering is disclosed.

[0079] Figure 2 A hierarchical clustering tree diagram is disclosed. DETAILED DESCRIPTION

[0080] For the purpose of better illustrating the object and advantages of the present application, the following further describes the application in connection with the drawings and examples.

[0081] Example 1

[0082] The experiment was performed on a computer with the following specific configuration: Intel Core i5-104002.9GHz CPU 2.40GHz, 16GB of memory, GPU GT 730, operating system is windows10, 64-bit; programming language is Python3.9, integrated development environment is Pycharm2022, and programming framework is Pytorch. The accuracy and F1-measure value are used to evaluate the results.

[0083] As shown in Figure 1 The specific implementation steps of the topic classification method based on the hyperbolic graph convolution network and the hierarchical clustering are as follows:

[0084] Step 1: Collect microblog text under a specified topic, and preprocess the text data, and label the topic category of the microblog text to obtain the labeled microblog text dataset, and use the microblog text dataset as the input for microblog text vectorization in subsequent step 2. The preprocessing includes data cleaning, Chinese word segmentation, and stop word removal.

[0085] Step 1.1: Collect microblog text under a specified topic. The microblog text data is crawled on the microblog website through a crawler.

[0086] Step 1.2: Clean the microblog text data collected in step 1.1 according to the pre-set standard information cleaning principle, remove irrelevant information, and obtain the microblog text with standard information. The pre-set standard information cleaning principle includes cleaning and filtering microblog text with a length of less than 10 characters, and microblog text containing only expressions without text information.

[0087] Step 1.3: Chinese word segmentation is performed on the microblog text with standard information obtained in step 1.2. Chinese text is different from English text, which is composed of words and separated by spaces. Chinese text is coherent into a sentence, so the Chinese word segmentation tool Jieba is used for word segmentation to obtain the Chinese word segmented microblog text used in step 1.4.

[0088] Step 1.4: According to the Harbin Institute of Technology stop list, meaningless information in the segmented microblog text, such as Chinese auxiliary words such as "of", "is", "in", and conjunctions such as "and", "but", etc. are removed. The microblog text without stop words is obtained, which reduces the amount of microblog text data processed by the subsequent hyperbolic graph convolution network, and improves the accuracy and classification efficiency of the hyperbolic graph convolution network classification.

[0089] Step 1.5: The microblog text data set after data cleaning, Chinese segmentation, and stop word removal preprocessing is labeled with topic categories, and the labeled microblog text data set is obtained. The microblog text data set is used as the input for the subsequent step two for microblog text vectorization.

[0090] Step two: For the microblog text in the microblog text data set obtained in step one, the feature vector of the microblog text is calculated in the Euclidean space E by the TF-IDF method, and the microblog text is converted into a vector form. Each dimension of the vector corresponds to the TF-IDF value of a word, that is, the feature vector x corresponding to each microblog text is obtained. E .

[0091] Step 2.1: For the microblog text in the microblog text data set obtained in step one, in the Euclidean space E, the number of times of occurrence of the words in the microblog text is normalized according to the term frequency TF normalization formula as shown in formula (1), to obtain the normalized term frequency TF of the microblog text. The numerator of the TF normalization formula is the number of occurrences of the word in the document, and the denominator is the total number of characters in the document.

[0092]

[0093] where n t,d is the number of occurrences of the word t in the document d, and the denominator is the sum of the number of occurrences of all words in the document d.

[0094] Step 2.2: For the microblog text in the microblog text data set obtained in step one, in the Euclidean space E, the inverse document frequency IDF is calculated according to formula (2).

[0095] IDF, also known as inverse document frequency, is the inverse of the document frequency, which is used to reduce the effect of common words in all documents that have little effect on the document. The fewer documents containing the word t, the more it can be distinguished by the word t. The formula of IDF is as follows:

[0096]

[0097] where |D| is the total number of documents in the corpus. The denominator is the number of documents containing the word t.

[0098] Step 2.3: Based on the term frequency (TF) obtained in Step 2.1 and the inverse document frequency (IDF) obtained in Step 2.2, calculate the feature vector x in the Euclidean space E of the Weibo text according to the TF-IDF feature vector formula shown in Equation (3). E Vector x E Each dimension corresponds to the TF-IDF value of a word.

[0099] The TF-IDF value is obtained by multiplying the two values, using the following formula:

[0100] TF-IDf(t,d,D)=TF(t,d)×IDFt,D)#(3)

[0101] Convert the text into vector form, with each dimension corresponding to the TF-IDF value of a word, thus obtaining the feature vector x for each Weibo post. E , where E represents Euclidean space.

[0102] Step 3: Based on the feature vector x corresponding to each Weibo post obtained in Step 2 E The similarity between word vectors is calculated according to formula (4) to establish the adjacency matrix M for hyperbolic graph convolutional networks.

[0103] An adjacency matrix M is constructed based on the similarity between the obtained word vectors. Therefore, cosine similarity is used to calculate their adjacency relationship. The formula for cosine similarity is:

[0104] cosine-similarity i,j =(x i ·x j ) / (||x i ||||x j ||)#(4)

[0105] Where x i ·x j It is a vector x i and x j The dot product, ||·||, is the magnitude of the vector.

[0106] The adjacency matrix M is an n*n matrix representing the relationships between n microblog texts. If two microblog texts x i With x j If the cosine similarity between two points is greater than 0.1, the corresponding position M in the adjacency matrix M is... i,j The value is 1.

[0107] Step 4: Use exponential mapping to transform the feature vector x corresponding to each Weibo post obtained in Step 2. E Mapping to hyperbolic space, transforming information between Euclidean space and hyperbolic space, while preserving the hyperbolic structural features of hyperbolic space. 0,HAs input to step five.

[0108] The processed data is mapped to hyperbolic space, which converts information between Euclidean space and hyperbolic space using exponential mapping while preserving the hierarchical structure characteristics of hyperbolic space.

[0109]

[0110] where x 0,E represents a feature in Euclidean space, H d,K is a d-dimensional hyperbolic sphere with constant negative curvature -1 / K (K>0), represents the origin used as a reference point in H d,K , which is used to perform tangent space operations.

[0111] Step five: take the hyperbolic feature x 0,H obtained in step four as input, and calculate the hyperbolic distance between nodes according to the hyperbolic distance formula between nodes after the mapping characteristics of hyperbolic space are calculated; the hierarchical clustering algorithm captures the relationship in the hierarchical structure data according to the hyperbolic distance, and obtains the adjacency matrix in the hyperbolic space as the input of the hyperbolic graph convolution network.

[0112] Step 5.1: take the hyperbolic feature x 0,H obtained in step four as input, and calculate the hyperbolic distance between nodes according to formula (6):

[0113]

[0114] where and are hyperbolic space node representations, ||·|| 2 is the Euclidean norm.

[0115] Step 5.2: according to the hyperbolic distance between nodes obtained in step 5.1, perform hierarchical clustering in hyperbolic space to obtain the adjacency matrix in hyperbolic space as the input of the hyperbolic graph convolution network.

[0116] The basic steps of hierarchical clustering in hyperbolic space are as follows:

[0117] 1) Initialization: At the beginning, each data point is regarded as a cluster, so if there are N data points, the number of initial clusters is also N.

[0118] 2) Calculate distance: Calculate the hyperbolic distance between all possible pairs of clusters.

[0119] 3) Merge clusters: find the two closest clusters and merge them into a new cluster.

[0120] 4) Update distance: Update the distance matrix to reflect the distance between the merged cluster and other clusters.

[0121] 5) Repeat steps 2) through 4): Repeat steps until all data points are aggregated into one cluster.

[0122] In each merging step of hierarchical clustering, the distance between any two sub-clusters is calculated according to equation (7):

[0123]

[0124] in the formula Sub-clusters C a C b The sample in n a n b C a C b The number of samples in the dataset. The two closest subclusters will be divided into C. b They are grouped together to form a new sub-cluster.

[0125] Hierarchical clustering is performed by calculating the distance in hyperbolic space to obtain a dendrogram, such as... Figure 2 As shown, each merging operation in the dendrogram is represented by a connecting line, and the height of the line represents the distance between the merged clusters. The data is partitioned by different hyperbolic distance scales x and y through the resulting dendrogram, and the corresponding clustering results are obtained. At the same time, the adjacency matrix X is constructed using the clustering information of hyperbolic spatial hierarchical clustering, and the calculation of each element in matrix X is shown in formula (8):

[0126]

[0127] If node x i With x j In hyperbolic hierarchical clustering, nodes belonging to the same class are assigned a value of 1 at row i, column j in matrix G; otherwise, they are assigned a value of 0. Similar or related nodes receive higher weights w during information aggregation. i,，j Establish an adjacency matrix G = X + M, merge the adjacency matrices of Euclidean space and hyperbolic space, and use them as the input to the hyperbolic graph convolutional network. i,j The calculation is shown in formula (9).

[0128]

[0129] Step Six: Convert the hyperbolic feature vector x 0,HWith the adjacency matrix input into the hyperbolic graph convolutional network, the hierarchical information extraction is completed in the hyperbolic graph convolutional network. The hyperbolic graph convolutional network realizes the update of node information through three steps of feature conversion, neighborhood aggregation and nonlinear activation. The topic classification layer uses the Softmax function to convert the value output by the hyperbolic graph convolutional network into a probability value, and the probability value represents the possibility of different topic categories and is used for topic classification. Finally, the topic with the highest probability is selected as the predicted classification result.

[0130] Step 6.1: Hyperbolic feature x 0,H is obtained by performing feature conversion on the initial input in the hyperbolic graph convolutional neural network to obtain the hyperbolic feature The feature conversion formula is as shown in formula (10).

[0131]

[0132] Wherein, l is the corresponding layer number, W l and b l are trainable parameters, and K is the curvature.

[0133] Step 6.2: Input the hyperbolic feature obtained in step 6.1 and pass w i,j obtained in step 5 to the node to aggregate information, in the aggregation operation, each node aggregates its own information with the information of its neighbor nodes, thereby updating its own representation, and obtaining the information aggregation representation of the node after information aggregation The information aggregation formula in the hyperbolic space is as shown in formula (11).

[0134]

[0135] Step 6.3: The feature obtained after aggregation needs to be nonlinearly activated, and nonlinear transformation is introduced to enhance the learning ability of the model, improve the expression ability of the model and improve the convergence speed of the model. When performing nonlinear activation in the hyperbolic space, the hyperbolic graph convolutional network realizes nonlinear processing of the graph data while maintaining the hyperbolic geometric properties through the mapping operation between the hyperbolic space and the tangent space, and the nonlinear activation formula in the hyperbolic space is as shown in formula (12).

[0136]

[0137] Step 6.4: The topic classification layer converts x l+1' output by the hyperbolic graph convolutional network into a probability value in the range of [0, 1] through the Softmax function, and the probability value represents the possibility of different topic categories and is used for topic classification. Finally, the topic with the highest probability is selected as the predicted classification result.

[0138]

[0139] where X is the hyperbolic vector output by the hyperbolic neural network step 6.3, W a is a parameter matrix, and b a is a bias term.

[0140] Step seven: training the hyperbolic graph convolutional network constructed in step six with reference to the sample, to obtain a trained hyperbolic graph convolutional network.

[0141] The loss function adopts a cross-entropy function, as shown in formula (15).

[0142]

[0143] where c is the total number of categories, represents the category of the topic, N is the total number of texts, y i,c is an indicator function, which is 1 when sample i belongs to category c, and 0 otherwise; is the probability that the model predicts that sample i belongs to category c.

[0144] Step eight: inputting the microblog text to be classified into the trained hyperbolic graph convolutional network, to obtain the topic classification result of the microblog text to be classified, that is, implementing topic classification of microblog text based on the hyperbolic graph convolutional network and hierarchical clustering.

[0145] Further comprising step nine: applying the topic classification result of the microblog text obtained in step eight to the field of natural language processing and machine learning to solve related technical problems.

[0146] In order to investigate the effect of the microblog text topic classification method based on hyperbolic hierarchical clustering and hyperbolic graph convolutional neural network, the present application is compared with the classical K-MEANS and Agglomerative clustering algorithms in Euclidean space and the neural network classification method based on Euclidean space. The experimental results on the Weibo and PubMed data sets are shown in Table 1.

[0147] Table 1 Experimental results

[0148]

[0149] The accuracy and F1 value of the present application on the Weibo and PubMed data sets are higher than those of other models, showing excellent text classification ability. When dealing with data with obvious hierarchical structure or complex relationship, the hyperbolic space method has potential advantages. The advantages of the present application in processing data sets show its efficient processing ability for social media text.

[0150] The above detailed description of the specific description, the purpose, technical scheme and beneficial effects of the application are further described in detail, it should be understood that the above description is only a specific embodiment of the application, and is not used to limit the protection scope of the application, any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application should be included in the protection scope of the application.

Claims

1. A microblog text topic classification method based on hyperbolic graph convolutional network and hierarchical clustering, characterized in that: The method comprises the following steps, Step one: collect microblog texts under a specified topic, pre-process the text data, and label the topic categories of the microblog texts to obtain a labeled microblog text dataset, which is used as input for microblog text vectorization in subsequent step two; the pre-processing includes data cleaning, Chinese word segmentation, and stop word removal; Step two: for the microblog text in the microblog text data set obtained in step one, the feature vector of the microblog text is calculated in the Euclidean space E through the TF-IDF method, the microblog text is converted into a vector form, and each dimension of the vector corresponds to a TF-IDF value of a word, that is, the feature vector x corresponding to each microblog text is obtained E ; Step three: according to the feature vector x corresponding to each micro-blog text obtained in step two E According to the formula of cosine similarity, the similarity between word vectors is calculated, and the adjacency matrix M for the hyperbolic graph convolution network is established; Step four: use exponential mapping to get the feature vector x corresponding to each micro-blog text in step two E Map to hyperbolic space, convert information between Euclidean space and hyperbolic space, while preserving the hyperbolic structure characteristics of hyperbolic space x 0,H As the input of step five; Map the processed data to hyperbolic space, use exponential mapping to convert information between Euclidean space and hyperbolic space, while preserving the hierarchical structure characteristics of hyperbolic space; where x 0,E represents a feature in Euclidean space, H d,K is a d-dimensional hyperbolic sphere with constant negative curvature -1 / K, K > 0, represents the origin used as a reference point in H d,K to perform tangent space operations; Step 5: Using the hyperbolic feature x obtained in Step 4 0,H As input, the hyperbolic distance between nodes is calculated based on the mapping features of hyperbolic space. The hierarchical clustering algorithm captures the relationships in the hierarchical data based on the hyperbolic distance to obtain the adjacency matrix representing the hyperbolic space, which is used as the input to the hyperbolic graph convolutional network. Step six: the hyperbolic eigenvector x 0,H The adjacency matrix G is input into the hyperbolic graph convolution network, and the hierarchical information is extracted by the hyperbolic graph convolution network; the hyperbolic graph convolution network realizes the update of node information through three steps of feature conversion, neighborhood aggregation and nonlinear activation; the value output by the hyperbolic graph convolution network is converted into a probability value by using a Softmax function in the topic classification layer, and the probability value represents the possibility of different topic categories and is used for executing topic classification; the topic with the highest probability is selected as the predicted classification result; Step seven: train the hyperbolic graph convolutional network constructed in step six in combination with the loss function to obtain a trained hyperbolic graph convolutional network; Step eight: input the microblog text to be classified into the trained hyperbolic graph convolutional network to obtain the topic classification result of the microblog text to be classified, that is, to realize the topic classification of microblog text based on the hyperbolic graph convolutional network and hierarchical clustering. 2.The microblog text topic classification method based on hyperbolic graph convolutional network and hierarchical clustering according to claim 1, wherein: It also includes step nine: applying the topic classification result of the microblog text obtained in step eight to the field of natural language processing and machine learning to solve related technical problems; The related technical problems include intelligent text mining and services, social network information mining, public opinion monitoring; The trained hyperbolic graph convolutional network is applied to network public opinion monitoring, the microblog text is classified by topic, the public opinion information related to the predetermined event or topic is monitored, the real-time monitoring and early warning of public opinion are realized, different coping strategies are adopted according to the topic classification result for different public opinion topics, the negative public opinion is handled, the spread of positive public opinion is strengthened, the accuracy and efficiency of the public opinion monitoring system are improved, the effect of public opinion monitoring work is improved, and the network public opinion is maintained positive and the society is stable. 3.The microblog text topic classification method based on hyperbolic graph convolutional network and hierarchical clustering according to claim 1 or 2, wherein: The implementation method of step one is, Step 1.1: collect microblog texts under a specified topic; Step 1.2: clean the microblog text data collected in step 1.1 according to the pre-set information cleaning principle to remove irrelevant information and obtain microblog text with standard information; the pre-set information cleaning principle includes cleaning and filtering microblog texts with a text length less than a pre-set length threshold and microblog texts containing only expressions without text information; Step 1.3: Chinese word segmentation is performed on the microblog text with standard information obtained in step 1.2 to obtain Chinese segmented microblog text for step 1.4; Step 1.4: referring to a stop word list, meaningless information in the segmented microblog text is removed, including Chinese auxiliary words and conjunctions, Chinese auxiliary words include "of", "is", "in", and conjunctions include "and", "but"; although Chinese auxiliary words and conjunctions play a role in the grammatical structure of a sentence, removing these high-frequency but low-information words allows the hyperbolic graph convolutional network to focus more on key words that have a substantial contribution to the meaning of the text, and obtains microblog text without stop words; Step 1.5: label the topic categories of the microblog text pre-processed by data cleaning, Chinese word segmentation, and stop word removal to obtain a labeled microblog text dataset, which is used as input for microblog text vectorization in subsequent step two. 4.The microblog text topic classification method based on hyperbolic graph convolutional network and hierarchical clustering of claim 3, wherein: The implementation method of step two is, Step 2.1: For the microblog text in the microblog text data set obtained in step one, the number of occurrences of words in the microblog text is normalized in the Euclidean space E according to the word frequency TF normalization formula as shown in formula (1) to obtain the word frequency TF of the normalized words in the microblog text. where n t,d is the number of occurrences of the word t in the document d, and the denominator is the sum of the occurrences of all words in the document d; Step 2.2: For the microblog text in the microblog text data set obtained in step one, the inverse document frequency IDF is calculated in the Euclidean space E according to formula (2). IDF is called inverse document frequency, which is the inverse of document frequency, used to reduce the effect of common words in all documents that have little effect on the document; the fewer documents containing the word t, the more it can distinguish documents through the word t; the formula of IDF is as follows: Where |D| is the total number of documents in the corpus; the denominator is the number of documents containing the word t; Step 2.3: According to the term frequency TF obtained in step 2.1 and the inverse document frequency IDF obtained in step 2.2, the feature vector x of the microblog text in the Euclidean space E is calculated according to the TF-IDF feature vector formula as shown in formula (3) E Vector x E Each dimension corresponds to the TF-IDF value of a word The TF-IDF value is obtained by the product of the two, and the formula is: TF-IDF(t,d,D)=TF(t,d)×IDF(t,D) (3) Convert the text into vector form, with each dimension corresponding to the TF-IDF value of a word, i.e. get the feature vector x corresponding to each microblog text E where E represents the Euclidean space.

5. The microblog text topic classification method based on hyperbolic graph convolutional network and hierarchical clustering of claim 4, wherein: In step three, The adjacency matrix M is established by the similarity between the obtained word vectors, so the cosine similarity is used to calculate their adjacency relationship, and the formula of the cosine similarity is: Cosine-similarity i,j = (x i · x j ) / (||x i || ||x j ||) (4) where x i · x j is the dot product of the vectors x i and x j , and || · || is the norm of a vector; The adjacency matrix M is an n*n matrix representing the relationship between n micro-blog texts. If the cosine similarity between two micro-blog texts x i and x j is greater than a set threshold, the value of the corresponding position M i,j of the adjacency matrix M is 1. 6.The microblog text topic classification method based on hyperbolic graph convolutional network and hierarchical clustering of claim 5, wherein: The implementation method of step five is Step 5.1: The hyperbolic features x obtained in step four are used as input to compute the hyperbolic distance between nodes according to equation (6): 0,H Step 5.1: The hyperbolic features x obtained in step four are used as input to compute the hyperbolic distance between nodes according to equation (6): 0,H Step 5. wherein with is a hyperbolic space node representation, ||·| 2 is the Euclidean norm; Step 5.2: According to the hyperbolic distance between the nodes obtained in step 5.1, hierarchical clustering in hyperbolic space is performed to obtain a representative adjacency matrix in hyperbolic space as the input of the hyperbolic graph convolution network. The basic steps of hierarchical clustering in hyperbolic space are as follows: 1) Initialization: At the beginning, each data point is regarded as a cluster, so if there are N data points, the initial number of clusters is also N; 2) Calculate the distance: Calculate the hyperbolic distance between all possible pairs of clusters; 3) Merge clusters: Find the two closest clusters and merge them into a new cluster; 4) Update the distance: Update the distance matrix to reflect the distance between the merged cluster and other clusters; 5) Repeat steps 2) to 4) until all data points are aggregated into one cluster; Wherein in each merging step of hierarchical clustering, the distance between any two sub-clusters is calculated according to formula (7): In the formula respectively, the number of samples in C a , C b , n a , n b respectively, the number of samples in C a , C b ; the two closest sub-clusters will be grouped together to become a new sub-cluster; The hierarchical clustering is realized by calculating the distance in hyperbolic space to obtain a tree diagram, and each merging operation in the tree diagram is represented by a connection line, and the height of the line represents the distance between the merged clusters; the division of different hyperbolic distance scales is realized by the result tree diagram to divide the data and obtain the corresponding clustering result; the clustering information of the hierarchical clustering in the hyperbolic space is used to construct the adjacency matrix X, and the elements in the matrix X are calculated as shown in formula (8): If node x i With x j In hyperbolic space hierarchical clustering, if it belongs to a class at the same time, the position of i row j column in matrix G is set to 1, otherwise 0; the adjacency matrix G=X+M is established, and the adjacency matrices of Euclidean space and hyperbolic space are combined as the input of the hyperbolic graph convolution network.

7. The microblog text topic classification method based on hyperbolic graph convolutional network and hierarchical clustering of claim 6, wherein: The implementation method of step six is Step 6.1: The double hyperbolic feature x is obtained by performing feature conversion on the double hyperbolic graph convolutional neural network with the double hyperbolic graph feature x 0,H For initial input, the double hyperbolic feature is obtained by performing feature conversion in the double hyperbolic graph convolutional neural network The feature conversion formula is as formula (9); where l is the corresponding layer number, W l with b l are trainable parameters, K is the curvature; Step 6.2: Input the hyperbolic feature of step 6.1 and the adjacency matrix G obtained from step five, the information aggregation of the nodes, in the aggregation operation, each node aggregates its own information with the information of its neighbor nodes, thereby updating its own representation, to obtain the information aggregation of the node information aggregation representation after aggregation The information aggregation formula of the hyperbolic space is as formula (10); w i,j is calculated as shown in equation (11); Step 6.3: Perform nonlinear activation on the features obtained after aggregation to introduce nonlinear transformation to enhance the learning ability of the hyperbolic graph convolution network model, improve the expression ability of the hyperbolic graph convolution network model, and improve the convergence speed of the model; when performing nonlinear activation in hyperbolic space, the hyperbolic graph convolution network realizes nonlinear processing of graph data while maintaining the geometric properties of hyperbolic space through the mapping operation between hyperbolic space and tangent space, and the nonlinear activation formula in hyperbolic space is shown in formula (12); Step 6.4: The topic classification layer transforms the hyperbolic graph convolutional network output x l+1′ into probability values in the range [0, 1] for performing topic classification by utilizing a Softmax function; Select the most probable topic as the predicted classification result; where X is the hyperbolic vector output by the hyperbolic neural network step 6.3, W a is a parameter matrix, and b a is a bias term. 8.The microblog text topic classification method based on hyperbolic graph convolutional network and hierarchical clustering of claim 7, wherein: In step seven, The loss function adopts a cross-entropy function formula as formula (15); where c is the total number of classes, represents the class of the topic, N is the total number of texts, y i,c is an indicator function that is 1 when sample i belongs to class c and 0 otherwise; is the probability that the model predicts that sample i belongs to class c.

Images