A sentiment tendency classification method for hot topics in social media

Abstract

The present invention discloses a sentiment tendency classification method for hot topics in social media, including: collecting public attribute information of social media users and blog posts that have been published and formatting them into strings to form documents to be classified; establishing word nodes and document nodes, establishing edges between word nodes and document nodes and between word nodes, calculating the value of word frequency-inverse document frequency and the value of positive point-by-point mutual information as the weight of the edge, embedding the document node using the RoBERTa model to obtain a document feature vector, initializing the word node feature to 0, and obtaining a node feature matrix of a text graph; respectively obtaining a classification probability vector based on the RoBERTa model and a classification probability vector based on the GCN model, performing weighted summation on the two, obtaining a joint classification probability vector, and selecting the category with the highest probability from the joint classification probability vector as the classification result. The present invention can objectively and comprehensively mine the sentiment tendencies implied by users facing different hot topics in social media.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and specifically relates to a sentiment tendency classification method for hot topics in social media, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] As one of the important bases for personalized recommendations and event trend prediction, the sentiment tendency classification of hot topics on social media is of great significance. At present, in the field of natural language processing, existing technologies focus more on tasks such as user profiling and knowledge graph construction, but rarely involve the classification of social media users' sentiment tendencies towards different hot topics. In addition, existing technologies have not yet effectively mined the public information and blog posts of users in social media, and users have different sentiment tendencies and opinions on different hot topics. The above information is one of the important bases for downstream tasks such as event trend prediction.

[0003] The sentiment classification task can be modeled as a text classification task. Currently, the text classification task is mainly based on a pre-trained language model. For example, as a prior art, CN114168709B discloses a text classification method based on a lightweight pre-trained language model, and CN113987171A discloses a news text classification method and system based on a variant of a pre-trained model. The method based on the pre-trained language model uses a variant of the Transformer and a self-attention mechanism to encode the text into a feature vector from three perspectives: token embeddings, segment embeddings, and position embeddings, to capture contextual dependencies and semantic information. Commonly used models include BERT, RoBERTa, ERNIE, etc. Whole Word Masking (WWM), which is more suitable for Chinese text, has also emerged. The output of the pre-trained language model is a vector representation, which can be classified through a fully connected layer for text classification tasks. However, for social media blog data, the text length is often short and the text itself does not contain enough information. The method of obtaining text encoding for classification only through pre-trained language models is difficult to capture feature vectors that meet classification requirements, resulting in poor classification results. Summary of the invention

[0004] The purpose of the present invention is to provide a sentiment tendency classification method, computer equipment, computer-readable storage medium and computer program product for hot topics in social media, which can objectively and comprehensively mine the sentiment tendencies implied by social media users towards different hot topics, and provide a reference basis for downstream tasks such as personalized recommendations and event trend prediction.

[0005] One aspect of the present invention provides a sentiment tendency classification method for hot topics in social media, characterized by comprising:

[0006] The data fusion step collects the public attribute information and previously published blog posts of social media users, and formats the public attribute information and previously published blog posts of users into strings to form multiple documents to be classified;

[0007] The text graph construction step is to use all non-repeated words in all documents as word nodes, and multiple documents to be classified as document nodes. Edges are established between word nodes and document nodes and between word nodes. The value of word frequency-inverse document frequency is calculated as the weight of the edge between word nodes and document nodes. The value of positive point-by-point mutual information is calculated as the weight of the edge between word nodes and word nodes. The document nodes are embedded using the RoBERTa model to obtain the document feature vector. The word node features are initialized to 0 to obtain the node feature matrix of the text graph.

[0008] In the sentiment tendency classification step, the document feature vector is used as the input of the RoBERTa model, and the classification probability vector based on the RoBERTa model is obtained after the softmax function. The node feature matrix is ​​used as the input of the GCN model, and the classification probability vector based on the GCN model is obtained after the softmax function. The classification probability vector based on the RoBERTa model and the classification probability vector based on the GCN model are weighted and summed to obtain the joint classification probability vector. The category with the highest probability is selected from the joint classification probability vector as the classification result.

[0009] Preferably, in the sentiment tendency classification step, the following joint classification probability vector is obtained:

[0010] logits=λlogits GCN +(1-λ)logits RoBERTa ,

[0011] Among them, logits RoBERTa is the classification probability vector based on the RoBERTa model, logits GCN is the classification probability vector based on the GCN model, and λ is the weight;

[0012] Classification probability vector logits based on RoBERTa model RoBERTa for:

[0013] logits RoBERTa =softmax(X doc W RoBERTa ),

[0014] Among them, Xdoc is the document feature vector, W RoBERTa are the RoBERTa model parameters,

[0015] Classification probability vector logits based on GCN model GCN for:

[0016]

[0017] Among them, ReLU is the activation function, X is the node feature matrix, the GCN model uses 2 layers of GCN, W0 and W1 represent the parameters of the 0th layer and the 1st layer of GCN respectively, and A is the adjacency matrix of the text graph. is the normalized adjacency matrix, Where D is the degree matrix of A.

[0018] Preferably, each element A in the adjacency matrix A i,j The value of is the weight E of the edge between nodes i and j i,j ,

[0019]

[0020] PPMI(i,j)=max(PMI(i,j),0),

[0021]

[0022] Where PPMI represents positive pointwise mutual information, TF-IDF represents term frequency-inverse document frequency, PMI represents pointwise mutual information, Window(i) is the number of windows containing node i in the sliding window, Window(j) is the number of windows containing node j in the sliding window, Window(i,j) is the number of windows in which nodes i and j co-occur, Window is the total number of sliding windows, p(i) is the proportion of windows in the sliding window that contain word node i, p(j) is the proportion of windows in the sliding window that contain word node j, and p(i,j) is the proportion of windows in which word nodes i and j co-occur.

[0023] Preferably, the node feature matrix X is:

[0024]

[0025] Among them, d is the dimension of the document feature vector, n doc and n word Represent the number of document nodes and the number of word nodes respectively.

[0026] Preferably, the weight λ is adjusted by the following loss function to optimize the joint classification probability vector:

[0027] loss = -ylog(logits),

[0028] Among them, loss is the loss function, y is the classification label, and the value of y is 0 or 1.

[0029] Preferably, λ=0.7.

[0030] Preferably, the public attribute information of the user includes a nickname and a personal signature.

[0031] Another aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0032] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0033] Another aspect of the present invention provides a computer program product, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0034] The sentiment tendency classification method, computer device, computer-readable storage medium and computer program product for social media hot topics according to the above aspects of the present invention can objectively and comprehensively mine the sentiment tendencies implied by social media users towards different hot topics, and provide a reference basis for downstream tasks such as personalized recommendations and event trend prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings used in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work:

[0036] Figure 1 A flowchart of a method for classifying sentiment tendency of hot topics in social media according to an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of the principle of a method for classifying sentiment tendency of hot topics in social media according to an embodiment of the present invention;

[0038] Figure 3 4 is a structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The embodiments of the present invention provide a sentiment tendency classification method for hot topics in social media, which can be applied to tasks such as personalized recommendation and event trend prediction. Figure 1 is a flow chart of a sentiment tendency classification method for hot topics in social media according to an embodiment of the present invention, Figure 2 The following is a schematic diagram of the principle of the emotional tendency classification method for social media hot topics in an embodiment of the present invention. Figure 1 and Figure 2 , and detailed description of each step of the sentiment tendency classification method of an embodiment of the present invention.

[0041] Step S1 is a data fusion step, in which the public attribute information and previously published blog posts of social media users are collected, and the public attribute information and previously published blog posts of users are formatted into character strings to form a plurality of documents to be classified.

[0042] Specifically, we first collect the public attribute information and blog posts of social media users, and structure the data into different attributes, including nicknames, personal signatures, etc.

[0043] For example, the user's public attribute information and the corresponding previously published blog posts are formatted into a string: "[Nickname], personal signature is [Personal Signature]. For the topic [Topic Name], the previously published blog post is [Published Content]." (The contents within "[]" in the string are all filled with corresponding attribute information) as the document required for sentiment tendency classification.

[0044] Step S2 is a text graph construction step. In this step, all non-repeated words in all documents are used as word nodes, and multiple documents to be classified are used as document nodes. Edges are established between word nodes and document nodes and between word nodes. The value of word frequency-inverse document frequency is calculated as the weight of the edge between word nodes and document nodes. The value of positive point-by-point mutual information is calculated as the weight of the edge between word nodes and word nodes. The document nodes are embedded using the RoBERTa model to obtain document feature vectors. The word node features are initialized to 0 to obtain the node feature matrix of the text graph.

[0045] Specifically, words and documents are used as nodes, where word nodes are all non-repeated words in all documents, and document nodes are the text to be classified. Edges are established between word nodes and document nodes, and between word nodes and word nodes, and the values ​​of term frequency-inverse document frequency (TF-IDF) and positive point-wise mutual information (PPMI) are calculated as edge weights. TF-IDF can indicate the importance of a word in a document. The higher the TF-IDF value, the more important the word is to the corresponding document; PPMI can indicate the semantic relevance between words. The higher the PPMI value, the stronger the semantic relevance.

[0046] The weight of the edge between nodes i and j is:

[0047]

[0048] Among them, PPMI is a variant of PMI (Point-wise Mutual Information). The calculation formula of PMI is as follows:

[0049]

[0050] Window(i) is the number of windows in the sliding window that contain node (word) i, Window(j) is the number of windows in the sliding window that contain node (word) j, Window(i,j) is the number of windows in which nodes i and j co-occur, Window is the total number of sliding windows, p(i) is the proportion of windows in the sliding window that contain node i, p(j) is the proportion of windows in the sliding window that contain node j, and p(i,j) is the proportion of windows in which nodes i and j co-occur. PPMI is the case where PMI is greater than or equal to 0:

[0051] PPMI(i,j)=max(PMI(i,j),0)

[0052] At this point, by constructing nodes and edges, the text graph has been established. It can be concluded that the adjacency matrix of the text graph is an n×n matrix A, where n=n doc +n word , which is the total number of document nodes and word nodes. Each element A in the adjacency matrix A i,j The value of is the edge weight E i,j .

[0053] Then the node features are initialized. Specifically, the RoBERTa model is used to extract text features from document nodes and used as their initial features. The word node features are initialized to 0, and the node feature matrix X is obtained:

[0054]

[0055] Among them, X doc represents the document feature vector obtained by embedding the document node using the RoBERTa model, d is the feature dimension after embedding by the RoBERTa model, that is, the document feature vector dimension, n doc and n word Represent the number of document nodes and word nodes respectively. It should be noted that in order to better apply to Chinese text, the RoBERTa model can adopt a pre-trained language model based on the full-word masking strategy and Chinese corpus. At this time, the node feature matrix is ​​obtained.

[0056] Step S3 is the sentiment tendency classification step. In this step, the document feature vector is used as the input of the RoBERTa model, and the classification probability vector based on the RoBERTa model is obtained after the softmax function. The node feature matrix is ​​used as the input of the GCN model, and the classification probability vector based on the GCN model is obtained after the softmax function. The classification probability vector based on the RoBERTa model and the classification probability vector based on the GCN model are weighted and summed to obtain a joint classification probability vector. The category with the highest probability is selected from the joint classification probability vector as the classification result.

[0057] In this step, the method of the embodiment of the present invention combines the advantages of RoBERTa model extraction features and GCN (Graph Convolutional Network) model generalization capabilities, while overcoming the problems of RoBERTa model and GCN model respectively:

[0058] The RoBERTa model encodes text into feature vectors to capture contextual dependencies and semantic information. However, for social media blog data, the text length is often short and the text itself contains insufficient information. The method of obtaining text encoding for classification through the RoBERTa model alone is difficult to capture feature vectors that meet the classification needs, and the feature vector dimension after the RoBERTa model is embedded is large. The GCN model constructs text into a graph and aggregates neighbor node information using convolution operations. When constructing the text graph, the TF-IDF algorithm is used to calculate the importance of words in the document and the PMI algorithm is used to calculate the co-occurrence frequency of words in the sliding window. The structural features between words and words and between words and documents are used, but the semantic features of the text are not used, resulting in insufficient representation of text data. It is also sensitive to the parameter of window size. If the window is too small, it cannot effectively reflect the co-occurrence information of global words. If the window is too large, a relationship will be established between words that are not related to each other. In addition, the constructed text graph may contain tens of thousands of nodes.

[0059] Therefore, the method of the embodiment of the present invention combines the advantages of the RoBERTa model in extracting features and the GCN model in generalization ability. To address the problem that the feature vector dimension is large after the RoBERTa model is embedded and there are too many nodes in the text graph, making it difficult to optimize the entire graph and the node feature vector at the same time, the RoBERTa model and the GCN model are used to respectively derive the predicted classification probability, and a weighted sum is taken of the two. The loss function is then calculated and gradient descent optimization is performed to achieve the purpose of jointly optimizing the RoBERTa model and the GCN model.

[0060] Specifically, in the sentiment tendency classification step S3, for the RoBERTa model, the predicted probability vector (the classification probability vector logits based on the RoBERTa model) RoBERTa )for:

[0061] logits RoBERTa =softmax(X doc W RoBERTa )

[0062] Among them, W RoBERTa It is the parameter that needs to be optimized in the RoBERTa model, logits RoBERTa is a 1×class number vector, with each class having a corresponding predicted probability.

[0063] For the GCN model, after obtaining the initialized node feature matrix X, X is used as the input of the GCN model. During the training iteration, the node representation is updated by transforming the feature matrix. Specifically, a two-layer GCN is used and the following feature calculation method is adopted for updating. The output features of the GCN model are used as the final representation of the text graph, and then classified through the softmax layer to obtain the predicted probability vector (based on the classification probability vector logits of the GCN model) GCN )for:

[0064]

[0065] Among them, ReLU is the activation function, W0 and W1 represent the parameters that need to be optimized for the 0th and 1st layer GCN respectively. is the normalized adjacency matrix, and the calculation formula is as follows:

[0066]

[0067] Where D is the degree matrix of A, and the calculation formula is D = ∑ column A row×column That is, sum A row by row.

[0068] logits GCN is a 1×class number vector, with each class having a corresponding predicted probability.

[0069] During model training, in order to better jointly optimize the RoBERTa model and the GCN model, the weight λ is used to weight the joint module loss, and then the gradient is updated. The overall convergence is achieved by adjusting the hyperparameter λ:

[0070] logits=λlogits GCN +(1-λ)logits RoBERTa

[0071] Among them, λ controls the ratio of the two models participating in the optimization. λ=1 means that only the GCN model is optimized, and λ=0 means that only the RoBERTa model is optimized. When λ∈(0,1), joint optimization is achieved by adjusting λ. The loss function is the cross entropy loss, and the calculation formula is as follows:

[0072] loss = -ylog(logits)

[0073] Among them, y is the classification label, and the log function is the natural logarithm function. Experiments have shown that the classification effect is best when λ=0.7.

[0074] The value of y can be 0 or 1, corresponding to the pre-set classification label. In one example, in the data fusion step, the sentiment polarity ("positive" or "negative") of different hot topics can be manually labeled, and the labeled labels are used as the basis for optimizing the classification effect.

[0075] Then, the category with the highest probability is selected from the classification probability vector logits through the argmax function as the final classification result, that is, pred in the formula below. In the embodiment of the present invention, two categories are defined, namely "positive" and "negative".

[0076] pred = argmax(logits).

[0077] The following uses a data example to further illustrate the sentiment tendency classification method of the embodiment of the present invention.

[0078] In the data fusion step, the public attribute information of a social media user is collected, and the user's public attribute information and published blog posts are formatted into strings:

[0079] A**, personal signature is **. The topic is ***, and the blog posts he has published are *****.

[0080] In the text graph construction step, all the text after the above formatting operation is used as document nodes, and all the non-repeated words in the document are used as word nodes, and a total of 11,918 nodes are constructed. TF-IDF and PPMI values ​​are calculated as the weights of the edges between words and words, and between words and documents, and a total of 895,226 edges are obtained.

[0081] In the sentiment classification step, the RoBERTa model is used to embed the document node to obtain a 768-dimensional feature vector. Since there are 2 categories, after the softmax function, a 2-dimensional predicted classification probability vector is obtained. Similarly, the GCN model obtains a 2-dimensional predicted classification probability vector after the softmax function. The two classification probability vectors are then weighted and summed to obtain a 2-dimensional joint classification probability vector. Finally, the category label with the highest probability is obtained through the argmax function: positive.

[0082] To sum up, the sentiment tendency classification method for hot topics in social media in the embodiment of the present invention organically combines the pre-trained language model (RoBERTa model) with the graph convolutional network model (GCN model), and effectively applies it to the sentiment tendency classification task of hot topics in social media. It objectively and comprehensively mines the implicit sentiment tendencies of social media users towards different hot topics, and provides a reference basis for downstream tasks such as personalized recommendations and event trend prediction.

[0083] An embodiment of the present invention further provides a computer device, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 3 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store operating parameter data of each framework. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps of the emotional tendency classification method of the embodiment of the present invention are implemented.

[0084] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0085] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the emotion tendency classification method of the embodiment of the present invention are implemented.

[0086] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the sentiment tendency classification method of the embodiment of the present invention.

[0087] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A sentiment tendency classification method for hot topics in social media, characterized by: include: The data fusion step collects the public attribute information and previously published blog posts of social media users, and formats the public attribute information and previously published blog posts of users into strings to form multiple documents to be classified; The text graph construction step is to use all non-repeated words in all documents as word nodes, and multiple documents to be classified as document nodes. Edges are established between word nodes and document nodes and between word nodes. The value of word frequency-inverse document frequency is calculated as the weight of the edge between word nodes and document nodes. The value of positive point-by-point mutual information is calculated as the weight of the edge between word nodes and word nodes. The document nodes are embedded using the RoBERTa model to obtain the document feature vector. The word node features are initialized to 0 to obtain the node feature matrix of the text graph. In the sentiment tendency classification step, the document feature vector is used as the input of the RoBERTa model, and the classification probability vector based on the RoBERTa model is obtained after the softmax function. The node feature matrix is ​​used as the input of the GCN model, and the classification probability vector based on the GCN model is obtained after the softmax function. The classification probability vector based on the RoBERTa model and the classification probability vector based on the GCN model are weighted and summed to obtain the joint classification probability vector. The category with the highest probability is selected from the joint classification probability vector as the classification result. In the sentiment tendency classification step, the following joint classification probability vector is obtained: logits=λlogits GCN +(1-λ)logits RoBERTa , Among them, logits RoBERTa is the classification probability vector based on the RoBERTa model, logits GCN is the classification probability vector based on the GCN model, and λ is the weight; Classification probability vector logits based on RoBERTa model RoBERTa for: logits RoBERTa =softmax(X doc W RoBERTa ), Among them, X doc is the document feature vector, W RoBERTa are the RoBERTa model parameters, Classification probability vector logits based on GCN model GCN for: Among them, ReLU is the activation function, X is the node feature matrix, the GCN model uses 2 layers of GCN, W0 and W1 represent the parameters of the 0th layer and the 1st layer of GCN respectively, and A is the adjacency matrix of the text graph. is the normalized adjacency matrix, Where D is the degree matrix of A; Each element A in the adjacency matrix A i,j The value of is the weight E of the edge between nodes i and j i,j , in, PPMI(i,j)=max(PMI(i,j),0), Where PPMI represents positive pointwise mutual information, TF-IDF represents term frequency-inverse document frequency, PMI represents pointwise mutual information, Window(i) is the number of windows in the sliding window that contain node i, Window(j) is the number of windows in the sliding window that contain node j, Window(i,j) is the number of windows in which nodes i and j co-occur, Window is the total number of sliding windows, p(i) is the proportion of windows in the sliding window that contain word node i, p(j) is the proportion of windows in the sliding window that contain word node j, and p(i,j) is the proportion of windows in which word nodes i and j co-occur; The node feature matrix X is: Among them, d is the dimension of the document feature vector, n doc and n word Represents the number of document nodes and the number of word nodes respectively; The weight λ is adjusted through the following loss function to optimize the joint classification probability vector: loss = -ylog(logits), Among them, loss is the loss function, y is the classification label, and the value of y is 0 or 1.

2. The method according to claim 1, characterized in that λ=0.

7.

3. The method according to claim 1, characterized in that The public attribute information of the user includes nickname and personal signature.

4. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 3.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images