A social text classification method, system, computer device and storage medium

Abstract

This application proposes a social text classification method based on a graph convolutional network, comprising the following steps: obtaining social text data, which includes users and their text content; calculating the text content of each user and applying a trained BERT-attention model to obtain a user text vector; constructing a user association graph with user text vectors as nodes and the number of text contents sent between users as edges; performing a graph convolution operation on the user association graph to obtain an associated text vector; and obtaining a classification label for the user's text content based on the associated text vector and the user text vector. Based on BERT and graph convolutional neural networks, the present invention not only focuses on the chat text of the characters themselves, but also mines and models information from the characters' overall social content, quantifies the associations between characters, and improves the accuracy of social text classification.

Description

Technical Field

[0001] The present application belongs to the field of natural language processing technology, and specifically relates to a social text classification method, system, computer device and storage medium based on graph convolutional network. Background Art

[0002] User tag classification refers to the process of labeling users based on multiple aspects such as their behavior patterns, speech content, and information data. In the prior art, there are three methods for categorizing user tags. One is user habit analysis based on recommendation algorithms, such as the use of matrix decomposition, factorization machines, and deep cooperative neural networks (DeepCoNN), which can construct character portraits of people's consumption habits and browsing habits in e-commerce, short videos, and other fields. The second is character portrait analysis based on keyword extraction, such as the LDA keyword extraction model, the tf-idf keyword extraction algorithm, and the BI-lstm-attenion deep learning word extraction model. The third is user portrait construction based on text classification models, such as using TextCnn, TextRnn, and transformer models to label user chat texts and categorize characters based on the labeling results. However, in actual applications, the above solutions still have problems such as inaccurate classification. Summary of the Invention

[0003] In response to the above problems, the first aspect of the present invention proposes a social text classification method based on a graph convolutional network, comprising the steps of: obtaining social text data, which includes users and their text content; calculating the text content of each user to obtain a user text vector; constructing a user association graph with the user text vector as a node and the number of text contents sent between users as an edge; performing a graph convolution operation based on the user association graph to obtain an associated text vector; and obtaining a classification label for the user's text content based on the associated text vector.

[0004] Preferably, the trained BERT-attention model is applied to obtain the user text vector.

[0005] Preferably, the input of the BERT-attention model is a sentence set X. The construction of the sentence set X includes the following steps: performing primary classification on the user's text content; extracting a certain number of sentences from each category to form the sentence set X, wherein the number of sentences extracted from each category is proportional to the proportion of the number of sentences in the category to the total number of sentences in the text content.

[0006] Preferably, the BERT-attention model calculates the input text content as a sentence vector, and applies a self-attention mechanism to perform weighted summation on the sentence vectors to obtain a user text vector.

[0007] Preferably, the edges of the user association graph are constructed only between users whose number of sent text contents is greater than a threshold.

[0008] Preferably, the classification label of the user's text content is obtained based on the associated text vector and the user text vector.

[0009] Preferably, the calculation of the classification label of the user's text content specifically includes the steps of: concatenating the associated text vector and the user text vector to obtain a concatenated vector; and classifying the concatenated vector to obtain the classification label of the user's text content.

[0010] The second aspect of the present invention proposes a social text classification system based on a graph convolutional network, comprising:

[0011] A data crawling module configured to obtain social text data, where the social text data includes users and text content of users;

[0012] A text content classification module configured to calculate the text content of each user to obtain a user text vector;

[0013] A user association graph construction module configured to construct a user association graph using user text vectors as nodes and the number of text contents sent between users as edges;

[0014] A graph convolution module is configured to perform graph convolution operations based on the user association graph to obtain associated text vectors;

[0015] The user profiling module is configured to obtain classification labels of user text content based on associated text vectors.

[0016] A third aspect of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method as described in any one of the first aspects when executing the computer program.

[0017] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which implements the method as described in any one of the first aspects when executed by a processor.

[0018] The solution presented in this paper leverages BERT and graph convolutional neural networks to fully mine the social discourse of social people online and assign labels to them. Compared to existing user profile labeling techniques based on text classification models, this solution not only focuses on the chat text of the person, but also mines and models information from the person's overall social content, quantifying the connections between people. The modeling process incorporates chat content information from related users, which is then quantified alongside the user's own text content. This results in a category-based person profiling method, improving the accuracy of social text classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings help further understand the present application. The elements of the drawings are not necessarily in scale with each other. For ease of description, only the parts relevant to the invention are shown in the drawings.

[0020] Figure 1 This is a step diagram of a social text classification method based on a graph convolutional network in one embodiment of the present invention;

[0021] Figure 2 This is a technical framework diagram of social text classification based on graph convolutional networks in one embodiment of the present invention;

[0022] Figure 3 This is a framework diagram of a social text classification model based on a graph convolutional network in one embodiment of the present invention;

[0023] Figure 4 This is a framework diagram of a social text classification system based on graph convolutional networks in another embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention.

[0025] BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained neural network model developed by Google. It adopts the Transformer architecture and uses a bidirectional encoder to generate contextual word vector representations. BERT is able to achieve state-of-the-art performance in multiple natural language processing tasks, including question answering, sentiment analysis, and named entity recognition. The training process of the BERT model can be summarized into two stages: pre-training and fine-tuning. In the pre-training stage, BERT uses massive amounts of unlabeled text data to learn universal language representations that can be applied to a variety of natural language processing tasks. In the fine-tuning stage, BERT is fine-tuned from labeled data in specific domains through supervised learning to improve its performance.

[0026] The main technology of this invention is to first use the BERT model to train a base class model classifier to perform category judgment and sentence vector conversion on the chat text. Secondly, taking the chatter as the unit, self-attention and graph convolutional neural network (GCN) are used to mine and analyze the chat text of both the chatter itself and the contacts. Finally, the chatter is labeled with a category based on the mining and analysis results.

[0027] Figure 1 This is a step diagram of a social text classification method based on a graph convolutional network in a specific embodiment. Figure 2 This is a diagram of the social text classification technology framework based on graph convolutional networks in this embodiment. The steps include:

[0028] S1, obtaining social text data, where the social text data includes users and their text content.

[0029] Generally, data can be obtained from social software such as Weibo and QQ by web crawlers. Specifically, in this embodiment, group chat data is obtained by web crawlers. The crawled data is divided according to the chat content and chatters to obtain the person set P = {p1, p2, ... p m} and text set S={s1,s2,...s n}, where m represents the number of chat characters and n represents the content of the chat text. The data was cleaned according to the standardized format and the results are shown in Table 1.

[0030] Table 1 Group chat data format example

[0031]

[0032] Typically, chat content contains some internet slang, so data cleaning and preprocessing are necessary. Specifically, the following steps are required: 1. Use regular expressions to filter out distracting text such as phone numbers, ID numbers, chat prompts, and web links; 2. Use collected emoticons, dictionaries, and stop word dictionaries to remove emoticons and stop words from the data; 3. Use a specialized dictionary collected for social text as a user dictionary and segment the chat content using the Jieba word segmentation tool.

[0033] S2, calculate the text content of each user to obtain the user text vector.

[0034] A dataset is formed by extracting some chat content from the preprocessed dataset and annotating the extracted dataset. The annotated dataset is divided into a training set, a test set, and a validation set, and a sentiment classification model is trained. The present invention uses the BERT model to train the base classification model and uses the annotated dataset to train a text classification model.

[0035] In this embodiment, the user text vector is synthesized based on the BERT-attention model. The user text vector is a text vector obtained by quantizing all the chat content of the chatter. The synthesis steps of the user text vector are:

[0036] First, use the trained BERT model to mark the user's chat content for text classification.

[0037] Second, set the number of sentences to be extracted m, and extract m pieces of data from the chat text according to the category ratio to form a sentence set X = {x i ,x2,...x m}.

[0038] Third, use the trained BERT classifier to perform embedding on the input sentence to obtain sentence_embedding, and perform self-attention weighted summation on the summarized sentence vectors to obtain the personal text vector v. The specific formula is as follows:

[0039]

[0040]

[0041]

[0042]

[0043] Where f represents the generating function of the BERT model, Represents the matrix formed by concatenating the generated sentence vectors, W t , b t To calculate the weight matrix and bias term when paying attention, v is the personal comprehensive vector, which is a comprehensive vector representation of the chat text of the user target.

[0044] S3, takes user text vectors as nodes and the number of text contents sent between users as edges to construct a user association graph.

[0045] Social text is a form of textual communication and interaction between people. In social text, a person's influence comes not only from themselves but also from their connections. A graph convolutional neural network (GCN) is a deep learning model that captures information between nodes in a graph format. In this example, a graph convolutional neural network is built using chat content as edges and the chatterers' user text vectors as node vectors.

[0046] The specific steps of constructing the user association graph include: using the chatter's user text vector as a node, and constructing the connecting edge based on the number of texts sent between the chatter and other chatters. In order to reduce the construction of invalid edges, the present invention sets a threshold value m. If the number of texts sent between users is greater than m, the edge is constructed. The constructed adjacency matrix a ij The formula is as follows:

[0047]

[0048]

[0049] Among them, n ij represents the number of sentences sent between user i and user j, a j It represents the normalization process using the value of the adjacent node j of node i.

[0050] S4, performs graph convolution operation based on the user association graph to obtain the associated text vector.

[0051] The global graph convolution iteration is performed based on the graph convolution and the user's corresponding comprehensive text vector. In the graph convolution, the output value of the node can also be used as the input value for multiple iterations. This embodiment uses 3 layers of graph convolution for iteration and performs global extraction of the nodes of the graph information multiple times. Let the dependency graph g = (v, ε), v and ε represent the node set and edge set, then the vector output of the node i in the kth layer is The calculation formula is as follows:

[0052]

[0053]

[0054] A ij represents the value of the adjacency matrix, represents the output of node j at the k-1 layer, w (k) represents the weight matrix, b (k) represents the deviation vector, 1 / d i represents the normalization operation for node i, and ρ(.) represents the ReLU activation function. The k-th layer node can receive not only the output of the k-1th layer node, but also the output of all previous layers. In order to sum the receptive fields of the output vectors of each layer node, a self-attention operation is performed on the output of the last layer. The main formula is:

[0055]

[0056]

[0057] h i represents the user association vector obtained by the i-th node, represents the output information of other layers carried by node i during the k-th layer graph convolution, v i represents the initial vector of the i-th node, which is also the personal comprehensive category vector v obtained in the previous step, g(.) represents the self-attention mechanism function, W i , B i is the weight matrix and bias term of the self-attention mechanism function when node i is the node i.

[0058] S5, based on the associated text vector, obtain the classification label of the user's text content. Select a suitable classification function to classify the associated text vector to obtain the classification label.

[0059] In a preferred embodiment, a person's label can be obtained based on the personal text vector and the associated text vector. Figure 3 This is the framework diagram of the model in this embodiment. In this embodiment, the associated text vector and the personal text vector are spliced, and the probability category is judged by using full connection and softmax function. The specific function formula is as follows:

[0060] x=concat(h i , v i )

[0061] z=fc(dropout(x))

[0062] pred=soft max(z)

[0063] Finally, an argmax judgment is performed based on the probability result of pred, and the category corresponding to the value with the largest probability is used as the category label of the user.

[0064] The loss function still uses the cross entropy loss function. The dropout function and L1 regularization coefficient are added to the fully connected layer to prevent the model from overfitting. The formula of the loss function is as follows:

[0065] L=-∑y i log(p i )+λ||Θ||

[0066] y i Represents the true label p i represents the predicted probability value, and Θ represents other parameters of the model.

[0067] Figure 4 In another specific embodiment, a social text classification system 400 based on a graph convolutional network includes:

[0068] A data crawling module 401 is configured to obtain social text data, where the social text data includes users and their text content;

[0069] A text content classification module 402 is configured to calculate the text content of each user to obtain a user text vector;

[0070] A user association graph construction module 403 is configured to construct a user association graph using user text vectors as nodes and the number of text contents sent between users as edges;

[0071] A graph convolution module 404 is configured to perform a graph convolution operation based on the user association graph to obtain an associated text vector;

[0072] The user portrait module 405 is configured to obtain a classification label of the user's text content based on the associated text vector.

[0073] This paper proposes a character profiling method based on BERT and GCN for social text classification. This method primarily analyzes the chatter's text and uses deep learning methods such as sentence embedding and self-attention to quantify the social text content of both the chatter and its associated people to generate a character profile. This method overcomes shortcomings of existing classification models, such as insufficient mining of chat content and failure to consider chatter relevance. This method has broad application prospects for character profiling and other functions in various terminal apps.

[0074] Although the contents of the present application are specifically shown and described in conjunction with the preferred embodiments, those skilled in the art should understand that various changes made to the present application in form and details without departing from the spirit and scope of the present application as defined by the appended claims and without making any creative work are within the scope of protection of the present application.

Claims

1. A social text classification method, characterized in that: Including steps: Acquire social text data, where the social text data includes users and text content of users; Applying the trained BERT-attention model to obtain a user text vector; wherein the BERT-attention model calculates the input text content as a sentence vector, and applies a self-attention mechanism to perform a weighted summation on the sentence vectors to obtain the user text vector; Constructing a user association graph using the user text vectors as nodes and the number of text contents sent between users as edges, wherein the edges of the user association graph are only constructed between users whose number of sent text contents is greater than a threshold; Performing a graph convolution operation based on the user association graph to obtain an associated text vector; Based on the associated text vector, a classification label of the user's text content is obtained.

2. The social text classification method according to claim 1, characterized in that The input of the BERT-attention model is a sentence set , the sentence set The construction includes the steps: Performing preliminary classification on the text content of the user; Extract in each category according to the category ratio m sentences, forming the sentence set , where the category ratio is the ratio of the number of sentences in the category to the total number of sentences in the text content.

3. The social text classification method according to claim 1, characterized in that The classification label of the user's text content is obtained based on the associated text vector and the user text vector.

4. The social text classification method according to claim 3, characterized in that The calculation of the classification label of the user's text content specifically includes the following steps: Splicing the associated text vector and the user text vector to obtain a spliced ​​vector; Classify the concatenated vector to obtain a classification label of the user's text content.

5. A social text classification system, characterized in that: include: A data crawling module configured to obtain social text data, wherein the social text data includes users and text content of users; A text content classification module configured to calculate the text content of each user to obtain a user text vector; A user association graph construction module configured to construct a user association graph using the user text vectors as nodes and the number of text contents sent between users as edges; A graph convolution module configured to perform a graph convolution operation based on the user association graph to obtain an associated text vector; The user portrait module is configured to obtain a classification label of the user's text content based on the associated text vector.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images