Social robot identification method combined with emoticon information
Through the RGCN-based graph neural network processing social media information, the problem of distinguishing between social robots and human users is solved, and the accuracy of detection and the authenticity of the information network are improved.
Patent Information
- Application Number
- CN202411044466.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art is difficult to effectively distinguish between social robots and human users. Social robots are difficult to be directly identified by imitating human behavior, resulting in the influence of bias and false information in the information network.
A social robot detection method based on RGCN graph neural network is proposed. Through expression preprocessing, user node feature processing and user relationship feature processing, expression and user information are input into the E-BotRGCN model, and graph convolution network and multi-layer The perceptron performs feature representation and classification to realize the detection of social robots.
It improves the accuracy of social robot detection and F1-score, significantly improves the ability to identify abnormal users, can more accurately distinguish between humans and social robots, and reduces bias in the information network.
Smart Images

Figure CN119088959A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of social engineering, and in particular to a social robot recognition method improved based on RGCN graph neural network. Background Art
[0002] The information exposure of users of online social networks depends on the content generated by the accounts they follow. Users obtain information by establishing relationships with other users in online social networks. Therefore, the close user network formed acts as an information filter to some extent. If social networks become diversified, the information received by users is also more diversified. However, due to the existence of social robots, there are many biases in our network. This biased information exposure is persistent, and the information we are exposed to will form an information network full of false and biased information, which will affect our understanding and cognition of the social environment. Studies have shown that on Twitter, 9% to 15% of accounts are fake. In online social networks, malicious robots are programs that automatically carry out social media activities. They have automation capabilities and can amplify the negative effects of harmful activities. Nowadays, there are even social robot networks, which are robot groups managed by a single robot administrator. Social robot networks can more effectively influence users in online social networks in a coordinated and synchronized manner.
[0003] In order to reduce the impact of these harmful bots, social media platforms have begun to focus on detecting and deleting these accounts. However, in reality, humans face great difficulties in distinguishing between bots and legitimate users because this distorts the way people perceive and interact in online social networks. Generally speaking, popular users post information to guide the public to support or question it, thereby influencing the way the public obtains and understands information. In this regard, social robots are equivalent to popular users. They have the ability to imitate legitimate users and adopt social behaviors similar to humans, such as participating in debates or commenting on specific topics, and promoting and forwarding targeted messages. These behaviors of social robots themselves are generated by learning from humans, so it is difficult for humans to directly distinguish between social robots and humans.
[0004] Existing methods for detecting abnormal user social robots can be divided into four categories: feature-based analysis methods, behavior analysis methods, graph-based methods, and deep learning-based methods. Feature analysis methods design manual features from user configurations and use labeled data sets to train machine learning models. For example, the Botometer model uses a random forest model to train and classify by analyzing the historical social information of ordinary users and social robot accounts. The model distinguishes ordinary users from social robots based on six features: network, user, friend, time, content, and emotion. Behavior analysis methods detect social robots by modeling and analyzing user behavior based on the differences in behavior patterns between social robots and real users. For example, malicious robots are detected by using the proportion of tweet forwarding, building a semi-supervised clickstream similarity graph model for user behavior, and identifying active, passive, and inactive users based on social interactions. These methods use supervised learning or unsupervised learning techniques to detect social robots by analyzing user behavior and features. Graph methods are based on the structure of social network graphs, using different graph-based features to divide graphs into two groups: ordinary graphs and attribute graphs, and using user nodes and relationship edges to detect social robots through graph segmentation. The deep learning-based method uses deep neural networks to detect social robots by training models to learn feature representation and classification rules. Compared with other methods, it has great advantages in processing large-scale data and complex patterns. Summary of the invention
[0005] This paper proposes a new detection method for social robots based on deep learning. This paper processes the expressions in tweets and re-integrates them into the deep learning framework. The framework consists of a graph model and a fully connected layer to capture various available data in the account. For tweets, this method can better analyze the semantics and other implicit features of tweets, better utilize the expression feature in tweets, capture the emotions and semantics contained in expressions, and capture the semantic similarity between tweets, and finally understand the semantics of account descriptions. By passing this information to the fully connected layer, this model can make the final prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] Figure 1 : Training model structure;
[0007] Figure 2 : Tweet extraction invention flow chart; DETAILED DESCRIPTION
[0008] In this section, this paper introduces how the entire E-BotRGCN framework is implemented. It is mainly divided into six parts: expression preprocessing, user node feature processing, user relationship feature processing, E-BotRGCN model construction, classification and experiment. The overall model is as follows Figure 1 shown.
[0009] Step 1: Expression preprocessing
[0010] Since the emojis in tweets are represented by Unicode, as shown in Table 1, traditional social robot detection will treat emojis as punctuation marks, resulting in the emojis being ignored or deleted, ignoring the important semantic information contained in the emojis. There are also great differences between humans and social robots in the number and types of emojis used, so this paper deals with emojis in Twitter separately. As mentioned above, there are significant differences in the types and number of emojis between social robots and human accounts, and the reason for this is the semantic information contained in the emojis. Because the difference between humans and social robots when using emojis lies in whether they truly understand the meaning of the emojis, it is necessary to explore the deep meaning of the emojis in order to actually distinguish between humans and social robots. Regarding the problem of expressing the meaning of emojis, this paper believes that the name of the emoji can summarize the action or content of the emoji, thereby expressing the meaning represented by the emoji to a certain extent. The traditional Python emojis library has few emojis and is no longer suitable for current emoji use; although the emoji2vec library obtains the embedded expression of emojis based on the emotional polarity of emojis, after research, it is found that the emotional polarity of the library is not suitable for social robot detection.
[0011] Finally, this article obtains a python database called emojiswitch. Compared with the previous emojis dataset, emojiswitch collects a wider range of expressions, and the translation of expressions includes both Chinese and English translations, and the translation of expressions is more accurate. Figure 2 As shown in the figure, this paper uses the emojiswitch library to convert the Unicode in the tweet into the corresponding emoticon expression and then puts it into the original tweet. Then, the new tweet features replace the original tweet features as the feature input into the model for training.
[0012] Step 2: User node feature processing
[0013] This paper classifies the processed data set according to the four features set in the data set, namely user description features, user tweet features, user numerical features and user classification features.
[0014] For user description feature r b, this paper uses the RoBERTa pre-trained language model for training. In this study, the pre-trained RoBERTa model is used to encode the new user feature tweet. As shown in formula (1), first, this paper converts user tweets and emoticons into the representation of the RoBERTa model. S represents the number of words in the user description, which is obtained by traversing each tweet of each user. The symbol b represents the vector representation of the user tweet, where D s (768 dimensions) represents the embedding dimension of RoBERTa. The tensor dimension uses the default 768 dimensions of the RoBERTa model. i represents each user's tweets, and R represents the real number domain:
[0015]
[0016] Next, we use the learnable parameter W B and b B , through linear transformation Mapped to a representation vector r of dimension D (128 dimensions) b , as shown in formula (2), which is the representative vector of the tweet after the user's expression is converted. B and b B Assign a random initial value and automatically learn through back propagation for more than 50 rounds to obtain a certain value. There is no limit on the dimension of D, and good results are obtained in 128 dimensions:
[0017]
[0018] After obtaining the vector representation of each emoticon and tweet, the vectors of each tweet and emoticon text are then summed and averaged to obtain the average vector representation of each tweet. The average vectors of all tweets of each user are then summed and averaged again to obtain the average vector representation of a single user.
[0019] For user tweet feature r t , this paper adopts and r b The same processing method is used to obtain the vector representation of the user description features.
[0020] As shown in Table 2, user information includes followers_count (number of fans), friends_count (number of followers), favorites (number of likes), status_counts (get or set the total number of tweets posted by the user), active_days (number of active days), and screen_name (number of characters in the screen name). The user's numerical features are obtained through user information. It contains some of the user's quantitative characteristics. This paper leaves the processing of user numerical attributes to the multi-layer perceptron (MLP) and graph neural network, and uses the fully connected layer to obtain user digital features. Representation.
[0021] As shown in Table 3, the classification characteristics of users are obtained through user information. It contains some user-defining features, including protected (whether the Twitter account is set to private), geo_enabled (whether geolocation is enabled), verified (whether the account is verified), contributors_enabled (whether account sharing is allowed), is_translator (whether automatic translation is allowed), is_translation_enabled (whether translation is allowed), profile_background_tile (whether the profile background is tiled), profile_user_background_image (whether the user uploaded a background image), has_extended_profile (whether there is an extension file), default_profile (whether the user uses the default theme background) and default_profile_image (whether the user is using the default profile image). For the classification features of users Similar to the user's numerical attributes, the user classification features can be directly encoded using multi-layer perceptron (MLP) and graph neural network. This paper uses the one-hot encoding method to represent these features, and connects and transforms them through the fully connected layer and RELU activation function to obtain the user classification features. Representation.
[0022] After obtaining the representation of the four features, this paper merges the four features, as shown in formula (3):
[0023]
[0024] The user node features are obtained to facilitate subsequent input into the E-BotRGCN model for processing.
[0025] Step 3: User relationship feature processing
[0026] The feature processing of user relationships mainly involves establishing the edge index and edge type of the graph. The relationship information between the neighbor nodes and the user is extracted from the user's neighbor nodes and converted into the form of edge index and edge type.
[0027] Specifically, for each neighbor, the index of the neighbor node and the index of the current node are used as the starting node and target node of the edge, and they are added to the edge index list. At the same time, according to the type of the neighbor (follower or fan), the corresponding edge type (0 or 1) is added to the edge type list.
[0028] Step 4: E-BotRGCN model construction
[0029] In terms of model selection, this paper chooses the Relational Graph Convolutional Network (RGCN), which is a neural network model used to process graph data with complex relationships.
[0030] In this paper, E-BotRGCN builds a heterogeneous graph based on RGCN, where Twitter users are represented as nodes. The model uses two types of edges, namely "following" and "follower", to reflect the interactive relationship between users. For each user, the model defines its following neighbors and follower neighbors, and constructs a heterogeneous graph reflecting the relationship between Twitter users. If the dataset provides more user relationship types, E-BotRGCN can further integrate these relationships.
[0031] The architecture of E-BotRGCN in this paper adopts RGCNs (Relational Graph Convolutional Networks) and learns user representation. Specifically, as shown in formula (4), this paper first transforms the user features to obtain the i-th node in the graph The initial hidden vector of .
[0032]
[0033] W1 and b1 are learnable parameters, and are randomly assigned initial values. The final result is obtained through more than 50 rounds of model training. i is the user’s feature vector, represents the activation function, and RELU is used as the activation function. Then this paper applies the l-th layer RGCN, as shown in formula (5):
[0034]
[0035] Where N r Represents the accounts that the user follows, obtained directly through user information, Θ self Represents the projection matrix of its own node, Θ rRepresents the projection matrix of adjacent nodes and randomly assigns initial values. Both projection matrices are learnable parameters in the model, and the final results are obtained through more than 50 rounds of model training. represents the user's adjacent node x j After the L layers of RGCN, this paper uses a multi-layer perceptron (MLP) to further transform the user representation. As shown in formula (6):
[0036]
[0037] h i is the final representation of user i, which is obtained by applying the weight matrix W2 and bias b2 to The result is obtained by linear transformation. W2 and b2 are learnable parameters and are assigned random initial values. The final result is obtained through more than 50 rounds of model training.
[0038] Step 5: Classification
[0039] This paper is about the identification method of social robots and converts it into a label classification problem. When designing the classification model, this paper uses a single-label classifier to handle this problem. Each social account is regarded as a binary value, that is, whether it is a social robot (0 represents human, 1 represents robot).
[0040] In this paper, a softmax layer is used to classify the user representations obtained from RGCN when detecting Twitter robots. In formula (7), represents the predicted probability after the softmax function processing, W O and b O It is a learnable parameter and is randomly initialized. When training the model, the value is automatically updated through back propagation. The final result is obtained through more than 50 rounds of model training. This paper converts the 128-dimensional vector into a 2-dimensional vector. For the 2-dimensional vector, this paper selects the largest value as the final output category. Specifically, this paper converts the matrix y∈R after feature extraction and RGCN training 128 With the weight matrix w∈R 128*2 Perform point multiplication and then add the corresponding bias coefficient b∈R 2 , so that the output z∈R 2 The initial values of the weight matrix and bias coefficients are generally random values. When training the model, this paper uses the back propagation algorithm to automatically update these values, as shown in formula (7):
[0041]
[0042] The loss function of E-BotRGCN consists of two parts. The first part is the cross entropy loss, which is calculated by dividing the predicted probability and the true label y i Compare and calculate the accuracy of the prediction. The second part is the weight decay term, which is used to control the complexity of the model. i∈y represents the sum of the marked users, log represents the natural logarithm, λ is the attenuation coefficient, ∑ w∈θ It represents the sum of all learnable parameters in the E-BotRGCN framework. The loss function of RGCN is constructed as follows:
[0043]
[0044] Where Z represents the loss of the model, Y represents the annotated users, which is directly obtained from the dataset, and θ is the set of learnable parameters in the BotRGCN framework, including but not limited to W in formula (2) B and b B , W1 and b1 of formula (4), Θ of formula (5) self and θ r , W2 and b2 in formula (6), W in formula (7) O and b O ,λ is the regularization parameter, which is adjusted according to the final effect of the experiment and finally takes a value of 0.005.
[0045] Step 6: Experiment
[0046] This paper uses the test results of replacing emojis with text on RGCN to compare with other baselines to verify whether the effect of converting emojis to text is better than other baselines. The experiment shows that the model E-BotRGCN in this paper is far ahead of other models. In terms of accuracy, the accuracy of E-BotRGCN is 0.8622, which is higher than other methods such as GCN (0.7295), GAT (0.7726), GIN (0.6181), Lee et al (0.7456), Yang et al.
[0047] (0.8191), Kudugunta et al. (0.8174), Wei et al (0.7126), Millerr et al (0.4801), Cresci et al (0.4793), Botometer (0.5584), Alhosseini et al (0.6813), SATAR
[0048] (0.8412) and BotRGCN (0.8495), which have higher accuracy, indicating that E-BotRGCN can detect abnormal user social robots more accurately.
[0049] In terms of the F1-score indicator, the value of E-BotRGCN is 0.8766, which is also better than other methods, showing that it has achieved a good balance between detection accuracy and recall rate.
[0050] In terms of the MCC (Matthews Correlation Coefficient) indicator, the value of E-BotRGCN is 0.7227, which is higher than most other methods, indicating that its detection results are more consistent with the actual situation.
[0051] In summary, the excellent performance of E-BotRGCN in terms of accuracy, F1-score, and MCC reflects its high performance and accuracy in detecting abnormal user social robots.
[0052] Table 1: The emoticons themselves and their Unicode and text expressions;
[0053] Table 2: User numerical characteristics;
[0054] Table 3: User classification characteristics;
[0055] Table 1
[0056]
[0057] Table 2
[0058]
[0059] Table 3
[0060]
Claims
1. A social robot recognition method combining emoticon information, characterized in that The following steps are involved: Step 1: Expression preprocessing The emojis in tweets are represented by Unicode. The Unicode in the tweets is converted into the corresponding emoji connotations and then put into the original tweets. Then, the new tweet features are used instead of the original tweet features as features to input into the model for training. Step 2: User node feature processing The dataset after expression preprocessing is classified according to the four features set in the dataset, namely user description feature, user tweet feature, user numerical feature and user classification feature; For user description feature r b , the RoBERTa pre-trained language model is used for training; the pre-trained RoBERTa model is used to encode the new user feature "etweet"; As shown in formula (1), first, the user tweets and emoticons are converted into the representation form of the RoBERTa model. S represents the number of words in the user description, which is obtained by traversing each tweet of each user. Symbol Represents the vector representation of a user's tweet, where the 768-dimensional D s represents the embedding dimension of RoBERTa. The tensor dimension uses the default 768-dimensional tensor of the RoBERTa model. i represents each user's tweets, and R represents the real number domain: Using learnable parameters W B and b B , through linear transformation Mapped to a representation vector r of dimension D b , as shown in formula (2), which is the representative vector of the tweet after the user's expression conversion; where W B and b B Assign a random initial value and automatically learn through more than 50 rounds of back propagation to obtain a certain value: After obtaining the vector representation of each emoji and tweet, we then sum and average the vectors of each tweet and emoji text to obtain the average vector representation of each tweet. Then we sum and average the average vectors of all tweets of each user to obtain the average vector representation of a single user. For user tweet feature r t , get the vector representation of user description features; User information includes followers_count, which is the number of followers; friends_count, which is the number of followers; favorites, which is the number of likes; statuses_counts, which is to get or set the total number of tweets posted by the user; active_days, which is the number of active days; and screen_name, which is the number of characters in the screen name. The user's numerical features are obtained through user information. It contains some of the user's quantitative characteristics. The processing of user numerical attributes is left to the multi-layer perceptron MLP and graph neural network, and the user digital features are obtained using the fully connected layer ' Obtain user classification characteristics through user information It contains some user-defining features, including protected, i.e., whether the Twitter account is set to private, geo_enabled, i.e., whether the geolocation is enabled, verified, i.e., whether the account is verified, contributors_enabled, i.e., whether account sharing is allowed, is_translator, i.e., whether automatic translation is allowed, is_translation_enabled, i.e., whether translation is allowed, profile_background_tile, i.e., whether the profile background is tiled, profile_user_background_image, i.e., whether the user has uploaded a background image, has_extended_profile, i.e., whether there is an extension file, default_profile, i.e., whether the user is using the default theme background, and default_profile_image, i.e., whether the user is using the default profile image. The user classification features are encoded using multi-layer perceptron MLP and graph neural network; these features are represented by one-hot encoding, and they are connected and transformed through fully connected layers and RELU activation functions to obtain user classification features. ' After obtaining the representation of the four features, the four features are merged as shown in formula (3): Get user node characteristics; Step 3: User relationship feature processing Extract the relationship information between the neighbor nodes and the user from the user's neighbor nodes, and convert it into the form of edge index and edge type; Specifically, for each neighbor, the index of the neighbor node and the index of the current node are used as the starting node and target node of the edge, and they are added to the edge index list; at the same time, according to the type of the neighbor, that is, follower or fan, the corresponding edge type, that is, 0 or 1, is added to the edge type list; Step 4: E-BotRGCN model construction In terms of model selection, we choose the relational graph convolutional network RGCN; In E-BotRGCN, a heterogeneous graph is constructed based on RGCN, where Twitter users are represented as nodes. The model uses two types of edges, namely "following" and "follower", to reflect the interactive relationship between users. As shown in formula (4), the user features are first transformed to obtain the i-th node in the graph The initial hidden vector of W1 and b1 are learnable parameters, and are randomly assigned initial values. The final result is obtained through more than 50 rounds of model training; i is the user’s feature vector, represents the activation function, and RELU is used as the activation function; then the l-th layer RGCN is applied, as shown in formula (5): Where N r Represents the account that the user follows, obtained directly through user information, Θ self Represents the projection matrix of its own node, Θ r Represents the projection matrix of adjacent nodes and randomly assigns initial values. Both projection matrices are learnable parameters in the model, and the final results are obtained through more than 50 rounds of model training. represents the user's adjacent node x j After passing through the L layer of RGCN, the user representation is further transformed using a multi-layer perceptron (MLP), as shown in formula (6): h i is the final representation of user i, which is obtained by applying the weight matrix W2 and bias b2 to The linear transformation is performed, W2 and b2 are learnable parameters, and are given random initial values. The final result is obtained through more than 50 rounds of model training; Step 5: Classification It is a method for identifying social robots and converting it into a label classification problem. When designing the classification model, a single-label classifier is used to handle this problem. Each social account is regarded as a binary value, that is, whether it is a social robot, 0 represents a human, and 1 represents a robot. When detecting Twitter robots, a softmax layer is used to classify users based on the user representations obtained from RGCN; in formula (7), represents the predicted probability after the softmax function processing, W O and b O It is a learnable parameter and is randomly initialized. When training the model, the value is automatically updated by back propagation. The final result is obtained through more than 50 rounds of model training. The 128-dimensional vector is converted to a 2-dimensional vector. For the 2-dimensional vector, the largest value is selected as the final output category. Specifically, the matrix y∈R after feature extraction and RGCN training is 128 With the weight matrix w∈R 128*2 Perform point multiplication and then add the corresponding bias coefficient b∈R 2 , so that the output z∈R 2 ; The initial values of the weight matrix and bias coefficients are random values, and when training the model, the back propagation algorithm is used to automatically update these values, as shown in formula (7): The loss function of E-BotRGCN consists of two parts; the first part is the cross entropy loss by converting the predicted probability and the true label y i Compare and calculate the accuracy of the prediction; the second part is the weight decay term, which is used to control the complexity of the model; i∈y represents the sum of the marked users, log represents the natural logarithm, λ is the attenuation coefficient, ∑ w∈θ It represents the sum of all learnable parameters in the E-BotRGCN framework; the loss function of RGCN is constructed as follows: Where Z represents the loss of the model, Y represents the annotated users, which is directly obtained from the dataset, and θ is the set of learnable parameters in the BotRGCN framework, including but not limited to W in formula (2) B and b B , W1 and b1 of formula (4), Θ of formula (5) self and θ r , W2 and b2 in formula (6), W in formula (7) O and b O , λ is the regularization parameter, and its value is 0.005.
