Social bot detection algorithm based on user semantics, attributes and neighborhood information

Abstract

The present application belongs to the technical field of big data mining, and specifically relates to a social robot detection algorithm based on user semantics, attributes and neighborhood information. The algorithm comprises: encoding the text content through a BERT model, modeling the semantic representation of the user, and combining the user attributes and neighborhood features to build a user relationship network with the user as a node and the forwarding relationship as an edge. The constructed graph data is subjected to self-supervised training using an improved graph attention network model to learn the social user representation. The model is subjected to parallelized calculation in the form of subgraph sampling, and auxiliary tasks are set using multi-task learning. In order to solve the data imbalance problem, a conditional adversarial generative network is used for data augmentation, and the final user vector representation is subjected to density-based clustering to obtain the discrimination result of whether it is a social robot. The algorithm adopts a self-supervised technology and introduces an adversarial idea, and is strong in generalizability and in line with the future technology development trend.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of big data mining, and particularly relates to a social robot detection algorithm based on user semantics, attributes and neighborhood information. BACKGROUND

[0002] With the continuous deepening of the network into people's life, online social platforms have become a public platform for people to express their opinions. Social media represented by Facebook, Twitter abroad and Sina Weibo at home has attracted billions of users worldwide. As of May 2021, Facebook has had 2.9 billion monthly active users, close to 40% of the world's population, which has brought about the vigorous development of social robot technology. Social robots interact with human users in an organized manner by imitating and simulating human behavior in social media, and influence target audiences according to the intention of human manipulators. They participate in the spread and discussion of social hot events like people, and participate in the dissemination for certain commercial purposes, while the general public are unaware. Since the early days of online social networks, social robots have coexisted with humans, but there has been a lack of an accurate and consistent definition of what a social robot is. The reason is that people who study them belong to different disciplines, leading to different definitions, and each definition focuses on different features. Today, social robots have become another actor in the network space, especially in social media platforms. Studies have shown that social robots account for 9%-15% of all Twitter accounts.

[0003] Social robots refer to artificial intelligence applications that play the role of humans in social networks, have different degrees of personality attributes, and interact with humans. According to the degree of automation, they can be divided into fully automatic robots and semi-automatic robots; according to the purpose, they can be divided into spam robots, public opinion robots, etc.

[0004] In the past decade, detection techniques for social robots have emerged, from manual recognition to machine learning using feature engineering, from individual-based analysis to group-based analysis, from supervised to unsupervised methods. Although detection techniques are constantly improving, their generalizability and generalization ability have not been improved. On the one hand, this is due to the evolution of social robots, and on the other hand, most detection techniques use supervised models, which are limited by the diversity and size of the sample set, which can easily lead to model overfitting; in addition, the collection of social robot sample labels for supervised models is also a difficulty. Therefore, in recent years, researchers have begun to consider moving away from supervised models and using unsupervised or adversarial methods to improve the generalization ability of the model.

[0005] The inventors found that the existing behavior feature extraction of social robots using unsupervised or adversarial methods is not comprehensive enough, resulting in that the generalization ability of the existing methods cannot reach the expected level when facing new social robots. SUMMARY

[0006] In view of the above, the purpose of the present application is to provide a social robot detection algorithm based on user semantics, attributes and neighborhood information with strong generalization ability and without sample labels.

[0007] The social robot detection algorithm based on user semantics, attributes and neighborhood information provided by the present application has the overall process as shown in FIG. 1, which includes data preprocessing of the social account data set to be identified; using a BERT model to encode the account text features to obtain a text feature vector, and encoding and embedding the user neighborhood features and attribute features, and finally constructing a complete social user representation vector through joint encoding. In order to solve the problem of sample imbalance in the original data set, the present application uses a conditional generative adversarial network for data augmentation: the user representation vector is subjected to density peak clustering and the noise and small user sets are removed, the clustering sequence labels are marked and used as the labels of each node user, the label user representation vector is input into an improved conditional adversarial generation network model, and the generated samples after training are combined with the original data to obtain augmented data. The augmented data is input into an improved graph attention network as the initial hidden vector of the node, parallel computing is performed through subgraph sampling, and the node closeness is calculated using the graph closeness matrix based on the pagerank algorithm and the neighbor nodes are selected. The model uses a multi-task learning method to construct auxiliary tasks for self-supervised training, and the output of the finally trained discriminant model is subjected to HDBSCAN clustering to obtain the identification result of whether it is a social robot. The specific steps are as follows:

[0008] Step 1: data preprocessing of the social account information to be identified; the social account information includes user text, attribute information and neighborhood information. The text information comes from the recent tweets published by the account; the attribute information comes from the public basic information of the social account; the neighborhood information includes the interaction relationship between the target account and other accounts, which can be forwarding, commenting, following, etc. The preprocessing process includes:

[0009] (1) text data preprocessing, the text is divided into semantic units to facilitate the processing of downstream tasks;

[0010] (2) attribute data preprocessing, the missing values and abnormal points that may exist in the original data are processed to improve the data quality and enhance the analysis reliability;

[0011] (3) Neighborhood data preprocessing, modeling the relationship between accounts to get a relationship graph network, and deleting or filling in the abnormal nodes in the network to facilitate further analysis of the user's field information.

[0012] Step 2: Embedding and aggregation of the preprocessed data in step 1 to obtain a complete user vector representation. Specifically, it includes:

[0013] (1) input the preprocessed text data into the pre-trained RoBERTa model [2] to obtain the user semantic feature vector;

[0014] (2) encode the preprocessed attribute data to obtain the user semantic feature vector;

[0015] (3) encode the processed neighborhood data to obtain the user neighborhood feature vector;

[0016] (4) aggregate the user semantic feature vector, user attribute feature vector and user neighborhood feature vector to obtain the final user representation vector.

[0017] Step 3: Density peak clustering of the user representation vector obtained in step 2, remove noise and small user sets, and assign a cluster sequence label to each node user as a label. Specifically, input the label user representation vector into an improved conditional generative adversarial network model [3], which introduces the Wasserstein distance and uses gradient penalty to improve the convergence condition of the conditional generative adversarial network. After multiple rounds of training, the generated samples are combined with the original data to obtain augmented data, which solves the problem of unbalanced data distribution.

[0018] Step 4: Construct an input graph for the augmented data obtained in step 3 and input it into a graph embedding model for training. The graph embedding model used in the present application is an optimized model based on the Graph-bert model [4], which uses subgraph sampling to optimize complexity and uses node intimacy to select neighbor nodes. In the pre-training process of the model, auxiliary signals related to the downstream social robot detection task are introduced, and multi-task learning is used for self-supervised training. The output of the finally trained model is clustered by HDBSCAN, and the clustering result is used to determine whether it is a social robot.

[0019] In the present application, the specific process of step 1 is:

[0020] Step 1-1: Preprocess the text information in the social account information to be identified, including: segmenting the text content, filtering stop words, deleting all meaningless symbols, deleting all missing lines, etc., to obtain the processed user text data;

[0021] Step 1-2: Preprocess the attribute information in the social account information to be identified, including: using one-hot encoding for some non-numeric features, filling in missing data, deleting duplicate data, etc., to finally obtain processed user attribute data;

[0022] Step 1-3: Preprocess the neighborhood information in the social account information to be identified, including: constructing a forwarding relationship network with users as nodes and forwarding relationships as edges; filling in missing user nodes, deleting duplicate and isolated nodes in the network, and self-loop, etc., to finally obtain processed user neighborhood data.

[0023] In the present application, the specific process of step 2 is as follows:

[0024] Step 2-1: For the preprocessed user text data, use the pre-trained RoBERTa model to encode it to obtain the user semantic feature vector:

[0025]

[0026] Where D is the embedding dimension, h i is the user text data.

[0027] Step 2-2: For the preprocessed user attribute data, perform normalization operation and connect a fully connected layer as the user attribute feature vector:

[0028]

[0029] Where D is the embedding dimension, p i is the user attribute data.

[0030] Step 2-3: For the preprocessed user neighborhood data, in order to obtain the global position information of the node, the WL algorithm [5] is used for embedding:

[0031]

[0032] Where D is the embedding dimension, WL(v j ) is the vector obtained by encoding the user field data v j using the WL algorithm, and i is the number of data entries.

[0033] Step 2-4: Aggregate the user semantic feature vector, attribute feature vector and network feature vector to obtain the complete user representation vector.

[0034]

[0035] In the present application, the specific process of step 3 is as follows:

[0036] Step 3-1: Density-based clustering is performed on the user representation vector obtained in step 2. Different clusters obtained by clustering are labeled as negative example samples, while clustering noise is divided into positive example samples. By analyzing the clustering results, it is found that there is a sample imbalance problem in which the number of positive example samples is much larger than that of negative example samples. In order to reduce the prior information of the sample proportion learned by the model, the number of negative example user data is extracted, and the cluster sequence label is marked and assigned to each node user in the cluster. The remaining nodes are uniformly labeled as a cluster sequence.

[0037] Step 3-2: The user representation vector with cluster sequence label obtained in step 3-1 is input into an improved conditional adversarial generation network for training. The traditional adversarial generation network consists of a generator G and a discriminator D, and the loss function of the entire training process is:

[0038]

[0039] where D is the discriminator, G is the generator, p z(z) is the input noise distribution, p data(x) is the original data distribution. The improved model introduces the Wasserstein distance to solve the model collapse and gradient disappearance problems in the traditional generative adversarial network, and uses gradient penalty to improve the convergence condition of the adversarial generation network. In order to meet the first-order Lipschitz continuity condition, the gradient length of the function at any position cannot exceed a fixed value, and the "arbitrary point" can be constructed by using linear interpolation:

[0040]

[0041] where ε is a random number uniformly distributed between 0 and 1, x r represents the real data, and x g represents the generated data. The generated by mixing samples is used to replace the "arbitrary point" to meet the gradient constraint condition.

[0042] At this time, the loss functions of the generator G and the discriminator D are respectively:

[0043]

[0044]

[0045] where p data is the distribution of real samples, p g is the distribution learned by the generator G, is the distribution of "arbitrary" points, and f is the gradient of the function.

[0046] Finally, the new samples generated by training are fused with the original samples to obtain augmented data, and an input graph is constructed according to the neighborhood information wherein, and ε represent the set of nodes and edges in the graph G respectively, and the mapping and are the projections of the features and labels of the node pair respectively, and the symbols x and y represent the feature space and the label space respectively, and for the node v i , the feature vector and the label vector thereof can be represented as and The size of the graph can be represented by the number of nodes in the graph.

[0047] In the present application, the specific process of step 4 is as follows:

[0048] Step 4-1: Subgraph sampling is performed on the input graph G obtained in step 3, and the input graph G is decomposed into |v| edgeless subgraphs Each subgraph contains a center node and a fixed number of neighbor nodes, wherein the neighbor nodes represent the context information of the center node. The present application adopts a top-k closeness sampling method, which is based on a graph closeness matrix for sampling, wherein S(i,j) measures the closeness score between the nodes v i and v j ; and for the measurement index of node closeness, the present application adopts a graph closeness matrix based on the PageRank algorithm[4], which is specifically as follows:

[0049]

[0050] wherein A is the adjacency matrix of the input graph, is the result of column normalization of the adjacency matrix A, and α∈[0,1], generally taking a value of 0.15. For any target node Based on the closeness matrix S, the neighbor node set can be defined as:

[0051]

[0052] wherein, "A\B" represents the difference set of set A minus set B, "∪" represents the union set, θi is the minimum closeness score threshold, and takes a value of the closeness score of the kth neighbor node arranged in descending order. By selecting the center node v i and the k neighbor nodes Γ(v i ) with the highest closeness score to jointly form an edgeless subgraph , which is represented as V iA subgraph is sampled for the target node, and then the input graph is sampled |v| times in sequence, and finally a subgraph set can be obtained

[0053] Step 4-2: For the subgraph set obtained in step 4-1 The network features of the nodes in the graph are embedded and represented. The present invention adopts the graph embedding method in the Graph-Bert model [4]. After position encoding fusion, the subgraph Midpoint v i The embedding vector of can be expressed as Then for the subgraph The matrix can be constructed:

[0054]

[0055] And input it into the graph Transformer encoder, update the node representation through multiple layers of iteration, and the final model output z i This is the learned node feature representation. The present invention pre-trains the model by designing and integrating three different tasks:

[0056] Task 1 is the node attribute reconstruction task, for the subgraph The target node v in i , after training the Graph-Bert model, a new node feature representation z is obtained i , and then after passing through the fully connected layer, the reconstructed original features can be obtained In order to ensure that the learned node feature representation can capture the original attribute information of the node, the loss function can be constructed as follows:

[0057]

[0058] Among them, x i For node v i The original feature vector of .

[0059] Task 2 is the graph structure recovery task. In order to ensure that the new node feature representation can also capture the graph structure information, a new node similarity matrix can be constructed for it. The residual similarity is used to calculate the intimacy score between nodes, that is, By comparing with the true value intimacy matrix S obtained in step 4-1, the loss function can be constructed as follows:

[0060]

[0061] in,

[0062] Task three, node classification task, in order to ensure that the learned node features can be better used for downstream social robot detection task, the label related to downstream social robot detection task As a self-supervised signal to train, specifically in node feature representation z i A layer of softmax is connected behind, and the output is:

[0063]

[0064] Wherein, z i is the learned user representation vector, W f and b f are learnable parameters, is the predicted probability of each category. The label of the user is After one-hot encoding, the true value y=[y1, y2,..., y D ] is obtained, and the loss function is constructed by minimizing cross entropy as follows:

[0065]

[0066] Wherein, D is the total number of label categories.

[0067] The present application is pre-trained by combining the above three self-supervised tasks, specifically constructing a joint loss function:

[0068]

[0069] Wherein, λ1, λ2, λ3 are all learnable parameters, in order to find the appropriate weight, the present application uses DWA algorithm

[10] for optimization.

[0070] Step 4-3: the node feature representation output by the pre-trained model Use HDBSCAN algorithm[5] for clustering, mark the noise in the clustering result as natural user; Exclude small clusters and noise interference, mark the remaining suspicious node cluster as robot account, and finally obtain the identification result of whether it is a social robot.

[0071] The innovation of the present application is that the present application proposes a self-supervised social user representation method based on user semantics, attributes and neighborhood information joint coding, and applies it to social robot detection. Unlike the previous social robot detection model based on user semantics and attribute information, which tends to use supervised machine learning methods, the present method pays more attention to the neighborhood information of the user and constructs a user relationship graph based on the neighborhood information. The method learns the potential user neighborhood features in the network through graph embedding. In view of the shortcoming of the previous graph-based social robot detection method that cannot handle large-scale graphs, the present application uses an optimized Graph-bert graph embedding model for parallel operation. By clustering the data based on density and assigning cluster labels to the nodes, on the one hand, the sample labels can be input into the conditional generative adversarial network to generate negative samples, thereby solving the problem of sample imbalance; on the other hand, the self-supervised labels can be used for pre-training of the model to improve the generalization ability of the model. The whole model framework does not need original label data, and the robustness of the model is enhanced by introducing an adversarial idea; at the same time, the model considers the application on large-scale data, fully catering to the development trend of future social robot detection technology. BRIEF DESCRIPTION OF DRAWINGS

[0072] Figure 1 The whole flow chart of the social robot detection algorithm based on user semantics, attributes and neighborhood information of the present application.

[0073] Figure 2 The model structure diagram in the social robot detection algorithm based on user semantics, attributes and neighborhood information of the present application. DETAILED DESCRIPTION

[0074] In order to make the above-mentioned purposes and innovations of the present application easier to understand, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0075] Step 1: data preprocessing is performed on the social user information to be identified, including: text data preprocessing, attribute data preprocessing, neighborhood data preprocessing; wherein:

[0076] Step 1-1: the text information in the social account information to be identified is preprocessed, including: word segmentation is performed on the text content, stop words are filtered, all meaningless symbols are deleted, and all missing lines are deleted; finally, the processed user text data is obtained;

[0077] Step 1-2: the attribute information in the social account information to be identified is preprocessed, including: one-hot encoding is used for some non-numeric features, missing data is filled, and duplicate data is deleted; finally, the processed user attribute data is obtained;

[0078] Step 1-3: Preprocess the neighborhood information in the social account information to be identified, including: constructing a forwarding relationship network with users as nodes and forwarding relationships as edges; filling in missing user nodes, deleting duplicate and isolated nodes in the network, and self-loops, etc., to obtain processed user neighborhood data.

[0079] Step 2: For the processed text data, encode and train through the pre-trained RoBERTa model to obtain user semantic feature vectors; extract node degree, forwarding level, and other network feature vectors from the constructed user forwarding network; and aggregate the user semantic feature vectors with user attributes and network feature vectors:

[0080] Step 2-1: For the preprocessed user text data, use the pre-trained RoBERTa model to encode it to obtain user semantic feature vectors:

[0081]

[0082] where D is the embedding dimension, h i is the user text data.

[0083] Step 2-2: For the preprocessed user attribute data, perform normalization and connect a fully connected layer as the user's attribute feature vector:

[0084]

[0085] where D is the embedding dimension, p i is the user attribute data.

[0086] Step 2-3: For the preprocessed user neighborhood data, in order to obtain the global position information of the node, use the WL algorithm [5] to embed it:

[0087]

[0088] where D is the embedding dimension, WL(v j ) is the vector obtained by encoding the user field data v j using the WL algorithm, and i is the number of data entries.

[0089] Step 2-4: Aggregate the user semantic feature vectors, attribute feature vectors, and network feature vectors to obtain the complete user representation vector.

[0090]

[0091] Step 3: The user representation vector obtained in step 2 is subjected to density peak value-based clustering to remove noise and small user sets, cluster sequence labels are assigned as labels of each node user, and the labeled user representation vector is input into an improved conditional generative adversarial network model to generate label samples.

[0092] Step 3-1: The user representation vector obtained in step 2 is subjected to density-based clustering, and different clusters obtained by clustering are labeled as negative example samples, and clustering noise is divided into positive example samples. By analyzing the clustering results, it is found that the data has a sample imbalance problem that the number of positive example samples is much larger than that of negative example samples; in order to reduce the prior information of the sample proportion learned by the model, therefore, after extracting the small number of negative example user data, sequence labels are assigned to each node user in the cluster; the remaining nodes are uniformly labeled as a cluster sequence.

[0093] Step 3-2: The user representation vector with cluster sequence labels obtained in step 3-1 is input into an improved conditional adversarial generative network as a minority class sample for training. In order to solve the problems of model collapse and gradient disappearance in the traditional generative adversarial network, the improved model introduces the Wasserstein distance and uses gradient penalty to improve the convergence judgment condition of the adversarial generative network. In order to meet the first-order Lipschitz continuous condition, the gradient length of the function at any position cannot exceed a fixed value, and a linear interpolation method can be used to construct "arbitrary points":

[0094]

[0095] where ε is a random number uniformly distributed between (0, 1), x r represents real data, and x g represents generated data. The generated is used to replace the "arbitrary point" to meet the gradient constraint condition.

[0096] At this time, the loss functions of the generator G and the discriminator D are respectively:

[0097]

[0098]

[0099] where p data is the distribution of real samples, p g is the distribution learned by the generator G, is the distribution of "arbitrary" points , and is the gradient of the function.

[0100] Finally, the new samples generated by training are fused with the original samples to obtain augmented data, and the input graph is constructed according to the neighborhood information where and ε represent the set of nodes and edges in graph G respectively; the mapping and are the projection of the features and labels of the node pair respectively; the symbols x and y represent the feature space and label space respectively; for node v i , the feature vector and label vector can be represented as and The size of the graph can be represented by the number of nodes in the graph.

[0101] Step 4: The input graph G obtained in step 3 is input to the graph embedding model after subgraph sampling, and the neighborhood information is learned by selecting neighbor nodes based on the pagerank algorithm-based closeness matrix. The training adopts a multi-task learning method, and auxiliary tasks are constructed for self-supervised training. The specific process is as shown in Figure 2 , and the model structure diagram of the social robot detection algorithm based on user semantics and neighborhood information.

[0102] Step 4-1: The input graph G obtained in step 3 is subgraph sampled, and the input graph G is decomposed into subgraphs Each subgraph contains a center node and a fixed number of neighbor nodes, where the neighbor nodes represent the context information of the center node. The present application adopts a top-k closeness sampling method, which is based on graph closeness matrix sampling, where S(i, j) measures the closeness score between node v i and node v j . For the measurement index of node closeness, the present application adopts a graph closeness matrix based on PageRank algorithm[4], which is specifically:

[0103]

[0104] where A is the adjacency matrix of the input graph, is the result of column normalization of the adjacency matrix A, and α ∈ [0, 1], generally taking the value of 0.15. For any target node Based on the closeness matrix S, the neighbor node set can be defined as:

[0105]

[0106] where "A\B" represents the difference set of set A minus set B, "∩" represents the union set, and θ iis the minimum closeness score threshold, i.e., the closeness score of the kth neighbor node arranged from large to small. The center node v i is selected i The k neighbor nodes with the highest closeness score form a subgraph with v i as the target node. Then the input graph is sampled in turn times, and finally the subgraph set

[0107] Step 4-2: Perform embedding representation of network features on the nodes in the subgraph set obtained in step 4-1. The graph embedding method in Graph-Bert model is adopted in the present application, and the embedding vector of node v i in the subgraph after position encoding fusion can be expressed as The matrix can be constructed for the subgraph and input into the graph Transformer encoder to update the representation of the nodes through multi-layer iteration. The output z i of the final model is the learned node feature representation. The present application pre-trains the model by designing and fusing three different tasks:

[0108] Task one is the node attribute reconstruction task. For the target node v i in the subgraph , the new node feature representation z i is obtained after training by the Graph-Bert model, and the reconstructed original feature can be obtained after the fully connected layer. In order to ensure that the learned node feature representation can capture the original attribute information of the node, the loss function can be constructed as follows:

[0109]

[0110] where x i is the original feature vector of node v i .

[0111] Task two is the graph structure recovery task. In order to ensure that the new node feature representation can also capture the graph structure information, a new node similarity matrix can be constructed. The residual similarity is used to calculate the closeness score between nodes, i.e., By comparing with the true closeness matrix S obtained in step 4-1, the loss function can be constructed as follows:

[0112]

[0113] where,

[0114] Task three is a node classification task; in order to ensure that the learned node features can be better used for the downstream social robot detection task, the label related to the downstream social robot detection task is As a self-supervised signal to train, specifically after the node feature representation zi, a layer of softmax layer is connected, and the output is:

[0115]

[0116] Where, zi is the learned user representation vector, W f and b f are learnable parameters, is the predicted probability of each class. After the label of the user is one-hot encoded, the true value y=[y1, y2,..., y D ] is obtained, and the loss function is constructed by minimizing the cross entropy as follows:

[0117]

[0118] Where, D is the total number of label categories.

[0119] The present application is pre-trained by combining the above three self-supervised tasks, specifically constructing a joint loss function:

[0120]

[0121] Where λ1, λ2, λ3 are all learnable parameters, in order to find the appropriate weight, the present application uses DWA algorithm

[10] for optimization.

[0122] Step 4-3: the node feature representation output by the pre-trained model is clustered using the HDBSCAN algorithm, and previous research [7] shows that due to the automation and coordination of social robots, social robots tend to gather in the form of communities on the network. Therefore, the noise in the clustering result is marked as a natural user, and after excluding small clusters and noise interference, the remaining node clusters are marked as robot accounts, and the final identification result is obtained.

[0123] The present application uses a non-public data set as an experimental object, and evaluates the algorithm of the present application. The data set comes from the micro-blog data mining competition data set. In order to facilitate the evaluation of the data set model, 500 accounts are manually labeled, and the statistics of the data set are as follows Table 1:

[0124] Table 1

[0125]

[0126]

[0127] The machine configuration used in the experiment is as shown in Table 2:

[0128] Table 2

[0129] Operating system Window 10 Memory 32768 MB CPU model AMD Ryzen 7 5800X 8-Core 3.80 GHZ Number of CPUs 8 GPU acceleration Yes GPU model NVIDIA GeForce RTX 3060Ti

[0130] The number of training rounds is set to 100 rounds, and the standards used are precision, recall, and F1-score; the calculation formula is as follows:

[0131]

[0132]

[0133] Among them, TP, TN, FP, and FN represent the number of true cases, the number of true negative cases, the number of false positive cases, and the number of false negative cases, respectively.

[0134] In the experiment, the model of the application is compared with several previous methods, and in order to verify the necessity of the components of the model, ablation experiments are also conducted, including a version (dg) in which the generative adversarial network is deleted and a version (dt) in which the multi-task pre-training is deleted.

[0135] The final experimental results are as shown in Table 3:

[0136] Table 3

[0137]

[0138]

[0139] Compared with the latest supervised learning algorithms SATAR, Ha-bot, and Botometer, and the unsupervised algorithm RTbust, we found that AbertBot showed better social robot detection accuracy and higher F1 value, and the recall rate of AbertBot was close to that of Ha-bot. This indicates the effectiveness of AbertBot in the task of social robot detection. In addition, the AbertBot algorithm is significantly better than the social robot detection methods that simply use semantic information or attribute information, such as Botometer, etc. This shows that AbertBot more fully mines the potential information of social users. In addition, we set up an ablation experiment and found that after deleting the generative adversarial network and the multi-task training module, the performance of the model will decrease significantly, which proves the necessity of each component of the model.

[0140] References:

[0141] [1] Feng S, Wan H, Wang N, et al. Satar: A self-supervised approach to twitter account representation learning and its application in bot detection [C] / / Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021:3808-3817.

[0142] [2] Liu Y, Ott M, Goyal N, et al. Roberta: A robustly optimized bert pretraining approach [J]. arXiv preprint arXiv:1907.11692, 2019.

[0143] [3] Wu B, Liu L, Yang Y, et al. Using improved conditional generative adversarial networks to detect social bots on twitter [J]. IEEE Access, 2020, 8:36664-36680.

[0144] [4] Zhang J, Zhang H, Xia C, et al. Graph-bert: Only attention is needed for learning graph representations [J]. arXiv preprint arXiv:2001.05140, 2020.

[0145] [5] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [J]. Advances in neural information processing systems, 2017, 30.

[0146] [6] McInnes L, Healy J, Astels S. hdbscan: Hierarchical density based clustering [J]. J. Open Source Softw., 2017, 2(11): 205.

[0147] [7] Mazza M, Cresci S, Avvenuti M, et al. Rtbust: Exploiting temporal patterns for botnet detection on twitter [C] / / Proceedings of the 10th ACM conference on web science. 2019: 183-192.

[0148] [8] Feng S, Tan Z, Li R, et al. Heterogeneity-aware Twitter Bot Detection with Relational Graph Transformers [J]. arXiv preprint arXiv:2109.02927, 2021.

[0149] [9] Clayton Allen Davis, Onur Varol, Emilio Ferrara, Alessandro Flammini, and Filippo Menczer. 2016. BotOrNot: A System to Evaluate Social Bots. In ACM WWW Companion 2016 Companion Publication of the World Wide Web Conference on Web Companion 2016. ACM, 491-492. https: / / doi.org / 10.1145 / 2911956.2912002

[0150]

[10] Liu S, Johns E, Davison A J. End-to-end multi-task learning with attention [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 1871-1880.

Claims

1. A social robot detection algorithm based on user semantics, attributes and neighborhood information, characterized by: include: The social account dataset to be identified is preprocessed; the BERT model is used to encode the account text features to obtain a text feature vector; the user neighborhood features and attribute features are encoded and embedded; finally, a complete social user representation vector is constructed through joint encoding; among them, the conditional generative adversarial network is used for data augmentation to solve the sample imbalance problem in the original dataset: the user representation vector is density-peak clustered and noise and small user sets are removed, the cluster sequence is marked and used as the label of each node user, and the label user representation vector is input into the improved conditional adversarial generative network model; the trained samples are merged with the original data to obtain augmented data; the augmented data is used as the initial hidden vector of the node and input into an improved graph attention network, and parallel calculation is performed through subgraph sampling. The graph intimacy matrix based on the pagerank algorithm is used to calculate the node intimacy and select neighbor nodes; the model uses a multi-task learning method to construct auxiliary tasks for self-supervised training, and the HDBSCAN clustering is performed on the output of the final trained discriminant model to obtain the identification result of whether it is a social robot; the specific steps are as follows: Step 1: Preprocess the social account information to be identified. The social account information includes user text, attribute information, and neighborhood information. The text information comes from the account's recent tweets; the attribute information comes from the public social account basic information; the neighborhood information includes the interaction between the target account and other accounts, including forwarding, commenting, and following behaviors. The preprocessing process includes: (1) Text data preprocessing: dividing text into semantic units to facilitate downstream task processing; (2) Attribute data preprocessing: processing the missing values ​​and outliers that may exist in the original data to improve data quality and enhance analysis reliability; (3) Preprocessing of neighborhood data: Modeling the relationships between accounts to obtain a relationship graph network, and deleting or filling in abnormal nodes in the network to facilitate further analysis of user domain information; Step 2: Embed and aggregate the pre-processed data in step 1 to obtain a complete user vector representation; specifically, the following steps are performed: (1) Input the pre-processed text data into the pre-trained RoBERTa model to obtain the user semantic feature vector; (2) Encode the preprocessed attribute data to obtain the user semantic feature vector; (3) Encode the processed neighborhood data to obtain the user neighborhood feature vector; (4) Aggregate the user semantic feature vector, user attribute feature vector, and user neighborhood feature vector to obtain the final user representation vector; Step 3: The user representation vectors obtained in Step 2 are clustered by density peaks. After removing noise and small user clusters, they are labeled with cluster sequences and assigned as labels to each node user. Specifically, the labeled user representation vectors are input into a conditional generative adversarial network model. This model introduces the Wasserstein distance and uses gradient penalties to improve the convergence judgment criteria of the conditional generative adversarial network. After multiple rounds of training, the generated samples are merged with the original data to obtain augmented data to address the imbalanced distribution of the original data. Step 4: Construct an input graph for the augmented data obtained in step 3 and input it into the graph embedding model for training; the graph embedding model used is an optimization model based on the Graph-BERT model; during the pre-training process of the model, auxiliary signals related to the downstream social robot detection task are introduced, and multi-task learning is used for self-supervised training; HDBSCAN clustering is performed on the output of the final trained model, and the clustering results are used to determine whether it is a social robot.

2. The social robot detection algorithm according to claim 1, characterized in that The specific process of data preprocessing of the social account information to be authenticated in step 1 is as follows: Step 1-1: Preprocess the text information in the social account information to be identified, including: segmenting the text content, filtering stop words, removing all meaningless symbols, and deleting all missing lines, ultimately obtaining processed user text data; Step 1-2: Preprocess the attribute information in the social account information to be identified, including using one-hot encoding for some non-numerical features, filling in missing data, and deleting duplicate data, ultimately obtaining processed user attribute data; Steps 1-3: Preprocess the neighborhood information in the social account information to be identified, including: constructing a forwarding relationship network with users as nodes and forwarding relationships as edges; filling in missing user nodes, deleting duplicate and isolated nodes and self-loops in the network, and finally obtaining processed user neighborhood data.

3. The social robot detection algorithm according to claim 1, characterized in that In step 2, the pre-processed data is embedded and aggregated to obtain a complete user vector representation. The specific process is as follows: Step 2-1: For the pre-processed user text data, use the pre-trained RoBERTa model to encode it and obtain the user semantic feature vector, which is expressed as: Where D is the embedding dimension, h i is user text data; Step 2-2: For the pre-processed user attribute data, normalize it and connect it to a fully connected layer as the user's attribute feature vector, which is expressed as: Where D is the embedding dimension, p i It is user attribute data; Step 2-3: For the pre-processed user neighborhood data, in order to obtain the information of the global position of the node, the WL algorithm is used to embed it, which is expressed as: Where D is the embedding dimension, WL(v j ) is to use the WL algorithm to analyze the user domain data v j The vector obtained after encoding, i is the number of data entries; Step 2-4: Aggregate the user's semantic feature vector, attribute feature vector, and network feature vector to obtain a complete user representation vector:

4. The social robot detection algorithm according to claim 1, characterized in that In step 3, the obtained user representation vectors are clustered by density peaks. After removing noise and small user sets, they are marked with cluster sequences and assigned as labels to each node user. The specific process is as follows: Step 3-1: Perform density-based clustering on the user representation vectors obtained in Step 2. Different clusters obtained from the clustering are labeled as negative samples, and cluster noise is classified as positive samples. The clustering results have a sample imbalance problem, where positive samples far outnumber negative samples. To reduce the prior information on the proportion of samples learned by the model, after extracting the user data of the relatively small number of negative samples, a cluster sequence label is labeled and assigned to each node user in the cluster. The remaining nodes are uniformly labeled as a cluster sequence. Step 3-2: The user representation vector with cluster sequence labels obtained in step 3-1 is used as a minority class sample and input into a conditional adversarial generative network for training. The adversarial generative network consists of two parts: the generator G and the discriminator D. The loss function of the entire training process is: Among them, D is the discriminator, G is the generator, and p z(z) is the input noise distribution, p data(x) is the original data distribution; in order to solve the model collapse and gradient vanishing problems in traditional generative adversarial networks, the model introduces the Wasserstein distance and uses gradient penalty to improve the convergence judgment condition of the adversarial generative network. In order to meet the first-order Lipsitz continuity condition, the gradient length of the function at any position must not exceed a fixed value. The "arbitrary point" is constructed using linear interpolation: Among them, ε is a random number uniformly distributed between (0, 1), x r represents the real data, x g Represents generated data; generated by mixing samples Instead of representing "any point" to satisfy the gradient constraint; At this time, the loss functions of the generator G and the discriminator D are: Among them, p data is the distribution of the real samples, p g is the distribution learned by the generator G, Any point The distribution of is the gradient of the function; Finally, the new samples generated by training are fused with the original samples to obtain augmented data, and the input graph is constructed based on the neighborhood information. in, and ε represent the set of nodes and edges in the graph G, respectively, mapping x: and They are the projections of nodes on their features and labels, respectively. and Represent feature space and label space respectively. For node v i , its feature vector and label vector can be expressed as and The size of a graph can be determined by the number of its nodes To express.

5. The social robot detection algorithm according to claim 1, characterized in that The specific process of step 4 is as follows: Step 4-1: Perform subgraph sampling on the input graph G obtained in step 3 and decompose the input graph G into Unbounded subgraph Each subgraph Each node contains a central node and a fixed number of neighbor nodes, where the neighbor nodes represent the context information of the central node; the top-k intimacy sampling method is used, which is based on the graph intimacy matrix Sampling is performed, where the element S(i,j) of the graph intimacy matrix measures the node v i and node v j The intimacy score between nodes; and for the measurement index of node intimacy, the graph intimacy matrix S based on the PageRank algorithm is used, which is specifically expressed as: Where A is the adjacency matrix of the input graph, is the result of column normalization of the adjacency matrix A, α∈[0,1], for any target node Based on the intimacy matrix S, the set of neighbor nodes is defined as: Among them, "A\B" represents the difference between set A and set B, "∧" represents the union, θ i is the minimum intimacy score threshold, which is the intimacy score of the kth neighbor node arranged from large to small; by selecting the central node v i With the k neighbor nodes with the highest intimacy score Γ(v i ) together form an unbounded subgraph That is, it is expressed as v i A subgraph sampled for the target node, and then the input graph is sampled |v| times in sequence, and finally a subgraph set is obtained Step 4-2: For the subgraph set obtained in step 4-1 The nodes in the network are embedded to represent the network features, specifically using the graph embedding method in the Graph-Bert model, and after position encoding fusion, the subgraph is Midpoint v i The embedding vector is represented as Then for the subgraph Construct the matrix: And input it into the graph Transformer encoder, update the node representation through multiple layers of iteration, and the final model output z i That is the learned node feature representation; The model is pre-trained by designing and integrating three different tasks: Task 1 is the node attribute reconstruction task, for the subgraph The target node v in i , after training the Graph-Bert model, a new node feature representation z is obtained i , and then pass through the fully connected layer to obtain the reconstructed original features In order to ensure that the learned node feature representation can capture the original attribute information of the node, the loss function is constructed as follows: Among them, x i For node v i The original eigenvector of ; Task 2 is the graph structure recovery task. In order to ensure that the new node feature representation can also capture the graph structure information, a new node similarity matrix is ​​constructed for it. Cosine similarity is used to calculate the intimacy score between nodes, that is, And by comparing with the true value intimacy matrix S obtained in step 4-1, the loss function is constructed as follows: in, Task three is the node classification task. In order to ensure that the learned node features can be better used for the downstream social robot detection task, the labels related to the downstream social robot detection task are As a self-supervisory signal for training, specifically, in the node feature representation z i Then connect a softmax layer, whose output is: Among them, z i is the learned user representation vector, W f and b f is a learnable parameter, is the predicted probability of each category; Perform one-hot encoding to get the true value y=[y1,y2,…,y D ], the loss function is constructed to minimize the cross entropy as follows: Where D is the total number of label categories; Pre-training is performed by combining the three self-supervised tasks mentioned above, and the joint loss function is: Among them, λ1, λ2, and λ3 are all learnable parameters; Step 4-3: Represent the node features output by the pre-trained model Use the HDBSCAN algorithm for clustering and mark the noise in the clustering results as natural users; exclude clusters that are too small and the interference of noise, and mark the remaining suspicious node clusters as robot accounts, and finally obtain the identification result of whether it is a social robot.

Images