Graph and structure perception based adversarial social robot detection method

Abstract

The present application belongs to the technical field of data analysis, and specifically relates to a method for detecting an adversarial social robot based on a graph and structure perception. The method comprises the following steps: constructing a user relationship network by taking social platform users as nodes and the forwarding relationship between the users as edges; performing representation learning on different structures in the network by using a structure feature extractor; performing structure similarity calculation between nodes by using a structure self-attention mechanism; training user representation vectors by using a graph Transformer algorithm; and inputting the user representation vectors into a category generative adversarial network for training to obtain a discrimination result of whether the user is a social robot. The method has strong generalization and high robustness by using adversarial thinking and unsupervised training, and can be used for detecting social robots in a large social network platform, thereby providing commercial value for user management, advertisement placement, public opinion safety control and the like of the social platform.

Description

Technical Field

[0001] This invention belongs to the field of data analysis technology, specifically relating to a method for detecting adversarial social robots based on graph and structure perception. Background Technology

[0002] In recent years, with the rapid popularization of the internet, social media has become an ideal platform for people to exchange ideas and freely disseminate news. The massive demand for information exchange has led to the booming development of social robot technology. Social robots are essentially automated agents that mimic human behavior on social media, interacting with human users in an organized manner and influencing target audiences according to the intentions of human manipulators. This has also led to some social robots being used for malicious activities such as spreading false information and fraud, with the aim of creating bias, influencing public opinion, and even political decisions and behaviors. This poses a serious cyber threat to society and public safety. To prevent this phenomenon, social robot detection technology has emerged.

[0003] Despite over a decade of development in the field of social robot detection, no decisive technological breakthroughs have been achieved to address the proliferation of social robots. On one hand, the rapid advancements in deep learning and large-scale language models have led to the continuous evolution of social robots, making them increasingly adept at evading detection, while technological innovation has resulted in a lag in our understanding of them. On the other hand, some complex social robot behaviors are directly driven by humans, blurring the lines between human and social robot behavior. These issues directly hinder the development and application of social robot detection technology, prompting researchers to explore more novel methods to address these challenges.

[0004] Traditional social bot detection methods often focus on individual account analysis. Given a set of accounts, a detector is applied to each account and a binary label is assigned. This approach assumes that bots and regular users are separable. However, with the emergence of highly sophisticated, human-driven bots, their identities cannot be defined by simple binary labels. This has led to a new research trend: detecting bots across entire account groups rather than individual accounts. This is because social bots often act collaboratively, forming networks to amplify their influence. Therefore, the automated and collaborative traces left by these networks are valuable for tracking. In recent years, many researchers have begun to consider constructing social relationship networks for social users, using graph-based methods to discover suspicious account connections. This approach is known as graph-based social bot detection.

[0005] Consequently, the research focus in the field of social bot detection has shifted from individuals to groups. However, the problem of how to deal with the evolution of social bots remains unsolved. Most existing detection schemes are passive, first observing the existence of bot accounts, then collecting and analyzing data to design detection strategies. As social bots continue to evolve, previous detection schemes will become inapplicable, leading to a new round of tug-of-war. In recent years, researchers have begun to consider proactive detection schemes. Adversarial machine learning is a technique specifically designed to provide security for machine learning under malicious attacks. Through adversarial training, it promptly identifies weaknesses and vulnerabilities in social bot detection systems and, in turn, develops more powerful models. This approach is known as adversarial social bot detection.

[0006] The inventors discovered during their research on social robot detection that existing methods, whether graph-based or adversarial, are not comprehensive enough in acquiring the features of social robots. The models have poor generalization and robustness, making it difficult for existing methods to achieve the expected results when facing the evolution of social robots. Summary of the Invention

[0007] In view of the above, the purpose of this invention is to provide a graph and structure-aware adversarial social robot detection method with strong generalization and robustness.

[0008] The proposed adversarial social robot detection method based on graph and structure awareness has the following overall process: Figure 1 As shown, the specific steps are as follows:

[0009] (1) The fully connected neural network and the pre-trained RoBERTa model [1] were used to preprocess the text and attribute information of social users to obtain the attribute feature representation and semantic feature representation of users; and a user relationship graph was constructed based on the user's neighborhood information.

[0010] (2) The user's attribute feature representation and semantic feature representation are fused together and then used as the initial vector representation of the node in the graph; the user relationship graph is input into the local structural feature extractor and the remote structural feature extractor respectively, and used as the sub-network to extract different network structure representations; the extracted network representations are pooled and fused, and then input into the graph Transformer network [2] as the node vector representation of the sub-network to update the user;

[0011] (3) Density-based clustering is performed on the updated user representation to obtain the hidden labels of the users, and then input into a conditional generative adversarial network CGAN[3] as a sub-network to generate artificial (synthetic) robot samples; the synthesized robot samples are fused with the original data and used as the input of the category generative adversarial network CatGAN[4] to conduct mutual adversarial training between the generator network and the discriminator network. Finally, the generator network outputs the generated robot adversarial samples, and the discriminator network outputs the social robot identification results.

[0012] In this invention, step 1 specifically involves the following process:

[0013] Step 1-1: The preprocessing of user attribute information includes: for numerical attribute features, after normalization, inputting them into a fully connected neural network to obtain the fully connected layer representation of user numerical attribute features; for categorical attribute features, after one-hot encoding, inputting them into a fully connected neural network, and using the leaky-relu activation function to transform them to obtain the fully connected layer representation of user categorical attribute features.

[0014] Step 1-2: The user text information is preprocessed, including: segmenting the user text information into words, correcting spelling errors, filtering stop words, and standardizing it. Then, it is encoded using a pre-trained RoBERTa model and input into a fully connected neural network to obtain a fully connected layer representation of the user's semantic features.

[0015] Steps 1-3: The step of constructing a user relationship graph based on the user's neighborhood information involves using social users as nodes and forwarding relationships between users as edges to construct a social relationship graph; concatenating the encoded user semantic features and user attribute features as a comprehensive feature representation of the user, and using a fully connected layer for transformation to derive the initial representation of the node features in the graph.

[0016] In this invention, step 2 specifically involves the following process:

[0017] Step 2-1: Take the user relationship graph constructed in Step 1 as input, first pass it through the structural feature extraction layer, and then use the local network builder and the remote network builder to extract network features. After pooling the vector representations output by the two, they are concatenated to obtain the updated node vector representation.

[0018] Step 2-2: Input the updated node vector representation into the graph Transformer layer for training, and the output is the final user vector representation.

[0019] In this invention, step 3 specifically involves the following process:

[0020] Step 3-1: Perform density-based clustering on the user vector representations obtained in Step 2 to obtain the latent labels of the users;

[0021] Step 3-2: Input the user vector representation with hidden labels into a conditional generative adversarial network as a sub-network to generate artificial (synthetic) robot data;

[0022] Step 3-3: Fuse the synthetic robot data with the original data and use it as input to the final class generative adversarial network CatGAN. In the class generative adversarial network, the generator network and the discriminator network train against each other. The discriminator network uses the information extracted from the entire user relationship graph by the fusion graph representation learning network, as well as the encoded user attribute information and semantic information, to make a predefined classification result for the input user. The output of the discriminator is the category of the input user, and finally, the result of whether it is a robot is obtained.

[0023] This invention proposes a structure-aware network structural feature extractor framework that can capture suspicious coordination behaviors of node users in the network. It updates the node representations of learned users using an improved graph Transformer network, quantifying the feature differences between social bots and ordinary users for social bot detection. Furthermore, the proposed unsupervised social bot detection framework based on class-based generative adversarial networks can achieve multi-class social bot detection tasks under unsupervised conditions, exhibiting strong generalization and robustness. Attached Figure Description

[0024] Figure 1 This is an overall flowchart of the adversarial social robot detection method based on graph and structure perception of the present invention.

[0025] Figure 2 This is a model structure diagram of the adversarial social robot detection method based on graph and structure awareness of the present invention.

[0026] Figure 3 This is a framework diagram of the LSE results.

[0027] Figure 4 This is a framework diagram of the results of RSE. Detailed Implementation

[0028] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0029] The proposed adversarial social robot detection method based on graph and structure awareness has the following overall process: Figure 1 As shown.

[0030] Step 1: Preprocess the text and attribute information of the social users to be identified, and construct a user relationship graph based on the user's neighborhood information: Where:

[0031] Step 1-1: Preprocess user attribute information. Specifically, for numerical features in user attributes, use numerical features that can be obtained from social platform APIs without feature engineering, and obtain user attribute numerical features after z-score normalization. The fully connected layer representation; for categorical features in user attributes, one-hot encoding is used followed by a fully connected layer and a leaky-ReLU activation function to derive the user categorical attribute features.

[0032] Step 1-2: Preprocess user semantic information, specifically: use the pre-trained RoBERTa model [1] to encode user tweets into vector form; RoBERTa is a pre-trained bidirectional language representation model based on the Transformer structure, which has achieved state-of-the-art performance in multiple NLP tasks; by averaging all the user's tweet representations, the user tweet embedding vector is obtained.

[0033]

[0034] in, This represents all of the user's tweets, totaling L words; D s This is the embedding dimension of RoBERTa, set to 128 dimensions; embedding user tweets into vectors. User semantic features are obtained through a fully connected layer:

[0035]

[0036] Among them, W B and b B These are learnable parameters; φ is the activation function, which is set to leak-relu here; D is the embedding dimension of the final user feature vector.

[0037] Steps 1-3: Concatenate the encoded user semantic features and user attribute features to obtain the user's comprehensive feature vector.

[0038]

[0039] Where D is the embedding dimension of the user feature vector, which is set to 128 in this invention;

[0040] Treating social users as nodes in the graph, and constructing an undirected network with user forwarding relationships as edges; for user u i eigenvector x i A fully connected layer is used for transformation to derive an initial representation of the node features in the graph:

[0041] xi =σ(W I ·x i +b I (4)

[0042] Among them, W I and b I These are all learnable parameters, where σ represents the activation function. This invention uses leaky-relu.

[0043] Step 2: Input the user relationship graph obtained in Step 1 into two different structural feature extractors as the network structure representation of the sub-network extracting nodes. After pooling the output representations of the two, fuse them and input them into a graph Transformer network as the feature representation of the sub-network updating node users.

[0044] Step 2-1: Input the user relationship graph obtained in Step 1 into two different structural feature extractors as the network structure representation of the sub-network extracted nodes. The structural feature extractors are as follows:

[0045] (1) Local Structure Extractor (LSE)

[0046] LSE first samples the input user relationship graph G; specifically, it performs k-hop subgraph sampling on each node in the input graph. If it directly performs complete neighbor sampling, the scale-free nature of the social relationship graph will lead to neighbor explosion. Therefore, this invention will adopt the neighbor sampling method in GraphSAGE[5], and perform multiple samplings of the k-hop neighbors of each node with replacement, that is, for node v i have:

[0047]

[0048] in, Indicates the relationship between node u i The k-hop subgraph obtained by neighbor sampling, where N represents the number of nodes sampled and k represents the number of hops sampled;

[0049] The sampled subgraphs are input into a graph neural network for feature extraction, thus obtaining the Local Structure Feature Extractor (LSE):

[0050]

[0051] in, For the sampled subgraph, v represents the neighboring nodes; the representations learned from all subgraphs are aggregated as the node u. i Learned local structural representation

[0052] The overall framework of LSE is as follows: Figure 3 As shown.

[0053] (2) Remote Structure Extractor (RSE)

[0054] RSE aims to learn the remote structural properties of nodes in the graph. Therefore, it performs edgeless subgraph sampling on the input user relationship graph. Specifically, it calculates the similarity between pairs of nodes in the graph by selecting a certain metric function, thereby sampling each node to obtain itself and a fixed number of neighboring nodes. The distance function can be any node distance metric function. Considering the need to preserve the network structure after sampling as much as possible, a graph affinity matrix based on the Pagerank algorithm [6] is adopted, which is as follows:

[0055]

[0056] Where A is the adjacency matrix of the input graph. This is the result of normalizing column A of the adjacency matrix, where α∈[0,1] and the default value is 0.15; for any target node Based on the proximity matrix S, the set of neighbor nodes is defined as follows:

[0057]

[0058] in, Let θ represent the set of nodes in a complete graph G, where \ represents the difference set, ∧ represents the union set, and θ represents the union set. i The minimum intimacy score threshold is defined and used to obtain the top k neighbor nodes with the highest intimacy.

[0059] By selecting the central node u i Γ(v) of the k neighbor nodes with the highest intimacy scores i Together they form node u i Unbounded sampling subgraph By sampling each node of the input graph in sequence, a set of subgraphs is finally obtained.

[0060] The sampled subgraph is input into a multi-layer Transformers network, iteratively updating the node representations. Unlike traditional Transformers applied to natural language processing, which aim to learn vector representations of all words, this invention aims to obtain the overall structural representation of the subgraph. Therefore, the outputs of all nodes in the subgraph are pooled as node u. i Learned long-range structural representation

[0061]

[0062] Wherein, PE(·) represents absolute position encoding, and the present invention adopts the WL role embedding algorithm [7];

[0063] The overall framework of RSE is as follows: Figure 4 As shown.

[0064] Step 2-2: Input the node network representations extracted by LSE and RSE into an improved graph Transformer network as a subnetwork to update the user node representations, specifically:

[0065] (1) Original Transformer Network

[0066] Unlike traditional GNNs, Transformers abandon explicit structures and instead infer relationships between nodes through an attention mechanism. Transformers consist of two modules: a multi-head self-attention module and a feedforward neural network. In the multi-head self-attention module, the input node feature matrix X is first linearly mapped to query (Q), key (K), and value (V), where Q = XW. Q K = XW K V = XW V Therefore, the formula for calculating self-attention can be derived:

[0067]

[0068] Where, d out Q is the dimension, softmax is the activation function, which can be used for multi-classification problems; multi-head self-attention is usually used to enhance multi-dimensional learning; the output of the self-attention module is connected to a module composed of residual connection and feed-forward network (FFN) [6], which together form a Transformer layer, and the changes in node features are shown below:

[0069] X′=X+Attn(X), (11)

[0070] X″=FFN(X′), (12) where X′ represents the node feature matrix X after passing through the multi-head self-attention module, and X″ represents the node feature matrix X after passing through the complete Transformer layer; multiple Transformer layers are stacked together to obtain the complete network, which is used to learn the node representation in the graph.

[0071] (2) Improved Graph Transformer Network

[0072] This invention will use a kernel smoothing approach to transform the self-attention computation form in the original Transformer network:

[0073]

[0074] Where, f(x) v ) = W V x is a linear function, κ exp W Q and W K Defined exponential kernel function:

[0075]

[0076] Where <·,·> is dot product multiplication. Based on this, it is transformed into structure-aware attention that captures structural feature similarity [8]:

[0077]

[0078] Among them, S G (u) represents a subgraph in graph G centered at node u, κ graph It can be any kernel function capable of comparing a pair of subgraphs. The new self-attention function considers not only the attribute similarity between nodes but also the structural similarity between nodes. κ graph (S G (u),S G (v)) can be transformed into:

[0079]

[0080] in, It can be a subgraph extractor network.

[0081] Based on this, the present invention proposes a multi-head structure perception self-attention mechanism:

[0082]

[0083] Among them, W O The transformation matrix is ​​linear, and different heads represent different structural representation extractors. In this invention, it is set to 2, which are the heads that focus on local structural features. lse and head that focuses on remote structural features rse .

[0084] (3) Update node representation

[0085] After passing through an L-layer (L=2) graph Transformer network, the node feature set X is obtained at this point. (L)Multilayer perceptron (MLP)[9] is used to transform the final user representation for user u. i have:

[0086]

[0087] Where W2 and b2 are learnable parameters, φ is the activation function, and r i That is, user u i The final vector representation.

[0088] Step 3: The user feature representations obtained in Step 2 are clustered and input into a Conditional Generative Adversarial Network (CGN) to generate artificial (synthetic) robot data as a sub-network. The synthetic robot data is then fused with the user feature representation data obtained in Step 2 and used as input to the final category-based CGN to obtain the final identification result of whether it is a social robot. The model structure of the graph- and structure-aware adversarial social robot detection algorithm is as follows: Figure 2 As shown.

[0089] Step 3-1: Perform minority class (robot sample) data augmentation on the user vector representation obtained in Step 2 to solve the class imbalance problem. First, use Gaussian kernel density peak clustering (GKDPCA) to cluster the original data and then generate pseudo labels.

[0090] Step 3-2: Input the labeled data obtained in Step 3-1 into a Conditional Generative Adversarial Network (CGN) as a subnetwork to expand the robot samples. The algorithm flow is as follows:

[0091] (1) Improved Conditional Generative Adversarial Network (CGAN)

[0092] The original CGAN suffers from problems such as low sample quality, difficulty in convergence, and mode collapse. The improved CGAN uses Wasserstein distance and gradient penalty to improve the convergence criteria of the conditional generative adversarial network. The Wasserstein distance is defined as:

[0093]

[0094] Where p data It is the true sample distribution, p g The distribution learned by the generator G, ∏(p data ,p g ) is p data and p gThe set of all joint distributions is given. For each possible joint distribution, the distance ||xy|| between the true sample x and the sample y generated by the generator G is calculated, thus allowing the computation of the joint distribution E. (x,y)～y The expected value of the sample distance under [‖xy‖]. The lower bound of this expected value in all possible joint distributions is defined as the Wasserstein distance.

[0095] In actual training, to satisfy the first-order Lipsitz continuity condition, the gradient length of the function at any position does not exceed a fixed value. Therefore, a linear interpolation method is used to construct the "arbitrary point":

[0096]

[0097] Where ε is a random number that follows a uniform distribution between (0, 1), and x r Represents real data, x g This represents the generated data. It was generated through a pooling of samples. This replaces the representation of "arbitrary point" to satisfy the gradient constraint. At this point, the loss functions of the generator G and the discriminator D are respectively:

[0098]

[0099]

[0100] Where, p data It is the distribution of the real samples, p g It is the distribution learned by the generator G. It is an "arbitrary" point The distribution Let p be the gradient of the function; where p data It is the distribution of the real samples, p g The distribution is learned by the generator G, and E represents the expected value. This indicates the loss that the discriminator uses to determine the generated samples. This indicates the loss in the discriminator's judgment of the true data. It is an "arbitrary" point The distribution λ is the gradient of the function, and λ is the weight factor, with a default value of 10, which can be adjusted according to the actual training process.

[0101] (2) CGAN oversampling algorithm

[0102] The user latent labels obtained in step 3-1 are used as auxiliary information and input into the improved conditional generative adversarial network (CGAN) for iterative training. At this time, the generator G generates synthetic robot samples. When the generator G generates synthetic robot samples that are not distinguished by the discriminator D, the synthetic robot samples are mixed with real samples as augmented data. The above steps are repeated until the sample classes in the dataset are balanced.

[0103] Step 3-3: The augmented dataset T obtained in Step 3-2... a The input is fed into a Category Generative Adversarial Network (CatGAN) for training. CatGAN aims to learn a discriminative classifier from the dataset in an unsupervised manner, which divides the data into classes based on the number of prior choices, rather than simply two categories: true or false. CatGAN improves upon the GAN framework by defining a new two-player game: instead of requiring the classifier D to predict the probability that a sample x belongs to the true dataset, it forces D to classify all data points into k classes, while the class assignment of the samples generated by G is uncertain. On the other hand, it requires G to generate samples that belong to exactly one of the k classes, rather than generating samples that belong to the dataset.

[0104] Therefore, this invention uses empirical entropy as the loss function for training, as shown in the following formula:

[0105]

[0106] in, It is the discriminator loss function. Here, H[·] represents the generator loss function, x is the real sample, z is random noise, and c is the class label. It is the distribution on the dataset; H represents the entropy of the class distribution on the real samples. G [p(c|D)] represents the entropy of the class distribution over the generated samples. This represents the estimated entropy of the predicted class distribution on the real samples. This represents the estimated entropy of the predicted class distribution on the generated samples.

[0107] The output of the final discriminator network is the result of determining whether the input user is a social bot.

[0108] This invention uses a dataset from a domestic social media platform as its experimental subject and evaluates it using the algorithm of this invention. The dataset is a labeled social user dataset provided by the social media platform, involving 539,967 account information entries. The labeling type is binary label (bot / ordinary user), where accounts labeled "bot" are all accounts that have been banned by the social media platform. Therefore, the labels of WB-21 have high credibility and can be regarded as basic facts. This invention first randomly samples the preprocessed WB-21 dataset, obtaining 10,000 sampled samples. After querying the labels, it was found that there are 7,214 human users and 2,786 social bots. The labels of these 2,786 social bot accounts are masked, and they are re-labeled (bot / ordinary user) through expert review, resulting in 1,218 samples labeled "bot" and 1,568 samples labeled "ordinary user". These are referred to as pseudo-label samples. By comparing the pseudo-labels of the samples with the basic facts, all "ordinary users" in the pseudo-label samples are regarded as a new type of evolved bot. The final dataset was divided into three categories: ordinary users (labeled 0), ordinary social bots (labeled 1), and the remaining category, novel bots (labeled 2). The dataset statistics are shown in Table 1.

[0109] Table 1

[0110]

[0111] The machine configuration used in the experiment is shown in Table 2:

[0112] Table 2

[0113] Memory 32,768MB CPU model AMD Ryzen 7 5800X 8-Core 3.80GHZ Number of CPUs 8 GPU acceleration yes GPU model NVIDIA GeForce RTX 3080 Ti .

[0114] This invention follows state-of-the-art hyperparameter settings, with both the generator and discriminator of CatGAN consisting of feedforward neural networks. A noise vector of dimension 128 is used, and the embedding layer size is set to 3 to match the data categories. The loss functions of both networks are constructed from empirical entropy, and both networks are trained in mini-batch using the Adam

[10] optimizer based on SGD with a learning rate of 0.0002 and 200 training epochs.

[0115] The evaluation indicators mainly include the following aspects:

[0116] (1) Accuracy: Accuracy refers to the ratio of the number of samples correctly predicted by the model to the total number of samples. Accuracy can be used to evaluate the overall performance of the model; however, in social robot detection tasks, accuracy does not accurately reflect the model's performance because, for imbalanced data, the class with the largest proportion often becomes the most significant factor affecting accuracy. Accuracy is defined as:

[0117]

[0118] Wherein, TP represents a predicted positive sample that is indeed a positive sample; TN represents a predicted negative sample that is indeed a negative sample; FP represents a predicted positive sample that is unfortunately a false positive sample; and FN represents a predicted negative sample that is unfortunately a false negative sample.

[0119] (2) Precision: Precision refers to the proportion of truly positive samples out of all samples detected as positive. Precision rate is a statistical measure of a subset of samples, focusing on the data that the classifier classifies as positive. Its definition is:

[0120]

[0121] Where TP represents the number of correctly classified positive samples, and TP+FP represents the number of samples predicted as positive samples.

[0122] (3) Recall: Recall is the proportion of correctly classified positive samples to the total number of true positive samples. Recall is also a statistic for a subset of samples, focusing on the true positive samples. Its definition is:

[0123]

[0124] Where TP represents the number of correctly classified positive samples, and TP+FP represents the number of samples that are actually positive samples.

[0125] (4) F1 score: The F1 score is the harmonic mean of precision and recall, aiming to achieve a balance between the two. It is defined as follows:

[0126]

[0127] The table below reports the results for the binary classification task, showing a performance comparison with several state-of-the-art unsupervised methods and two supervised methods:

[0128] DNA-Sequence

[11] applies an unsupervised method based on the longest common substring, which can compare different sequences (users).

[0129] Ahmed et al.

[12] proposed a Markov clustering-based method to identify spam activity on Facebook and Twitter networks.

[0130] MulBot

[13] proposed an unsupervised social robot detector based on multivariate time series, which mainly uses clustering to complete the recognition task.

[0131] The final experimental results are shown in Table 3.

[0132] Table 3

[0133]

[0134]

[0135] As can be seen from the table, the algorithm of this invention achieves excellent performance across all metrics, outperforming all other unsupervised and supervised methods. This demonstrates that the user representations learned by the algorithm of this invention have a significant impact on social robot detection tasks.

[0136] For the multi-classification task, the results are shown in Table 4 below.

[0137] Table 4

[0138]

[0139] As can be seen, this invention exhibits high precision and recall for normal users, indicating that the algorithm is more reliable in real-world applications and makes more careful judgments regarding normal users. Meanwhile, the model's precision for advanced robots is only 0.76, reflecting the complexity of the features of such advanced robots. For ordinary robots, the model performs well, with an F1 score of 0.82 and a recall of 0.81.

[0140] References:

[0141] [1]Liu Y,Ott M,Goyal N,et al.Roberta:Arobustly optimized bertpretraining approach[J].arXiv preprint arXiv:1907.11692,2019.

[0142] [2]Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need[J].Advances in neural information processing systems,2017,30.

[0143] [3]Wu B, Liu L, Yang Y, et al. Using improved conditional generative adversarial networks to detect social bots on Twitter[J]. IEEE Access, 2020, 8: 36664-36680.

[0144] [4]Liu Z,Wang J,Liang Z.Catgan:Category-aware generative adversarialnetworks with hierarchical evolutionary learning for category text generation[C] / / Proceedings of the AAAI Conference on Artificial Intelligence.2020,34(05):8425-8432.

[0145] [5]Hamilton W,Ying Z,Leskovec J.Inductive representation learning onlarge graphs[J].Advances in neural information processing systems,2017,30.

[0146] [6]Zhang J,Zhang H,Xia C,et al.Graph-bert:Only attention is neededfor learning graph representations[J].arXiv preprint arXiv:2001.05140,2020.

[0147] [7]Morris C,Ritzert M,Fey M,et al.Weisfeiler and leman go neural:Higher-order graph neural networks[C] / / Proceedings of the AAAI conference onartificial intelligence.2019,33(01):4602-4609.

[0148] [8]Chen D,O’Bray L,Borgwardt K.Structure-aware transformer for graphrepresentation learning[C] / / International Conference on MachineLearning.PMLR,2022:3469-3489.

[0149] [9]Tolstikhin I O,Houlsby N,Kolesnikov A,et al.Mlp-mixer:An all-mlparchitecture for vision[J].Advances in neural information processing systems,2021,34:24261-24272.

[0150]

[10] Diederik P.Kingma and Jimmy Lei Ba.Adam:Amethod for stochasticoptimization.Proc.of ICLR,2015.

[0151]

[11] Cresci S,Di Pietro R,Petrocchi M,et al.Social fingerprinting:detection of spambot groups through DNA-inspired behavioral modeling[J].IEEETransactions on Dependable and Secure Computing,2017,15(4):561-576.

[0152]

[12] Ahmed F,Abulaish M.Ageneric statistical approach for spamdetection in online social networks[J].Computer Communications,2013,36(10-11):1120-1129.

[0153]

[13] Cai Z,Tan Z,Lei Z,et al.LMBot:Distilling Graph Knowledge intoLanguage Model for Graph-less Deployment in Twitter Bot Detection[J].arXivpreprint arXiv:2306.17408,2023。

Claims

1. A method for detecting adversarial social robots based on graph and structure awareness, characterized in that, The specific steps are as follows: (1) The fully connected neural network and the pre-trained RoBERTa model are used to preprocess the text and attribute information of social users to obtain the attribute feature representation and semantic feature representation of users; and user relationship graphs are constructed based on the neighborhood information of users; (2) The user's attribute feature representation and semantic feature representation are fused together and then used as the initial vector representation of the node in the graph; the user relationship graph is input into the local structural feature extractor and the remote structural feature extractor respectively, and used as the sub-network to extract different network structure representations; the extracted network representations are pooled and fused, and then input into the graph Transformer network, and used as the node vector representation of the sub-network to update the user. (3) Density-based clustering is performed on the updated user representation to obtain the latent labels of the users, and then input into a conditional generative adversarial network CGAN as a sub-network to generate artificial robot samples; the synthesized robot samples are fused with the original data and used as input to the category generative adversarial network CatGAN to conduct mutual adversarial training between the generator network and the discriminator network. Finally, the generator network outputs the generated robot adversarial samples, and the discriminator network outputs the social robot identification results. The Local Structural Feature Extractor (LSE) is specifically: First, analyze the input user relationship diagram. Sampling is performed; specifically, k-hop subgraph sampling is performed on each node in the input graph, using the neighbor sampling method in GraphSAGE, which samples the k-hop neighbors of each node multiple times with replacement. have: ，(5) in, Indicates a node The k-hop subgraph obtained by neighbor sampling This indicates the number of nodes sampled. Indicates the number of hops in the sampling; The sampled subgraphs are input into a graph neural network for feature extraction, thus obtaining the Local Structure Feature Extractor (LSE): ，(6) in, For sampling sub-images, Represent neighboring nodes; aggregate the representations learned from all subgraphs and use them as nodes. Learned local structural representation .

2. The detection method according to claim 1, characterized in that, The specific process for step 1 is as follows: Step 1-1: The preprocessing of user attribute information includes: for numerical attribute features, after normalization, inputting them into a fully connected neural network to obtain the fully connected layer representation of user numerical attribute features; for categorical attribute features, after one-hot encoding, inputting them into a fully connected neural network, and using the leaky-relu activation function to transform them to obtain the fully connected layer representation of user categorical attribute features. Step 1-2: The user text information is preprocessed, including: segmenting the user text information into words, correcting spelling errors, filtering stop words, and standardizing it. Then, it is encoded using a pre-trained RoBERTa model and input into a fully connected neural network to obtain a fully connected layer representation of the user's semantic features. Steps 1-3: The step of constructing a user relationship graph based on the user's neighborhood information involves using social users as nodes and forwarding relationships between users as edges to construct a social relationship graph; concatenating the encoded user semantic features and user attribute features as a comprehensive feature representation of the user, and using a fully connected layer for transformation to derive the initial representation of the node features in the graph.

3. The detection method according to claim 2, characterized in that, The specific process for step 2 is as follows: Step 2-1: Take the user relationship graph constructed in Step 1 as input, first pass it through the structural feature extraction layer, and then use the local structural feature extractor and the remote structural feature extractor to extract network features. After pooling the vector representations output by the two, they are concatenated to obtain the updated node vector representation. Step 2-2: Input the updated node vector representation into the graph Transformer layer for training, and the output is the final user vector representation.

4. The detection method according to claim 3, characterized in that, The specific process for step 3 is as follows: Step 3-1: Perform density-based clustering on the user vector representations obtained in Step 2 to obtain the latent labels of the users; Step 3-2: Input the user vector representation with hidden labels into a conditional generative adversarial network as a subnetwork to generate artificial robot data; Step 3-3: Fuse the synthetic robot data with the original data and use it as input to the final class generative adversarial network CatGAN; In class generative adversarial networks, the generator network and the discriminator network train against each other. The discriminator network integrates information extracted from the entire user relationship graph by the graph representation learning network, as well as encoded user attribute information and semantic information, to classify the input user into a predefined category. The output of the discriminator is the category of the input user, and finally, the result of whether it is a robot is obtained.

5. The detection method according to claim 4, characterized in that, In step 2-1, the remote structural feature extractor (RSE) specifically comprises: First, edgeless subgraph sampling is performed on the input user relationship graph. Specifically, a metric function is selected to calculate the similarity between pairs of nodes in the graph, thereby sampling each node to obtain a subset containing itself and a fixed number of neighboring nodes. The metric function is a distance function. To preserve the network structure after sampling as much as possible, a graph affinity matrix based on the PageRank algorithm is used, specifically: ， (7) in, It is the adjacency matrix of the input graph. It is an adjacency matrix The result of column normalization For any target node Based on the intimacy matrix Define the set of neighbor nodes as: ，(8) in, Representing a complete graph The set of nodes in " " represents the difference set," " represents the union, The defined minimum intimacy score threshold is used to obtain the top... The neighbor node with the highest affinity; By selecting the central node With the highest intimacy score neighboring nodes Together they form a node Unbounded sampling subgraph By sampling each node of the input graph in turn, a set of subgraphs is finally obtained. ; The sampled subgraph is input into a multi-layer Transformers network, and the node representations are iteratively updated. The outputs of all nodes in the subgraph are pooled and used as node representations. Learned long-range structural representation : ，(9) in, This represents absolute position encoding, specifically using the WL role embedding algorithm.

6. The detection method according to claim 5, characterized in that, Step 2-2 describes inputting the updated node vector representation into the improved graph Transformer network for training, and the output is the final user vector representation; specifically: (1) Original Transformer Network Transformers consists of two modules: a multi-head self-attention module and a feedforward neural network module. In the multi-head self-attention module, the input node feature matrix X is first linearly mapped to query (Q), key (K), and value (V), where Q = XW. Q K=XW K V=XW V Therefore, the formula for calculating self-attention is derived: ，(10) in, It is the dimension of Q. The activation function can be used for multi-class classification problems; multi-head self-attention is used to enhance multi-dimensional learning; the output of the self-attention module is connected to a module composed of residual connections and a feedforward neural network (FFN), which together form a Transformer layer, and the changes in node features are shown below: ， (11) ，(12) in, X represents the node feature matrix after passing through the multi-head self-attention module. X represents the node feature matrix after passing through a complete Transformer layer; multiple Transformer layers are stacked to obtain the complete network, which is used to learn the node representations in the graph; Atnn is used for attention calculation. (2) Improved Graph Transformer Network The self-attention computation in the original Transformer network is transformed using a kernel smoothing approach: ， (13) in, It is a linear function. It is by and Defined exponential kernel function: ， (14) in, It's a dot product multiplication; based on this, it's transformed into structure-aware attention that captures structural feature similarity: ，(15) in, Representation diagram China-Israel node The subgraph centered on the subgraph It is any kernel function that can compare a pair of subgraphs. The new self-attention function considers not only the attribute similarity between nodes, but also the structural similarity between nodes. Transformed into: ，(16) in, For a certain type of subgraph extractor network; Based on this, a multi-head structured perception self-attention mechanism is adopted: ， (17) Among them, different This represents different structural representation extractors, specifically set to 2, which focus on local structural features respectively. and focusing on remote structural features ; (3) Update node representation After passing through an L-layer graph Transformer network, the node feature set is obtained at this point. A multilayer perceptron (MLP) is used to transform the data into the final user representation. have: ， (18) in, and These are all learnable parameters. For activation function, i.e., user The final vector representation.

7. The detection method according to claim 6, characterized in that, Step 3-2 describes inputting the user vector representation with pseudo-labels into a conditional generative adversarial network (GAN) as a sub-network for generating synthetic robot data; the specific process is as follows. (1) Improved Conditional Generative Adversarial Network (CGAN) The improved conditional generative adversarial network (GAN) uses Wasserstein distance and gradient penalty to improve the convergence criteria. The Wasserstein distance is defined as: (19) in, It is the true sample distribution. It is the distribution learned by the generator G. yes and The set of all joint distributions, for each possible joint distribution, calculate the distance between the true sample x and the sample y generated by the generator G. Therefore, the joint distribution can be calculated. The expected value of the subsample distance; the lower bound of this expected value in all possible joint distributions is defined as the Wasserstein distance; To satisfy the first-order Lipsitz continuity condition, the gradient length of the function at any location does not exceed a constant value. A linear interpolation method is used to construct the "arbitrary point": ，(20) in, It is a random number that follows a uniform distribution between (0, 1). Represents real data. This refers to the generated data; data produced through pooling samples. To replace the representation of "arbitrary point" to satisfy the gradient constraint condition; At this point, the loss functions for the generator G and the discriminator D are respectively: ，(21) ，(22) in, It is the distribution of the real samples. It is the distribution learned by the generator G. Represents the mathematical expectation. This indicates the loss that the discriminator uses to determine the generated samples. This indicates the loss in the discriminator's judgment of the true data. It is an "arbitrary" point The distribution The gradient of the function. As a weighting factor; (2) Conditional Generative Adversarial Network Oversampling Algorithm The user latent labels obtained in step 3-1 are used as auxiliary information and input into the improved conditional generative adversarial network. The conditional generative adversarial network is trained iteratively. At this time, the generator G generates synthetic robot samples. When the generator G generates synthetic robot samples that are not distinguished by the discriminator D, the synthetic robot samples will be mixed with real samples as augmented data. The above steps are repeated until the sample categories of the dataset are balanced.

8. The detection method according to claim 7, characterized in that, In step 3-3, the augmented dataset obtained in step 3-2 is denoted as... The input is fed into the class generative adversarial network CatGAN for training, which aims to learn a discriminative classifier from the dataset in an unsupervised manner. This classifier divides the data into categories based on the number of prior choices; CatGAN improves upon the GAN framework by defining a new two-player game: on the one hand, it does not require the classifier to... To predict samples The probability of belonging to the real dataset is not the forced classifier. Divide all data points into Class, while the generator is uncertain. The class assignment of the generated samples; on the other hand, it requires... The generation exactly belongs to Instead of generating samples belonging to the dataset, it generates samples from one of the classes. Therefore, empirical entropy is used as the loss function for training CatGAN, and the specific formula is as follows: ， (23) in, It is the discriminator loss function. It is the generator loss function. Let x represent the empirical entropy, and x be a real sample. It is random noise. It is a class tag. It is the distribution on the dataset; The entropy represents the class distribution on the real samples. The entropy represents the class distribution over the generated samples. This represents the estimated entropy of the predicted class distribution on the real samples. This represents the estimated entropy of the predicted class distribution on the generated samples; The output of the final discriminator network is the result of determining whether the input user is a social bot.

Images