A Social Robot Detection System and Method Based on Graph Neural Networks and Cutmix

Abstract

This invention belongs to the field of social network technology, and specifically relates to a social bot detection system and method based on graph neural networks and Cutmix. The system includes a node shuffling module, a node cutting and mixing module, and an attribute connection module. The node shuffling module shuffles the account nodes and corresponding edges in the original social network graph, obtaining its isomorphic graph. The node cutting and mixing module sequentially mixes the node features and labels of the original social network graph and its isomorphic graph. The attribute connection module inputs the mixed node features into a graph convolutional layer, which generates features based on local aggregation. These features are then combined with the original features of the two graphs to generate new features. Finally, the generated new features are input into a classifier for social bot detection. This invention has high accuracy in social bot detection, and is particularly suitable for situations with a limited number of labeled nodes.

Description

Technical Field

[0001] This invention belongs to the field of social network technology, and specifically relates to a social robot detection system and method based on graph neural networks and Cutmix. Background Technology

[0002] Currently, social bot detection technologies can be categorized into three types: feature-based detection methods, graph-based detection methods, and deep learning-based detection methods. Feature-based detection methods have been the dominant approach since the development of social bot detection. This method involves extracting and designing features from user metadata, such as Twitter content, user data, and features like timeliness, language, and sentiment extracted from friends and neighbors, and then using traditional classifiers for social bot detection. However, social bots can modify their registration information based on features designed for detection, thus avoiding the pitfalls of feature-based detection methods.

[0003] Graph-based detection methods utilize the graph structure of social networks to predict the label for each remaining node. These techniques do not require large amounts of language-dependent text and media data. Based on their core algorithms, they can be broadly categorized into two types: random walk methods and recurrent belief propagation (LBP) methods. Both methods start with a set of labeled nodes and predict unknown labels through semi-supervised learning. Effectively utilizing internal data on social platforms can significantly improve their detection capabilities.

[0004] Deep learning-based detection methods primarily rely on graph representation learning or GNN models, while leveraging user attribute and structural information to improve detection performance. TrustGCN utilizes the "friend request-response" relationship to form a social graph and combines social graph-based defense concepts with GNNs to enhance robustness against adversarial attacks. Bot2vec is an improved Node2vec-based social bot detection algorithm. It combines community detection algorithms and graph representation learning algorithms for social bot detection. The proposed BotRGCN model applies graph convolutional networks to a Twitter social graph with multimodal user semantics, attributes, and neighborhood information. SATAR is a self-supervised demonstration learning framework for Twitter users, utilizing user tweets, metadata, and community information.

[0005] Recently, data augmentation has proven highly effective in convolutional neural networks; however, research on data augmentation in graph neural networks (GNNs) remains limited. For node classification tasks, Rong et al. proposed a data augmentation technique called DropEdge, the core of which involves randomly removing a certain number of edges from the input graph at each training stage. Wang et al. proposed a two-stage mixing method for graphs (MixupForGraph), first performing a feedforward similar to existing GNNs with no initial mixing, and then performing bi-branch mixed graph convolution on paired nodes at each layer in the second stage. Furthermore, Han et al. proposed a data augmentation technique called G-Mixup for graph classification, which interpolates graphs across different categories. Summary of the Invention

[0006] The purpose of this invention is to solve the problems in the prior art and propose a social robot detection system and method based on graph neural networks and Cutmix, which has a high accuracy in social robot detection and is particularly suitable for situations with fewer labeled nodes.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] This invention provides a social robot detection system based on graph neural networks and Cutmix, comprising:

[0009] The node shuffling module shuffles the account nodes and corresponding edges in the original social network graph, and obtains its isomorphic graph based on the original social network graph.

[0010] The node cutting and blending module sequentially blends the node features and labels of the original social network graph and its isomorphic graph.

[0011] The attribute connection module inputs the mixed node features into the graph convolutional layer. The graph convolutional layer generates features based on local aggregation. These features are then combined with the original features of the two graphs to generate new features. Finally, the generated new features are input into the classifier for social robot detection.

[0012] According to the social robot detection system based on graph neural networks and Cutmix of the present invention, further, in the original social network graph... Based on this, we obtain its isomorphism graph. include:

[0013] In the original social network graph Top pair of nodes Shuffle the cards to obtain the shuffled nodes. According to the node Reorder In the nodes, λ represents the proportion of nodes shuffled, where λ∈[0,1].

[0014] According to the social robot detection system based on graph neural networks and Cutmix of the present invention, the isomorphic graph is further defined as follows: given two graphs and Assume there exists a bijective m: Makes all Both have uv∈E A Equivalent to m(u)m(v)∈E B Then it is called and They are isomorphic.

[0015] According to the social robot detection system based on graph neural networks and Cutmix of the present invention, further, social network graphs are used. and social network graphs The i-th account node in the construct node pair (h) A,i ,h B,i ), where 1≤i≤N, and N represents the number of nodes in the graph; the cutting and mixing of node pairs is represented as:

[0016]

[0017] Where M∈{0,1} H This is a binary mask representing the positions to be deleted and filled from two vectors, and and

[0018] According to the social robot detection system based on graph neural networks and Cutmix of the present invention, further, The output is obtained after graph convolution layer. The output of the convolutional layer and the features obtained by passing the original features through the linear layer are shown in the figure. and Adding them together gives and The expression is as follows:

[0019]

[0020]

[0021] After that, the result and The nodes are sent to the node cutting and blending module of the subsequent layer; among them, It is a social network graph Adjacency matrix, It is a social network graph The adjacency matrix, AGGREGAT, represents the aggregation function that aggregates the hidden representations of neighboring nodes.

[0022] According to the social robot detection system based on graph neural networks and Cutmix of the present invention, the corresponding labels should be mixed when calculating the loss function:

[0023]

[0024] in, Indicates the mixed label, y A express The tag, y B express The label is α∈[0,1], where α represents the hyperparameter controlling the ratio.

[0025] This invention also provides a social robot detection method based on graph neural networks and Cutmix, comprising the following steps:

[0026] By shuffling the account nodes and their corresponding edges in the original social network graph, we obtain its isomorphic graph based on the original social network graph.

[0027] The node features and labels of the original social network graph and its isomorphic graph are mixed sequentially;

[0028] The mixed node features are input into the graph convolutional layer, which generates features based on local aggregation. These features are then combined with the original features of the two graphs to produce new features.

[0029] Finally, the generated new features are input into the classifier for social robot detection.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] This invention's social robot detection system based on graph neural networks and Cutmix uses a node cutmix module to sequentially mix node features and labels from the original social network graph and its isomorphic graph. The mixed features are then input into a graph convolutional layer, where convolution generates features based on local aggregation. These features are then combined with the original features from both graphs to produce a new feature layer. The Cutmix module uses region-level cut and paste mixing techniques, forcing the graph neural network to focus more on the global contextual information of the image rather than just local information, maintaining information continuity compared to traditional mixup. This invention achieves high accuracy in social robot detection, particularly suitable for scenarios with few labeled nodes. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram illustrating the working principle of the social robot detection system based on graph neural networks and Cutmix according to an embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram illustrating the working principle of the node cutting and mixing module according to an embodiment of the present invention.

[0035] Figure 3 This is an accuracy graph of using different training set sizes with follower relationship graphs in an embodiment of the present invention;

[0036] Figure 4 This is an accuracy graph of using different training set sizes of the friend user relationship graph in an embodiment of the present invention;

[0037] Figure 5 This is an accuracy graph of G-CutMix under different α values ​​according to an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] First, let's introduce neural networks, data augmentation techniques, and social network graph structures.

[0040] (1) Graph Neural Network

[0041] GNNs are primarily used to process non-Euclidean graph data. They generate node-level feature representations through message passing. Specifically, the feature representation of a node at layer l+1 is obtained by aggregating l-layer 1-hop neighborhoods:

[0042]

[0043] in, is the adjacency matrix of the social network graph, AGGREGAT is the aggregation function that aggregates the hidden representations of neighboring nodes, σ(·) is the non-linear activation function, and W is the trainable weight matrix.

[0044] (2) Data Augmentation

[0045] Data augmentation plays a crucial role in neural network training, manipulating input data to significantly improve performance. For example, in image classification, data augmentation techniques such as horizontal flipping and random erasure have proven effective. Mixup is an effective data augmentation method for image classification. Recently, several mixture-based methods have been proposed to enhance model performance for graph data. G-Mixup is a data augmentation technique for graph classification that generates a new graph by using a linear combination of two graphs. MixupForGraph combines two graphs in a way that preserves the graph structure. GraphMix combines fully connected networks with graph neural networks and leverages parameter sharing and interpolation-based regularization for better performance.

[0046] Cutmix is ​​a data augmentation technique that randomly mixes input data and labels from two different samples and trains the model on the combined data. It offers a more powerful data augmentation method than Mixup because Cutmix encourages the network to learn from different parts of multiple training examples. Therefore, Cutmix is ​​one of the simplest and most effective techniques for improving the performance of graph neural networks, significantly enhancing their ability to learn graph representations. Cutmix uses region-level cut and paste blending techniques, forcing graph neural networks to focus more on the global contextual information of an image rather than just local information, maintaining the continuity of information compared to traditional Mixup.

[0047] (3) Social Network Graph

[0048] set up Represent a social network graph, where V = {v1, v2, ..., v} N} is a set of nodes; E is a set of edges representing relationships between users. E contains various types of edges, such as friend relationships and follower relationships between users; H∈R N×d Let be the feature matrix, where d is the dimension of the node attributes, and N is the number of nodes in the graph; y∈R n yes The label of the node.

[0049] Based on the above, this embodiment proposes a social robot detection system based on graph neural networks and Cutmix, such as... Figure 1 As shown, it includes a node shuffling module, a node cutting and mixing module, and an attribute connection module.

[0050] The node shuffling module shuffles the account nodes and their corresponding edges in the original social network graph, thus reshaping the original social network graph. Based on this, we obtain its isomorphism graph. Specifically, in the original social network graph Top pair of nodes Shuffle the cards to obtain the shuffled nodes. According to the node Reorder The nodes in the array. Let λ represent the shuffling ratio of the nodes, where λ∈[0,1].

[0051] The definition of an isomorphic graph is as follows: given two graphs and Assume there exists a bijective m: Makes all Both have uv∈E A Equivalent to m(u)m(v)∈E B Then it is called and They are isomorphic.

[0052] The node cutting and blending module sequentially blends the node features and labels of the original social network graph and its isomorphic graph. and social network graphs Node pairs are formed by matching nodes with corresponding sequence numbers. For example, a node pair (h) can be constructed using the i-th account node from two social network graphs. A,i ,h B,i ), where 1≤i≤N, and N represents the number of nodes in the graph, such as Figure 2 As shown. The shearing and blending of node pairs is represented as follows:

[0053]

[0054] Where M∈{0,1} H This is a binary mask representing the positions to be deleted and filled from two vectors, and and and Obtained through the Cutmix module

[0055] The attribute connection module inputs the mixed node features into the graph convolutional layer (GConv). The graph convolutional layer generates features based on local aggregation. These features are then combined with the original features of the two graphs to generate new features. Finally, the generated new features are input into the classifier for social robot detection.

[0056] Specifically, The output is obtained after passing through a graph convolutional layer (GConv). The output of the convolutional layer and the features obtained by passing the original features through the linear layer are shown in the figure. and Adding them together gives and The expression is as follows:

[0057]

[0058]

[0059] After that, the result and The nodes are sent to the node cutting and blending module of the subsequent layer; among them, It is a social network graph Adjacency matrix, It is a social network graph The adjacency matrix, AGGREGAT, represents the aggregation function that aggregates the hidden representations of neighboring nodes. Once Cutmix is ​​used to enhance the training distribution features, the corresponding labels should be mixed when calculating the loss function:

[0060]

[0061] in, Indicates the mixed label, y A express The tag, y B express The label is α∈[0,1], where α represents the control ratio parameter.

[0062] Corresponding to the aforementioned social robot detection system based on graph neural networks and Cutmix, this embodiment also proposes a social robot detection method based on graph neural networks and Cutmix, comprising the following steps:

[0063] Step S1: Shuffle the account nodes and corresponding edges in the original social network graph to obtain its isomorphic graph based on the original social network graph.

[0064] Step S2: In sequence, the node features and labels of the original social network graph and its isomorphic graph are mixed.

[0065] Step S3: Input the mixed node features into the graph convolutional layer. The graph convolutional layer generates features based on local aggregation. Then, combine these features with the original features of the two graphs to generate new features.

[0066] Step S4: Finally, the generated new features are input into the classifier for social robot detection.

[0067] The effectiveness of the present invention G-CutMix in detecting social robots is verified through experiments below.

[0068] 1. Dataset

[0069] We evaluated the performance of our Twitter bot detection model on three graph-structured datasets: Cresci-15, Twibot-20, and MGTAB. For these datasets, a social network graph was constructed using all labeled users. Details of these three datasets are shown in Table 1. Random partitioning of 0.5:1:8.5 was performed on all datasets for training, validation, and testing purposes, respectively.

[0070] Table 1. Statistics of the robot detection dataset used in this paper.

[0071]

[0072]

[0073] 2. Baseline Method

[0074] To evaluate the effectiveness of the proposed method, it was compared with commonly used GNN methods.

[0075] GCN: is a local first-order approximation of spectral graph convolution, used for semi-supervised learning of graph-structured data.

[0076] SAGE: This involves sampling and aggregation. It first samples neighboring nodes using the connectivity between nodes, and then continuously fuses the information of neighboring nodes through multiple aggregation functions. The fused information is used to predict node labels.

[0077] GAT: Employs an attention mechanism to determine the weights of a node's neighbors. By adaptively assigning weights to different neighbors, it improves the performance of graph neural networks.

[0078] DropEdge is a technique that randomly removes a certain number of edges from the input graph during each training phase.

[0079] MixupForGraph is a two-stage method for GNNs. It first performs a feedforward in the normal way, and then performs a two-branch mixed graph convolution on the paired nodes of each layer.

[0080] 3. Configuration

[0081] All models consist of two graph convolutional layers with ReLU activation and a dropout rate of 0.5. We optimize the models using cross-entropy loss and the Adam optimizer, with a learning rate of 1e-3 and weight decay of 5e-4. For the Cresci-15, Twibot-20, and MGTAB datasets, we utilize follower relationships and friend relationships, respectively. We set α to 0.4 and trained the models for 200 epochs. The attention heads of GAT were set to 4.

[0082] 4. Experimental Results

[0083] In the experiment, we created a social network graph using follower and friend user relationships, and then used a graph neural network model to detect bot accounts. Each baseline was executed five times with a different seed to mitigate randomness. The experimental results are shown in Tables 2 and 3.

[0084] Table 2 shows the performance of robot detection using follower relationship graphs; Acc and F1 represent accuracy and f1-score, respectively, and bold text is used to mark the best results.

[0085]

[0086] Table 3 shows the performance of robot detection using a friend-user relationship graph; Acc and F1 represent accuracy and f1-score, respectively, and bold text is used to mark the best results.

[0087]

[0088] 5. Discussion

[0089] 5.1 Training set size

[0090] In the field of robot detection, obtaining large-scale labeled datasets is often challenging. The G-CutMix method of this invention produces more significant improvements when trained on smaller datasets. The results are shown in [link to example]. Figure 3 and Figure 4 .

[0091] Observing the performance of G-CutMix and the original GNN, it is clear that our proposed G-CutMix method improves performance across different training set sizes. Our method shows a more significant improvement on smaller training sets, as the model struggles to achieve sufficient training with limited data.

[0092] 5.2 Parameter Sensitivity Analysis

[0093] The parameter α controls the Cutmix ratio. Figure 5 The performance of G-CutMix is ​​illustrated as α varies from 0.1 to 0.9; from the figure, we observe that G-CutMix exhibits good performance when α is 0.3 across all datasets.

[0094] 5.3 Cutmix of Heterogeneous Graphs

[0095] In the field of social bot detection, it has been observed that using multiple relationships instead of just one can lead to better performance. Our proposed method can be extended to multi-relation graphs, where multiple relationships are used simultaneously. To this end, we constructed a social network graph with two types of relationships using a bot detection dataset: followers and friends. The comparison results of G-CutMix with classic heterogeneous graph neural networks RGCN and RGAT are shown in Table 4. This approach proves advantageous because G-CutMix enables the model to generalize better to unseen data by leveraging information from multiple relationships.

[0096] Table 4 shows the performance on a multi-relationship graph using friend-user relationships and follower relationships. Bold text indicates the best results.

[0097]

[0098] 5.4 Ablation Experiment

[0099] We divided the dataset into training, validation, and test sets. To evaluate the impact of each module in G-CutMix, we used a follower graph. Specifically, we conducted experiments by removing the node shuffle module, the node cutmix module, and the attribute join module, respectively. The settings "w / o shuffle", "w / o cutmix", and "w / o attribute" correspond to methods without the node shuffle module, node cutmix module, and attribute join module, respectively.

[0100] Table 5 Accuracy of G-CutMix and its variants

[0101]

[0102] Extensive experiments have demonstrated the superior performance of the proposed G-CutMix method in detecting social robots, especially when the number of labeled nodes is limited.

[0103] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0104] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0105] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0107] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0108] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A social robot detection system based on graph neural networks and Cutmix, characterized in that, include: The node shuffling module shuffles the account nodes and corresponding edges in the original social network graph, and obtains its isomorphic graph based on the original social network graph. The node cutting and blending module sequentially blends the node features and labels of the original social network graph and its isomorphic graph. The attribute connection module inputs the mixed node features into the graph convolutional layer. The graph convolutional layer generates features based on local aggregation. These features are then combined with the original features of the two graphs to generate new features. Finally, the generated new features are input into the classifier for social robot detection.

2. The social robot detection system based on graph neural networks and Cutmix according to claim 1, characterized in that, In the original social network graph Based on this, we obtain its isomorphism graph. include: In the original social network graph Top pair of nodes Shuffle the cards to obtain the shuffled nodes. According to the node Reorder In the nodes, λ represents the proportion of nodes shuffled, where λ∈[0,1].

3. The social robot detection system based on graph neural networks and Cutmix according to claim 2, characterized in that, The isomorphic graph is defined as follows: given two graphs and Assume there exists a bijection Makes all Both have uv∈E A Equivalent to m(u)m(v)∈E B Then it is called and They are isomorphic.

4. The social robot detection system based on graph neural networks and Cutmix according to claim 3, characterized in that, Using social network graphs and social network graphs The i-th account node in the construct node pair (h) A,i ,h B,i ), where 1≤i≤N, and N represents the number of nodes in the graph; the cutting and mixing of node pairs is represented as: Where M∈{0,1} H This is a binary mask representing the positions to be deleted and filled from two vectors, and and The output is obtained after graph convolution layer. The output of the convolutional layer and the features obtained by passing the original features through the linear layer are shown in the figure. and Adding them together gives and The expression is as follows: After that, the result and The nodes are sent to the node cutting and blending module of the subsequent layer; among them, It is a social network graph Adjacency matrix, It is a social network graph The adjacency matrix, AGGREGAT, represents the aggregation function that aggregates the hidden representations of neighboring nodes, and W is a trainable weight matrix.

5. The social robot detection system based on graph neural networks and Cutmix according to claim 4, characterized in that, Appropriate labels should be included when calculating the loss function: in, Indicates the mixed label, y A express The tag, y B express The label is α∈[0,1], where α represents the hyperparameter controlling the ratio.

6. A social robot detection method based on graph neural networks and Cutmix, characterized in that, Includes the following steps: By shuffling the account nodes and their corresponding edges in the original social network graph, we obtain its isomorphic graph based on the original social network graph. The node features and labels of the original social network graph and its isomorphic graph are mixed sequentially; The mixed node features are input into the graph convolutional layer, which generates features based on local aggregation. These features are then combined with the original features of the two graphs to produce new features. Finally, the generated new features are input into the classifier for social robot detection.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 6.

Images