A social robot detection method based on directed relationship graph contrastive learning

Abstract

The application discloses a social robot detection method based on directed relation graph contrast learning, comprising the following steps: taking each user as a user node, obtaining personal attribute information and personal text information of each user node, and converting the personal attribute information and the personal text information into a feature vector after splicing; constructing a user directed relation graph, first learning the features of adjacent users under different relations by using a directed relation graph convolutional neural network, and then performing relation feature fusion based on an attention mechanism; constructing contrast views of the directed relation graph from two aspects of attribute features and graph topological structure combined with direction information, and then performing training and prediction. The application improves the accuracy of robot account identification of a social network platform by using a directed relation graph attention mechanism and directed graph contrast learning, and can effectively utilize direction information of relations of the social network platform in real life and importance information of relations between users by using the directed relation graph combined with the attention mechanism.

Description

Technical Field

[0001] This invention belongs to the field of social network analysis technology and relates to a social bot detection method based on directed graph contrastive learning. Specifically, this method is mainly used to identify automated bot accounts in social networks that mimic the behavior of real users. Background Technology

[0002] With the rapid rise of social networking platforms such as Twitter (now known as X), a large number of social bot accounts have emerged. Some malicious bots spread harmful information, engage in false propaganda, and even publish inflammatory remarks and incite confrontation on these platforms. The existence of malicious bot accounts poses a serious threat to cybersecurity. Therefore, effective detection of social bots is of great significance for purifying online content and maintaining a genuine public opinion environment.

[0003] This invention primarily focuses on methods for detecting social bots on social networking platforms, with broad application prospects. This technology can help identify automated accounts on various social media platforms, providing strong support for preventing the spread of misinformation and manipulation of public opinion. In customer service, analyzing bot interactions with users can optimize customer experience. In the financial sector, detection technology can identify abnormal transaction patterns and reduce financial risks. Furthermore, this technology also has important applications in cybersecurity, public opinion analysis, and political election monitoring. With technological advancements, the application scenarios for social bot detection will further expand, becoming one of the key means of maintaining a healthy online environment.

[0004] In recent years, graph structures have been widely used in social network research. They can describe direct or indirect relationships between social network accounts, such as following relationships and follower relationships. A typical approach to this type of method is to first model the relationships between accounts as nodes and the relationships between accounts as edges, then use graph neural networks to extract information from the social network graph, ultimately achieving binary classification of accounts. However, current advanced social bots often attempt to mimic the characteristics of real users through feature manipulation (stealing real users' personal information) and actively follow or befriend real accounts, resulting in unsatisfactory detection accuracy for most existing graph neural network-based social bot detection methods. Summary of the Invention

[0005] One objective of this invention is to address the insufficient accuracy of existing social bot detection technologies by providing a social bot detection method based on directed graph contrastive learning. This method includes: acquiring initial feature vectors for user nodes, using user personal attribute information and semantic information extracted from user text via RoBERTa as node initialization features; constructing a user directed graph, first using a directed graph convolutional neural network to learn features of adjacent users under different relationships, and then fusing relationship features based on an attention mechanism; constructing two contrastive views of attribute features and graph topology in the user directed graph, and using graph contrastive learning combined with relationship direction information to amplify the similarity between users of the same category and the differences between users of different categories. This invention improves the accuracy of social media platform bot account identification by combining a directed graph attention mechanism and graph contrastive learning with relationship direction information.

[0006] The technical solution adopted in this invention is as follows:

[0007] Step (1): Treat each user as a user node, obtain the personal attribute information and personal text information of each user node i, and concatenate the personal attribute information and personal text information to transform them into a feature vector X, where X = {x i}={x1,x2,…,x N}, x i Let N represent the initial feature vector of user node i, and N represent the number of user nodes.

[0008] Step (2): Based on user nodes, construct directed edges in different directions according to different types of relationships between users. For example, if there is a follow relationship between node i and node j, there will be a directed edge from node i to node j. Construct a heterogeneous graph of directed relationships G = (V, E), where V is the set of user nodes and E is the set of directed edges; based on the heterogeneous graph of directed relationships, generate a first comparison view G from the perspective of node attributes. α Generate a second comparison view G from the perspective of graph topology. β ;

[0009] Step (3): Construct a directed relational graph convolutional neural network based on the attention mechanism to learn the features of neighboring nodes j under different relations around the target user node i, and obtain the user's high-order feature vector. and user vector

[0010] Step (4): In the first comparison view G, respectively α Second comparison view G β Pre-training of a directed graph convolutional neural network based on an attention mechanism is performed on x: i Input to a directed graph convolutional neural network based on an attention mechanism, respectively in G αand G β The corresponding user vector is obtained after L1 learning.

[0011] Step (5), Fine-tuning of the directed graph convolutional neural network based on the attention mechanism: adjusting the user vector As input to the directed relational heterogeneous graph G, L2 iterations of a directed relational graph convolutional neural network based on an attention mechanism are performed for fine-tuning to obtain the final user feature vector.

[0012] Step (6): Extract user feature vectors from users using a fine-tuned directed graph convolutional neural network based on an attention mechanism. This allows us to determine the user category.

[0013] Another object of the present invention is to provide a social robot detection system for implementing the above method, comprising:

[0014] The user node feature extraction module is used to obtain the feature vector representation of user nodes;

[0015] The directed graph construction module is used to transform social networks into graph structures, with users as nodes and different types of relationships between users as directed edges, to obtain a directed heterogeneous graph.

[0016] The graph comparison view building module is used to generate comparison views of directed relational heterogeneous graphs from the perspectives of node attributes and graph topology.

[0017] The user node social robot detection module uses a pre-trained and fine-tuned attention-based directed relational graph convolutional neural network to detect the category of user node social robots.

[0018] Another object of the present invention is to provide a computer storage medium having a corresponding computing program stored thereon, which, when run in a computer, is used to execute the above-described method.

[0019] Another object of the present invention is to provide a computing device including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method described above.

[0020] The technical solution provided by this invention has the following beneficial effects:

[0021] This invention jointly encodes multimodal user information, including vectorized personal attribute information and semantic features obtained by extracting information from user text through a pre-trained language model RoBERTa, which are concatenated as an initial user vector. Jointly encoding multimodal user information can effectively improve the robustness of robot detectors.

[0022] This invention proposes a social robot detection model based on directed relation graphs combined with an attention mechanism. It effectively extracts the features of neighboring nodes of user nodes under different relationships in the social network graph, while considering the directional information in different relationships to obtain higher quality user features. By using directed relation graphs combined with an attention mechanism, it can effectively utilize the directional information of relationships on social network platforms in real life and the importance information of relationships between users, thus solving the problem of social robots evading detection by imitating the features of real users.

[0023] This invention proposes a social robot detection method based on graph contrastive learning. It uses supervised graph contrastive learning to guide the aggregation of node information and proposes three relationship enhancement strategies for enhancing the graph topology structure, addressing the problem of social robots deliberately associating with normal accounts. This achieves the effect of increasing the similarity between users of the same category and the difference between users of different categories, thereby improving the accuracy of social media platform robot account identification. Attached Figure Description

[0024] Figure 1 This is an overall flowchart of the method of the present invention;

[0025] Figure 2 This is a schematic diagram of the method structure of the present invention;

[0026] Figure 3 This is the structure diagram of the directed relational graph Transformer. Detailed Implementation

[0027] The present invention will now be further analyzed with reference to the accompanying drawings.

[0028] A social robot detection method based on directed relation graph contrastive learning, such as Figure 1-2 As shown, it includes the following steps:

[0029] Step (1): Obtain the personal attribute information (including Boolean type attributes and numeric type attributes) and personal text information (including post text and personal description text) of each user to obtain the feature vector X of all user nodes.

[0030] Let X = {x1, x2, ..., x} i ,…,x N} is a set obtained by concatenating vectorized personal attribute information of all users with semantic features extracted from user text using the pre-trained language model RoBERTa, where Let N represent the initial feature vector of user node i, and N represent the number of nodes. in Let T represent the self-introduction text feature vector, post feature vector, numerical attribute information vector, and Boolean attribute information vector of user node i, respectively, where T is the dimension of the user vector.

[0031] Step (2): Construct a heterogeneous graph G of directed user relationships, and generate a comparison view G from the perspectives of node attributes and graph topology. α and G β .

[0032] During graph construction, nodes are represented using user feature vectors, and edges are represented using different types of relationships between users.

[0033] Suppose we have a heterogeneous graph G = (V, E) containing a set of nodes V and a set of relation edges E, where the node set V = {x1, ..., x...} j}, where x j Equals the initialized user vector and relation edge set obtained in step (1). in Let R represent the set of edges pointing from user node i to user node j under relation r1, where all edges are in relation R = {r1, r2, ..., r...} k}

[0034] During the construction of the graph comparison view, G α The view, from the perspective of node attributes, randomizes the feature vectors of user nodes in the original graph and swaps the attributes of nodes of the same category. G β The view, from the perspective of graph topology and combined with real-world social network scenarios, obtains a result by deleting and adding relationships between different types of edges in the original graph's edge set. The following three strategies are proposed for graph structure enhancement:

[0035] 1. If there is a one-way relationship between node i and node j, and one of their accounts is a robot account, the edge between node i and node j can be deleted;

[0036] 2. If node i and node j have a bidirectional relationship, they are considered to be strongly related, and the edge should not be deleted;

[0037] 3. If there is a bidirectional relationship between node i and node j, and a bidirectional relationship between node j and node k, an edge can be added between node i and node k.

[0038] For Strategy 1: We can assume that traditional social bots, like official accounts such as Microsoft's, are followed by many real users, but as bot accounts, they don't follow other real accounts. Similarly, many web-based social bots follow many users, but few users follow them back. This shows that many social relationships are unequal, especially between real humans and social bots. Strategy 2: If a two-way relationship exists, it can be assumed that the two individuals are strongly associated. Since one-way and two-way relationships differ significantly in practical terms, if a relationship is accidentally deleted during adjustments, turning a two-way relationship into a one-way one, the users involved will be affected by noise, leading to misclassification. Therefore, we need to strengthen this relationship. Strategy 3: If A and B have a two-way relationship, and B and C also have a two-way relationship, there is a possibility of overlap between A and C, much like social relationships in real life. If A and B are friends, and B and C are also friends, then A and C are likely to know each other. Therefore, we can add social relationships between them.

[0039] Step (3) employs a message-passing and attention mechanism in a directed relational graph convolutional neural network to learn the features of neighboring user nodes under different relationships around the target user node. The directional information of the target user node and its neighboring nodes under different relationships is also considered.

[0040] like Figure 3 As shown, the directed relation graph convolutional neural network includes the following modules: relation extraction module, neighbor information aggregation module, gating mechanism module, and global relation fusion module.

[0041] First, the relationship extraction module separates user nodes with different relationships within their respective views.

[0042] Next, the neighbor information aggregation module calculates the attention weights between user node i and neighbor node j based on a multi-head attention mechanism:

[0043]

[0044] Where q, k, and v are the query, key, and value in the attention mechanism, respectively, c represents the c-th head in multi-head attention, and (l) represents the l-th layer of the graph neural network. and Let i and j represent the feature vectors of user node i and neighbor node j in the (l-1)th layer, respectively. This represents the query with relation type r under the c-th header. This represents the key of type r under the c-th header. This represents the value of relation type r under the c-th header. All W cand b c These are all learnable parameters of the weight matrix and bias terms for different relationships under the c-th head. Then, the influence of heterogeneity is simulated by calculating the attention weights between the target node and its neighboring nodes:

[0045]

[0046] in This represents the attention weights between user node i and its neighbor node j. It refers to the scale dot product operation in the attention mechanism, where N... r (i) represents the set of neighbor nodes of user node i in a relation of type r.

[0047] Then, the gating mechanism module divides all neighboring nodes of user node i into outgoing and incoming edges according to their directions. in It is the set of neighboring nodes of type r with outgoing edges of user node i. This is the set of neighboring nodes of type r with incoming edges for user node i. This partitioning is based on the assumption that outgoing edges represent active associations between the target user node and its neighbors, while incoming edges represent passive associations. Therefore, information aggregation of nodes associated with outgoing edges is given higher weight, while information aggregation of nodes associated with incoming edges is given lower weight.

[0048]

[0049] in The hidden representation of user node i at level l of relation type r, where β is the weight of outgoing edges requiring a value greater than 0.5, and C is the number of attention heads. Then, a gating mechanism is applied to obtain the representation of user node i at level l of relation type r.

[0050]

[0051] Where ⊙ represents the Hadamard Product. and These are the initial and hidden representations of user node i at the l-th layer of relation type r, as described above.

[0052] Finally, the global relation fusion module uses a semantic attention network to aggregate user node representations across relations. Get the final First, obtain the weight of each relationship through a global view of all nodes in the heterogeneous graph:

[0053]

[0054] in This represents the weight of the d-th attention head in relation of type r. Let V represent the semantic attention vector of the d-th attention head in the l-th layer. Let V be the set of nodes, and W be the set of nodes. d,s (l) and b d,s (l) and represent the learnable weight matrix and bias term parameters of the d-th head in the l-th layer, respectively.

[0055] Next, the softmax function is used to assign weights to each relation. (Right now Normalize:

[0056]

[0057] in This represents the normalized weight for relation type r, and k represents the index used to traverse each relation type. This represents the weight of the d-th attention head in layer l for relation type k;

[0058] Finally, the node representations under different types of relationships are fused with these weights to obtain the final user vector representation.

[0059]

[0060] Where D is the number of heads in the multi-head semantic attention network.

[0061] Step (4): Take the initial user node vector obtained in step (1) and, based on the two comparison views G obtained in step (2),... α and G β The graph neural network model obtained by performing step (3) above L1 times is pre-trained to obtain... View G α Taking node i as an example, we consider the corresponding node and nodes of the same class in its other view as positive samples, and nodes of different classes as negative samples, and calculate the contrastive loss function between them:

[0062]

[0063] in and They are x i In G α and G βThe representation of node i in both views is learned through an L1-order directed graph combined with an attention mechanism. τ is the temperature coefficient in the contrastive loss that adjusts the degree of distribution uniformity. cos(·,·) is the cosine similarity function for calculating the user feature vector. Then, the loss function for all nodes in both views is calculated in the same way, and the average value is taken.

[0064]

[0065] Finally, the model parameters and user node feature vectors will be combined. Save locally.

[0066] Step (5): Transfer the feature vector of user i The final user feature vector is obtained by fine-tuning the input feature vector used in step (3). Finally, the softmax function is used to classify each user:

[0067]

[0068] The fine-tuned loss function combines cross-entropy loss and L2 regularization:

[0069]

[0070] Where y i This represents the actual label of user i. Let λ represent the predicted label for user i, λ be the hyperparameter of the loss function, and θ be the set of all trainable parameters.

[0071] The performance of this invention was evaluated on three datasets supporting graph-based social bot detection methods: Cresci-15, TwiBot-20, and MGTAB-22. The Cresci-15 dataset, proposed in 2015, contains 5301 Twitter users and their two types of relationships: followers and friends. The TwiBot-20 dataset includes account information and tweets for 229580 users, as well as a graph structure consisting of two types of user relationships, but only 11826 labeled users were actually used. The MGTAB-22 dataset contains 410199 users and seven types of relationships, with 10119 users using expert-annotated data. The table below shows the account type and relationship type data for the three datasets:

[0072] Table 1 shows the account type and relationship type data in the dataset.

[0073]

[0074]

[0075] This task can be considered a binary classification task. The experiment uses F1-score and accuracy as the performance evaluation indicators for social robot detection.

[0076] The formula for the accuracy evaluation index is as follows:

[0077]

[0078] Accuracy reflects the overall classification accuracy of the model. TP, TN, FP, and FN represent the number of samples correctly predicted as positive, the number of samples correctly predicted as negative, the number of samples incorrectly predicted as positive, and the number of samples incorrectly predicted as negative, respectively.

[0079] The formula for the F1-score evaluation metric is as follows:

[0080]

[0081] In this context, precision and recall represent classification accuracy and recall, respectively, evaluating whether the model accurately classifies positive examples and the proportion of positive examples classified by the classifier to all positive examples. From the above definition, we can see that F1-score is an evaluation metric that comprehensively considers the classifier's precision and recall. By combining these two metrics, F1-score can more comprehensively evaluate the classifier's performance in positive example classification.

[0082] Table 2 below shows the results of social robot detection performed using the above three datasets, which were randomly split into training, validation, and test sets at 80%, 10%, and 10% respectively. The Acc and F1-score mentioned above were used as evaluation metrics.

[0083] Table 2. Social Robot Detection Results

[0084]

[0085] The analysis in the table above shows that the present invention represents a significant improvement over the current best model. On large datasets such as TwiBot-20 and MGTAB-22, the method of the present invention demonstrates substantial improvements in both Acc and F-score.

Claims

1. A social robot detection method based on directed graph contrastive learning, characterized in that, Includes the following steps: Step (1): Treat each user as a user node, obtain the personal attribute information and personal text information of each user node i, and concatenate the personal attribute information and personal text information to transform them into a feature vector X, where X = {x i }={x1,x2,…,x N }, x i Let N represent the initial feature vector of user node i, and N represent the number of user nodes. Step (2): Based on user nodes, construct directed edges in different directions according to different types of relationships between users, and construct a heterogeneous graph of directed relationships G = (V, E), where V is the set of user nodes and E is the set of directed edges; according to the heterogeneous graph of directed relationships, from the perspective of node attributes, randomize the feature vector X and exchange the attributes of nodes of the same category to obtain... Let it be the first comparison view G α ; From the perspective of graph topology, removing relations from edges of different types in the set E of directed edges yields... This is denoted as the second comparison view G. β ; Step (3): Construct a directed relational graph convolutional neural network based on the attention mechanism to learn the features of neighboring nodes j under different relations around the target user node i, and obtain the user's high-order feature vector. and user vector Step (4): In the first comparison view G, respectively α Second comparison view G β Pre-training of a directed graph convolutional neural network based on an attention mechanism is performed on x: i Input to a directed graph convolutional neural network based on an attention mechanism, respectively in G α and G β The corresponding user vector is obtained after L1 learning. Specifically, define view G α For node i in the dataset, treat its corresponding node and nodes of the same class in its other view as positive samples, and treat nodes of different classes as negative samples, and calculate the contrastive loss function between them: in and They are x i In G α and G β The representation of node i in both views is learned through an L1-order directed graph combined with an attention mechanism. τ is the temperature coefficient in the contrastive loss that adjusts the degree of distribution uniformity. cos(·,·) is the cosine similarity function for calculating the user feature vector. Then, the loss function for all nodes in both views is calculated in the same way, and the average value is taken. Finally, the model parameters and user node feature vectors will be combined. Save locally; Step (5), Fine-tuning of the directed graph convolutional neural network based on the attention mechanism: adjusting the user vector As input to the directed relational heterogeneous graph G, L2 iterations of a directed relational graph convolutional neural network based on an attention mechanism are performed for fine-tuning to obtain the final user feature vector. Step (6): Extract user feature vectors from users using a fine-tuned directed graph convolutional neural network based on an attention mechanism. This allows us to determine the user category.

2. The social robot detection method based on directed graph contrastive learning according to claim 1, characterized in that, The personal attribute information includes Boolean type attributes and numeric type attributes, and the personal text information includes post text and personal description text.

3. The social robot detection method based on directed graph contrastive learning according to claim 1, characterized in that, The attention-based directed relation graph convolutional neural network includes a relation extraction module, a neighbor information aggregation module, a gating mechanism module, and a global relation fusion module, wherein: The relationship extraction module is used to separate user nodes with different relationships; The neighbor information aggregation module calculates the attention weights of user node i and neighbor node j based on a multi-head attention mechanism; The gating mechanism module divides all neighboring nodes of user node i into outgoing and incoming edges according to their direction. It aggregates the information of nodes associated with outgoing edges and assigns higher weights, while aggregating the information of nodes associated with incoming edges and assigns lower weights. This process calculates the hidden representation, and then uses the hidden representation to obtain the user's higher-order feature vector. The global relationship fusion module will integrate user high-order feature vectors The user vector is obtained by fusing its weights.

4. The social robot detection method based on directed graph contrastive learning according to claim 3, characterized in that, The calculation formula for the neighbor information aggregation module is as follows: Where q, k, and v are the query, key, and value in the attention mechanism, respectively, c represents the c-th head in the multi-head attention mechanism, and (l) represents the l-th layer of the directed graph convolutional neural network based on the attention mechanism. and Let i and j represent the feature vectors of user node i and their neighbor node j in the (l-1)th layer, respectively. This represents the query with relation type r under the c-th header. This represents the key of type r under the c-th header. This represents the value of relation type r under the c-th header, and all W c and b c These are all learnable parameters of the weight matrix and bias terms for different relations under the c-th head; Then, the influence on heterogeneity is simulated by calculating the attention weights between user node i and its neighbor node j: in This represents the attention weights between user node i and its neighbor node j. It refers to the scaled dot product operation in the attention mechanism, N r (i) represents the set of neighbor nodes of user node i in a relation of type r.

5. The social robot detection method based on directed graph contrastive learning according to claim 4, characterized in that, The gating mechanism module performs the following steps: Divide all neighboring nodes of user node i into outgoing edges and incoming edges according to their directions. in It is the set of neighboring nodes of type r with outgoing edges of user node i. This is the set of neighboring nodes of type r with the incoming edge relationship of user node i; the information of nodes associated with the outgoing edge is aggregated and assigned a higher weight, while the information of nodes associated with the incoming edge is aggregated and assigned a lower weight, thus obtaining the hidden representation. in The hidden representation of user node i at level l of relation type r, where β is the weight of outgoing edges that must be greater than 0.5, and C is the number of attention heads; then, a gating mechanism is applied to obtain the representation of user node i at level l of relation type r. Let this be denoted as the user's higher-order feature vector: Where ⊙ represents the Hadamard product, and These are the initial and hidden representations of user node i at the l-th level of relation type r, respectively.

6. The social robot detection method based on directed graph contrastive learning according to claim 5, characterized in that, The global relationship fusion module performs the following steps: First, obtain the weights for each type of relation by using a global view of all nodes in the directed relation heterogeneous graph G: in This represents the weight of the d-th attention head in the l-th layer for relation type r. Let V represent the semantic attention vector of the d-th attention head in the l-th layer, where V is the set of nodes and W is the semantic attention vector of the d-th attention head. d,s (l) and b d,s (l) These represent the learnable weight matrix and bias term parameters of the d-th head in the l-th layer, respectively. Next, the softmax function is used to assign weights to each relation. Normalize: in The value represents the normalized weight of relation type r; k represents the index used to traverse each relation type. This represents the weight of the d-th attention head in layer l for relation type k; Finally, the higher-order feature vectors of users under different types of relationships are analyzed. With corresponding weights The user vector is obtained by fusion Where D is the number of heads in the multi-head semantic attention network.

7. A social robot detection system implementing the method of any one of claims 1-6, characterized in that, include: The user node feature extraction module is used to obtain the feature vector representation of user nodes; The directed graph construction module is used to transform social networks into graph structures, with users as nodes and different types of relationships between users as directed edges, to obtain a directed heterogeneous graph. The graph comparison view building module is used to generate comparison views of directed relational heterogeneous graphs from the perspectives of node attributes and graph topology. The user node social robot detection module uses a pre-trained and fine-tuned attention-based directed relational graph convolutional neural network to detect the category of user node social robots.

8. A computer storage medium storing a corresponding computing program thereon, wherein the computing program, when run in a computer, is used to perform the steps of the method according to any one of claims 1-6.

9. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the steps of the method according to any one of claims 1-6.

Images