Community robot detection system based on local perception multi-view fusion

Abstract

The application discloses a community robot detection system based on local perception multi-view fusion, and comprises the following steps: a feature coding module is designed, multi-modal features of an account are extracted, and feature coding of a user is acquired; based on the feature coding module, StarBlokc and HGT modules are used respectively to extract multi-view features of the user; based on double-view features, a local similarity perception module is designed to extract a local similarity score of the user, and the local similarity score is used to fuse the multi-view features; a 3-stage community and label perception contrast learning method is designed to train the whole model, and the model is used for community robot identification. The application designs a multi-view feature extractor to extract low-frequency and high-frequency features respectively, models local similarity of a user, and proposes a local similarity module to dynamically weight and fuse a final representation of the user, so as to solve the problem of heterogeneity imbalance caused by data sampling and robot camouflage.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer graph neural networks, and in particular to a community robot detection system based on local similarity perception multi-view fusion network and community-label contrast learning. BACKGROUND

[0002] Community robots are automated accounts that perform various functions. They are manipulated by malicious actors and pose a significant risk on various online social platforms, especially in spreading false information and facilitating fraudulent activities. They are constantly involved in spreading misinformation and manipulating public opinion. If not regulated, this increasingly popular and evolving technology can greatly expand its influence. Therefore, it is of great significance and urgency to develop effective community robot detection methods.

[0003] In order to effectively detect social robots, researchers have proposed feature-based, text-based, and graph-based detection methods that use user metadata, text information, and social information to identify robot accounts. Feature-based methods extract features from user metadata, user timelines, and relationships and feed them into traditional classifiers such as Random Forest for bot detection. However, robots often deliberately tamper with handcrafted features to evade feature-based detection methods.

[0004] Researchers have also proposed text-based methods that use NLP techniques such as word embeddings, recurrent neural networks, and pre-trained language models to encode tweet content and identify malicious intent. However, text-based methods have limitations when encountering new Twitter robots, as they interleave malicious tweets with normal tweets stolen from real users. The recent large language model ChatGPT has demonstrated intelligence close to that of humans, and using ChatGPT can simulate human speech, posing new challenges to Twitter-based methods.

[0005] As the effectiveness of graph neural networks has been proven, researchers have also used users' social network relationships to form graph relationships and proposed various graph-based Twitter robot detection methods such as GCN, RGCN, and RGT. These graph-based detection methods have achieved the best performance.

[0006] Despite the significant progress made by graph-based Twitter robot detection methods, there are still some problems. For example, Figure 1As shown, in terms of connection node types, there are actually three types of connections: connections between two people, connections between two social robots, and connections between a person and a social robot. The connections between a person and his / her related person and the connections between social robots built for the purpose of expanding their discourse influence belong to the same type of nodes, and the connections formed between the same type of nodes are called homophily connections. The connection between a person and a robot is formed between nodes of different types, and such a connection is called a heterophily connection.

[0007] Existing graph-based bot detection methods are based on the homophily assumption that users (nodes) with similar features or the same label are easily connected, and a low-pass filter is used to smooth the features of users within the neighborhood. This assumption smoothes the representation of connected users, making it easy for bot accounts to evade detection by tracking real users. Current graph neural networks are mainly constructed based on the attention and follower relationship of people, however, due to the complexity and largeness of the social relationship of people in twitter, the current dataset can only contain part of the relationship of an account using a sampling method, for example, the Twitter-20 dataset only contains 10 attention relationships and 10 follower relationships of an account, which may cause the proportion of homophily connections and heterophily connections of part of the accounts to be imbalanced; on the other hand, the heterogeneous graph disguises robot accounts by increasing social relationships with normal accounts to avoid existing graph detection methods. This phenomenon causes the existing GNN classifier to be unable to well detect these highly heterogeneous accounts. SUMMARY

[0008] The purpose of the present application is to realize effective community robot detection based on a local similarity perception multi-view fusion network and a community-label contrast learning community robot detection system, and the entire model graph is as shown in Figure 3 ;

[0009] The community robot detection method based on local perception multi-view fusion comprises the following steps:

[0010] S1: design a feature encoding module to extract multi-modal features of an account and obtain feature encodings of users;

[0011] S2: based on the feature encoding module designed in step S1, use StarBlokc module and HGT module to extract multi-view features of users respectively;

[0012] S3: based on the double-view features of step S2, design a local similarity perception module (LocalSim) to extract the local similarity score of the user, and fuse the multi-view features with the local similarity score;

[0013] S4: Design a 3-stage community and label-aware contrastive learning method to train the whole model, and use the model for community bot identification.

[0014] Further, a community bot account U mainly contains four kinds of information, user description information D, user uploaded post list T, user meta information P, and user neighbor information N. Among them, the user description information D = {D represents a user description with L words. The user uploaded post list T = {T represents a user's M posts, and the i-th post T contains Q i words. User meta information P = {P num , P cat} represents the user's digital attribute set and user classification attribute set. The user's neighbor information N = {N represents a user's neighbor set constructed by R kinds of neighbor relationships, and Y represents the user's label. Y = 0 represents that the user is a human, and Y = 1 represents that the user is a bot.

[0015] For user description features, a pre-trained language model Roberta is used to encode the user's description information:

[0016]

[0017] wherein represents the expression of the user description, D s represents the dimension of the Roberta embedding. Then its final expression is obtained through a linear transformation:

[0018]

[0019] wherein W D and b D represent the learnable weight and bias respectively, D d is the embedding dimension of the final expression, and Relu represents the activation function, which uses the Leaky-Relu activation function here.

[0020] Similar to the user description feature, RoBERTa is used to obtain the embedding of a single post, and the average expression of all posts is obtained to obtain the expression of the user post feature , and then a linear transformation is used to obtain its final expression.

[0021]

[0022]

[0023] wherein WT and bT are learnable parameters and biases, D tis the embedding dimension of user twitter features. For meta information features P = {P num , P cat}, directly map them to target dimension using linear transformation:

[0024]

[0025]

[0026] where W and b represent learnable parameters, and b represent biases, and b represent embedding dimensions.

[0027] Finally, concatenate the above representations as user's node representation and map to target embedding dimension using a linear transformation:

[0028]

[0029] where W r and b r are learnable weights and biases, and D is the user node embedding dimension.

[0030] Further, use the StarBlock neural network and the heterogeneous graph neural network GNN shown in Figure 1 to extract the multi-view features of the model respectively. Specifically, first use l StarBlock modules as high-frequency feature encoders to extract the high-frequency features of the model:

[0031] O l = O l-1 + dropout(Star(O l-1 ))

[0032] where dropout is a dropout operation, Star is the output after passing through a Star module, and O l is the output of the i-th star module.

[0033] Then use GNN to aggregate the commonality between neighbors to extract the low-frequency features of the user. In order to be able to process multiple relationships, here HGT is used as a low-frequency feature encoder to extract the low-frequency features of the user:

[0034] G m = G m-1 + dropout(HGT(G m-1 ))

[0035] where Gm represents the output of each GNN layer.

[0036] Furthermore, using Figure 2 The LocalSim local similarity perception module in the model performs weighted fusion of features from two views to obtain the final representation of the model. Specifically, the local similarity of a node is calculated as follows:

[0037]

[0038] in Let be the set of neighbors of node i, and sim(·,·) be a similarity measure function, here using the Euclidean distance similarity measure function:

[0039] sim(x i x j )=-||x i -x j || 2

[0040] To enhance the discriminative power of the MLP layer, a nonlinear representation is incorporated into the local similarity:

[0041]

[0042] MLPLS is a 2-layer perceptron, and Sigmoid(*) is the sigmoid activation function.

[0043] Then, the mixed weights are obtained by calculating the local similarity between low-frequency features and high-frequency features respectively:

[0044] α=sigmoid(MLP(LS(O l )))

[0045] β = sigmoid(MLP(LS(G) m )))

[0046] Finally, obtain the mixed expression:

[0047] r ls =α·O l +β·G m

[0048] Furthermore, a novel graph contrastive learning framework utilizing community and label information of the graph is employed, employing a three-stage training process to acquire more robust graph node representations. Specifically, it uses... Let e ​​represent the edge set of the entire graph G, where e ij This represents the connection between the i-th user and the j-th user. The Louvain method is used to partition the entire graph into communities, obtaining community partitioning information. Where c iis the community number of the ith user. Sample a subgraph G sub from G containing n edges Compute the contrastive learning loss on the subset.

[0049] For each edge e ij , compute the contrastive loss based on the label and the community respectively:

[0050]

[0051]

[0052] where x i and x j are the feature representations of users i and j respectively. Finally, the total contrastive loss is:

[0053]

[0054] where λ is the weight coefficient to control the proportion of the two contrastive losses.

[0055] Next, use a 3-stage training to train the entire model.

[0056] Stage 1: High-frequency feature extractor training: Use an additional MLP to obtain the classification result to learn the classification loss, and set the output of the lth layer starblock as O l Then:

[0057]

[0058] Use the contrastive learning method to expand the difference between the feature representations of different labels. Here, use O l as the user's representation to obtain the user's contrastive loss Then the final loss is:

[0059]

[0060] where μ is the weight coefficient.

[0061] Stage 2: Low-frequency feature extractor training. In this stage, freeze the user node information encoder and starblocks, and only train HGTs. Similar to stage 1, the output G m of HGTs is used as the user's representation, and an additional MLP stage2 is used to obtain the predicted label, and finally the classification loss is obtained. At the same time, the contrastive learning loss is also obtained. Finally, the final loss is:

[0062]

[0063] Stage 3: Finally, use the mixed representation R ls As the final representation, use an MLP to obtain the predicted labels. Here, the whole model is unfrozen and only the classification loss is used:

[0064]

[0065] where G train is the training set subgraph.

[0066] Compared with the prior art, the present application is designed to extract low-frequency and high-frequency features from multi-view features, model the local similarity of users, and propose a local similarity module to dynamically weight and fuse the final representation of users to solve the problem of heterogeneous imbalance caused by data sampling and robot camouflage. Then a community and label-aware contrastive learning network and a 3-stage training method are designed, which helps to learn more robust node representations and enhance the generalization ability of the model. Through testing on the dataset, it is proved that the community robot detection method designed by the present application has better performance. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 is the architecture of the StarBlock module.

[0068] Figure 2 is the architecture of the LocalSim module.

[0069] Figure 3 is the overall architecture of the designed network.

[0070] Figure 4 is the effect comparison of the model of the present application and other models. DETAILED DESCRIPTION

[0071] The technical solutions and implementation details in the implementation process of the present application will be described clearly and completely below. Obviously, the described implementation examples are only a part of the implementation examples of the present application, not all the embodiments; based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0072] The implementation scheme of the experiment is as follows:

[0073] Step S1: Constructing the dataset. Twibot20 dataset: The dataset includes 11826 community accounts, of which 7451 are human accounts and 7451 are robot accounts, labeled with manual annotation. Twibot-20 not only contains the basic information of users (such as username, personal profile, number of followers), but also includes social network structure (such as follow / fan relationship), user published text, interaction information and other multi-dimensional features.

[0074] Step S2: Data preprocessing of the dataset. For the meta-information features of the dataset, including numerical features (such as the number of followers, the number of fans, the number of likes, the number of active days, etc.) and classification features (such as gender, account protection status, authentication status, whether to use the default profile and default avatar, etc.), they are extracted respectively, the numerical features are normalized, and the classification features are converted into onehot vectors. Robert is used to encode the user's profile and text features, and the user's fan list and follow list are used to construct a graph, and finally the feature encoding of all users and the directed graph of the follow-fan relationship between users are obtained.

[0075] Step S3: Network training. The 3-stage training process described above is implemented using Pytorch on a server with 2 NVIDIA RTX2080Ti GPUs, pre-trained for 300 epochs with a batch size of 128. The AdamW optimizer is used with an initial learning rate of 0.001 and a warmup number of 70. The number of StarBlock and HGT is set to 3 and 2 respectively, and the embedding dimension is set to 32.

[0076] Step S4: Model testing: The Twibot20 dataset is divided into 7:2:1 for training, validation and testing respectively. The accuracy ACC and F1 value of the model are calculated to measure the performance of the model:

[0077]

[0078] Where TP is the number of true positive classes that are also predicted to be positive; TN is the number of true negative classes that are also predicted to be negative; FP is the number of true negative classes that are predicted to be positive; FN is the number of true positive classes that are predicted to be negative; and accuracy ACC represents the proportion of samples predicted correctly by the model to the total number of samples.

[0079]

[0080] Where Precesion represents the proportion of true positive classes in the samples predicted to be positive:

[0081]

[0082] Recall the proportion of samples that are actually positive classes and are predicted to be positive classes.

[0083]

[0084] The F1 value balances between precision and recall, and is a comprehensive index. When the precision and recall are quite different, the F1 value is low.

[0085] The main innovation of the application is to design a multi-view feature extractor to extract low-frequency and high-frequency features respectively, and to model the local similarity of users, and to propose a local similarity module to dynamically weight and fuse the final representation of users to solve the problem of heterogeneity imbalance caused by data sampling and robot camouflage. Then a community and label aware contrastive learning network and a 3-stage training method are designed, which helps to learn more robust node representation and enhance the generalization ability of the model. Through the test on the data set, it is proved that the community robot detection method designed by the application has better performance.

[0086] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application; therefore, the embodiments should be regarded as exemplary and non-limiting, the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application; any reference signs in the claims should not be regarded as limiting the claims involved.

[0087] In addition, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description manner of the specification is only for the sake of clarity, those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be properly combined to form other embodiments that those skilled in the art can understand.

Claims

1. A community robot detection method based on local perception multi-view fusion, characterized in that, The method comprises the following steps: S1: design a feature encoding module, extract multi-modal features of an account, and obtain feature encoding of a user; S2: based on the feature encoding module designed in S1, multi-view features of the user are extracted by using StarBloC modules and HGT modules respectively; S3: based on the double-view features of S2, a local similarity perception module is designed to extract the local similarity score of the user, and the multi-view features are fused by using the local similarity score; S4: a 3-stage community and label-aware contrastive learning method is designed to train the whole model, and the model is used for community robot identification; StarBlock neural network and heterogeneous graph neural network GNN are used to extract multi-view features of the model; first, l StarBlock modules are used as high-frequency feature encoders to extract high-frequency features of the model: O l =O l-1 +dropout(Star(O l-1 )) wherein dropout is a dropout operation, Star is the output after one Star module, O l is the output of the lth Star module. GNN is used to aggregate the commonality between neighbors to extract low-frequency features of the user; HGT is used as a low-frequency feature encoder to extract low-frequency features of the user: G m = G m-1 + dropout(HGT(G m-1 )) where G m represents the output of each GNN layer; The LocalSim local similarity perception module is used to weight and fuse two view features to obtain the final expression of the model, and the local similarity of a node is calculated as follows: where is the set of neighbors of node i6, sim(·, ·) is a similarity measure function, here the Euclidean distance similarity measure function is used: In order to enhance the discriminant ability of the MLP layer, a nonlinear expression is added in the local similarity: where MLP ls is a 2-layer perceptron, Sigmoid(*) is the sigmoid activation function; Then, the mixed weight is obtained by calculating the local similarity of the low-frequency features and the high-frequency features respectively: a = sigmoid(MLP(LS(O l ))) β = sigmoid(MLP(LS(G m ))) Finally, the mixed expression is obtained: r ls = a · O l + β · G m A 3-stage training procedure is used to obtain more robust graph node representations; using denotes the edge set of the entire graph G, where denotes the connection between the ith7user and the jth3user; the community partition information is obtained using the louvain method to partition the entire graph where is the community number of the ith8user; a subgraph G sub is sampled from G, containing n edges The contrastive learning loss is calculated on the subset; For connection Compute contrastive loss based on label and community respectively: where and denote the feature representation of user i, user j, respectively; and the total contrastive loss is finally obtained as: Where λ is a weight coefficient for controlling the proportion of the two contrastive losses; The whole model is trained by using a 3-stage training method; Stage 1: High-frequency feature extractor training: Use an extra MLP to get the classification result in order to learn the classification loss, set the output of the l-th layer starblock as O l Then: where G sub is a subgraph sampled from the entire social network graph G by single round training, is the true label of node i9; The contrastive learning method is used to expand the difference of feature expression of different labels. Here, O l The user's contrastive loss is obtained as the user's expression The final loss is: Where μ is a weight coefficient; Stage 2: Low-Frequency Feature Extractor Training; In this stage, the user node information encoder and starsblocks are frozen first, and the HGTs are trained. The output G of the HGTs... m As a representation of the user, use an additional MLP stage2 To obtain the predicted labels, and finally to obtain the classification loss. At the same time, we also obtain the contrast learning loss. Ultimately, the final loss is: Stage 3: Finally, use a hybrid expression R ls As a final expression, use an MLP to obtain the predicted labels; here the whole model is unfrozen and only the classification loss is used: where G train is a training set subgraph. 2.The community robot detection method based on local perception multi-view fusion according to claim 1, wherein, A community robot account U contains four kinds of information, user description information D, user uploaded post list T, user meta information P, and user neighbor information N; where, user description information D = {D represents a user description with L words; user uploaded post list T = {T represents M posts of a user and the i2th post is T contains Q i words; user meta information P = {P num , P cat} represents the user's digital attribute set and user classification attribute set; user's neighbor information N = {N represents a user's neighbor set constructed by R kinds of neighbor relationships, and Y is used to represent the user's label; Y = 0 indicates that the user is a human, and Y = 1 indicates that the user is a robot; For user description features, a pre-trained language model Roberta is used to encode the description information of the user: where, D represents the user description expression, s represents the dimension of Roberta embedding; then get its final expression through linear transformation: where W D and b D represent learnable weights and bias, respectively, D d is the embedding dimension of the final representation, and Relu represents an activation function, using a Leaky-Relu activation function; Similar to the user description feature, RoBERTa is used to obtain the embedding of a single post, and the average representation of all posts is obtained to obtain the representation of the user post feature Then a linear transformation is used to obtain its final representation; where W T and b T are learnable parameters, D t is the embedding dimension of the user twitter features; For meta-information features P = {P num ,P cat}, a linear transformation is used to directly map them to the target dimension: wherein and respectively represent learnable parameters, and respectively represent biases, and respectively represent embedding dimensions; Finally, the above expressions are spliced as the node expression of the user, and a linear transformation is used to map to the target embedding dimension: where W r and b r are learnable weights and biases, respectively, and D is the user node embedding dimension.

Images