Community robot detection system based on local perception multi-view fusion

Abstract

The invention discloses a community robot detection system based on local perception multi-view fusion, and the system comprises the following steps: designing a feature coding module, extracting a multi-modal feature of an account, and obtaining a feature code of a user; based on the feature coding module, extracting multi-view features of the user by using a StarBlokc module and an HGT module; based on the double-view features, a local similarity perception module is designed to extract local similarity scores of the users, and the local similarity scores are fused with the multi-view features; and designing a three-stage community and label perception comparative learning method to train the whole model, and carrying out community robot identification by using the model. According to the method, a multi-view feature extractor is designed to extract low-frequency and high-frequency features, modeling is carried out on local similarity of users, and a local similarity module is provided to carry out dynamic weighted fusion on final representation of the users to solve the problem of heterogeneity imbalance caused by data sampling and robot camouflage.

Description

Technical Field

[0001] The present invention relates to the field of computer graph neural networks, and in particular to a community robot detection system based on a local similarity-aware multi-view fusion network and community-label comparative learning. Background Art

[0002] Community bots are automated accounts that perform various functions. They are manipulated by malicious actors and pose significant risks on various online social platforms, especially in spreading false information and facilitating fraudulent activities. They are constantly involved in spreading misinformation and manipulating public opinion. If left unregulated, this increasingly popular and evolving technology could significantly expand its impact. Therefore, developing effective community bot detection methods is of great significance and urgency.

[0003] In order to effectively detect social robots, researchers have proposed feature-based, text-based, and graph-based detection methods that use user metadata, text information, and social information to identify robot accounts. Feature-based methods extract features from user metadata, user timelines, and relationships, and feed them into traditional classifiers such as Random Forest for bot detection. However, robots often deliberately tamper with handcrafted features to evade feature-based detection methods.

[0004] Researchers have also proposed text-based methods, which use NLP techniques such as word embedding, recursive neural networks, and pre-trained language models to encode tweet content and identify malicious intent. However, when encountering new Twitter robots, text-based methods are insufficient because they intersperse malicious tweets with normal tweets stolen from real users. Recently, the large language model ChatGPT has shown intelligence close to that of humans. Using ChatGPT, it is possible to simulate human speech, which poses new challenges to Twitter-based methods.

[0005] As the effectiveness of graph neural networks has been proven, researchers have also used users' social network relationships to form graph relationships and proposed various graph-based Twitter robot detection methods, such as GCN, RGCN, and RGT. These graph-based detection methods have achieved the best performance.

[0006] Although graph-based Twitter bot detection methods have made significant progress, some problems still exist. Figure 1As shown in the figure, from the perspective of connection node types, there are actually three types of connections: connections between two people, connections between two social robots, and connections between a person and a social robot. Connections between people and people related to them and connections between social robots to expand the influence of their discourse are both types of connections formed between nodes of the same type, called homogeneous connections. Connections between people and robots are formed between nodes of different types, and this type of connection is called heterogeneous connections.

[0007] Existing graph-based bot detection methods are based on the homogeneity assumption, that is, users (nodes) with similar features or the same labels are easily connected, and use low-pass filters to smooth user features in the neighborhood. This assumption smooths the representation of connected users, making it easy for bot accounts to evade detection by tracking real users. Current graph neural networks are mainly built based on people's follow and fan relationships. However, due to the complexity and large size of people's social relationships on Twitter, the current dataset can only contain part of the relationships of an account using sampling methods. For example, the Twitter-20 dataset only contains 10 follow relationships and 10 fan relationships of an account, which may cause an imbalance in the ratio of homogeneous connections to heterogeneous connections for some accounts; on the other hand, heterogeneous graph disguised robot accounts will evade existing graph detection methods by increasing social relationships with normal accounts. This phenomenon causes the existing GNN classifier to be unable to detect these highly heterogeneous accounts well. Summary of the invention

[0008] The purpose of this invention is to realize effective community robot detection based on the community robot detection system of local similarity perception multi-view fusion network and community-label comparative learning. The whole model is shown in the figure Figure 3 As shown;

[0009] The community robot detection method based on local perception multi-view fusion specifically includes the following steps:

[0010] S1: Design a feature encoding module to extract the multimodal features of an account and obtain the user's feature encoding;

[0011] S2: The feature encoding module designed based on step S1 uses the StarBlokc module and the HGT module to extract the multi-view features of the user;

[0012] S3: Based on the dual-view features in step S2, a local similarity perception module (LocalSim) is designed to extract the local similarity scores of users, and the local similarity scores are used to fuse multi-view features;

[0013] S4: Design a three-stage community and label-aware contrastive learning method to train the entire model, and use the model for community robot identification.

[0014] Furthermore, a community robot account U mainly contains four types of information: user description information D, user uploaded post list T, user meta information P, and user neighbor information N. Represents a user description with L words. List of posts uploaded by the user Indicates that a user has M posts and the i-th post Included Q i words. User meta information P = {P num , P cat} represents the user's digital attribute set and user classification attribute set. User's neighbor information represents a user's neighbor set constructed by R kinds of neighbor relationships, and Y represents the user's label. Y = 0 means the user is a human, and Y = 1 means the user is a robot.

[0015] For user description features, the pre-trained language model Roberta is used to encode the user's description information:

[0016]

[0017] in represents the expression of user description, D s Represents the dimension of Roberta embedding. Then a linear transformation is performed to obtain its final expression:

[0018]

[0019] Where W D and b D Represent the learnable weights and biases, D d is the embedding dimension of the final expression, Relu represents the activation function, and the Leaky-Relu activation function is used here.

[0020] Similar to the user description feature, RoBERTa is used to obtain the embedding of a single tweet, and finally the average expression of all tweets is obtained to obtain the expression of the user tweet feature. , and then a linear transformation is applied to obtain its final representation.

[0021]

[0022]

[0023] Where WT and bT are learnable parameters and biases, D tis the embedding dimension of the user’s Twitter feature. For the meta-information feature P = {P num , P cat}, and use a linear transformation to directly map it to the target dimension:

[0024]

[0025]

[0026] in and Represent the learnable parameters, and Represent the bias, and represent the embedding dimensions respectively.

[0027] Finally, concatenate the above expressions as the user's node expression and use a linear transformation to map it to the target embedding dimension:

[0028]

[0029] Where W r and b r are the learnable weights and biases respectively, and D is the user node embedding dimension.

[0030] Further, using Figure 1 The StarBlock neural network and heterogeneous graph neural network GNN shown in the figure are used to extract the multi-view features of the model respectively. Specifically, l StarBlock modules are first used as high-frequency feature encoders to extract the high-frequency features of the model:

[0031] O l =O l-1 +dropout(Star(O l-1 ))

[0032] Where dropout is the dropout operation, Star is the output after passing through the Star module once, and O l is the output of the star module.

[0033] Then use GNN to aggregate the commonalities between neighbors and extract the low-frequency features of users. In order to handle multiple relationships, HGT is used as a low-frequency feature encoder to extract the low-frequency features of users:

[0034] G m =G m-1 +dropout(HGT(G m-1 ))

[0035] Where Gm represents the output of each GNN layer.

[0036] Furthermore, using Figure 2 The LocalSim local similarity perception module in the model is used to perform weighted fusion of the two view features to obtain the final expression of the model. Specifically, the local similarity of a node is calculated as follows:

[0037]

[0038] in is the neighbor set of node i, sim(·,·) is a similarity measurement function, and the Euclidean distance similarity measurement function is used here:

[0039] sim(x i , x j )=-||x i -x j || 2

[0040] In order to enhance the discriminative ability of the MLP layer, a nonlinear expression is added to the local similarity:

[0041]

[0042] Where MLP1s is a 2-layer perceptron and Sigmoid(*) is the sigmoid activation function.

[0043] Then the mixed weight is obtained by calculating the local similarity of low-frequency features and high-frequency features respectively:

[0044] α=sigmoid(MLP(LS(O l )))

[0045] β=sigmoid(MLP(LS(G m )))

[0046] Finally get the mixed expression:

[0047] r ls =α·O l +β·G m

[0048] Furthermore, a novel graph contrastive learning framework that utilizes the community information and label information of the graph is used, and a three-stage training process is used to obtain a more robust graph node representation. Specifically, represents the edge set of the entire graph G, where e ij Represents the connection between the i-th user and the j-th user. Use the Louvain method to divide the entire graph into communities and obtain community division information where c iis the community number of the i-th user. Sample a subgraph G from G sub , contains n edges Compute the contrastive learning loss on the subsets.

[0049] For connection ij , respectively, based on the label and the community to calculate the contrast loss:

[0050]

[0051]

[0052] where x i and x j They represent the feature expressions of user i and user j respectively, and finally the total contrast loss is obtained:

[0053]

[0054] Where λ is the weight coefficient, which is used to control the proportion of the two contrast losses.

[0055] Next, a three-stage training process is used to train the entire model.

[0056] Stage 1: High-frequency feature extractor training: Use an additional MLP to obtain the classification results to facilitate learning the classification loss. Let the output of the l-th layer starblocks be O l ,but:

[0057]

[0058] Use contrastive learning to expand the differences in feature expressions of different labels. Here we use O l As the user's expression, the user's contrast loss is obtained The final loss is:

[0059]

[0060] Where μ is the weight coefficient.

[0061] Stage 2: Low-frequency feature extractor training. In this stage, the user node information encoder and starsblocks are frozen and only HGTs are trained. Similar to stage 1, the output G of HGTs m As the user's expression, an additional MLP is used stage2 To obtain the predicted label, the final classification loss We also get the contrastive learning loss Finally, the final loss is:

[0062]

[0063] Stage 3: Finally, use the mixed expression R ls As a final expression, an MLP is used to obtain the predicted labels. Here we unfreeze the entire model and use only the classification loss:

[0064]

[0065] Among them G train is the training set graph.

[0066] Compared with the prior art, the present invention designs a multi-view feature extractor to extract low-frequency and high-frequency features respectively, and models the local similarity of users. It proposes a local similarity module to dynamically weight the final representation of users to solve the problem of heterogeneity imbalance caused by data sampling and robot disguise. Then, a community and label-aware contrast learning network and a three-stage training method are designed to help learn more robust node representations and enhance the generalization ability of the model. Through testing on the data set, it is proved that the community robot detection method designed by the present invention has better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is the architecture of the StarBlock module.

[0068] Figure 2 It is the architecture of LocalSim module.

[0069] Figure 3 It is the overall architecture of the designed network.

[0070] Figure 4 This is a comparison of the effects of the model invented in this article and other models. DETAILED DESCRIPTION

[0071] The following is a clear and complete description of the technical solutions and implementation details in the specific implementation process of the present invention. Obviously, the described implementation examples are only part of the implementation examples of the present invention, not all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] The implementation plan of this experiment is as follows:

[0073] Step S1: Construct a dataset. Twibot20 dataset: The dataset includes 11,826 community accounts, including 7,451 human accounts and 7,451 robot accounts, which are annotated manually. Twibot-20 not only contains basic information of users (such as username, profile, number of followers), but also includes features of multiple dimensions such as social network structure (such as follow / fan relationship), text posted by users, and interaction information.

[0074] Step S2: Preprocess the data set, extract the metadata features of the data set, including numerical features (such as number of followers, number of fans, number of posts liked, number of active days, etc.) and categorical features (such as gender, account protection status, authentication status, whether to use the default profile and default avatar, etc.), normalize the numerical features, and convert the categorical features into onehot vectors. Use Robert to encode the user's profile and text features, and use the user's fan list and follow list to build a graph, and finally obtain the feature encoding of all users and the directed graph of the follow-fan relationship between users.

[0075] Step S3: Network training. The three-stage training process described above was implemented using Pytorch on a server with two NVIDIA RTX2080Ti GPUs, with 300 epochs of pre-training at a batch size of 128. The AdamW optimizer was used, with an initial learning rate of 0.001 and 70 warmup rounds. The number of StarBlocks and HGTs was set to 3 and 2 respectively, and the embedding dimension was set to 32.

[0076] Step S4: Model testing: Divide the Twibot20 dataset into 7:2:1 for training, validation, and testing respectively. And calculate the model's accuracy ACC and F1 value to measure the performance of the model:

[0077]

[0078] Among them, TP is the number of samples that are actually positive and predicted to be positive; TN is the number of samples that are actually negative and predicted to be negative; FP is the number of samples that are actually negative but predicted to be positive; FN is the number of samples that are actually positive but predicted to be negative. The accuracy rate ACC indicates the proportion of samples predicted correctly by the model to the total samples.

[0079]

[0080] Precesion represents the proportion of samples predicted to be positive to samples that are actually positive:

[0081]

[0082] Recall is the ratio of samples predicted to be positive among samples that are actually positive.

[0083]

[0084] The F1 value is a comprehensive indicator that strikes a balance between precision and recall. When the difference between precision and recall is large, the F1 value will be lower.

[0085] The main innovation of the present invention is to design a multi-view feature extractor to extract low-frequency and high-frequency features respectively, and to model the local similarity of users. A local similarity module is proposed to dynamically weight the final representation of users to solve the problem of heterogeneity imbalance caused by data sampling and robot disguise. Then a community and label-aware contrast learning network and a three-stage training method are designed to help learn more robust node representations and enhance the generalization ability of the model. Through testing on the data set, it is proved that the community robot detection method designed by the present invention has better performance.

[0086] It is obvious to those skilled in the art that the present invention is not limited to the details of the above-mentioned exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention; therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, and therefore it is intended to include all changes within the meaning and scope of the equivalent elements of the claims within the present invention; any figure mark in the claims should not be regarded as limiting the claims involved.

[0087] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A community robot detection method based on local perception and multi-view fusion, characterized in that: The following steps are included: S1: Design a feature encoding module to extract the multimodal features of an account and obtain the user's feature encoding; S2: The feature encoding module designed based on S1 uses the StarBlokc module and the HGT module to extract the multi-view features of the user; S3: Based on the dual-view features of S2, a local similarity perception module is designed to extract the local similarity scores of users, and the local similarity scores are used to fuse multi-view features; S4: Design a three-stage community and label-aware contrastive learning method to train the entire model, and use the model for community robot identification.

2. The community robot detection method based on local perception multi-view fusion according to claim 1 is characterized in that: A community robot account U contains four types of information: user description information D, user uploaded post list T, user meta information P, and user neighbor information N; among them, user description information, Represents a user description with L words; a list of posts uploaded by the user Indicates that a user has M posts and the i-th post Contains Q i words; user meta information P = {P num , P cat } represents the user's digital attribute set and user classification attribute set; the user's neighbor information represents a user's neighbor set constructed by R kinds of neighbor relationships, and Y represents the user's label; Y = 0 means the user is a human, and Y = 1 means the user is a robot; For user description features, the pre-trained language model Roberta is used to encode the user's description information: in, represents the expression of user description, D s represents the dimension of Roberta embedding; then the final expression is obtained through linear transformation: Where W D and b D Represent the learnable weights and biases, D d is the embedding dimension of the final expression, Relu represents the activation function, and the Leaky-Relu activation function is used; Similar to the user description feature, RoBERTa is used to obtain the embedding of a single tweet, and finally the average expression of all tweets is obtained to obtain the expression of the user tweet feature. Then a linear transformation is used to obtain its final expression; Among them, WT and bT are learnable parameters and biases, D t is the embedding dimension of the user’s Twitter feature; for the meta-information feature P = {P num , P cat }, and use a linear transformation to directly map it to the target dimension: in and Represent the learnable parameters, and Represent the bias, and Represent the embedding dimensions respectively; Finally, concatenate the above expressions as the user's node expression and use a linear transformation to map them to the target embedding dimension: Where W r and b r are the learnable weights and biases respectively, and D is the user node embedding dimension.

3. The community robot detection method based on local perception multi-view fusion according to claim 2 is characterized in that: Use StarBlock neural network and heterogeneous graph neural network GNN to extract multi-view features of the model respectively; first use StarBlock modules as high-frequency feature encoders to extract high-frequency features of the model: O l =O l-1 +dropout(Star(O ι-1 )) Among them, dropout is the dropout operation, Star is the output after passing through the Star module once, O l is the output of the star module; Use GNN to aggregate the commonalities between neighbors and extract the low-frequency features of users; use HGT as a low-frequency feature encoder to extract the low-frequency features of users: G m =G m-1 +dropout(HGT(G m-1 )) Where Gm represents the output of each GNN layer.

4. The community robot detection method based on local perception multi-view fusion according to claim 3 is characterized in that: The LocalSim local similarity perception module is used to perform weighted fusion of the two view features to obtain the final expression of the model. The local similarity of a node is calculated as follows: in is the neighbor set of node i, sim(·,·) is a similarity measurement function, and the Euclidean distance similarity measurement function is used here: yes(x i ,x j )=-‖x i -x j || 2 In order to enhance the discriminative ability of the MLP layer, a nonlinear expression is added to the local similarity: Where MLPls is a 2-layer perceptron, Sigmoid(*) is the sigmoid activation function; Then the mixed weight is obtained by calculating the local similarity of low-frequency features and high-frequency features respectively: α=sigmoid(MLP(LS(O l ))) β=sigmoid(MLP(LS(Gm))) Finally get the mixed expression: r ls =a·O l +β·G m 。 5. The community robot detection method based on local perception multi-view fusion according to claim 4 is characterized in that: A novel graph contrastive learning framework that uses community information and label information of the graph is used, and a three-stage training process is used to obtain a more robust graph node representation; represents the edge set of the entire graph G, where e ij Represents the connection between the i-th user and the j-th user; Use the Louvain method to divide the entire graph into communities and obtain community division information where c i is the community number of the i-th user; sample a subgraph G from G sub , contains n edges Compute contrastive learning loss on the subset; For connection ij , respectively, based on the label and the community to calculate the contrast loss: where x i and x j They represent the feature expressions of user i and user j respectively, and finally the total contrast loss is obtained: Where λ is the weight coefficient, which is used to control the proportion of the two contrast losses.

6. The community robot detection method based on local perception multi-view fusion according to claim 5 is characterized in that: Use 3 stages of training to train the entire model; Stage 1: High-frequency feature extractor training: Use an additional MLP to obtain the classification results to facilitate learning the classification loss. Set the output of the first layer of starblocks to O l ,but: Use contrastive learning to expand the differences in feature expressions of different labels. Here we use O l As the user's expression, the user's contrast loss is obtained The final loss is: Where μ is the weight coefficient; Stage 2: Low-frequency feature extractor training. In this stage, the user node information encoder and starsblocks are frozen, HGTs are trained, and the output Gm of HGTs is used as the user expression. An additional MLP is used stage2 To obtain the predicted label, the final classification loss We also get the contrastive learning loss Finally, the final loss is: Stage 3: Finally, use the mixed expression R ls As a final representation, an MLP is used to obtain the predicted labels; here we unfreeze the entire model and use only the classification loss: Among them G train is the training set graph.

Images