Social robot detection method and system based on adaptive model learning

Through adaptive model learning and multimodal feature fusion, automatic search for model structure and adjust network layers, the problem of existing social robot detection methods relying on expert knowledge and heterogeneous camouflage interference is solved, and the detection performance and model generalization ability are improved .

CN118898526APending Publication Date: 2024-11-05Chinese People's Liberation Army Cyberspace Force Information Engineering University
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411082200.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-05

Smart Images

  • Figure CN118898526A_ABST
    Figure CN118898526A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of social robot detection, in particular to a social robot detection method and system based on adaptive model learning, and the method comprises the steps: carrying out the preprocessing of multi-modal features of each account, and obtaining the initial features of each account; abstracting each account as a node in the social relation topological graph, and constructing an undirected graph G according to the attention relation between the accounts; performing digital coding on the basic GNN model to form a genotype of a genetic algorithm, and searching a model structure by using the genetic algorithm in a search space with the maximum length of L; and optimizing the searched social robot detection model by minimizing the loss function, and detecting the social robot by using the trained model. According to the method, the number of layers of the network is deepened by automatically searching the model structure, the network structure does not need to be manually designed, the receptive field of the model is improved, and the risk of overfitting is effectively reduced by automatically searching and optimizing the model parameters.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of social robot detection, and in particular to a social robot detection method and system based on adaptive model learning. Background Art

[0002] Online social networks (OSNs) have become an integral part of people's lives. However, there are a large number of social robots in OSNs. The purposes of these automated programs include advertising promotion, topic speculation, emotional incitement, and other malicious behaviors, which have a negative impact on real users. Due to the ever-evolving nature of social robots, detecting these robots remains a persistent challenge. Early robot detection methods relied on extracting or designing features and training machine learning or neural network classifiers. These features covered various aspects, including the number of followers and friends, tweet frequency, and creation date. Subsequent research turned to extracting features from robot-generated posts to improve detection accuracy. Traditional feature- and text-based detection methods have three problems: first, the features used for detection are difficult to design and require high expert knowledge; second, the model is difficult to scale to ultra-high-dimensional and large-scale data; and third, the interactive relationship features between social robots are not utilized.

[0003] Compared with traditional feature-based methods, GNN-based methods can effectively exploit user interaction patterns and achieve superior detection performance. However, there are still two key issues to be addressed:

[0004] (1) Social robot classification models rely on experts to design network structures.

[0005] Social bot detection is essentially a node classification task. By constructing a network based on a given dataset, where some accounts are labeled and others are unlabeled, GNN can learn a robust model that effectively identifies the class labels of unlabeled accounts. Despite the popularity of GNN, the literature relies heavily on manual hyperparameters or architecture design to achieve optimal performance in bot detection. This leads to expensive manual work when a large number of models are used for various social account datasets. At the same time, when aggregating neighborhood node features, graph neural networks need to aggregate features through directly connected edges, and the k-th layer of graph neural networks is equivalent to the aggregation of the k-th hop neighbors of the nodes in the graph, which results in the inability of manually designed shallow neural networks to extract more global features, while too many layers will cause over-smoothing. To overcome such problems, some literature uses fixed and fine-grained subgraph neural networks to extract rules within the subgraphs where some nodes are located, which requires heavy reliance on experience or domain knowledge and is not effective enough.

[0006] (2) There is heterogeneous camouflage interference when aggregating a large number of neighboring nodes.

[0007] Since 2016, a new generation of social bots has emerged that blends human manipulation and automated behavior. This hybrid allows them to effectively disguise themselves and evade detection by traditional classifier-based platforms, resulting in an ongoing “cat and mouse game”. This dynamic has stimulated the continuous development of innovative solutions, disguises, and countermeasures. Although existing graph-based social bot detection methods have achieved advanced performance, most of them implicitly follow the homogeneity assumption, which assumes that users with the same tags are more likely to be connected and uses a low-pass filter to smooth user features within a neighborhood, resulting in similar predictions for neighboring nodes. Recent studies have revealed heterogeneous disguise behaviors in social bots, which deliberately imitate human accounts and disguise themselves by following a large number of real users, while suspicious features are smoothed by the normal features of neighboring real users, and inter-class edge interference is increased, thereby evading detection based on homogeneous graphs. As Figure 1 We describe such heterogeneous camouflage, where inter-class edges negatively impact the neighborhood aggregation of GNNs, making it more challenging to detect bots. Summary of the invention

[0008] The purpose of the present invention is to solve the problems existing in the prior art and to propose a social robot detection method and system based on adaptive model learning, which deepens the number of network layers and improves the receptive field of the model by automatically searching the model structure, while solving the problem that the GNN architecture classification model relies on expert knowledge.

[0009] To achieve the above purpose, the technical solution adopted is:

[0010] A social robot detection method based on adaptive model learning, comprising:

[0011] Preprocess the multimodal features of each account to obtain the initial features of each account;

[0012] Abstract each account as a node in the social relationship topology graph, and construct an undirected graph G based on the attention relationship between accounts;

[0013] The basic GNN model is digitally encoded to form the genotype of the genetic algorithm, and the genetic algorithm is used to search for the model structure in a search space with a maximum length of L;

[0014] The social robot detection model obtained by searching is optimized by minimizing the loss function, and the trained model is used to detect social robots.

[0015] According to the social robot detection method based on adaptive model learning of the present invention, further, when preprocessing the multimodal features of each account, the semantic features of the account are encoded using a natural language processing model, the community structure features of the account are encoded using a community detection model, and the attribute features of the account are encoded using One-hot encoding and normalization technology, and the features of the three modalities are spliced ​​together to form the initial features of the account.

[0016] According to the social robot detection method based on adaptive model learning of the present invention, the RoBERTa model is further used to encode the tweets and account description information and concatenate them together to form the semantic feature P S ; Use Z-score normalization to encode the category features in the attribute features, use One-hot to encode the category features in the attribute features, and finally concatenate to obtain the attribute feature P p ; Use DANMF social detection algorithm to extract community structure features P com ; Finally, the initial features of the lth account are composed of semantic features Attribute characteristics and community structure characteristics Splicing to get

[0017] According to the social robot detection method based on adaptive model learning of the present invention, further, the undirected graph G=(V, E, X), wherein V represents the set of all accounts, N represents the number of nodes, E represents the set of edges, and when there is a follow relationship between node i and node j, E i,j =1, otherwise, E i,j =0; X represents the initial feature of each node.

[0018] According to the social robot detection method based on adaptive model learning of the present invention, further, the basic GNN model uses three basic graph convolution blocks, namely, a 2-layer JK-Net block, a 2-layer residual GCN block and a 2-layer IncepGCN block, which are represented by "0", "1" and "2" respectively, and the model structure sequence is represented by S: S = {S1, S2, ..., S l},l <L,Si∈{0,1,2}。

[0019] According to the social robot detection method based on adaptive model learning of the present invention, further, each model structure sequence is equivalent to a genotype of a genetic algorithm, and the genetic algorithm specifically includes:

[0020] Initialize the population P t , each individual represents a social robot detection model structure, and each individual is represented by a gene sequence g i express;

[0021] Evaluate the fitness of each individual f(g i );

[0022] A binary tournament method with replacement is used to select parent individuals with higher fitness.

[0023] Generate the offspring population through crossover mutation operation t ;

[0024] The parent population P t and the offspring population Offspring t Merge to form a candidate population, use the binary tournament method without replay for environmental selection, and select individuals with higher fitness from the candidate population to enter the new population;

[0025] Repeat the above parent selection, genetic operation and environment selection process until the preset termination condition is met;

[0026] The individual with the highest fitness in the last generation of the population is taken as the optimal solution; it is mapped into a social robot detection model through a decoding strategy, including network topology, connection method and hyperparameter configuration.

[0027] According to the social robot detection method based on adaptive model learning of the present invention, further, the process of detecting the social robot using the social robot detection model obtained by searching is as follows:

[0028] First, a layer of graph convolution is used to aggregate the features of graph G to obtain the hidden state H1;

[0029] Then, the hidden state H1 is encoded using the social robot detection model obtained by automatic search, and the node features are aggregated through a layer of GCNConv for message passing to obtain the final representation of the node;

[0030] Finally, the softmax function is used to predict the category of each node to detect social robots.

[0031] Furthermore, the present invention also proposes a social robot detection system based on adaptive model learning, comprising:

[0032] The initial feature module is used to preprocess the multimodal features of each account to obtain the initial features of each account;

[0033] The social relationship graph construction module is used to abstract each account into a node in the social relationship topology graph and construct an undirected graph G based on the attention relationship between accounts;

[0034] The model structure search module is used to digitally encode the basic GNN model to form the genotype of the genetic algorithm, and use the genetic algorithm to search for the model structure in a search space with a maximum length of L;

[0035] The model training module is used to optimize the social robot detection model obtained through search by minimizing the loss function, and use the trained model to detect social robots.

[0036] The beneficial effects achieved by adopting the above technical solution are:

[0037] 1. In order to address the limitations of manually designed network structures, the present invention proposes a social robot detection method based on adaptive model learning, AML4BD, which introduces automatic search model structure to solve the problem that the GNN architecture classification model relies on expert knowledge.

[0038] 2. In order to solve the problem of heterogeneous camouflage interference of large-scale neighbor nodes, the present invention adopts multiple strategies; first, by integrating multimodal features, including community structure features, semantic features and attribute features, for example, when social robots establish a large number of connections with real users, it is difficult to detect them by structural features alone, and the comprehensive use of multimodal features can effectively meet this challenge. Secondly, the basic GNN model can be automatically stacked according to the characteristics of data distribution, the number of network layers can be deepened, and low-order and high-order information can be integrated in an adaptive manner to improve the receptive field of the model, thereby effectively reducing the impact of heterogeneous camouflage interference.

[0039] 3. The present invention can effectively reduce the risk of overfitting by automatically searching and optimizing model parameters. Traditional manual parameter adjustment methods tend to rely on specific training sets, resulting in poor performance of the model on new data. The social environment is highly dynamic and changeable. Robot accounts and human accounts on different platforms or even on the same platform in different time periods have different behavior patterns. The present invention can dynamically adjust the model structure and parameters based on new data. This adaptive ability enables the model to continuously learn and adapt to new social situations and maintain a high level of generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention, wherein the drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.

[0041] Figure 1 It is a schematic diagram of GNN aggregation under heterogeneous camouflage;

[0042] Figure 2 is a flow chart of a social robot detection method based on adaptive model learning according to an embodiment of the present invention;

[0043] Figure 3 It is a basic framework diagram of a social robot detection method based on adaptive model learning according to an embodiment of the present invention;

[0044] Figure 4 is a network model structure diagram corresponding to the code "01221" of an embodiment of the present invention;

[0045] Figure 5 is a T-SNE visualization graph of the embedding vectors of each model in the embodiment of the present invention on the Twibot-20 dataset;

[0046] Figure 6 This is an embodiment of the present invention that uses a genetic algorithm to model search results in different search spaces on the Twibot-20 dataset. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings of specific embodiments of the present invention to clearly and completely describe the exemplary scheme of the embodiment of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field.

[0048] Based on the genetic algorithm, a social robot detection method based on adaptive model learning is proposed, in which the structure of the GNN is searched in a variable-length search space and the hyperparameters of the model are searched in a fixed-length search space. Figure 2 and Figure 3 As shown, the following steps are included:

[0049] Step S101: Preprocess the multimodal features of each account, encode the semantic features of the account using a natural language processing model, encode the community structure features of the account using a community detection model, encode the attribute features of the account using One-hot encoding and normalization technology, and splice the features of the three modalities together to form the initial features of each account.

[0050] Specifically, in social networks, accounts contain rich information, including tweet content, personal attributes, and social relationships. The social robot detection method AML4BD of this embodiment utilizes multiple different types of features. Studies have found that a single type of feature is easily imitated by robots. For example, Cresci et al. pointed out that new social robots can copy personal profile information of real accounts, making it difficult to distinguish between robots and real users in terms of attribute characteristics. In addition, the research of Fazil et al. shows that when robot accounts establish a large number of social relationships with real user accounts, the effectiveness of traditional graph-based detection methods may be affected. Therefore, in order to improve the accuracy of detection, account semantic features, attribute features, and community structure features are comprehensively considered.

[0051] In Twitter, account description information is a paragraph of text that the account owner uses to briefly introduce himself or his account. This description is usually used to summarize the account owner's identity, occupation, hobbies, or other related information. It can help other users understand the content and positioning of the account more quickly. This embodiment uses the method mentioned in BotRGCN to vectorize semantic features, that is, using the RoBERTa model to encode tweets and account description information and splice them together to form semantic features P S For the description information of the lth account, it represents the vector Such as the formula:

[0052]

[0053] Among them, d i represents the i-th word in the account description information, n represents the length of the account description information, σ represents the activation function, and W d and b d is a learnable parameter.

[0054] Use Z-score normalization to encode the category features in the attribute features, use One-hot to encode the category features in the attribute features, and finally concatenate to obtain the attribute feature P p .

[0055] Using DANMF social detection algorithm to extract community structure features P com , as shown in the following formula:

[0056]

[0057] Among them, W com and d com is a learnable parameter, is the community structure feature representation vector of the l-th account, DANMF(G) represents the extraction of community structure features from graph G, and its input is the graph adjacency matrix of the entire graph.

[0058] Finally, the initial features of the lth account are composed of semantic features Attribute characteristics and community structure characteristics Splicing to get

[0059] Step S102: abstract each account as a node in a social relationship topology graph. In order to more comprehensively capture the relationship between nodes, an undirected graph G is constructed based on the attention relationship between accounts.

[0060] Undirected graph G = (V, E, X), where V represents the set of all accounts, Let \(N\) denote the number of nodes and \(E\) denote the set of edges. When there is a following relationship between node \(i\) and node \(j\), \(E\) i,j = 1; otherwise, \(E\) i,j = 0; \(X\) represents the initial features of each node, where \(X = [P\) S , \(P\) p , \(P\) com . After constructing the social relationship topology graph, it is necessary to design a model to learn the node representation vectors suitable for social robot detection to accurately detect robot accounts.

[0061] Step S103: Digitally encode the basic GNN model to form the genotype of the genetic algorithm, and search for the model structure using the genetic algorithm within a search space with a maximum length of \(L\).

[0062] Generally, when the genetic algorithm performs the crossover operation, it exchanges the genes of two parents to generate new individuals in the next generation. In order to utilize the advantages of the genetic algorithm to adaptively search for the network structure, balance the over-smoothing problem of deep graph convolution, and extract more global features, this example adopts a variable-length coding strategy to encode the model structure, representing it as a digital sequence pattern. Considering the limitation of computing resources, a search space based on three basic graph convolution blocks with a maximum length of \(L\) is constructed. The purpose of such a design is to adaptively learn a social robot detection model adapted to the data distribution. The model searches for the model structure and model hyperparameters based on the genetic algorithm, as follows:

[0063] The three basic graph convolution blocks are the 2-layer JK-Net block, the 2-layer residual GCN block, and the 2-layer IncepGCN block, which are represented by "0", "1", and "2" respectively. The model structure sequence is represented by \(S\): \(S=\{S_1,S_2,\cdots,S\) l}), \(l < L\), \(S_i\in\{0,1,2\}\). Taking Figure 4 as an example, its model structure sequence is composed of a 2-layer JK-Net block, a 2-layer residual GCN block, a 4-layer IncepGCN block, and a 2-layer residual GCN block connected in series. When there are consecutive identical encodings in the encoding sequence, twice the number of consecutive repetitions represents the number of layers of the module.

[0064] Note that in order to maintain the dimension of the hidden layer, a network layer is added after the JK-Net block and the IncepGCN block to reduce the dimension.

[0065] The setting of hyperparameters is also a key factor affecting the performance of the neural network. AML4BD searches for several common hyperparameters such as the dimension of the hidden layer, the learning rate, the Dropout rate, and the weight decay in the defined hyperparameter search space; each hyperparameter is directly encoded with real numbers, and the search space of the hyperparameters is represented by a set of constant values.

[0066] Each model structure sequence is equivalent to the genotype of the genetic algorithm. In order to evaluate the survival advantage of each model structure sequence, AML4BD first converts the individual's genotype into the corresponding network model (i.e., the phenotype in the genetic algorithm) through a decoding strategy; in the decoding strategy, the variable-length model structure sequence is converted into the corresponding network model, and the hyperparameters of the model are represented by a fixed-length encoding. Then, the network model obtained by adaptive search of the genetic algorithm is trained with training data, and its loss on the validation set is calculated to evaluate the fitness of each individual (i.e., each search result).

[0067] The specific steps of genetic algorithm are as follows:

[0068] (1) Population Initialization P t , each individual represents a social robot detection model structure, and each individual is represented by a gene sequence g i express;

[0069] (2) Fitness Evaluation: evaluate the fitness of each individual f(g i ), the fitness function is designed based on the detection accuracy of the robot detection task.

[0070] (3) Parent Selection uses a binary tournament method with replacement to select parent individuals with higher fitness:

[0071] From the current population P t Randomly select two individuals g1 and g2.

[0072] Compare their fitness values ​​and select the individual with higher fitness as one of the parents.

[0073] Repeat the above process until the required number of parents are selected.

[0074]

[0075] (4) Genetic Operations

[0076] Crossover: Randomly select a crossover point c from two parent individuals, exchange the gene fragments on both sides of the crossover point of the parent individuals, and generate new offspring individuals.

[0077] Crossover(g1,g2,c)=(g1[1:c]||g2[c+1:n],g2[1:c]||g1[c+1:n])

[0078] Mutation: with a certain probability pm , randomly modify certain gene positions of offspring individuals.

[0079]

[0080] (5) Environmental Selection: The parent population P t and the offspring population Offspring t The candidate populations are merged to form a candidate population, and the individuals with higher fitness are selected from the candidate populations to enter the new population using the binary tournament method without replay.

[0081] Randomly select two individuals from the candidate population.

[0082] Compare their fitness values ​​and select individuals with higher fitness to enter the new population.

[0083] Repeat the above process until a new population is selected.

[0084]

[0085] (6) Iterative Evolution: The parent selection, genetic operation, and environment selection process are repeated until the preset termination condition is met, such as reaching the maximum number of iterations T or the average fitness of the population reaches the threshold θ.

[0086]

[0087] (7) Decoding the Optimal Solution: Decode the individual g with the highest fitness in the last generation of population best As the optimal solution; it is mapped into a social robot detection model through a decoding strategy, including network topology, connection method and hyperparameter configuration.

[0088] By adopting the binary tournament method with replacement to select parents, generating a rich offspring population through crossover and mutation operations, and performing environmental selection through the binary tournament method without replay, AML4BD is able to retain excellent individuals in the population while exploring new solution spaces, achieving a balance between population diversity and convergence speed, and improving the efficiency and effectiveness of the search.

[0089] Step S104: Optimize the social robot detection model obtained through search by minimizing the loss function, and use the trained model to detect the social robot.

[0090] The above steps S101-S103 respectively obtain the initial representation X of each account init , the social relationship graph G = (V, E, Xinit ) and the graph model Net* obtained by automatic search based on genetic algorithm.

[0091] First, a layer of graph convolution is used to aggregate the features of graph G to obtain the hidden state H1:

[0092] H1=GCNConv(X init ).

[0093] Then, the graph model Net obtained by automatic search is used * The hidden state H1 is encoded, and then a layer of GCNConv is used to aggregate node features through message passing to obtain the final representation of the node:

[0094] X Final =GCNConv(Net * (H1)).

[0095] Finally, the softmax function is used to predict the category of each node to detect social robots:

[0096]

[0097] in, Indicates the predicted node category.

[0098] This embodiment selects the cross entropy loss function commonly used in binary classification models as the loss function of the social robot detection model:

[0099]

[0100] The goal during training is to minimize the average cross entropy loss L. The model parameters θ will be updated along the negative gradient direction of the loss function to reduce the value of the loss function:

[0101]

[0102] Where η is the learning rate, is the gradient of the loss function L with respect to the model parameters θ.

[0103] By iteratively performing gradient descent updates, the model will continue to adjust its parameters so that the predicted probability As close as possible to the true label y i This process will minimize the cross entropy loss and improve the performance of the model on the binary classification task.

[0104] Corresponding to the above method, this embodiment also proposes a social robot detection system based on adaptive model learning, which is characterized by comprising:

[0105] The initial feature module is used to preprocess the multimodal features of each account to obtain the initial features of each account.

[0106] The social relationship graph construction module is used to abstract each account into a node in the social relationship topology graph and construct an undirected graph G based on the attention relationship between accounts.

[0107] The model structure search module is used to digitally encode the basic GNN model to form the genotype of the genetic algorithm, and use the genetic algorithm to search for the model structure in a search space with a maximum length of L.

[0108] The model training module is used to optimize the social robot detection model obtained through search by minimizing the loss function, and use the trained model to detect social robots.

[0109] The present invention has conducted a large number of experiments on multiple data sets to verify the effectiveness of the proposed method. The performance of the proposed method has been verified by comparative experiments with existing methods. In addition, the advantages of the present model (AML4BD) in the social robot detection task are intuitively illustrated based on the analysis of the embedding vectors of various models using T-SNE visualization technology. The influence of the automatic search network structure and hyperparameters on the model performance is verified by search space ablation and feature ablation experiments, so that the effectiveness of the model can be evaluated more comprehensively.

[0110] ①Dataset

[0111] Three public and graph-structured datasets are used to evaluate the Twitter robot detection model: Cresci-2015, Twibot-20 and Twibot-2, as shown in Table 1.

[0112] Table 1 Dataset

[0113] Dataset Account Robot Accounts Real user accounts Cresci-2015 5301 3351 1950 Twibot-20 229573 5273 6589 Twibot-22 1000000 139943 860057

[0114] Here is a brief overview of each dataset:

[0115] Cresci-2015: This dataset is a social robot detection dataset released by Cresci et al. in 2015. It is mainly used to study and analyze robot accounts on social media (especially Twitter), which are usually used for various purposes, including but not limited to increasing the number of followers, spreading specific information, and influencing public opinion.

[0116] Twibot-20: This dataset contains 229,580 users and 227,979 edges. It includes 11,826 accounts that are divided into automated users and real users. A notable feature of this dataset is that each account has at most 20 neighbors (10 followers and 10 followings). The dataset provides information on the friend and follow relationships between these users.

[0117] Twibot-22: This dataset shows richer relationship types between accounts, as well as more tweets and attributes. It includes 1 million Twitter accounts, including 139,943 bots and 860,057 human users, with a total of 86,764,167 tweets and 170,185,937 follow relationships. All labeled users are utilized to construct the user social graph of these datasets.

[0118] For Twibot-20 and Twibot-22, 16 user attribute features are used, including protected, geo_enabled, verified, contributors_enabled, is_translator, is_translation_enabled, profile_background_title, profile_user_background_image, has_extended_profile, default_profile, and default_profile_image. Two 768-dimensional user description features and user tweet features extracted using RoBERTa are combined.

[0119] ②Baseline model

[0120] In order to verify the effectiveness of the proposed model AML4BD, it is compared with several baseline models Node2vec, Bot2vec, Accu2vec, BotGCN, BotGAT, ResGCN, JK-Net, IncepGCN, BotRGCN, SATAR, and RF-GNN, among which RF-GNN, Accu2vec, and SATAR are the latest social robot detection models. Node2vec only encodes structural features. Compared with Node2vec, Bot2vec and Accu2vec additionally consider community structure features and use the Louvain method to extract community structure features. In the feature encoding stage, AML4BD uses the same method as SATAR and BotRGCN to extract semantic and attribute features.

[0121] In addition, AML4BD also uses a community detection algorithm based on non-negative matrix decomposition to extract community structure features. GCN, GAT, and BotRGCN share similar structures, the main difference is that they use different GNN backbone models. In contrast, SATAR enhances the multi-type feature aggregation method by introducing a multi-head attention mechanism for feature aggregation. AML4BD also combines genetic algorithms to search for network structures and hyperparameters in the network, and can adaptively select appropriate network structures to detect social robots according to the characteristics of the data set.

[0122] Node2vec: Node2vec uses a flexible strategy to explore the similarities between different types of nodes. Specifically, it uses two parameters p and q to control the selection of nodes during the random walk. Parameter p controls the possibility of returning to the previous node, while parameter q controls the possibility of moving to a node far away from the previous node. By adjusting these two parameters, a balance can be found between "exploration" and "exploitation" to better capture the similarities between nodes. Similarly, it was used by Pham et al. for social robot detection tasks.

[0123] Bot2vec: Bot2vec is a graph representation learning algorithm proposed by Pham et al. in 2022 based on the improvement of Node2vec. It takes into account the community aggregation of robots. First, the community is extracted using the community detection algorithm, and then different wandering rules are used within and between communities to characterize the different influence of accounts outside the community and accounts within the community on a specific account.

[0124] Accou2vec: It divides the social graph into multiple subgraphs, designs community walking rules to control the difference between walking within a community and walking between communities, and finally uses graph representation learning to learn the representation vector of each account.

[0125] GCN: It pioneered the convolution operation from image processing to processing graph structured data. It adopts the layer-by-layer propagation principle rooted in the graph structure and promotes the acquisition of node representation by aggregating adjacent information.

[0126] BotGAT: BotGAT is a social robot detection model based on GAT encoding. Compared with the GCN model, GAT introduces an attention mechanism that can pay more attention to important features in the graph. Its performance in social robot detection tasks is better than GCN.

[0127] ResGCN: In standard GCN, node representation is learned by aggregating and transferring information between nodes and their neighbors. ResGCN improves the performance of the model by introducing residual connections. Specifically, residual connections can alleviate the over-smoothing problem of deep graph neural networks and increase the receptive field of convolution kernels, thereby improving model performance.

[0128] JK-Net: JK-Net is mainly proposed to solve the over-smoothing problem of deep graph neural networks. Traditional GCN models may cause over-smoothing of information in deep layers, causing node representations to become too similar, thereby losing fine-grained information. JK-Net introduces the Jumping Knowledge mechanism to combine node representations at different levels, thereby better capturing multi-level graph structure information.

[0129] IncepGCN: IncepGCN is a model that combines the features of the Inception module and the graph convolutional network (GCN), aiming to improve the performance of graph data processing tasks. IncepGCN can better capture structural information of different scales in the graph by drawing on the multi-scale feature extraction idea of ​​the Inception network and combining the graph structure characteristics of GCN.

[0130] BotRGCN: BotRGCN is a social robot detection model based on RGCN. It uses RGCN to encode nodes and can process directed graphs, while GCN and GAT are usually only applicable to undirected graphs.

[0131] SATAR: STATA is a self-supervised social robot detection model proposed by Feng et al. in 2021. It simultaneously utilizes the semantic information, attribute information, and structural information of the account.

[0132] RF-GNN: RF-GNN is a social robot detection model based on random forest enhanced graph neural network. Different subgraphs are constructed as different training sets through node sampling, feature selection and edge dropout. GNN base classifiers are trained with various subgraphs, and the remaining features are used to train a fully connected neural network (FCN).

[0133] Node2vec and Bot2vec only use structural features, while GCN, GAT, BotRGCN, SATAR, RF-GNN, and AML4BD all use attribute features, semantic features, and structural features.

[0134] ③ Experimental results

[0135] Experimental configuration: The experiment was conducted on a server with two A800-80GB GPUs, 100GB memory, 14-core CPU model Xeon(R) Gold 6348, and Ubuntu 20.04. For the partitioning of the two datasets Twibot-20 and Twibot-22, in order to make a fair comparison, the partitioning method (7:1:2) consistent with the BotRGCN, SATAR and other models was adopted. In the experiment, the maximum offspring of each initial population was set to 20, and the maximum genotype length of each offspring was set to 30. The genetic algorithm was used to search in the basic graph convolution module of [0,1,2] to obtain the model structure and hyperparameters, and learn the social robot detection model suitable for the training dataset used. Among them, 0 represents 2-layer JK-Net, 1 represents 2-layer ResGCN, and 2 represents 2-layer IncepGCN.

[0136] Table 2 Accuracy, F1 and Precision values ​​of each model on Cresci-2015, Twibot-20 and Twibot-22 datasets

[0137]

[0138]

[0139] Table 2 shows the detection accuracy, F1 value and precision of each model on the Cresci-2015 dataset, Twibot-20 dataset and Twibot-22 dataset. On the Cresci-2015 dataset, each model achieved a detection accuracy of more than 95%, which may be because the Cresci-2015 dataset is a classic social robot detection dataset, in which the traces of robot account automation are relatively obvious. The Cresci-2015 dataset uses the data processed in the Bot2vec paper, which only contains the interaction graph information between accounts. The GCN-based method sets the initial features of the account to a fixed-length 0 vector on this dataset. It can be seen that on the Cresci-2015 dataset, Bot2vec, Accu2vec, and AML4BD of the present invention, which simultaneously consider the neighbor structure features and community structure features, all achieve relatively good detection results. Among them, Accu2vec achieved the best results on the Cresci-2015 dataset, which may be due to the fact that its advanced community wandering rule design makes full use of community structure information. In AML4BD, community structure features are used as a kind of multimodal features. At this time, community structure features have achieved a certain effect on social robot detection, but the effect is not as obvious as that of community structure in Acco2vec. On the Twibot-20 and Twibot-22 datasets, Node2vec, Bot2vec and Accu2vec, which only use graph structure features, performed poorly. This may be due to the fact that in Twibot-20 and Twibot-22 data, robot accounts have more interactions with normal human accounts, making it difficult to identify robot accounts only by structural features. AML4BD achieved the best results on the Twibot-20 and Twibot-22 datasets, showing the advantages of automatic graph learning.

[0140] In order to visually observe the training effect of each model, T-sne 2D visualization technology is used on the Twibot-20 dataset to observe the clustering effect of each model embedding vector (such as Figure 5 ), and the corresponding homogeneity score is given. The stronger the homogeneity score, the better the clustering effect. Among them, 10,000 nodes are randomly selected from the Twibot-20 dataset for visualization when making the T-sne graph. From the homogeneity score, it can be seen that the embedding vector obtained by the model training of the present invention is more conducive to the social robot detection task.

[0141] Ablation experiment:

[0142] Model search space ablation: First, we conducted a model search space ablation experiment, and the results are as follows: Figure 6 And as shown in Table 3.

[0143] from Figure 6 It can be seen from the figure that when the search space is only a single GNN model, the social robot detection performance of the searched model is poor compared to the detection performance of the model searched in a richer search space. This shows that different basic GNN models can learn different features that are conducive to detecting social robots, and the features extracted by various basic GNNs can complement each other. When the search space is ResGCN+JK-Net+IncepGCN([0,1,2]), the best performance is achieved on the Twibot-20 dataset. The search space in Table 3 represents the search space of the genetic algorithm, and genotype is the optimal genotype (i.e., model structure) searched by the genetic algorithm in a specific search space. hidden_dim, dropout, lr and weight decay are the hidden layer dimensions, dropout rate, learning rate and weight decay rate of the optimal social robot detection model searched by the genetic algorithm in a specific search space. It can be seen from Table 3 that in the search space of a single GCN block, ResGCN(“1”) performs better, which shows that ResGCN is more suitable for social robot detection tasks than JK-Net and IncepGCN. When the search space increases, the detection performance is improved compared to the single search space. For example, when the search space is [0,1], the detection accuracy is improved compared to the search space [0] or [1]. This shows that the combination of different GCN modules can improve the social robot detection performance, and that different modules can learn different features that are beneficial to social robot detection. The best social robot detection performance was achieved when the search space was [0,1,2]. The model structure layer was ResGCN+2 layers of JK-Net+1 layer of IncepGCN+1 layer of JK-Net, indicating that on the Twibot-20 dataset, the combination of the three basic GCN modules can maximize the advantages of each model in detecting social robots.

[0144] In the social robot detection task, most GNN-based models (such as GCN, BotRGCN, DeepSBD, etc.) usually use 2-layer GNN, and the number of layers of the GNN learned by automatic machine learning in the present invention is different from the number of layers of the general GNN. In Table 3, the detection accuracy of the 2-layer ResGCN is 0.8301, while the number of layers of the ResGCN learned by automatic learning is 8 layers, and the detection accuracy is 0.8338. The detection accuracy of the 2-layer JK-Net is 0.8286, while the number of layers of the JK-Net learned by automatic learning is 6 layers, and the detection accuracy is 0.8312. This shows that the model structure and parameter configuration learned by automatic learning have better detection performance than the traditional fixed parameter configuration.

[0145] Table 3. Model search space ablation results on the Twibot-20 dataset

[0146]

[0147] Feature ablation experiment:

[0148] When the model search space is [0,1,2], feature ablation experiments are conducted on the Twibot-20 dataset. The meanings of the parameters in Table 4 are the same as those in Table 3. As can be seen from Table 4. When only GCNs are used, only the local neighborhood structure information of the account can be captured through message passing. At this time, the detection accuracy is only 0.6315, and its detection performance is slightly lower than bot2vec and Accu2vec, which can simultaneously capture macro community structure features and neighborhood structure features. On the basis of GCN, adding any one of the community structure feature C, account attribute feature P, and account text feature T can improve the detection performance of the model, which shows that the various features of the account complement each other in social robot detection. When the community structure feature C, account attribute feature P, and account text feature T are added to GCN at the same time, the model achieves the best performance, indicating that multimodal feature fusion can improve the detection performance of the model's social robot and avoid interference caused by the manipulation of a single feature by the account manipulator.

[0149] Table 4 Feature ablation results on the Twibot-20 dataset under the condition that the search space is [0, 1, 2]

[0150]

[0151]

[0152] This paper proposes a social robot detection method based on adaptive model learning. This method uses automatic search of model structure to deepen the number of network layers, and uses automated search and optimization of model parameters to effectively reduce the risk of overfitting. This ensures that the graph neural network does not need to manually design the network structure when aggregating neighbor information, improves the receptive field of the model and the generalization ability of the model. A large number of experiments on three public datasets show that the method of the present invention outperforms the most advanced models.

[0153] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0154] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0155] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.

[0156] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.

[0157] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A social robot detection method based on adaptive model learning, characterized in that: Include: Preprocess the multimodal features of each account to obtain the initial features of each account; Abstract each account as a node in the social relationship topology graph, and construct an undirected graph G based on the attention relationship between accounts; The basic GNN model is digitally encoded to form the genotype of the genetic algorithm, and the genetic algorithm is used to search for the model structure in a search space with a maximum length of L; The social robot detection model obtained by searching is optimized by minimizing the loss function, and the trained model is used to detect social robots.

2. The social robot detection method based on adaptive model learning according to claim 1, characterized in that: When preprocessing the multimodal features of each account, the natural language processing model is used to encode the semantic features of the account, the community detection model is used to encode the community structure features of the account, and the one-hot encoding and normalization technology are used to encode the attribute features of the account. The features of the three modalities are spliced ​​together to form the initial features of the account.

3. The social robot detection method based on adaptive model learning according to claim 2 is characterized in that: Use the RoBERTa model to encode tweets and account description information and concatenate them together to form the semantic feature P S ; Use Z-score normalization to encode the category features in the attribute features, use One-hot to encode the category features in the attribute features, and finally concatenate to obtain the attribute feature P p ; Use DANMF social detection algorithm to extract community structure features P com ; Finally, the initial features of the lth account are composed of semantic features Attribute characteristics and community structure characteristics Splicing to get 4. The social robot detection method based on adaptive model learning according to claim 1, characterized in that: Undirected graph G = (V, E, X), where V represents the set of all accounts, N represents the number of nodes, E represents the set of edges, and when there is a follow relationship between node i and node j, E i,j =1, otherwise, E i,j =0; X represents the initial feature of each node.

5. The social robot detection method based on adaptive model learning according to claim 1, characterized in that: The basic GNN model uses three basic graph convolution blocks, namely, a 2-layer JK-Net block, a 2-layer residual GCN block, and a 2-layer IncepGCN block, which are represented by "0", "1", and "2" respectively. The model structure sequence is represented by S: S = {S1, S2, ..., S l },l <L,Si∈{0,1,2}。 6. The social robot detection method based on adaptive model learning according to claim 5, characterized in that: Each model structure sequence is equivalent to the genotype of the genetic algorithm, which specifically includes: Initialize the population P t , each individual represents a social robot detection model structure, and each individual is represented by a gene sequence g i express; Evaluate the fitness of each individual f(g i ); A binary tournament method with replacement is used to select parent individuals with higher fitness. Generate the offspring population through crossover mutation operation t ; The parent population P t and the offspring population Offspring t Merge to form a candidate population, use the binary tournament method without replay for environmental selection, and select individuals with higher fitness from the candidate population to enter the new population; Repeat the above parent selection, genetic operation and environment selection process until the preset termination condition is met; The individual with the highest fitness in the last generation of the population is taken as the optimal solution; it is mapped into a social robot detection model through a decoding strategy, including network topology, connection method and hyperparameter configuration.

7. The social robot detection method based on adaptive model learning according to claim 1, characterized in that: The process of detecting social robots using the social robot detection model obtained through search is as follows: First, a layer of graph convolution is used to aggregate the features of graph G to obtain the hidden state H1; Then, the hidden state H1 is encoded using the social robot detection model obtained by automatic search, and the node features are aggregated through a layer of GCNConv for message passing to obtain the final representation of the node; Finally, the softmax function is used to predict the category of each node to detect social robots.

8. A social robot detection system based on adaptive model learning, characterized in that: Include: The initial feature module is used to preprocess the multimodal features of each account to obtain the initial features of each account; The social relationship graph construction module is used to abstract each account into a node in the social relationship topology graph and construct an undirected graph G based on the attention relationship between accounts; The model structure search module is used to digitally encode the basic GNN model to form the genotype of the genetic algorithm, and use the genetic algorithm to search for the model structure in a search space with a maximum length of L; The model training module is used to optimize the social robot detection model obtained through search by minimizing the loss function, and use the trained model to detect social robots.

9. An electronic device, characterized in that: include: at least one processor, and a memory coupled to the at least one processor; The memory stores a computer program, and the computer program can be executed by the at least one processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 can be implemented.

Citation Information

Cited By

  • Robot user detection method, system and equipment based on combination of large language model and user relation network analysis and medium

    CN119669695A

  • Social robot detection method based on uncertainty driving and agent dynamic arrangement, electronic equipment and storage medium

    CN121958892A

  • Social robot detection method based on uncertainty driving and agent dynamic arrangement, electronic device and storage medium

    CN121958892B