Social network fake account detection method based on structure-aware graph transformer

Abstract

The application provides a social network false account detection method based on a structure perception graph Transformer. First, based on each user node of a social network, the centrality, clustering coefficient and feature vector corresponding to the first several minimum non-zero eigenvalues of the graph Laplacian of the node are calculated, the node features are spliced with the original node features to form structure-enhanced features, the cosine similarity matrix between nodes is calculated based on the structure perception feature matrix, then, TopK sampling is performed on each target node to select the first K similar nodes to form an input sequence, the input sequence is input into the Transformer for modeling to obtain a new representation after modeling, and a learnable bias based on the initial similarity score and a learnable bias based on the shortest path distance between nodes are introduced into the multi-head attention of the Transformer. Subsequently, the new representation after modeling is processed by a full connection layer to obtain a final prediction result, and finally, a neural network model is trained based on the deep learning theory. The trained network model is used to obtain the prediction result on the test set. The method can predict false accounts in a social network under the condition that the attribute features and relationship topological structure of the social network users are known.

Description

Technical Field

[0001] This invention relates to the fields of deep learning, social network analysis, and graph data processing, specifically to a method for detecting fake accounts on social networks based on structure-aware graph Transformers. Background Technology

[0003] Early methods for detecting fake accounts primarily relied on rule engines and manual data statistical analysis, such as filtering based on account registration duration, follower / follower ratio, and number of posts per unit time. While simple and lightweight, these methods were easily bypassed when faced with accounts exhibiting deliberately fabricated behavioral characteristics. In recent years, graph neural networks (GNNs) have demonstrated powerful modeling capabilities in social network analysis. These methods treat users on social platforms as nodes and follower or interaction relationships as edges, constructing a social network. Then, models like Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs) are used to aggregate node features through neighborhood aggregation and message passing, thereby detecting fake accounts. These methods utilize network topology information to some extent, offering a significant improvement over pure attribute models. However, due to the inherent oversmoothing problem of GNNs, node representations tend to become consistent after multiple layers of stacking, making it difficult for the model to effectively aggregate information from distant but structurally highly similar nodes. Transformers, with their global attention mechanism, overcome the receptive field limitations of GNNs, enabling direct modeling of dependencies between arbitrary node pairs and achieving superior results in node classification compared to traditional GNNs. However, most existing graph Transformer methods still follow the approach in natural language processing, directly inputting nodes into the model after linearly arranging them according to random walks or neighbor sampling order, lacking explicit encoding of the topological position of nodes in the graph. On the other hand, existing Transformer models rely entirely on the inner product between node features when calculating attention scores, lacking the injection of prior topological constraints, resulting in an insufficiently robust attention distribution that is easily affected by noisy neighbors. Summary of the Invention

[0004] To address the shortcomings of existing methods for detecting fake accounts on social networks, this invention proposes a structure-aware graph-based Transformer method. By fusing node degree centrality, clustering coefficients, and Laplacian feature vectors, a structure-enhanced node representation is constructed. Then, a learnable bias based on shortest path distance and similarity scores is incorporated into the Transformer's attention mechanism, enabling the model to aggregate distant nodes with similar structures to the target node, thereby improving the accuracy and robustness of fake account detection on social networks. A method for detecting fake accounts on social networks based on structure-aware graph Transformer, comprising the following steps.

[0005] Step S1: Input the topology graph, feature matrix, and adjacency matrix of the social network. The social network graph represents the social relationships between user nodes in the network, such as comments, likes, and follows. The feature matrix consists of the initial vector features of all user nodes in the network.

[0006] Step S2: Calculate three types of structural features for each user node: degree centrality, clustering coefficient, and Laplacian eigenvector. Then, concatenate these three types of features with the original features of the feature matrix to obtain the structure-enhanced feature matrix. The feature matrix is ​​then Z-score standardized.

[0007] Step S3: Based on structural enhancement features Calculate the similarity matrix by using cosine similarity to calculate the similarity score between any two nodes, thus obtaining the similarity matrix S.

[0008] Step S4: For each node From the similarity matrix No. Select the K nodes with the highest similarity from the row to form a sampling set. The sequence length is K+1. Then from... Selecting from and The node feature vectors of the corresponding rows are stacked to obtain the Token matrix. The first position in the token matrix represents the target node itself, and the rest are arranged in descending order of similarity.

[0009] Step S5: The Token matrix described in step S4 The input L-layer Transformer encoder contains a multi-head self-attention mechanism and a feedforward network in each layer, employing residual connections and layer normalization, as shown in the formula: in, Indicates the first Layer Transformer, This indicates the bulls' self-attention. It is a feedforward network; In the multi-head self-attention computation at each layer, two learning bias terms can be introduced into the attention score: (1) Similarity bias: for target node and nodes in the sequence Based on the four similarities between two nodes It can be mapped to a learnable scalar bias. ; (2) Shortest path distance offset: Calculate the target node and nodes in the sequence Shortest path length between Then, it is mapped to a scalar bias through a learnable embedding table. Finally, the target node. and nodes in the sequence The formula for attention between them is: .

[0010] Step S6: After L layers of encoding, extract the output representation of the first position of the sequence (i.e., the target node itself). The prediction results are then output after passing through a fully connected layer.

[0011] Step S7: Based on the prediction results calculated in Step S6 and the corresponding real data, calculate the loss value of the network model prediction, and use the cross-entropy loss function to train the model end-to-end. Repeat steps S3-S7 to train the model. When the loss value of the network model converges to the minimum or the training reaches the specified number of iterations, the training ends, and the trained network model is output as the prediction model.

[0012] Step S8: Use the prediction model to test its performance on the test set.

[0013] Step S9: Calculate and output the final prediction result using a fully connected layer.

[0014] Because traditional Transformer models can only capture the multi-dimensional relationships between node features while ignoring the information-rich network topology, this invention introduces three-dimensional structural data—degree centrality, clustering coefficient, and Laplace eigenvectors—into the original node features, enabling the model to consider network structural features while encoding features. Degree centrality reflects the breadth of a user's connections in the network; normal users typically have moderate degree values, while fake accounts often exhibit extremely low degree (e.g., isolation) or abnormally high degree (e.g., actively following, liking, or commenting on others). The clustering coefficient measures the tightness of a user's social circle: real users' social relationships are homogeneous, and friends are interconnected, resulting in a high clustering coefficient; fake accounts have extremely low probabilities of neighbors following, liking, and commenting on each other, leading to a low clustering coefficient. The Laplace vector in the topology can encode the structural embedding of nodes in the global network, such as distinguishing between bridging nodes and edge nodes.

[0015] The Transformer model naturally treats input tokens as equally important. Although this invention uses TopK sampling to form the token sequence, there are differences in the similarity between different nodes. Using the similarity score as the model's attention bias can provide prior knowledge of similarity, thereby improving the model's convergence speed.

[0016] The shortest path distance bias uses topological distance information to adjust attention weights, reducing the interference of noisy neighbors with similar features but no structurally unrelated ones. This not only improves the robustness of the model under sparse labeling conditions, but also provides the model with distance priors between different nodes.

[0017] This method can predict fake accounts in a social network when the user attribute characteristics and relationship topology of the social network are known. Attached Figure Description

[0018] Figure 1 A flowchart of the traffic flow prediction method provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the overall algorithm framework of the present invention. Detailed Implementation

[0019] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that all other embodiments obtained by those skilled in the art without creative effort, as described herein, fall within the scope of protection of the present invention.

[0020] refer to Figure 1 and Figure 2 A method for detecting fake accounts on social networks based on structure-aware graph Transformer, comprising the following steps.

[0021] Step S1: Input the topology graph, feature matrix, and adjacency matrix of the social network, where the social network graph is represented as follows: , Represents a set of user nodes. Indicates the number of nodes. The set of edges is represented by the characteristic matrix. , As the initial feature dimension, The adjacency matrix is ​​represented as... , Represents a node and nodes There are edges (social relationships) between them. It indicates no boundary.

[0022] Step S2, for each user node The steps to calculate structural features are as follows: (1) Degree centrality: ,in Indicates the degree of a node; (2) Clustering coefficient: ,in For nodes The number of edges between neighbors; (3) Laplacian eigenvectors: Calculate the graphical Laplacian matrix Take the eigenvectors corresponding to the first s=16 smallest non-zero eigenvalues. Then, these three types of features are concatenated with the original features of the feature matrix to obtain the structure-enhanced feature matrix. The features are then Z-score standardized.

[0023] Step S3: Based on structural enhancement features Calculate the similarity matrix The cosine similarity score is used to calculate the similarity score between any two nodes. and nodes The calculation formula is: .

[0024] Step S4: For each node From the similarity matrix No. Select the K=7 nodes with the highest similarity from the row to form a sampling set. The sequence length is 8. Then from... Selecting from and The node feature vectors of the corresponding rows are stacked to obtain the Token matrix. The first position in the token sequence is the target node itself, and the rest are arranged in descending order of similarity.

[0025] Step S5: The Token matrix from step S5... The input L=2-layer Transformer encoder contains a multi-head self-attention mechanism and a feedforward network in each layer, employing residual connections and layer normalization. Each layer includes: (1) Multi-head self-attention, number of heads Dimensions per head Total hidden dimensions (2) Feedforward network, containing two linear transformation layers with dimensions of 128, 256, and 128 respectively, using the ReLU activation function. In the multi-head self-attention computation at each layer, two learning bias terms can be introduced into the attention score: (1) Similarity bias: for target node and nodes in the sequence Based on the four similarities between two nodes It can be mapped to a learnable scalar bias. ; (2) Shortest path distance offset: Calculate the target node and nodes in the sequence Shortest path length between Then, it is mapped to a scalar bias through a learnable embedding table. Finally, the target node and nodes in the sequence The formula for attention between them is: in, , For the projection matrix of the query and key, For the dimension of attention head.

[0026] Step S6: After L layers of encoding, extract the output representation of the first position of the sequence (i.e., the target node itself). The prediction result is output after the fully connected layer, and the formula is: .

[0027] Step S7: Based on the prediction results calculated in Step S6 and the corresponding real data, calculate the loss value predicted by the network model, and use the cross-entropy loss function to train the model end-to-end. The formula is: in, For real labels, This is the training set of nodes. The AdamW optimizer is used for parameter updates. Optimizer: AdamW, weight decay coefficient 1e−4, initial learning rate 1e−3; Learning rate scheduling: cosine annealing strategy, learning rate decays to 1e−5 after 100 training epochs; Regularization: Dropout=0.4 is added inside the encoder to prevent overfitting to graph data; Early stopping mechanism: if the validation set AUC does not improve for 10 consecutive epochs, training is terminated, and the optimal weight model is saved. Determine whether the current network model loss value has converged to the minimum, or whether the specified number of iterations has been reached; if not, return to step S3 and continue training the model; if it has been reached, end the training and output the trained model.

[0028] Step S8: Test the data to be predicted using the trained network model. The prediction method is the same as that used in steps S3 to S7.

[0029] Step S9: Output the trained model.

[0030] The performance of this model was evaluated using MAE (Mean Absolute Error), which improved the accuracy by approximately 25% compared to existing superior prediction algorithms. Furthermore, degree centrality, clustering coefficients, and Laplacian eigenvectors effectively encode the local and global topological structure of nodes. The introduction of similarity bias and distance bias enhances the interpretability of the deep learning model's prediction algorithm while also improving prediction accuracy.

[0031] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Other variations and modifications are possible without departing from the technical solutions described in the claims.

Claims

1. A method for detecting fake accounts on social networks based on structural enhancement graph Transformer, characterized in that, Includes the following steps: S1. Input the topological graph, feature matrix, and adjacency matrix of the social network, where the social network graph is represented as follows: , Represents a set of user nodes. Indicates the number of nodes. The set of edges is represented by the characteristic matrix. , Let be the initial feature dimension; the adjacency matrix is ​​represented as . , Represents a node and nodes There are edges (social relationships) between them. Indicates no boundary; S2, Calculate each user node Degree centrality Clustering coefficient and Laplacian eigenvectors And concatenate it with the original feature matrix X to obtain the feature-enhanced feature matrix. ; S3, Based on the structural enhancement feature matrix described in S2 Calculate the cosine similarity among all user nodes to obtain the similarity matrix. ; S4. For each target node From the similarity matrix described in S3 The Select the K nodes with the highest similarity in the row, and together with the target node i, form a token sequence. And from the structural enhancement feature matrix Extract the corresponding feature vectors according to their serial numbers and stack them to obtain the Token matrix. H i = v i v j 1 v j 2 , . . . . , v j k ; S5. Transfer the Token matrix from step S4. The input is encoded into an L-layer Transformer encoder to obtain a new representation of the Token. In each layer of the Transformer encoder's multi-head self-attention, a learnable bias based on similarity values ​​is simultaneously added to the attention score. and learnable bias based on shortest path distance between nodes The formula for calculating attention score is: ; S6. Based on the new representation of the Token described in S5, extract its first and second sequence vectors. As the final representation of target node i, it is input into a fully connected classification layer for classification, and the output is the prediction result of fake accounts, as shown in the formula: ; S7. The cross-entropy loss function and AdamW optimizer are used to train the neural network model end-to-end.

2. The method according to claim 1, characterized in that, The Laplacian eigenvectors described in step S2 The calculation method is as follows: Take the eigenvectors corresponding to the first s smallest non-zero eigenvalues, where s ranges from 8 to 32.

3. The method according to claim 1, characterized in that, The shortest path distance mentioned in step S5 For nodes With nodes The shortest path length on a social network graph.

4. The method according to claim 1, characterized in that, The clustering coefficients mentioned in step S2 The calculation formula is: ,in, Let be the number of edges between node i and its neighbors. For nodes The degree.

5. The method according to claim 1, characterized in that, Degree centrality described in step S2 The calculation formula is: ,in Indicates the degree of a node.

6. The method according to claim 1, characterized in that, In step S4, the value of k ranges from 5 to 15, the length of the Token sequence is K+1, and the other nodes in the sequence, except for the target node itself, are arranged in descending order of similarity value from high to low.

7. The method according to claim 1, characterized in that, In step S5, the number of layers L of the Transformer encoder is 2 to 4, and the number of heads for multi-head self-attention is 4 to 8.

8. The method according to claim 1, characterized in that, The loss function described in step S6 uses the cross-entropy loss function to train the model end-to-end, and the formula is as follows: L c e = - ∑ i ∈ V t r a i n [ y i l o g y i p r e d + 1 - y i l o g 1 - y i p r e d ] in, For real labels, This is the training set of nodes. The AdamW optimizer is used for parameter updates. The AdamW optimizer, with a learning rate of 5e-4, adjusts the weight parameters in the network model based on the gradient descent algorithm, reducing the loss value for the next prediction and thus further optimizing the network model.

9. A social network fake account detection system based on structural enhancement graph Transformer, used to implement the method according to any one of claims 1 to 8, characterized in that, include: (1) Data acquisition module, used to collect user attribute data and social relationship data from social platform databases; (2) Graph construction module, used to construct a social network graph based on the acquired data and generate a feature matrix. Adjacency Matrix ; (3) The structural feature calculation module calculates the degree centrality, clustering coefficient, and Laplacian eigenvector of each node, and concatenates them with the original features to generate a structural enhancement feature matrix. ; (4) Similarity calculation module, based on Calculate the cosine similarity matrix S between nodes; (5) Token construction module: Performs TopK sampling on each target node based on the similarity matrix to construct the token sequence and token matrix. ; (6) Transformer encoding module, which performs L-layer encoding on the Token matrix and simultaneously adds similarity bias and shortest path distance bias to the multi-head self-attention; (7) Classification output module, used to extract the output representation of the target node and output the probability of fake accounts after passing through the fully connected layer; (8) Model training module, used for end-to-end training with cross-entropy loss and AdamW optimizer, and finally outputs the trained model.

Images