Two-channel social network abnormal user detection method based on mutual distillation

By designing multi-scale node attribute reconstruction module and connection-enhanced topological structure reconstruction module in social networks, and adopting mutual distillation mechanism, the DualAD framework solves the mutual interference problem between node attribute and topological structure abnormal detection in the prior art. , significantly improves the accuracy and performance of abnormal user detection.

CN119025969APending Publication Date: 2024-11-26HENAN UNIVERSITY
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411137589.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing social network abnormal user detection methods have mutual interference problems when identifying node attribute abnormalities and topological structure abnormalities, resulting in performance degradation. It is difficult for a single model to optimize two abnormal detections at the same time, resulting in a "tug-of-war" phenomenon and reducing overall performance.

Method used

A dual-dimensional graph neural network anomaly detection framework (DualAD) based on mutual distillation is proposed. By designing a multi-scale node attribute reconstruction module and a connection-enhanced topological structure reconstruction module, it learns representations from attribute and structural dimensions respectively, and through mutual communication The distillation mechanism promotes knowledge transfer and iterative optimization between the two modules.

Benefits of technology

It effectively alleviates the mutual interference between node attributes and topological structures, improves the abnormal detection performance, and significantly improves the detection accuracy and overall performance through dual-dimensional detection and mutual distillation mechanisms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025969A_ABST
    Figure CN119025969A_ABST
Patent Text Reader

Abstract

The invention discloses a mutual distillation-based dual-channel social network abnormal user detection method. The method comprises the following steps: constructing a mutual distillation-based dual-dimensional graph neural network anomaly detection framework DualAD; the DualAD comprises a multi-scale node attribute reconstruction module and a connection enhanced topological structure reconstruction module; the multi-scale node attribute reconstruction module performs information propagation by using a plurality of networks with different scales so as to detect attribute abnormity in the social network; the connection-enhanced topological structure reconstruction module performs effective message transmission on free nodes by introducing a connection enhancement graph so as to detect structural abnormality in the social network; the multi-scale node attribute reconstruction module and the connection enhanced topological structure reconstruction module jointly improve the performance through a mutual distillation mode; and performing social network abnormal user detection based on the trained DualAD. The detection framework provided by the invention can effectively improve the social network abnormal user detection performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of abnormal user detection in social networks, and in particular to a dual-channel abnormal user detection method in social networks based on mutual distillation. Background Art

[0002] With the rapid development of the Internet, online social networks have become an indispensable part of people's daily lives. In the intricate social relationship network, there are various forms of abnormal users, including false news publishers, fraudsters, and spammers, which have caused many negative impacts on society. For example, on social media, false news may mislead people's ideas, causing panic and confusion. In online review systems, unreliable user reviews may affect consumers' shopping choices. Network intrusions may lead to the leakage of personal privacy information, while financial fraud may cause significant damage to the economic system. These phenomena pose serious security and economic challenges to our society. It is of great practical significance to use technical means to efficiently and accurately detect and discover these abnormal users. Graph anomaly detection is currently an effective method for identifying abnormal users in social networks. The basic idea of ​​this method is to regard social network users as nodes, and the interactions between them (such as friend relationships, comments and views on the same content or topic, etc.) as edges, and abstract the social network into a graph. This transformation transforms the problem of detecting abnormal users in social networks into an anomaly detection problem in graphs, providing an innovative approach to solving problems related to false information publishers and fraudulent users.

[0003] Network anomaly detection aims to identify abnormal nodes that deviate significantly from the majority of the network. In real-world scenarios, the difference between abnormal nodes and normal nodes is mainly reflected in the attribute information of the nodes and the topological structure of the nodes. Correspondingly, abnormal nodes can be divided into two categories: attribute anomalies and topological anomalies. Given the high cost of collecting true labels for abnormal data, the main challenge of abnormal user detection in online social networks is to distinguish between attribute anomalies and structural anomalies of users in an unsupervised manner.

[0004] To address this challenge, researchers have proposed using graph neural networks (GNNs) to detect abnormal users in social networks. These models can be divided into two main categories: reconstruction-based methods and contrastive learning-based methods. Reconstruction-based methods reconstruct attribute information and topological structures by learning representations of users (nodes) and evaluate anomalies through reconstruction errors. Contrastive learning-based methods use contrastive learning to learn distinguishable representations and identify anomalies by distinguishing the consistency between nodes and their neighborhoods.

[0005] Although these methods have improved performance, they face the problem of mutual interference between node attributes and topological structures in the network. Recent studies have shown that this mutual interference is a key factor that leads to the performance degradation of GNNs. In anomaly detection tasks, this interference also weakens its performance. Existing methods usually use a single model to simultaneously identify node attribute anomalies and topological structure anomalies. However, simultaneously identifying different types of anomalies may lead to a "tug-of-war" problem, that is, the optimization for one anomaly detection task may interfere with or reduce the optimization for another anomaly, ultimately reducing the overall performance. Summary of the invention

[0006] In order to overcome the above problems, the present invention proposes a dual-channel social network abnormal user detection method based on mutual distillation, and specifically proposes a dual-dimensional graph neural network anomaly detection framework based on mutual distillation (Mutualdistillation-based Dual-dimension networkAnomaly Detection framework, referred to as DualAD). In this framework, we designed two channels (modules) to learn the attributes and structural representations of nodes from different dimensions, respectively, for identifying attribute information anomalies and topological structure anomalies. At the same time, the two channels promote each other, and knowledge transfer and iterative optimization are performed through the teacher-student distillation mechanism to improve the performance of the model.

[0007] In terms of attribute dimension, for a given target node, we first mask its attribute information, then reconstruct the masked attribute information based on its topological structure and neighbor information, and finally judge attribute anomalies by comparing the reconstruction error. In order to effectively reconstruct node attribute information and capture long-distance interactions between nodes, we designed a multi-scale node attribute information reconstruction module. This module uses feature propagation at different scales to alleviate the over-smoothing problem in GNNs to a certain extent and strengthen the learned representation for better attribute anomaly detection.

[0008] In terms of the topology dimension, we first mask the topology (edges) of the target node, and then reconstruct these masked edges based on their attribute information and neighbor information. In order to enhance the accuracy of topology reconstruction, we designed a connection-enhanced topology reconstruction module. This module generates an enhanced network based on the local topology abstracted from the actual network, so that information can be propagated to free nodes to enhance topology reconstruction. Through this module, free nodes caused by masked edges can receive sufficient information and provide strong support for topology anomaly detection. It can effectively improve the results of topology anomaly detection.

[0009] In terms of mutual distillation, we propose a mutual distillation scheme to promote the interaction between the multi-scale node attribute reconstruction module and the connection-enhanced topology reconstruction module by introducing a triplet distillation loss. Since these two modules try to learn the representation of the same node from different dimensions, they can complement each other. The mutual distillation scheme regards the connection-enhanced topology reconstruction module and the multi-scale node attribute reconstruction module as a teacher and a student, respectively, and distills the knowledge of the former into the latter to improve the accuracy of attribute anomaly detection. Similarly, the connection-enhanced topology reconstruction module also performs distillation by exchanging the roles of teacher and student. Finally, the detection results of attribute information and topology structure are connected and work together to jointly determine the final abnormal users.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions:

[0011] A dual-channel social network abnormal user detection method based on mutual distillation, comprising:

[0012] A mutually distilled dual-dimensional graph neural network anomaly detection framework DualAD is constructed; the DualAD includes a multi-scale node attribute reconstruction module and a connection-enhanced topological structure reconstruction module; the multi-scale node attribute reconstruction module uses multiple networks with different scales to propagate information to detect attribute anomalies in social networks; the connection-enhanced topological structure reconstruction module introduces a connection-enhanced graph to perform effective message transmission on free nodes to detect structural anomalies in social networks; the multi-scale node attribute reconstruction module and the connection-enhanced topological structure reconstruction module jointly improve performance through mutual distillation;

[0013] Abnormal user detection in social networks based on trained DualAD.

[0014] Furthermore, the input of the multi-scale node attribute reconstruction module is a network that has masked the attribute information of the target user node, and the output is a reconstructed attribute information representation. The module generates multiple enhanced views based on different feature propagation scales, and then uses the enhanced views to obtain multiple representations, and uses the attention mechanism to merge the multiple representations to form the final user node representation, and finally uses the final user node representation to reconstruct the attributes of the target user node.

[0015] Furthermore, the input of the connection-enhanced topological structure reconstruction module is a network with masked structural information, and the output is a network after structural reconstruction. The module first connects user nodes with similar semantics to generate an enhanced network, and then inputs the original network and the enhanced network into a propagation process based on graph diffusion and a conversion process based on a multi-layer perceptron to calculate the representation of the user nodes. Finally, the user node representation is used to reconstruct the topological structure of the target user node.

[0016] Further, the mutual distillation method includes:

[0017] Firstly, the multi-scale node attribute reconstruction module is regarded as a student and the connection-enhanced topological structure reconstruction module is regarded as a teacher to improve the performance of the multi-scale node attribute reconstruction module. Then, the performance of the connection-enhanced topological structure reconstruction module is improved by exchanging the roles of teacher and student. Finally, the node attributes and structure reconstruction results are combined to determine the abnormal situation.

[0018] Furthermore, in the multi-scale node attribute reconstruction module, feature propagation with different scales is performed to generate L enhanced feature matrices Each feature matrix is ​​generated as follows:

[0019]

[0020] Where α is the restart probability, X is the node attribute matrix, is the normalized adjacency matrix;

[0021] Then, the obtained enhanced feature matrix is ​​transformed into To encode:

[0022]

[0023] in is the embedding representation of N nodes in the lth scale, h is the length of the embedding representation, and W (1) is a learnable parameter matrix for feature matrix encoding;

[0024] Through multi-scale feature propagation, L feature representations are obtained {Z (l)} l=1 Learn from local view to capture node attribute information of neighboring nodes, while Learn from a set of high-order views to capture enhanced node attribute information of l-hop-ahead nodes;

[0025] Use the attention mechanism to learn the corresponding attention scores:

[0026] (α (1) ,…,α (L) )=Attention(Z (1) ,…,Z (L) ),

[0027]

[0028]

[0029]

[0030] in It's Z (1) ,…,Z (L) Attention score; It's Z (l) The i-th row of W represents the feature representation of level l; Z and W X represents the weight matrix, x i Represents the target user node v i The attribute vector of , q is the shared attention vector, yes The attention score,

[0031] The final representation Z is obtained by combining representations at different scales:

[0032] Z=∑ l α (l) Z (l) ,

[0033] Then decode the feature representation Z into the attribute information of the node, and use the fully connected layer to reconstruct the node attributes:

[0034]

[0035] in, is the reconstructed node attribute matrix, W (2) is a learnable parameter matrix and B is the corresponding bias term.

[0036] Furthermore, in the connection-enhanced topological structure reconstruction module, a k-nearest neighbor graph is used as an enhanced graph, and the node representation is calculated based on the obtained enhanced graph:

[0037] The information propagation of the masked structure network and the enhanced network is expressed as:

[0038]

[0039]

[0040] in, and Represent the normalized adjacency matrices of the masked structure network and the enhanced network respectively, X represents the initial feature matrix, β∈(0,1] is the restart probability, and t represents the number of iterations;

[0041] After T iterations, the propagated feature matrix is ​​obtained:

[0042]

[0043] Then, we use a multi-layer perceptron-based transformation to transform and Mapped to the intermediate representations H2 and H′2 respectively, the conversion feature matrices of the masked structure network and the enhanced network are expressed as:

[0044]

[0045]

[0046] Among them, W1 and W2 are learnable parameter matrices;

[0047] Finally, the representations H2 and H′2 from the two networks are combined to reconstruct the adjacency matrix

[0048]

[0049] Among them, f d (·) is the reconstruction function and AGG(·) represents the aggregation function.

[0050] Furthermore, the training loss of the multi-scale node attribute reconstruction module is for:

[0051]

[0052]

[0053] Where η1 and η2 represent the balancing hyperparameters, is the reconstruction loss, is the reconstructed node attribute matrix, X real is the real node attribute matrix, is the triplet distillation loss for the corresponding knowledge transfer.

[0054] Furthermore, the training loss of the connection-enhanced topological structure reconstruction module is for:

[0055]

[0056]

[0057]

[0058]

[0059] Where η1 and η2 represent the balancing hyperparameters, represents the reconstruction error between the reconstructed adjacency matrix and the true adjacency matrix, represents the consistency alignment loss, γ is a weight coefficient, A real is the true adjacency matrix, is the reconstructed adjacency matrix, is the triplet distillation loss for the corresponding knowledge transfer.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] This paper proposes a dual-dimensional graph neural network anomaly detection framework (DualAD) based on mutual distillation. In this framework, two channels are designed to learn two independent representations from different dimensions. By learning two representations, they are used to detect user attribute anomalies and topological structure anomalies, respectively, which effectively alleviates the mutual interference between attribute information and topological structure. In addition, the two channels are mutually enhanced by the teacher-student distillation method, which further improves the detection performance. The main contributions of this invention are summarized as follows:

[0062] (1) A new anomaly detection model DualAD for social networks is proposed. The model is a dual-dimensional model that uses two modules to detect attribute anomalies and structural anomalies in social networks respectively. At the same time, the two modules are connected to each other through knowledge distillation.

[0063] (2) A multi-scale node attribute information reconstruction module is designed. This module uses multiple networks with different scales to propagate information to alleviate the over-smoothing problem in GNNs and improve the detection effect of attribute anomalies.

[0064] (3) A connection-enhanced topology reconstruction module is designed. This module introduces a connection-enhanced graph to achieve effective message passing on free nodes and enhance the effect of structural anomaly detection.

[0065] (4) A mutual distillation scheme is proposed, which enables the above two modules to jointly improve the performance by mutual enhancement. Extensive experiments have shown that DualAD shows higher effectiveness compared with baseline methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a flow chart of a method for detecting abnormal users on a social network based on dual channels of mutual distillation according to an embodiment of the present invention;

[0067] Figure 2 A schematic diagram of the architecture of DualAD, a dual-dimensional graph neural network anomaly detection framework based on mutual distillation, constructed for an embodiment of the present invention;

[0068] Figure 3 The performance of different selection strategies provided by the embodiments of the present invention;

[0069] Figure 4The influence of the balance parameters η1 and η2 based on AUC evaluation provided by the embodiment of the present invention on three data sets;

[0070] Figure 5 The sensitivity analysis (AUC) of the number of neighbors k and the parameter λ provided in the embodiment of the present invention is shown. DETAILED DESCRIPTION

[0071] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0072] The formal statement of the abnormal user detection task in social networks is as follows: Given a social network with network user information in is the set of network users, ε={e1,e2,…,e m} is the set of interactive relationships between users, X = {x1, x2, …, x n} is the attribute information matrix containing n entities. Adjacency matrix Contains user collection and the topological structure information of the user relationship set ε, where A ij =1 indicates user v i and v j There is an interaction relationship between them, otherwise A ij = 0. In this context, the task of a social network anomaly detection model is to rank all users according to the calculated anomaly score, so that users that deviate significantly from the majority of reference users will have a higher anomaly score and a higher ranking.

[0073] like Figure 1 As shown, a dual-channel social network abnormal user detection method based on mutual distillation of the present invention includes:

[0074] A dual-dimensional graph neural network anomaly detection framework DualAD based on mutual distillation is constructed; the DualAD includes a multi-scale node attribute reconstruction module and a connection-enhanced topological structure reconstruction module; the multi-scale node attribute reconstruction module uses multiple networks with different scales to propagate information to detect attribute anomalies in social networks; the connection-enhanced topological structure reconstruction module introduces a connection-enhanced graph to perform effective message transmission on free nodes to detect structural anomalies in social networks; the multi-scale node attribute reconstruction module and the connection-enhanced topological structure reconstruction module jointly improve performance through mutual distillation;

[0075] Abnormal user detection in social networks based on trained DualAD.

[0076] The present invention is specifically introduced as follows:

[0077] 1DualAD Model Overview

[0078] In order to solve the problem of mutual interference between attribute anomaly detection and structural anomaly detection, we propose a dual-dimensional graph neural network anomaly detection framework (DualAD) based on mutual distillation. Figure 2 As shown in the figure, the framework consists of two channels: a multi-scale node attribute reconstruction module (left channel) and a connection-enhanced topological structure reconstruction module (right channel), which identify anomalies from two dimensions: node attributes and structure. At the same time, these two modules collaborate through a mutual distillation scheme (connecting the parts between the two channels).

[0079] The input of the multi-scale node attribute reconstruction module is the network that has masked the attribute information of the target node, and the output is the reconstructed attribute information representation. In order to capture both global information (i.e., long-distance interactions) and local information without causing over-smoothing problems, the module generates multiple enhanced views based on different feature propagation scales. Then, the enhanced views are used to obtain multiple representations, and the attention mechanism is used to merge these representations to form the final node representation. Finally, the node representation is used to reconstruct the attributes of the target node.

[0080] The input of the connection-enhanced topology reconstruction module is the network with masked structural information, and the output is the network after structural reconstruction. Since the structure (edge) of the target node is masked, the target node becomes an isolated node, which will lead to the inability to propagate information during the neighbor aggregation process of the graph neural network. To solve this problem, the module first connects nodes with similar semantics to generate an enhanced network. Next, the module inputs the original network and the enhanced network into the propagation process based on graph diffusion and the conversion process based on multi-layer perceptron to calculate the representation of the node. Finally, the topology of the target node is reconstructed using the node representation.

[0081] The mutual distillation strategy connects the two channels, the multi-scale node attribute reconstruction module and the connection-enhanced topology reconstruction module, to further improve their performance. The scheme uses a ternary distillation loss based on the intermediate representation of the two channels to encourage one module to help the other module for anomaly detection. The strategy first treats the multi-scale node attribute reconstruction module as a student and the connection-enhanced topology reconstruction module as a teacher to improve the performance of the multi-scale node attribute reconstruction module. Then, the performance of the connection-enhanced topology reconstruction module is improved by exchanging the teacher and student roles. Finally, the node attributes and structure reconstruction results are combined to determine the anomaly.

[0082] 2 Multi-scale node attribute reconstruction module

[0083] Node-abnormal users usually have normal neighborhood topology, but their user attribute information may be corrupted. Therefore, for a target entity, we mask its node attributes and assume that its neighborhood topology is known. Then, we reconstruct its node attributes and identify attribute anomalies based on the attribute reconstruction error.

[0084] We regard nodes with large reconstruction errors as abnormal nodes. The rationality of this assumption is that the model is good at capturing the characteristics of most normal nodes, that is, the model can accurately reconstruct the attribute information of normal nodes, while the reconstruction error of abnormal nodes is large. Based on this idea, considering global information (that is, long-distance interactions between users) is very important for effective node attribute reconstruction, especially when abnormal nodes are surrounded by other abnormal nodes. The reason is that the number of normal users far exceeds the number of abnormal users. Considering the interaction between long-distance users, it means that more normal users are included in the process of node attribute information reconstruction. This can make the reconstructed node attributes close to the normal node attribute information, while keeping a certain distance from abnormal users, thereby obtaining better anomaly detection results.

[0085] However, directly stacking multiple GNN layers to consider global information not only leads to information distortion caused by over-smoothing problems, but also introduces additional training parameters, reducing the training efficiency of the model. To address this problem, we design a multi-scale node attribute reconstruction module that can capture the interactions between distant nodes without suffering from the over-smoothing problem. Different from traditional GNNs, our module considers propagation features at different scales to produce multiple representations, and uses an attention mechanism to synthesize these representations to form the final node representation. This module can reconstruct the user's node attributes by capturing node attribute information from neighbors of different hops through different attention scores of each hop neighbor.

[0086] To capture the node attribute information of L-hop neighbors, we perform feature propagation with different scales to generate L enhanced feature matrices Each feature matrix is ​​generated as follows:

[0087]

[0088] Where α is the restart probability, X (0) =X is the node attribute matrix. is the normalized adjacency matrix, where yes The diagonal matrix of Here, the masked attributes of the target node are initialized to the average of the attributes of its neighboring nodes. Subsequently, the obtained enhanced feature matrix is To encode:

[0089]

[0090] in is the embedding representation of N nodes in the lth scale, h is the length of the embedding representation, and W (1) is a learnable parameter matrix for feature matrix encoding. (l)} l=1 Learn from local view to capture node attribute information of neighboring nodes, while Learn from a set of high-order views to capture enhanced node attribute information of l-hop-ahead nodes.

[0091] Through multi-scale feature propagation, we now have L feature representations Considering the differences between neighboring nodes of different hops, we use the attention mechanism to learn the corresponding attention scores:

[0092] (α (1) ,…,α (L) )=Attention(Z (1) ,…,Z (L) ), (3)

[0093] here It's Z (1) ,…,Z (L) Specifically, for the target node v i , which has a feature representation of level l (i.e. Z (l) (the i-th row of ), we calculate its attention score as follows:

[0094]

[0095] Among them, W Z and W X represents the weight matrix, x i represents the attribute vector of the target node, and q is the shared attention vector. The target node v i The final attention score is obtained by Normalized using Softmax(·):

[0096]

[0097] Larger Represents the target node v i It tends to pay more attention to the attribute information of the l-th hop neighbor. Definition We obtain the final representation Z by combining representations at different scales:

[0098] Z=∑ l α(l) Z (l) , (6)

[0099] Then we decode the feature representation Z into the attribute information of the node. We use a simple fully connected layer to reconstruct the node attributes:

[0100]

[0101] Among them, W (2) is a learnable parameter matrix and B is the corresponding bias term.

[0102] Finally, we compare the reconstructed node attribute matrix with the real node attribute matrix X real Compare to calculate the reconstruction loss:

[0103]

[0104] 3. Connection-enhanced topology reconstruction module

[0105] Structural anomaly nodes usually have incorrect connections with other nodes but have normal node attributes. Therefore, for a target node, we mask its topological structure information and assume that its node attribute information is known. Then, we reconstruct its topological structure and identify topological anomalies based on the topological structure reconstruction error.

[0106] In the masked network topology, the target node becomes a free node after masking its interaction relationship. This free node cannot obtain information from its neighbor nodes during the learning process, resulting in poor performance. To solve this problem, we create an enhanced network by connecting semantically similar nodes so that information can propagate to the free nodes. We use the original network and the enhanced network to learn node representations at the same time. The learned representations can effectively capture neighbor information and improve the effect of topology reconstruction. Finally, these representations are used for topology reconstruction.

[0107] Specifically, we use the k-nearest neighbor graph (kNN) as the augmented graph, which connects isolated nodes with semantically similar nodes. In the augmented graph, each isolated node has at least k neighbor nodes. The kNN graph A′ uses the propagated features To build:

[0108]

[0109] Here is with v i The set of top k nodes with the highest similarity. The propagation characteristics It is calculated by propagation diffusion (refer to formula (12)) after masking the topological relationship.

[0110] Based on the obtained enhanced network graph, we calculate the node representation through two steps: diffusion based on interaction relations and conversion based on multi-layer perceptron (MLP). and Represent the normalized adjacency matrices of the masked structure network and the enhanced network respectively, then the information propagation of the two networks can be expressed as:

[0111]

[0112]

[0113] Among them, X (0) =X represents the initial feature matrix, β∈(0,1] is the restart probability. After T iterations, we get the propagated feature matrix:

[0114]

[0115] We then use a multi-layer perceptron (MLP)-based transformation using two fully connected layers to transform and Mapped to the intermediate representations H2 and H′2 respectively. The conversion feature matrix of the masked network and the enhanced network can be expressed as:

[0116]

[0117]

[0118] Among them, W1 and W2 are learnable parameter matrices. Finally, we combine the representations H2 and H′2 from the two networks to reconstruct the adjacency matrix

[0119]

[0120] in, d (·) is the reconstruction function, which can reconstruct the adjacency matrix by calculating the similarity or inner product between node representations, and AGG(·) represents the aggregation function. We minimize the reconstructed adjacency matrix during training. and the true adjacency matrix A real The reconstruction error between

[0121]

[0122] However, this simple objective function has limitations. Since the masked network and the augmented network share model parameters, the learned representations (especially for free nodes) may be significantly different due to the difference in input structure. To bridge this gap, we introduce a consistency alignment loss to enforce semantic consistency between the two network representations. The loss is defined as:

[0123]

[0124] This loss function enhances the consistency between the original network and the enhanced network. It helps the model extract more useful information, especially for the free nodes.

[0125] During training, we combine these two loss functions as the overall goal of the topology reconstruction module:

[0126]

[0127] Where γ is a weight coefficient.

[0128] 4 Distillation and model training

[0129] The constructed multi-scale node attribute reconstruction module and connection-enhanced topology reconstruction module can identify node attribute anomalies and topology anomalies, respectively. The two modules are highly related and complementary because they try to learn the representation of the same node from different dimensions. For the target node, its attribute information is masked in the multi-scale node attribute reconstruction module, but the topology structure remains intact. Therefore, the multi-scale node attribute reconstruction module can extract knowledge from the connection-enhanced topology reconstruction module for learning, and vice versa. Therefore, we introduce a teacher-student distillation method to distill the knowledge in one module into another. Conventional distillation methods mainly focus on minimizing the difference between the teacher and the student, while our method requires learning representations that belong to the same category but have certain differences. We propose a mutual distillation scheme that uses the triplet loss of metric learning to embed the knowledge of the teacher model into the output space of the student model and simultaneously clarify the difference between their outputs. This scheme enables the two modules to enhance each other to improve the overall performance.

[0130] Specifically, to enhance the multi-scale node attribute reconstruction module, we use it as the student model and the connection-enhanced topology reconstruction module as the teacher model. Then, we select an anchor sample x a , a positive sample x p and a negative sample x n , forming a triple (x a ,x p ,x n ). Here, the anchor point x aand positive sample x p are the representations of the same target node in the student model and the teacher model, respectively, and the negative sample x n is a sample randomly selected from the teacher model that is different from the target node. The mutual distillation scheme learns an embedding based on this triplet so that the distance from the anchor to the positive sample is relatively closer than the distance from the anchor to the negative sample. Therefore, this goal can make the representations from the same node in the two modules close but keep some differences, satisfying the requirements of two modules with different goals and distilling knowledge from one module to enhance the other.

[0131] The triplet distillation loss for knowledge transfer is formally defined as:

[0132]

[0133] Where t(·) and s(·) are the teacher and student outputs, respectively, and m is a hyperparameter that defines the distance of the difference. Ω is a set of indices representing each corresponding anchor point, positive sample, and negative sample. When we train the student model, the weights of the teacher model are not updated. That is, during the training process, t(·) is treated as a constant. It should be noted that this loss is still valid even if the nodes have normal attribute information and abnormal topological structure. This is because the objectives of both modules produce normal attribute information and topological structure. Regardless of whether the original user is normal or not, the anchor points and positive samples calculated by the two modules should belong to the normal category and be closer than the distance from the anchor point to the negative sample.

[0134] For the multi-scale node attribute reconstruction module, its final training target is the combination of the reconstruction loss in formula (8) and the distillation loss in formula (19):

[0135]

[0136] Among them, η1 and η2 represent the balancing hyperparameters.

[0137] Similarly, to enhance the connection-enhanced topology reconstruction module, we swap the roles of the two modules, that is, the connection-enhanced topology reconstruction module is regarded as a student and the multi-scale node attribute reconstruction module is regarded as a teacher. Subsequently, the distillation loss of the connection-enhanced topology reconstruction module is derived according to formula (19), which is expressed as Therefore, the final training loss of the connection-enhanced topology reconstruction module is the reconstruction loss and the distillation loss in formula (18): Combination of:

[0138]

[0139] Among them, η1 and η2 are the same as in formula (20), that is, they represent balanced hyperparameters.

[0140] 5. Anomaly Score

[0141] We use reconstruction error to distinguish abnormal nodes from normal nodes. That is, the larger the reconstruction error, the more likely it is an abnormal node. This is because abnormal nodes conform to the pattern of most data and therefore cannot be accurately reconstructed. For a given user v i , we calculate its node attributes and topology reconstruction errors as follows:

[0142]

[0143]

[0144] After combining these two types of errors, the final anomaly score is calculated as follows:

[0145] AS(v i )=λ·AS att (v i )+(1-λ)·AS str (v i ), (twenty four)

[0146] Among them, λ is a balancing hyperparameter. Abnormal nodes are distinguished by anomaly scores, i.e., nodes with higher scores are more likely to be considered as anomalies.

[0147] To verify the effect of the present invention, the following experiment was performed:

[0148] In this section, we evaluate the effectiveness of the DualAD model in terms of anomaly detection performance in graph networks and demonstrate the modularity of our experimental design.

[0149] (a) Experimental setup

[0150] Datasets: We conduct experiments on two types of datasets, the details of which are shown in Table 1. Real anomaly graph: Amazon is a shopping website and YelpChi is a review website, both of which have abnormal real labels; Injected anomaly graph: Flickr is a social network with injected anomaly labels. Specifically, user attribute anomalies and topological structure anomalies are injected into the Flickr dataset.

[0151] Table 1 Descriptive statistics of the data set

[0152]

[0153] Baseline methods: We compare DualAD with various baseline models in four main categories: (1) Traditional methods: AMEN, Radar, and ANOMALOUS use shallow traditional methods to achieve anomaly detection; (2) Reconstruction-based methods: DOMINANT, ALARM, and ComGA perform anomaly detection through the reconstruction technique of autoencoders; (3) Filling models: ITR and S2GAE are graph filling methods. To detect anomalies, we use them to fill in the masked parts of user attributes and topology structures, and then distinguish anomalies based on the filling error; (4) Contrastive methods: CoLA and GRADATE perform anomaly detection through contrastive learning.

[0154] Experimental setup: We use the Adam optimizer to sequentially optimize the user attributes and topology reconstruction modules. For all experiments, we use grid search to find the best hyperparameters such as learning rate and number of training rounds.

[0155] (b) Anomaly detection results

[0156] We show the anomaly detection results of DualAD and the baseline model in Table 2, where AROC, APR and F1 denote AUC-ROC, AUC-PR and F1-macro, respectively.

[0157] Table 2 Performance comparison of DualAD and baseline methods on three datasets

[0158]

[0159]

[0160] From Table 2 we make the following observations:

[0161] First, our proposed model achieves the best detection performance on all datasets. Although GRADATE's AUC-PR is slightly higher than our model, its AUC-ROC and F1 scores are significantly lower than our model. Compared with the baseline model, the performance improvement of our model lies in learning two representations through mutual distillation, which alleviates the problem of mutual interference between attribute information and topological structure.

[0162] Second, the filling methods (ITR and S2GAE) achieved relatively good performance compared to traditional methods. These results show that these filling models are promising as potential anomaly detection tools. Although our model attempts to reconstruct the masked user attribute information and topological structure, it can be classified as a filling model to a certain extent. Our model outperforms these two state-of-the-art filling methods, highlighting the effectiveness of the proposed multi-scale node attribute information reconstruction module and connection-enhanced topological structure reconstruction module.

[0163] Third, the two main anomaly detection methods, reconstruction-based and contrastive learning-based, show remarkable performance, especially the contrastive learning-based method performs well in anomaly detection. However, the limitation of these methods is that they rely on a single representation to detect both node attribute and topological structure anomalies, which may lead to potential modality interference. In contrast, DualAD performs well on unconventional boundaries, validating our motivation of adopting a dual-dimensional strategy to detect both anomalies.

[0164] (c) Ablation experiment

[0165] We conducted ablation experiments on three important modules of DualAD by removing one of the modules from the entire model to evaluate the role of important modules in DualAD. (1) In DualAD-Att, we removed the multi-scale node attribute reconstruction module; (2) In DualAD-Str, we removed the connection-enhanced topological structure reconstruction module; (3) In DualAD-Dis, we removed the mutual distillation mechanism. Figure 3 As shown, removing the multi-scale node attribute reconstruction module (DualAD-Att) or the connection-enhanced topology reconstruction module (DualAD-Str) both leads to performance degradation, which indicates that these two modules are beneficial for accurately detecting anomalies in social networks. In addition, the variant DualAD-Dis does not perform mutual distillation and ignores the knowledge interaction between the two modules. In the absence of guidance from the multi-scale node attribute (or connection-enhanced topology) reconstruction module, the multi-scale node attribute (or connection-enhanced topology) reconstruction module produces suboptimal intermediate representations, resulting in performance degradation. It is worth noting that DualAD consistently outperforms all variants, highlighting the effectiveness of our designed attribute information and topology reconstruction modules with the mutual distillation mechanism.

[0166] (d) Sensitivity analysis

[0167] Balance parameters η1 and η2. Figure 4 As shown, for a given η1 (refer to Equation 20 and Equation 21), setting η2 to 0 leads to the worst performance on all datasets. This observation highlights the importance of the distillation loss we designed. On the other hand, setting η1 to a higher value generally improves performance. On the contrary, an excessively large value of η2 is counterproductive - over-emphasizing the distillation loss blurs the difference between the attribute and topology reconstruction modules, making the corresponding reconstruction a very challenging task. Therefore, appropriately setting the values ​​of η1 and η2 is crucial to the performance of the model.

[0168] The number of neighbors k and the parameter λ are selected. We change their values ​​in kNN (refer to formula (9)) and the balance parameter λ (refer to formula (24)) and Figure 5 Their impact on the performance of all datasets is reported in. Our observations lead to the following conclusions: (1) Too large or too small k values ​​usually weaken the performance. Since k determines the number of neighbor nodes connected to the target node, a smaller k may hinder the model from capturing complex information in the graph, while a larger k may introduce noise into the learning process. (2) The choice of the balance parameter λ (see formula (24)) depends on the dataset. The proportion of node attribute anomalies and topological structure anomalies in different datasets is different, which affects the detection performance. Therefore, when selecting k and λ values, it is necessary to consider the characteristics of the actual dataset and make adjustments and optimizations.

[0169] Selected number of neighbors k and parameter γ. We vary the selected number of neighbor nodes k (see Eq. (11)) and the weighting coefficient γ (see Eq. (20)) and report their impact on the performance of all datasets in the graph (see Figure 5 ). Our observations lead to the following conclusions: (1) Too large or too small a value of k usually weakens the performance. Since k determines the number of neighbor nodes connected to the target node, a smaller k may hinder the model from capturing complex information in the graph, while a larger k may introduce noise into the learning process. (2) It can be observed that when the value of γ is too high, the proportion of alignment loss significantly exceeds that of reconstruction loss, resulting in a decrease in anomaly detection performance. This is because when the model aligns the intermediate representations of the two graphs, it loses the ability to make effective structure estimates based on the reconstruction loss.

[0170] In summary, this paper proposes a new dual-dimensional graph neural network anomaly detection framework (DualAD) based on mutual distillation. This framework effectively alleviates the mutual interference between attribute information and topological structure by learning two representations, which are used to detect user attribute anomalies and topological structure anomalies respectively. In terms of attribute dimension, we designed a multi-scale node attribute reconstruction module, which generates multiple enhanced networks based on different feature propagation scales to alleviate the over-smoothing problem in traditional GNNs and improve the performance of node attribute anomaly detection. In terms of structural dimension, we designed a connection-enhanced topological structure reconstruction module, which uses enhanced networks to solve the problem of free nodes and improve the effect of topological structure anomaly detection. In addition, we proposed a mutual distillation scheme to improve the two modules using teacher-student distillation. A large number of experiments have proved that the framework we proposed can effectively improve the anomaly detection performance.

[0171] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A dual-channel social network abnormal user detection method based on mutual distillation, It is characterized in that include: A dual-dimensional graph neural network anomaly detection framework DualAD based on mutual distillation is constructed; the DualAD includes a multi-scale node attribute reconstruction module and a connection-enhanced topological structure reconstruction module; the multi-scale node attribute reconstruction module uses multiple networks with different scales to propagate information to detect attribute anomalies in social networks; the connection-enhanced topological structure reconstruction module introduces a connection-enhanced graph to perform effective message transmission on free nodes to detect structural anomalies in social networks; The multi-scale node attribute reconstruction module and the connection-enhanced topology structure reconstruction module jointly improve the performance through mutual distillation; Abnormal user detection in social networks based on trained DualAD.

2. According to claim 1, the dual-channel social network abnormal user detection method based on mutual distillation, It is characterized in that The input of the multi-scale node attribute reconstruction module is a network that has masked the attribute information of the target user node, and the output is a reconstructed attribute information representation. The module generates multiple enhanced views based on different feature propagation scales, and then uses the enhanced views to obtain multiple representations. The multiple representations are merged using the attention mechanism to form the final user node representation, and finally the final user node representation is used to reconstruct the attributes of the target user node.

3. According to claim 1, the dual-channel social network abnormal user detection method based on mutual distillation, It is characterized in that The input of the connection-enhanced topological structure reconstruction module is a network with masked structural information, and the output is a network after structural reconstruction. The module first connects user nodes with similar semantics to generate an enhanced network, and then inputs the original network and the enhanced network into a propagation process based on graph diffusion and a conversion process based on a multi-layer perceptron to calculate the representation of the user nodes. Finally, the user node representation is used to reconstruct the topological structure of the target user node.

4. According to claim 1, the dual-channel social network abnormal user detection method based on mutual distillation, It is characterized in that The mutual distillation method includes: Firstly, the multi-scale node attribute reconstruction module is regarded as a student and the connection-enhanced topological structure reconstruction module is regarded as a teacher to improve the performance of the multi-scale node attribute reconstruction module. Then, the performance of the connection-enhanced topological structure reconstruction module is improved by exchanging the roles of teacher and student. Finally, the node attributes and structure reconstruction results are combined to determine the abnormal situation.

5. According to claim 1, the dual-channel social network abnormal user detection method based on mutual distillation, It is characterized in that In the multi-scale node attribute reconstruction module, feature propagation with different scales is performed to generate L enhanced feature matrices Each feature matrix is ​​generated as follows: Where α is the restart probability, X is the node attribute matrix, is the normalized adjacency matrix; Then, the obtained enhanced feature matrix is ​​transformed into To encode: in is the embedding representation of N nodes in the lth scale, h is the length of the embedding representation, and W (1) is a learnable parameter matrix for feature matrix encoding; Through multi-scale feature propagation, L feature representations are obtained Learn from local view to capture node attribute information of neighboring nodes, while Learn from a set of high-order views to capture enhanced node attribute information of l-hop-ahead nodes; Use the attention mechanism to learn the corresponding attention scores: (a (1) ,…,a (L) )=Attention(Z (1) ,…,Z (L) ), in It's Z (1) ,…,Z (L) Attention score; It's Z (l) The i-th row of W represents the feature representation of level l; Z and W X represents the weight matrix, x i Represents the target user node v i The attribute vector of , q is the shared attention vector, yes The attention score, The final representation Z is obtained by combining representations at different scales: Z=∑ l α (l) WITH (l) , Then decode the feature representation Z into the attribute information of the node, and use the fully connected layer to reconstruct the node attributes: in, is the reconstructed node attribute matrix, W (2) is a learnable parameter matrix and B is the corresponding bias term.

6. According to claim 1, the dual-channel social network abnormal user detection method based on mutual distillation, It is characterized in that In the connection-enhanced topology reconstruction module, a k-nearest neighbor graph is used as an enhanced graph, and node representation is calculated based on the obtained enhanced graph: The information propagation of the masked structure network and the enhanced network is expressed as: in, and Represent the normalized adjacency matrices of the masked structure network and the enhanced network respectively, X represents the initial feature matrix, β∈(0,1] is the restart probability, and t represents the number of iterations; After T iterations, the propagated feature matrix is ​​obtained: Then, we use a multi-layer perceptron-based transformation to transform and Mapped to the intermediate representation H 2 and H′ 2 , the transformation feature matrix of the masked structure network and the enhanced network is expressed as: Among them, W 1 and W 2 is a learnable parameter matrix; Finally, the representations H from the two networks are combined 2 and H′ 2 Combine to reconstruct the adjacency matrix Among them, f d (·) is the reconstruction function and AGG(·) represents the aggregation function.

7. The method for detecting abnormal users in social networks based on dual channels of mutual distillation according to claim 1, It is characterized in that The training loss of the multi-scale node attribute reconstruction module for: where η 1 and η 2 represents the balancing hyperparameter, is the reconstruction loss, is the reconstructed node attribute matrix, X real is the real node attribute matrix, is the triplet distillation loss for the corresponding knowledge transfer.

8. The method for detecting abnormal users in social networks based on dual channels of mutual distillation according to claim 1, It is characterized in that The training loss of the connection-enhanced topology reconstruction module for: where η 1 and η 2 represents the balancing hyperparameter, represents the reconstruction error between the reconstructed adjacency matrix and the true adjacency matrix, represents the consistency alignment loss, γ is a weight coefficient, A real is the true adjacency matrix, is the reconstructed adjacency matrix, is the triplet distillation loss for the corresponding knowledge transfer.

Citation Information

Cited By

  • Social network abnormal user detection method and system based on external graph data

    CN119762262A

  • Node anomaly detection method and device, equipment, medium and program product

    CN120145245A