Network Node Depth Anomaly Detection Method Based on Density Estimation

Abstract

The present invention belongs to the technical field of communication network detection, and particularly relates to a method for deep anomaly detection of network nodes based on density estimation. The node attribute matrix and structure matrix in the attribute network are obtained, and the attribute matrix and structure matrix are used as the input of the graph convolutional neural network. The graph convolutional neural network is used to obtain the joint embedding vector of the node structure information and attribute information in the attribute network. The node attribute matrix and structure matrix input to the graph convolutional neural network are respectively reconstructed by using the joint embedding vector, and the node reconstruction error is obtained. For the joint embedding vector and the node reconstruction error, a probability distribution model is used to detect the node anomalies in the attribute network. Based on the idea of detecting abnormal nodes by reconstruction error, the present invention reconstructs the structure information and attribute information of the attribute network respectively, and uses density estimation to realize the detection of abnormal nodes based on the reconstruction error of the nodes and their own embedding vectors, which has good robustness and is convenient for practical scenario applications.

Description

Technical Field

[0001] The present invention belongs to the technical field of communication network detection, and particularly relates to a method for deep anomaly detection of network nodes based on density estimation. Background Art

[0002] From academic networks to social networks, from protein interaction networks to healthcare systems, attributed networks widely exist in various fields of the real world. Simple networks can only present the interdependent relationships between nodes. Attributed networks not only depict the dependencies between nodes, but also often contain rich feature information for each node itself. For example, in an academic network, an attributed network can not only reflect the cooperation relationships between scholars, but also characterize information such as the research directions, the number of publications, and the citation status of each scholar; in a social network, an attributed network can simultaneously depict the friendship or communication relationships between users and the feature information of the users themselves, such as the user's interests, posting status, and like status. The information contained in the nodes themselves plays an important role in knowledge mining on network data. Anomaly detection on network data is a hot topic in network science and has a wide range of application scenarios in real life, such as intrusion detection in network space security, detection of abnormal accounts on social networks, fraud detection in the financial field, etc. In the actual anomaly detection process, not only the interdependent relationships between network nodes need to be considered, but also the feature information of the nodes themselves. How to effectively process this multi-modal information is the key to anomaly detection on attributed networks. Attributed networks have great advantages in data modeling for data containing different modal information. Therefore, in recent years, many methods for anomaly detection on attributed network data have emerged. An abnormal node on an attributed network can be described as a node whose node pattern significantly deviates from that of most other nodes. The anomalies in the node pattern are mainly reflected in two aspects: one is the abnormal connection structure around the node; the other is the abnormal attributes of the node itself.

[0003] According to the different information considered when detecting anomalies, existing anomaly detection methods can be divided into three categories: one is anomaly detection based on the characteristics of nodes themselves, the second is anomaly detection based on ego-centric networks or community partitioning, and the third is anomaly detection based on network embedding. Anomaly detection methods based on the characteristics of nodes themselves need to select key characteristics that can accurately identify abnormal nodes, and then use statistical methods for anomaly detection. For anomaly detection methods based on ego-centric networks or community partitioning, for each node, a specific function model is used to depict the dependence relationship between a certain aspect of the characteristics of this node and the characteristics of other nodes in the ego-centric network or the same community. If the dependence relationship function corresponding to the node significantly deviates from the dependence relationship functions corresponding to most other nodes, it is regarded as an abnormal node. For anomaly detection methods based on network embedding, first, a specific network embedding method needs to be used to embed network nodes into a low-dimensional vector space. Each node corresponds to a point in the low-dimensional vector space, and it is called the embedding vector of the node. Then, anomaly detection is carried out based on the embedding vectors of the nodes. Common practices include anomaly detection using density estimation, anomaly detection based on reconstruction error, etc.

[0004] Although existing methods have achieved good results under specific conditions, they still face the following challenges when dealing with attributed network data: (1) Data sparsity. The sparsity of data is mainly caused by two reasons: one is the sparsity of the data itself. For example, the famous "Dunbar number" in social network research shows that "generally, a human individual can maintain stable social relationships with at most 150 other people". Therefore, as the scale of the social network expands, it will inevitably lead to the social network becoming sparser and sparser. The second is incomplete data acquisition. Due to privacy protection and the cost of data acquisition, it is difficult to ensure that all data of a specific network is obtained. The first two types of methods are highly dependent on data, and sparse network data will pose great challenges to the first two types of anomaly detection methods. (2) Complex modality interaction. The connection relationships between nodes and the characteristic information of nodes in an attributed network often come from different information sources, and these information are not independent of each other. That is, the characteristic information of the node itself and the topological structure around the node are mutually influential to a certain extent. How to depict the interaction relationship between different information and integrate it into the same feature space is a difficult point in dealing with attributed networks. (3) Mixture of structural anomalies and attribute anomalies. In reality, structural anomalies and attribute anomalies are often not equivalent. The attribute characteristics of a structurally abnormal node are not necessarily abnormal, and vice versa. If anomaly detection can be carried out separately for attribute anomalies and structural anomalies, not only can the classification of anomalies be refined, but for structurally abnormal nodes, the interference of attribute characteristics on anomaly determination can be reduced, and for attribute abnormal nodes, the interference of structural characteristics on anomaly determination can be reduced. Summary of the Invention

[0005] To this end, the present invention provides a method for deep anomaly detection of network nodes based on density estimation. Based on the idea of detecting abnormal nodes by reconstruction error, it reconstructs the structural information and attribute information of the attribute network respectively, and uses density estimation to detect abnormal nodes based on the reconstruction error of the nodes and their own embedding vectors, which has good robustness and is convenient for practical scenario applications.

[0006] According to the design scheme provided by the present invention, a method for deep anomaly detection of network nodes based on density estimation is provided, including the following content:

[0007] Obtain the node attribute matrix and structure matrix in the attribute network, and use the attribute matrix and structure matrix as the input of the graph convolutional neural network, and use the graph convolutional neural network to obtain the joint embedding vector of the node structure information and attribute information in the attribute network;

[0008] Use the joint embedding vector to reconstruct the node attribute matrix and structure matrix input to the graph convolutional neural network respectively, and obtain the node reconstruction error;

[0009] For the joint embedding vector and the node reconstruction error, use the probability distribution model to detect the node anomalies in the attribute network.

[0010] As the method for deep anomaly detection of network nodes based on density estimation of the present invention, further, the attribute network is represented as G = <V, E, X>, where V represents the node set of the attribute network G, E represents the edge set of the attribute network, X represents the node attribute matrix of the attribute network, and A represents the attribute network structure matrix.

[0011] As the method for deep anomaly detection of network nodes based on density estimation of the present invention, further, when obtaining the joint embedding vector, use the graph convolutional neural network composed of k stacked neural network layers and obtain the node embedding vector containing neighbor information through iterative calculation.

[0012] As the method for deep anomaly detection of network nodes based on density estimation of the present invention, further, use the graph convolutional neural network stacked with three neural network layers, and the iterative calculation process is expressed as follows:

[0013] Among them, h0 is the feature dimension of the attribute matrix X, and h1 is the feature dimension of the hidden layer; h (i) is the node feature matrix after the i-th layer of calculation, and i ∈ {1, 2, 3}, Z is the finally obtained node embedding vector, h3 is the dimension of the node embedding vector, and the feature vector in H (3) contains the node three-order neighbor information.

[0014] As the network node depth anomaly detection method based on density estimation of the present invention, further, among the node reconstruction errors, the activation functions tanh(·) or sigmoid(·) are selected as the prediction functions of the structure matrix, and a graph convolutional neural network stacked by three neural network layers is used to predict the attribute matrix; the reconstruction errors regarding the structural information and the attribute information are obtained based on the structure matrix, the attribute matrix, and their prediction results.

[0015] As the network node depth anomaly detection method based on density estimation of the present invention, further, the similarity between node vectors is used to measure the reconstruction errors between the node structure matrix and the attribute matrix and their corresponding prediction results.

[0016] As the network node depth anomaly detection method based on density estimation of the present invention, further, a Gaussian mixture model is used as the probability distribution model for detecting node anomalies in the attribute network, and the density estimation of the attribute information and the structural information in the attribute network is completed by solving the Gaussian mixture model; the abnormal nodes are obtained based on the probability distribution in the density estimation results.

[0017] As the network node depth anomaly detection method based on density estimation of the present invention, further, in the process of solving the Gaussian mixture model, the node joint embedding vector is used as the input, and the probability that the node belongs to each sub-distribution of the Gaussian mixture model is used as the output, and a multi-layer perceptron is used for model solving.

[0018] As the network node depth anomaly detection method based on density estimation of the present invention, further, the density estimation process of the node attribute information is expressed as: The density estimation process of the node structural information is expressed as: where z x , z a respectively represent the representation vectors of the attribute information and the structural information of the attribute network nodes, z xr , z ar respectively represent the reconstruction errors of the attribute information and the structural information, and MLP() represents the solution by the multi-layer perceptron.

[0019] As the network node depth anomaly detection method based on density estimation of the present invention, by constructing a depth anomaly detection model framework, the joint embedding vector and the node reconstruction error are obtained by using this depth anomaly detection model framework and the network node anomaly detection is carried out; among them, the depth anomaly detection model framework includes: a graph convolutional neural network for obtaining the joint embedding vector, a probability distribution model for detecting node anomalies, and an information reconstruction network for obtaining the node reconstruction error; the objective loss function of the model framework is expressed as: where N represents the number of samples, θ e , θ d , θm respectively represent the network parameters of the graph convolutional neural network, the probability distribution model, and the information reconstruction network; R A / X represents the structural information reconstruction error R A or the attribute information reconstruction error R X , and its value is R X or R A ; λ1, λ2 are preset hyperparameters, and E(z i ) represents the energy of the sample value z i . Among them, K represents the number of Gaussian components in the Gaussian mixture model, d refers to the dimension of the node representation vector, represents the j-th element on the main diagonal of the covariance matrix of the k-th Gaussian component.

[0020] Advantages of the present invention:

[0021] The present invention respectively performs structural anomaly node detection and attribute anomaly node detection on the attribute network, uses GCN to realize the joint embedding of the structural information and attribute information of the attribute network, and further realizes the reconstruction of the network structural information by means of the hyperbolic tangent function and the reconstruction of the network attribute information by means of GCN. The node reconstruction error is concatenated with its own embedding vector as the final vector representation of the node. Based on the final vector representation of the node, density estimation is performed through GMM, and then abnormal nodes are discovered. It can consider the connection between the structural information and attribute information in the attribute network and can well distinguish the types of abnormal nodes, that is, attribute anomalies or structural anomalies; and further conduct experimental verification on common data sets, and has good robustness and practicality. Description of the drawings

[0022] Figure 1 is a schematic diagram of the deep anomaly detection process of network nodes based on density estimation in the embodiment;

[0023] Figure 2 is a schematic diagram of the deep anomaly detection model framework in the embodiment;

[0024] Figure 3 is a schematic diagram of the curves of the sigmoid(·) function and the tanh(·) function in the embodiment;

[0025] Figure 4 is a schematic diagram of the curve of the change of AIC with the GMM parameter K under different data in the embodiment. Specific implementation manners

[0026] In order to make the purpose, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the drawings and technical solutions.

[0027] The research on the problem of detecting abnormal nodes in attribute networks has a wide range of application fields in life, such as social networks, cyberspace security, financial fields, etc. Most of the existing methods for detecting abnormal nodes in attribute networks ignore the connection between the structural information and attribute information in the attribute network; although some methods consider the connection between the two, they cannot well distinguish the types of abnormal nodes, that is, attribute anomalies or structural anomalies. In the embodiments of the present invention, see Figure 1 as shown, a method for deep anomaly detection of network nodes based on density estimation is provided, which includes the following contents:

[0028] S101. Obtain the node attribute matrix and structure matrix in the attribute network, and use the attribute matrix and structure matrix as the input of the graph convolutional neural network. Use the graph convolutional neural network to obtain the joint embedding vector of the node structure information and attribute information in the attribute network;

[0029] S102. Use the joint embedding vector to reconstruct the node attribute matrix and structure matrix input to the graph convolutional neural network respectively, and obtain the node reconstruction error;

[0030] S103. For the joint embedding vector and the node reconstruction error, use the probability distribution model to detect the node anomalies in the attribute network.

[0031] For the attribute network, structural anomaly node detection and attribute anomaly node detection are respectively carried out. The GCN is used to realize the joint embedding of the structural information and attribute information of the attribute network. Based on the idea of detecting abnormal nodes based on the reconstruction error, the structural information and attribute information of the attribute network are respectively reconstructed, and based on the reconstruction error of the node and its own embedding vector, density estimation is used to realize abnormal node detection, which has good robustness and is convenient for practical scenario applications.

[0032] For the convenience of description, denote the attribute network G = <V, E, X>, where V = {v1, v2, v3,..., v n} represents the node set of the network G, v i represents the i-th node, n = |V| represents the total number of nodes in the network G, represents the network edge set. If v i and v j have an edge, then this edge is denoted as e i,j , the node attribute The adjacency matrix of the network G is denoted as A, A i,j = 1 means that v i and v j have an edge, otherwise A i,j = 0.

[0033] By constructing a deep anomaly detection DADDE model framework, see Figure 2As shown, the deep anomaly detection model framework is used to obtain joint embedding vectors and node reconstruction errors for network node anomaly detection. Among them, the deep anomaly detection model framework includes: a graph convolutional neural network for obtaining joint embedding vectors, a probability distribution model for detecting node anomalies, and an information reconstruction network for obtaining node reconstruction errors. In the model framework, a graph convolutional neural network composed of k stacked neural network layers can be used to obtain node embedding vectors containing neighbor information through iterative calculations. Using the structure matrix A and attribute matrix X of the attribute network as inputs, GCN is used to achieve the joint embedding of the attribute network structure information and attribute information. The attribute information of the node itself is used as the initial feature representation of the node. During each round of GCN calculation with the help of the structure matrix A, the new node features contain the information of the node's first-order neighbors. In this way, the node features after n rounds of GCN calculation contain the information of the node's n-order neighbors. After obtaining the embedding vectors of the nodes, the input structure matrix A and attribute matrix X are reconstructed respectively based on the embedding vectors. Considering the difference in data characteristics between the structure matrix A and the attribute matrix X, the values in the structure matrix A are continuous, and the GCN network is selected to reconstruct the attribute information. The values in the attribute matrix X can only be 0 or 1, and the sigmoid function is selected to reconstruct the structure information. Considering that the information contained in the representation vector of the node itself has an auxiliary effect on the determination of abnormal nodes, therefore, in the embodiments of this case, the node reconstruction error can be spliced with the node representation vector as the final representation vector of the node. Based on the final representation vector of the node, the density estimation method can be used to detect abnormal nodes. First, it is assumed that the data points satisfy a specific probability distribution model, and the best fitting parameters of the probability model are obtained by maximizing the likelihood function, and then nodes that deviate significantly from the probability model are found, that is, abnormal nodes.

[0034] The attribute network contains both network structure and node attribute information. In most cases, these two aspects of information are not isolated from each other. The network structure information and node attribute information influence each other. For example, in a social network, attribute information such as a user's hobbies, gender, and age will affect the user's online dating behavior, thus affecting the evolution of the network structure. Vice versa, the user's social network friends also affect the user's attribute information such as hobbies to a certain extent. Therefore, when performing network embedding, the connection between the two cannot be severed and information embedding is performed separately. This will cause the embedding vector to only retain one-sided information and is prone to overfitting, ignoring the information on the other hand, which does not conform to the actual situation of the attribute network. Therefore, in the embodiments of this case, GCN is selected to achieve the joint embedding of the attribute network structure information and attribute information.

[0035] From a mathematical perspective, GCN extends the convolution operation to the spectral domain of network data and calculates a hierarchical latent representation vector of the network through the spectral convolution function. From the perspective of neural networks, GCN is composed of k stacked neural network layers, where k determines the order of neighbor information contained in the final embedding vector when GCN performs network embedding. The propagation method between layers in GCN can be expressed by the following formula:

[0036]

[0037] where, H (l) is the input matrix of the l-th convolutional layer, H (0) = X, where H (0) represents the input matrix of the input layer, and X represents the node attribute matrix of the attribute network; A is the adjacency matrix corresponding to the network, and I is the identity matrix; the matrix is the degree matrix of the matrix , that is W (l) is the parameter matrix learned by the l-th convolutional layer, which determines the influence degree of neighbor node information on the node's own information; σ(·) is a non-linear activation function, and the activation function that can be used is the ReLU function, ReLU(x) = max(0, x). The calculation process of a GCN layer can be divided into two stages: information aggregation and information activation. In formula (1), the operation performed in the information aggregation stage is In this stage, the attribute information of the node itself and the attribute information of the node's direct neighbors are aggregated together, and the parameter matrix W (l) is used to achieve the compression (or expansion) of the feature dimension. Since the operations in the information aggregation stage are all implemented by matrix multiplication, the operations in the information aggregation stage are all linear; the operation in the information activation stage is to apply the non-linear activation function σ(·) to the result of information aggregation. The non-linear transformation introduced thereby is the key for GCN to learn the non-linear dependence relationship between data. For formula (1), there are two points to note: one is that for a given attribute network, the structure matrix A and the attribute matrix X are determined, so in the formula can be directly calculated before model training, which can reduce the calculation time during model training; the other is that the attribute matrix A in GCN is shared by all layers, while the parameter matrix W (l) is shared by all nodes when aggregating neighbor information in the l-th layer. In the embodiments of this case, the neural network layer used can be composed of three stacked neural networks, that is, the final node representation vector considers the information of neighbors within 3 orders of the node, and the calculation process can be expressed as:

[0038]

[0039]

[0040]

[0041] Among them, h0 is the feature dimension corresponding to the attribute matrix X, and h1 is the feature dimension corresponding to the first hidden layer; similarly, H (i) , i ∈ {1, 2, 3} is the node feature matrix after being calculated by the i-th layer, and H (3) is the final embedding vector Z, h3 is the dimension of the final embedding vector, and the feature vectors in H (3) contain the information of the node's third-order neighbors.

[0042] When performing network embedding, GCN takes into account the influence of the node's high-order neighbors, which enables the speculation of the attribute information of some nodes even when their attribute information is missing, alleviating the challenges brought by data sparsity to a certain extent. On the other hand, when aggregating the information of neighbor nodes, GCN uses the non-linear activation function ReLU function, which can learn the non-linear dependence relationship between data, and the step of aggregating attribute information based on the structure matrix A considers the dependence relationship between the attribute network structure information and the attribute information. Therefore, GCN can be used to realize the joint embedding of the attribute network structure information and the attribute information.

[0043] Furthermore, in the embodiments of this case, when obtaining the node reconstruction error, the activation function tanh(·) or sigmoid(·) is selected as the prediction function of the structure matrix, and the graph convolutional neural network stacked by three neural network layers is used to predict the attribute matrix; the reconstruction errors of the structure information and the attribute information are obtained based on the structure matrix, the attribute matrix, and their prediction results. The similarity between node vectors is used to measure the reconstruction errors between the node structure matrix and the attribute matrix and their corresponding prediction results.

[0044] The difference between the original data and the estimated data, also known as the "reconstruction error", is an important indicator to measure the abnormality degree of data points. For the convenience of description, the predicted structure information is denoted as The predicted attribute information is The structure information reconstruction error is The attribute information reconstruction error is

[0045] The problem of predicting the network structure information can be reduced to: given nodes v i and v j corresponding embedding vectors z i and z j , determine nodes v i and vj Whether there is an edge connection between them, that is Whether it is equal to 1. Inspired by the work of Ding et al. [7], this paper uses the activation function f(·) as the prediction function of the structural information A, that is:

[0046]

[0047] Different from the sigmoid(·) activation function used, in the embodiments of this case, the hyperbolic tangent activation function tanh(·) can be used. The final form of the structural information prediction function is shown in formula (6):

[0048]

[0049] See Figure 3 As shown, the choice of the activation function tanh(·) or sigmoid(·) as the prediction function of the structural information A is determined by the special properties of the matrix A. The elements A in the matrix A i,j ∈{0,1}, and |tanh(·)|∈[0,1), sigmoid(·)∈(0,1). The absolute value of the function is always within the interval [0,1]. Therefore, the problem of predicting the element values in the matrix A can be reduced to finding a suitable threshold p th such that when Otherwise, The main difference between the sigmoid(·) function and the tanh(·) function is that the tanh(·) function is symmetric about the origin of coordinates. In the real number domain, the probability that the dot product result of two vectors is positive or negative is equal, that is, the dot product result of two vectors is symmetric about the value 0. Therefore, using the absolute value of the tanh(·) function symmetric about the origin as the probability of the edge connection between two nodes is more in line with the actual situation.

[0050] The process of using GCN for network embedding can be regarded as a process of dimension reduction of the attribute information of each node with the help of the adjacency matrix A. Therefore, the process of predicting the node attribute information from the final node embedding vector can be regarded as a process of dimension expansion of each node embedding vector. Inspired by this, in the embodiments of this case, the prediction of the node attribute information is realized with the help of GCN. Similar to the attribute network embedding process, the attribute information prediction network still uses a three-layer GCN network to implement, but the input layer in the attribute network embedding process is used as the output layer of the prediction network, and the output layer in the attribute network embedding process is used as the input layer of the prediction network. The calculation process of the attribute information prediction network is as follows:

[0051]

[0052]

[0053]

[0054] Prediction matrix based on attribute information and prediction matrix of structural information The reconstruction error of structural information can be expressed as The reconstruction error of attribute information is where, ||·|| F represents the Frobenius norm of the matrix It should be noted that R A and R X respectively refer to the sum of the reconstruction errors of the structural information and attribute information of all nodes in the entire network, which measures the reconstruction error at the network level and can be used as part of the loss function to train the GCN network, so as to obtain a more accurate embedding vector Z. In the process of generating the final representation vector of the node by splicing the reconstruction error with the embedding vector Z, the reconstruction error refers to the reconstruction error at the node level, and the similarity between vectors should be used for measurement, such as Euclidean distance, Manhattan distance, cosine similarity, Jaccard similarity, etc. In the embodiments of this case, z xr , z ar respectively represent the reconstruction errors of the attribute information and structural information at the node level. When multiple different vector similarity measurement methods are adopted, z xr , z ar are multi-dimensional.

[0055] Furthermore, in the embodiments of this case, a Gaussian mixture model is adopted as the probability distribution model for detecting node anomalies in the attribute network, and the density estimation of the attribute information and structural information in the attribute network is completed by solving the Gaussian mixture model; the abnormal nodes are obtained according to the probability distribution in the density estimation results.

[0056] Currently, many related works have discovered abnormal nodes by studying the reconstruction error and achieved good results. However, when the complexity of the representation learning model used is relatively high or the data itself has structurally complex noisy data, a large number of abnormal samples will show normal-level errors. Considering the embedding vector of the node itself and the reconstruction error can successfully cope with the challenges brought by this. Inspired by this, in the embodiments of this case, the form of splicing the node reconstruction error with the node's own embedding vector is used as the final representation vector of the node. The density estimation method is a common method for discovering abnormal nodes based on low-dimensional embedding vectors. Density estimation means: given a finite number of sample points x1, x2, x3,..., x n , the probability distribution p(x) of the variable x is modeled, and the abnormal points refer to the sample points with relatively low occurrence probabilities in the sample set.

[0057] In the embodiments of this case, the probability distribution model selected for modeling the node embedding vectors can be the Gaussian Mixture Model (GMM). The GMM is a commonly used model for simulating the joint distribution of multiple random variables. The GMM is a combination of K Gaussian distributions, and its probability distribution form is p(x|Θ) = ∑ k α k N(x; μ k , σ k ), where is the probability coefficient of the k-th sub-distribution, and N(x; μ k , σ k ) is a Gaussian distribution with an expectation of μ k and a standard deviation of σ k . Θ = (α1, α2, …, α K ; μ1, μ2, …, μ K ; σ1, σ2, …, σ K ). The process of density estimation is the process of finding the GMM parameter Θ, and the process of solving Θ is usually solved alternately by means of the Expectation-Maximization (EM) algorithm. However, this idea of alternating solution separates the embedding step of the attribute network and the density estimation process based on the node embedding vectors, and it is difficult to achieve joint optimization during the training process. To address this challenge, a separate neural network can be used to solve the GMM model, and good results have been achieved. Therefore, in the embodiments of this case, a Multi-Layer Perceptron (MLP) is used to solve the GMM model. The MLP takes the embedding vectors of the nodes as input and outputs the probabilities that the nodes belong to each sub-distribution of the GMM. The process of density estimation of the network attribute information can be expressed as:

[0058]

[0059] The process of density estimation of the network structure information can be expressed as:

[0060]

[0061] where z x , z a respectively represent the final representation vectors of the network attribute information and the structure information, and z xr , z ar respectively represent the reconstruction errors of the attribute information and the structure information. Given N samples and their probabilities of belonging to the sub-distribution k, 1 ≤ k ≤ K the parameters of the GMM model can be estimated by the following formula:

[0062]

[0063] where is the final embedding vector z of node v i and the probabilities corresponding to each sub - distribution i respectively are the probability coefficient, expectation, and covariance corresponding to the k - th sub - distribution of the GMM model. Here, the sub - distribution is regarded as a multivariate Gaussian distribution. Based on the predicted GMM parameter values, the sample energy can be calculated by the following formula: where |·| represents the determinant of a square matrix. The sample energy reflects the probability of the sample appearance. The larger the sample energy, the lower the probability of the sample appearance.

[0064]

[0065]

[0066] Given N samples x1, x2, x3, …, x n , in the DADDE model framework, anomaly detection from the perspective of attribute information and anomaly detection from the perspective of structural information are carried out separately, but the form of the loss function is the same. Therefore, the loss function can be expressed as:

[0067]

[0068] where θ e refers to the parameters of the GCN network in the attribute network embedding step, θ m refers to the parameters of the MLP network used to solve the GMM parameters, θ d refers to the parameters of the information reconstruction network in the reconstruction error calculation step, that is: when reconstructing attribute information, θ d refers to the GCN network used for attribute information reconstruction; when reconstructing structural information, θ d refers to the decision threshold p in the tanh(·) function used for structural information reconstruction th . Equation (14) consists of three parts: In the first part, R A / X represents the structural information reconstruction error R A or the attribute information reconstruction error R X . When detecting anomalies for attribute information, R A / X is taken as R X ; when detecting anomalies for structural information, R A / X is taken as R A . Intuitively, the smaller the reconstruction error, the more the node embedding vector can contain the key information in the node, and the more accurately the embedding vector depicts the node. In the second part, E(z i ) represents the energy of the sample value. By minimizing the sample energy, the observation probability of the sample value can be maximized, which can make the estimation of the GMM parameters by the MLP network more accurate. The third part introduces ​It is to solve the singularity problem existing in GMM: when the diagonal terms of the covariance matrix approach 0, it may lead to the appearance of trivial solutions, making the parameters of the GMM model no longer updated. Here, it can be set that To avoid problems caused by too small numerical values of the diagonal terms of the covariance matrix. The parameters λ1 and λ2 in formula (14) can be hyperparameters specified manually. In the experiment, λ1 = 0.1 and λ2 = 0.001 can be set.

[0069] To verify the effectiveness of the solution in this case, the following will make further explanatory notes in combination with common datasets:

[0070] Select three datasets widely used in existing research in this field. As shown in Table 1, since there are no abnormal data in the datasets themselves, for experimental verification, abnormal data can be injected into the datasets. Select the commonly used abnormal injection method: by changing the topology of part of the network

[0071] Table 1: Statistical Table of Dataset Information

[0072] BlogCatalog Flickr ACM Number of nodes 5196 7575 16484 Number of edges 171743 239738 71980 Total number of abnormal nodes 270 390 810 Proportion of abnormal nodes 5.2％ 5.1％ 4.9％

[0073] In the experiment, the precision P, recall rate R, and F1 value commonly used in the field of data classification are used as evaluation indicators. According to the confusion matrix of the classification results, the abnormal categories in the data are regarded as positive examples, and the precision P, recall rate R, and F1 value can be calculated respectively:

[0074] In the experiment, the learning rate of the DADDE model is set to 0.0001, and the GCN network used in the attribute network embedding step is set to a three-layer structure. The number of neurons in each layer of the network is respectively The GCN network used to predict the node attribute information in the reconstruction error calculation is also set to a three-layer structure. The number of neurons in each layer of the network is respectively And it satisfies The node reconstruction error is measured by the Manhattan distance and cosine similarity: Given the sample x and its predicted value x′, the calculation formula of the Manhattan distance is ∑ i |x i -x′ i |, and the calculation formula of the cosine similarity is The number of GMM sub-distributions in the density estimation step is denoted as K, and the MLP is set to a three-layer network structure. Denote the number of neurons in each layer of the network as respectively Set dropout = 0.2, and use the Adam algorithm to optimize the model during the training process. Mark the top 3% of the nodes with the highest anomaly degree as abnormal nodes, and in the DOMINANT model, mark the top 6% of the nodes with the highest anomaly degree as abnormal nodes. Among them, the DOMINANT model can detect both attribute abnormal nodes and structural abnormal nodes at the same time. The number of the two types of abnormal nodes in the dataset is equal. Therefore, the threshold of the DOMINANT model should be twice that of the DADDE model. The number of neurons in each network layer of DADDE and the value of K are shown in Table 2:

[0075] Table 2: The number of neurons in each network layer of DADDE

[0076]

[0077]

[0078] In the experiment, the value of K can be selected according to the Akaike information criterion. The Akaike information criterion (AIC) is a standard for measuring the complexity of a statistical model and the goodness of fit of the model. The smaller the AIC value, the better the model performance. Figure 4 is the curve graph of the change of the AIC value with the GMM model parameter K under the structural information data and attribute information data of BlogCatalog, Flickr, and ACM. From Figure 4 it can be seen that the structural information data of BlogCatalog achieves the best fitting effect when K = 3, and the rest of the data achieve the best fitting effect when K = 2.

[0079] The experiment separately counted the structural anomalies and attribute anomalies in the dataset, and the results are shown in Table 3:

[0080] Table 3: Precision P, recall R, and F1 value corresponding to different anomaly detection methods

[0081]

[0082] The horizontal line "-" in the table indicates that the corresponding anomaly detection method cannot be applied to the corresponding data type. For example, the SCAN method is used to detect anomalies in network structure information and cannot be applied to network attribute data. As can be seen from Table 3, the DADDE model achieved the best results on 4 (4 / 6) types of data, the DOMINANT model and OC-SVM each achieved the best results on 1 (1 / 6) type of dataset. The precision P, recall R, and F1 values of SCAN and LOF were poor and did not achieve the best results on any dataset. The OC-SVM based on kernel trick performed better than the traditional density-based anomaly detection method LOF. However, due to the challenges brought by data sparsity, it is difficult for OC-SVM to accurately characterize the key features of network node attribute information, which limits the anomaly detection performance of the method. The DADDE model and DOMINANT model essentially sort network nodes according to the degree of anomaly from high to low and determine abnormal nodes through a manually specified threshold. Therefore, in application scenarios that require a high recall rate, the recall rate of the model can be increased by lowering the determination threshold. Conversely, the precision of the model can be increased by raising the determination threshold.

[0083] Unless otherwise specifically stated, the relative steps, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0084] Finally, it should be noted that the above-described embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, and are not intended to limit it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions described in the foregoing embodiments or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention and should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for detecting deep anomalies in network nodes based on density estimation, characterized in that, It includes the following content: Obtain the node attribute matrix and the structure matrix in the attribute network, and use the attribute matrix and the structure matrix as the input of the graph convolutional neural network. Utilize the graph convolutional neural network to obtain the joint embedding vectors of the node structure information and the attribute information in the attribute network. Among them, the graph convolutional neural network is composed of k stacked neural network layers, and the node embedding vectors containing neighbor information are obtained through iterative calculation. The iterative calculation is as follows: h0 is the feature dimension of the attribute matrix X, and h1 is the feature dimension of the hidden layer; H (i) is the node feature matrix after the calculation of the i-th layer, and i ∈ {1, 2, 3}. Z is the finally obtained node embedding vector, h3 is the dimension of the node embedding vector, and the feature vectors in H (3) contain the node three-order neighbor information; The node attribute matrix and the structure matrix input to the graph convolutional neural network are respectively reconstructed by using the joint embedding vector. The activation function tanh(·) or sigmoid(·) is selected as the prediction function of the structure matrix, and the graph convolutional neural network stacked by three neural network layers is used to predict the attribute matrix; the node reconstruction errors regarding the structure information and the attribute information are obtained based on the structure matrix, the attribute matrix and their prediction results; For the joint embedding vector and the node reconstruction error, a probability distribution model is used to detect node anomalies in the attribute network.

2. The method for detecting network node depth anomalies based on density estimation according to claim 1, wherein, The attribute network is represented as G = <V, E, X>, where V represents the node set of the attribute network G, E represents the edge set of the attribute network, X represents the node attribute matrix of the attribute network, and A represents the structure matrix of the attribute network.

3. The method for detecting abnormal depth of network nodes based on density estimation according to claim 1, characterized in that The similarity between node vectors is used to measure the reconstruction errors between the node structure matrix and the attribute matrix and their corresponding prediction results.

4. The method for detecting abnormal depth of network nodes based on density estimation according to claim 1, wherein The Gaussian mixture model is used as the probability distribution model for detecting node anomalies in the attribute network. The density estimation of the attribute information and the structure information in the attribute network is completed by solving the Gaussian mixture model; the abnormal nodes are obtained based on the probability distribution in the density estimation results.

5. The method for detecting the depth anomaly of a network node based on density estimation according to claim 4, wherein In the process of solving the Gaussian mixture model, the node joint embedding vector is used as the input, and the probability that the node belongs to each sub-distribution of the Gaussian mixture model is used as the output, and a multi-layer perceptron is used for model solution.

6. The method for detecting network node depth anomalies based on density estimation according to claim 5, wherein The density estimation process of node attribute information is expressed as: The density estimation process of node structure information is expressed as: where z x and z a respectively represent the representation vectors of the attribute information and the structure information of the nodes in the attribute network, and z xr and z ar respectively represent the reconstruction errors of the attribute information and the structure information, and MLP() represents the solution of the multi-layer perceptron.

7. The method for detecting network node depth anomalies based on density estimation according to claim 1, wherein, By constructing a deep anomaly detection model framework, the deep anomaly detection model framework is used to obtain joint embedding vectors and node reconstruction errors and perform network node anomaly detection; among them, the deep anomaly detection model framework includes: a graph convolutional neural network for obtaining joint embedding vectors, a probability distribution model for detecting node anomalies, and an information reconstruction network for obtaining node reconstruction errors; the objective loss function of the model framework is expressed as: where N represents the number of samples, θ e , θ d , θ m respectively represent the network parameters of the graph convolutional neural network, the probability distribution model, and the information reconstruction network; R A / X represents the structural information reconstruction error R A or the attribute information reconstruction error R X , and its value is R X or R A ; λ1, λ2 are preset hyperparameters, E(z i ) represents the energy of the sample value z i ; where K represents the number of Gaussian components in the Gaussian mixture model, d refers to the dimension of the node representation vector, represents the j-th element on the main diagonal of the covariance matrix of the k-th Gaussian component.

Images