Multi-view unsupervised anomaly detection method based on optimal graph filtering

Abstract

The invention discloses a multi-view unsupervised anomaly detection method based on optimal graph filtering, and belongs to the technical field of anomaly detection. Aiming at the problems of low accuracy and poor robustness in an abnormal point detection process of an existing detection method, anomaly detection of multi-view data is carried out by introducing a high-order graph filtering mechanism and combining the consistency of a feature space and a neighborhood space. The method comprises the following steps: firstly, constructing a K neighbor graph of each view, and calculating the similarity by using a thermonuclear; then, a low-pass filtering mechanism is designed, and attribute abnormal points are detected through interaction of high-order neighborhoods; for category abnormal points, filtering parameters are learned by minimizing filtering differences between various views while sharing the parameters. And finally, calculating an overall anomaly score by combining the scores of the attribute anomaly and the category anomaly. The noise can be effectively eliminated, the detection precision is improved, and the method has wide applicability and efficient calculation performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of anomaly detection, and in particular relates to a multi-view unsupervised anomaly detection method based on optimal graph filtering. Background Art

[0002] Anomaly detection has always been a core technology in the fields of machine learning, data science, and network security. The discovery and mining of abnormal data and abnormal phenomena can reduce losses, reduce risks, and prevent security incidents. In recent years, research in the field of anomaly detection has attracted widespread attention from the whole society. Time series data represented by industrial manufacturing, power systems, and network operation and maintenance often have multi-variable and multi-attribute characteristics. Such data is complex and has many elements, and it is often difficult to discover anomalies in such data.

[0003] Unsupervised anomaly detection aims to identify unlabeled anomalies in a given dataset. This task is of great significance in the field of data mining and machine learning, and has been widely used in many applications such as social media, fraud detection, and information security. Traditional anomaly detection methods include distance-based, density-based, and clustering-based methods, which mainly target data from a single source, i.e., single-view data.

[0004] Multi-view data has complementary information from different views, depicts the features of objects from different angles and has richer information. Detecting outliers from multi-views has attracted more and more attention. The structure of outliers is also more complex than that of single-view data. Even if an instance is close to most instances, it may still be an outlier if it behaves inconsistently in all views. The three types of multi-view data outliers widely used in previous literature are defined as follows:

[0005] Class outliers: This type of outliers behave inconsistently in all views. Specifically, this type of outliers are usually close to their surrounding neighbors, but they may belong to different clusters or classes in different views.

[0006] Attribute outliers: This type of outlier is characterized by the fact that it exhibits significant differences from most other instances in each view, i.e., consistent abnormal behavior.

[0007] Class-attribute outliers: This type of outliers is a mix of the above two types. This means that in some views, they have the characteristics of class outliers, while in other views, they behave like attribute outliers.

[0008] So far, many multi-view anomaly detection methods have been proposed, which can be roughly divided into the following categories:

[0009] Clustering-based methods: These methods rely on clustering assumptions and may fail when the data does not have a clear clustering structure or the clustering method used cannot reveal the clustering structure of the data.

[0010] Methods based on reconstruction error: NCMOD uses the autoencoder reconstruction error to judge attribute anomalies and uses latent features to characterize the neighborhood errors of different views. SRLSP represents node features through nearest neighbor combination and minimizes the reconstruction error. IAMOD uses the prediction error of cross-view semantic information to give an inconsistency score.

[0011] Consensus-based methods: SRLSP learns a consensus neighborhood and minimizes the difference between it and each view to identify category anomalies. MODGD learns a clean consensus neighborhood to separate the structured anomaly representation of each view, and then measures the anomaly through the anomaly matrix. They are all inspired by MUVAD and compare the differences in local neighborhoods. CRMOD learns a consensus clustering indicator matrix based on DMOD to reduce parameters, and the relationship between MLRA and LDSR is similar.

[0012] However, these methods have several obvious shortcomings: 1. These methods often rely on low-order neighborhood information and ignore deeper high-order structural information, resulting in misjudgment when detecting category anomalies; 2. Most methods cannot effectively balance the information between different views and are easily disturbed by noisy views or poor quality views and complex weight optimization problems. 3. In reconstruction-based methods, most of them damage the information of abnormal nodes. For example, methods based on autoencoders, due to the powerful fitting ability of neural networks, will damage abnormal information when reconstructing abnormal nodes. In methods based on self-representation term reconstruction errors, abnormal points may still be well represented by their neighbors and thus misjudged as normal. Summary of the invention

[0013] In view of the problems of low accuracy and poor robustness of existing detection methods in the process of outlier detection, the present invention provides a multi-view unsupervised anomaly detection method based on optimal graph filtering.

[0014] In order to achieve the above object, the present invention adopts the following technical solutions:

[0015] A multi-view unsupervised anomaly detection method (MODGD) based on optimal graph filtering, the method comprising the following steps:

[0016] Step 1: Given a multi-view dataset X = {X 1 ,X 2 ,…,X V}, normalize the view data, you can use zscore or min-max normalization, where Represents the feature matrix of view v, where V represents the number of views;

[0017] Step 2: Construct the K nearest neighbor matrix of the view data, and use the heat kernel to calculate the similarity matrix of the data points at the corresponding positions Through the similarity matrix Construct the propagation matrix P of each view v , according to the propagation matrix P v , initialize weights and construct graph filter G v ;

[0018] Step 2.1: For view v, select data points The k nearest neighbor data points are calculated and the similarity matrix S is constructed based on the distance between neighbors using the heat kernel calculation. v , which is used to quantify the degree of connection between each data point and its neighboring data points; the similarity matrix element is defined as:

[0019]

[0020] in, and are the i-th and j-th data points in view v, respectively, θ is the bandwidth parameter, Represents the set of k nearest neighbors of data point i in view v;

[0021] Step 2.2: Based on the similarity matrix Sum degree matrix D v , construct the propagation matrix P v , used to propagate data point information and capture high-order neighborhood structure, the propagation matrix P v is defined as:

[0022]

[0023] Among them, I n is the n×n identity matrix;

[0024] Step 2.3: According to the propagation matrix P v , initialize weights and construct graph filter G v ; Graph filter G v is the propagation matrix P of each order v The polynomial combination of T-order graph filter is shown as follows:

[0025]

[0026] in, is the weight of the v-th view on the t-th order propagation matrix, which is used to adjust the contribution of graph filters of different orders in each view;

[0027] Step 3: Construct and optimize the objective function, use the relevant entropy induced metric to enhance the objective function, and obtain the optimized objective function;

[0028] Step 3.1: Construct the objective function as:

[0029]

[0030] Among them, λ is a balancing parameter that controls the contribution of filter differences between views; G v , G u It is a different view filter; The square of the F-norm representing the difference in filtering between different views; represents the weight matrix; the minimization objective function aims to find the optimal combination of filtering parameters for different views, so that the difference before and after filtering in the same view is minimized, and the filtering of different views is as close as possible;

[0031] Step 3.2: The relevant entropy induced metric loss function is:

[0032]

[0033] Here, δ is the same parameter as the Gaussian kernel width; is the loss of a single data point;

[0034] Step 3.3: According to the objective function and the related entropy induced metric loss function, the optimized objective function is obtained:

[0035]

[0036] Among them, δ v and δ vu Represent the kernel parameters of the two parts of loss respectively; and They represent the relevant entropy-induced metric forms of the two parts of the loss function in step 3.1 respectively.

[0037] Furthermore, an additional auxiliary variable z is introduced into the objective function in step 3 to transform the non-quadratic optimization problem into a quadratic optimization problem with an enhanced objective function:

[0038]

[0039] in, is a diagonal matrix that describes the auxiliary variables of the two parts under different views and view combinations; φ(Z v ) and φ(Q vu ) represent the conjugate functions of the two parts of loss respectively;

[0040] Among them, the diagonal elements and for:

[0041]

[0042] Among them, δ v and δ vu They represent the kernel parameters for constructing two auxiliary variables respectively;

[0043] Set the kernel parameter to the mean error:

[0044]

[0045] Where n represents the number of nodes;

[0046] When two auxiliary variables z and Q are given, the following quadratic programming problem about W is obtained:

[0047]

[0048] An alternating optimization strategy is used to optimize each variable in the above formula. For each view v, there are T filter parameters In each iteration, the filter parameters of the vth view are adjusted while the filter parameters of the remaining v-1 views are fixed.

[0049] Step 4: Calculate attribute anomaly scores separately and category anomaly scores The two are weighted and combined by adjusting the weighting parameter α to obtain the final anomaly score;

[0050] Step 4.1: Calculate attribute anomaly scores

[0051]

[0052] in, is the original feature of data point i in view v, is the embedding representation of the data point after graph filtering, and ∥·∥2 represents the L2 norm;

[0053] Step 4.2: Introduce the weight coefficient matrix W ij Weaken the impact of abnormal attribute scores:

[0054]

[0055] where ∈ is a smoothing parameter and ∈>0; used to prevent Too much weight caused by too small;

[0056] The category anomaly score for:

[0057]

[0058] in, and Denote the graph filter G v and G u The element at the i-th row and j-th column; n represents the number of nodes;

[0059] Combination and Calculate the final anomaly score:

[0060]

[0061] where α∈(0,1) is a balancing parameter.

[0062] Compared with the prior art, the present invention has the following advantages:

[0063] The present invention does not rely on clustering assumptions, but captures the high-order structural information of the data in each view through graph filtering. Through low-pass filtering, while obtaining the denoising effect, the anomaly is judged by comparing the differences between the nodes before and after filtering. At the same time, the attribute anomaly and category anomaly judgment are unified in the same objective function. In addition, the correlation entropy induced measure (CIM) is introduced to obtain the enhanced robust loss function, thereby compensating for the shortcomings of the traditional square loss function. In the optimization stage, the non-quadratic optimization problem is converted into a quadratic optimization problem with an enhanced objective function, and the semi-quadratic optimization technique is used to solve it. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are schematic diagrams of the present invention, which are used to better illustrate the content and implementation of the invention and to help understand the following description. The elements in the drawings are for illustrative purposes only and are not intended to be a detailed description of the content of the invention.

[0065] Figure 1 A schematic diagram of abnormal point types of the background technology of the present invention;

[0066] Figure 2 It is a two-dimensional space schematic diagram of the principle of detecting abnormal attributes;

[0067] Figure 3 Schematic diagram of the comparison between CIM loss and traditional L2 loss;

[0068] Figure 4 Schematic diagram of parameter sensitivity experiment. DETAILED DESCRIPTION

[0069] In order to gain a deeper understanding of the present invention, a comprehensive and detailed description will be given. However, the present invention has multiple implementations and is not limited to the specific examples listed herein. The presentation of these examples is intended to deepen the comprehensive understanding of the disclosure of the present invention.

[0070] The present invention provides a multi-view unsupervised anomaly detection method based on optimal graph filtering. In this example, experiments are carried out on four widely used real-world three-view datasets Handwritten, COIL20, WebKB, and Yale, which have 2000, 1600, 203, and 165 instances and 649, 6640, 2163, and 14150 features, respectively.

[0071] As shown in Table 1 below. The anomaly dataset is named by connecting the first letter of the dataset with the ID of the anomaly configuration. ρ1, ρ2, and ρ3 represent the proportions of attribute anomalies, category anomalies, and class attribute anomalies, respectively. For example, Y1 represents the Yale dataset, which contains 2%, 5%, and 8% of attribute anomalies, category anomalies, and class attribute anomalies, numbered 1. Repeat the experiment 5 times and record the average results.

[0072] Table 1 Abnormal proportions under different IDs

[0073]

[0074] Step 1: Given a multi-view dataset X = {X 1 ,X 2 ,…,X V},in Represents the feature matrix of view v, where V represents the number of views and the view data is normalized using zscore;

[0075] Step 2: Construct the K nearest neighbor matrix of the view data, and use the heat kernel to calculate the similarity matrix of the data points at the corresponding positions Through the similarity matrix Construct the propagation matrix P of each view v , according to the propagation matrix P v , initialize weights and construct graph filter G v ;

[0076] Step 2.1: For view v, select data points The k nearest neighbor data points are calculated and the similarity matrix is ​​constructed based on the distance between neighbors using the heat kernel calculation. This matrix is ​​used to quantify the degree of connection between each data point and its neighboring data points; the similarity matrix is defined as:

[0077]

[0078] in, and are the i-th and j-th data points in view v, respectively, θ is the bandwidth parameter, Represents the set of k nearest neighbors of data point i in view v;

[0079] Step 2.2: Based on the similarity matrix Sum degree matrix D v , construct the propagation matrix P v , used to propagate data point information and capture high-order neighborhood structure, the propagation matrix P v is defined as:

[0080]

[0081] Among them, I n is the n×n identity matrix;

[0082] Step 2.3: According to the propagation matrix P v , initialize weights and construct graph filter G v ; Graph filter G v is the propagation matrix P of each order v The polynomial combination of T-order graph filter is shown as follows:

[0083]

[0084] in, is the weight of the v-th view on the t-th order propagation matrix, which is used to adjust the contribution of graph filters of different orders in each view;

[0085] Step 3: Construct and optimize the objective function, use the relevant entropy induced metric to enhance the objective function, and obtain the optimized objective function;

[0086] Step 3.1: Construct the objective function as:

[0087]

[0088] Among them, λ is a balancing parameter that controls the contribution of filter differences between views; G v , G u It is a different view filter; The square of the F-norm representing the difference in filtering between different views; represents the weight matrix; the minimization objective function aims to find the optimal combination of filtering parameters for different views, so that the difference before and after filtering in the same view is minimized, and the filtering of different views is as close as possible;

[0089] Step 3.2: The relevant entropy induced metric loss function is:

[0090]

[0091] Here, δ is the same parameter as the Gaussian kernel width; is the loss of a single data point;

[0092] Step 3.3: According to the objective function and the related entropy induced metric loss function, the optimized objective function is obtained:

[0093]

[0094] Among them, δ v and δ vu Represent the kernel parameters of the two parts of loss respectively; and They represent the relevant entropy-induced metric forms of the two parts of the loss function in step 3.1 respectively;

[0095] An additional auxiliary variable z is introduced into the objective function to transform the non-quadratic optimization problem into a quadratic optimization problem with an enhanced objective function:

[0096]

[0097] in, is a diagonal matrix that describes the auxiliary variables of the two parts under different views and view combinations; φ(Z v ) and φ(Q vu ) represent the conjugate functions of the two parts of loss respectively;

[0098] Among them, the diagonal elements and for:

[0099]

[0100] Among them, δ v and δ vu They represent the kernel parameters for constructing two auxiliary variables respectively;

[0101] Set the kernel parameter to the mean error:

[0102]

[0103] Where n represents the number of nodes;

[0104] When two auxiliary variables z and Q are given, the following quadratic programming problem about W is obtained:

[0105]

[0106] An alternating optimization strategy is used to optimize each variable in the above formula. For each view v, there are T filter parameters In each iteration, the filter parameters of the vth view are adjusted while the filter parameters of the remaining v-1 views are fixed.

[0107] The present invention includes three hyperparameters: the number of neighbors K, the balance parameter λ, and the fusion parameter α. Fixed λ = 10 -5 Instead of tuning, a suitable K is selected from [5, 15, 25, 35]. The value of K is related to the size of the data set. For α, a good fusion parameter is usually selected from [0.3, 0.7]. In fact, unlike other methods that are sensitive to hyperparameters, the present invention is relatively stable to different hyperparameters and will not produce too much fluctuation.

[0108] When using quadprog in Matlab for optimization, we only need to calculate the coefficients of the quadratic term and the linear term. More specifically, for the first term:

[0109]

[0110] Where Tr represents the trace of the matrix; then the quadratic term:

[0111]

[0112] One-time term:

[0113]

[0114] For the second item:

[0115]

[0116] Then the quadratic term:

[0117]

[0118] One-time term:

[0119]

[0120] The alternating optimization subproblem is a standard convex quadratic optimization problem, that is, the Hessian matrices of the two parts are semi-positive definite. Take the proof of the first part as an example:

[0121]

[0122] in, represents the quadratic form of the matrix H;

[0123] The parameters of each view are optimized alternately, and the filter parameter matrix W is obtained.

[0124] Step 4: Calculate the anomaly score

[0125]

[0126] Fusion score:

[0127]

[0128] Table 2 shows the experimental results, which are compared with several popular multi-view anomaly detection (MVOD) methods, including CRMOD, DMOD, LDSR, MLRA, MUVAD-FSR, MUVAD-QPR, NCMOD, SRLSP, and MODGD (the present invention), as shown in the following table:

[0129] Table 2 Experimental results

[0130]

[0131] From the experimental results, it can be seen that the present invention is superior to the existing multi-view anomaly detection methods in most cases. It correctly utilizes the characteristics of neighbors of different orders to make the abnormal nodes change their behavior patterns more significantly. At the same time, focusing on high-order neighbors makes the present invention more reliable. At the same time, low-pass filtering smoothes the node noise and obtains a more robust feature representation.

[0132] from Figure 4 It can be seen that the present invention is stable over a wide range. Table 3 shows ablation experiments, demonstrating the importance of different components of the present invention, as shown in the following table:

[0133] Table 3 Ablation experiment

[0134]

[0135] Where: s a : Only use the scores in the feature space to identify anomalies;

[0136] Only unconstrained neighborhood scores are used to identify anomalies;

[0137] S c : Only constrained neighborhood scores are used to identify anomalies;

[0138] Unconstrained combined scores, using a combination of feature space scores and unconstrained neighborhood scores;

[0139] s a +s c : Constrained combined score;

[0140] Through the above implementation examples, the present invention provides a method and system that can efficiently detect and locate time series data anomalies. This implementation is only an example and does not limit the scope of the invention. It can be modified and changed according to specific needs.

[0141] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in the field. Although the illustrative specific embodiments of the present invention are described above to facilitate the understanding of the present invention by the technical personnel in the field, it should be clear that the present invention is not limited to the scope of the specific embodiments. For the ordinary technical personnel in the field, as long as various changes are within the spirit and scope of the present invention defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A multi-view unsupervised anomaly detection method based on optimal graph filtering, characterized in that: The method comprises the following steps: Step 1: Given a multi-view dataset X = {X 1 ,X 2 ,…,X V }, normalize the view data, where Represents the feature matrix of view v, where V represents the number of views; Step 2: Construct the K nearest neighbor matrix of the view data, and use the heat kernel to calculate the similarity matrix of the data points at the corresponding positions Through the similarity matrix Construct the propagation matrix P of each view v , according to the propagation matrix P v , initialize weights and construct graph filter G v ; Step 3: Construct and optimize the objective function, use the relevant entropy induced metric to enhance the objective function, and obtain the optimized objective function; Step 4: Calculate attribute anomaly scores separately and category anomaly scores The two are weighted and combined by adjusting the weighting parameter α to obtain the final anomaly score.

2. The multi-view unsupervised anomaly detection method based on optimal graph filtering according to claim 1, characterized in that: The step 2 constructs the K nearest neighbor matrix of the view data, and the heat kernel is used to calculate the similarity matrix of the data points at the corresponding positions. Through the similarity matrix Construct the propagation matrix P of each view v , according to the propagation matrix P v , initialize weights and construct graph filter G v The specific operations are: Step 2.1: For view v, select data points The k nearest neighbor data points are calculated and the similarity matrix S is constructed based on the distance between neighbors using the heat kernel calculation. v , similarity matrix elements is defined as: in, and are the i-th and j-th data points in view v, respectively, θ is the bandwidth parameter, Represents the set of k nearest neighbors of data point i in view v; Step 2.2: Based on the similarity matrix Sum degree matrix D v , construct the propagation matrix P v , used to propagate data point information and capture high-order neighborhood structure, the propagation matrix P v is defined as: Among them, I n is the n×n identity matrix; Step 2.3: According to the propagation matrix P v , initialize weights and construct graph filter G v ; Graph filter G v is the propagation matrix P of each order v The polynomial combination of T-order graph filter is shown as follows: in, is the weight of the v-th view on the t-th order propagation matrix, which is used to adjust the contribution of graph filters of different orders in each view.

3. The multi-view unsupervised anomaly detection method based on optimal graph filtering according to claim 2, characterized in that: The step 3 constructs the objective function, and uses the relevant entropy induced metric loss function to optimize the objective function. The specific operation of obtaining the optimized objective function is: Step 3.1: Construct the objective function as: Among them, λ is a balancing parameter that controls the contribution of filter differences between views; G v , G u It is a different view filter; The square of the F-norm representing the difference in filtering between different views; represents the weight matrix; Step 3.2: The relevant entropy induced metric loss function is: Here, δ is the same parameter as the Gaussian kernel width; is the loss of a single data point; Step 3.3: According to the objective function and the related entropy induced metric loss function, the optimized objective function is obtained: Among them, δ v and δ vu Represent the kernel parameters of the two parts of loss respectively; and They represent the relevant entropy-induced metric forms of the two parts of the loss function in step 3.1 respectively.

4. The multi-view unsupervised anomaly detection method based on optimal graph filtering according to claim 3, characterized in that: The step 4 calculates the attribute anomaly scores respectively. and category anomaly scores The specific operation of weighting the two by adjusting the weighting parameter α to obtain the final anomaly score is: Step 4.1: Calculate attribute anomaly scores in, is the original feature of data point i in view v, is the embedding representation of the data point after graph filtering, and ||·||2 represents the L2 norm; Step 4.2: Introduce the weight coefficient matrix W ij Weaken the impact of abnormal attribute scores: where ∈ is a smoothing parameter and ∈>0; used to prevent Too much weight caused by too small; The category anomaly score for: in, and Denote the graph filter G v and G u The element at the i-th row and j-th column; n represents the number of nodes; Combination and Calculate the final anomaly score: where α∈(0,1) is a balancing parameter.

5. The multi-view unsupervised anomaly detection method based on optimal graph filtering according to claim 4, characterized in that: An additional auxiliary variable z is introduced into the objective function in step 3 to transform the non-quadratic optimization problem into a quadratic optimization problem with an enhanced objective function: Among them, Z v , is a diagonal matrix that describes the auxiliary variables of the two parts under different views and view combinations; φ(Z v ) and φ(Q vu ) represent the conjugate functions of the two parts of loss respectively; Among them, the diagonal elements in the diagonal matrix and for: Among them, δ v and δ vu They represent the kernel parameters for constructing two auxiliary variables respectively; Set the kernel parameter to the mean error: Where n represents the number of nodes; When two auxiliary variables are given, the following quadratic programming problem about the weight matrix W is obtained: An alternating optimization strategy is used to optimize each variable in the above formula. For each view v, there are T filter parameters In each iteration, the filter parameters of the vth view are adjusted while the filter parameters of the remaining v-1 views are fixed.

Images