Public service venue data and government affair data anomaly detection model based on unsupervised difficult sample mining

Abstract

The invention relates to a public service venue data and government affair data anomaly detection model based on unsupervised difficult sample mining. A public service venue data and government affair data anomaly detection model based on unsupervised difficult sample mining comprises a neighborhood structure optimization module which jointly optimizes a neighborhood structure through a selective edge deletion strategy and structure enhancement of an input original graph; the feature extraction module comprises a comparison instance pair sampling module, a GNN module and a feature reading module and is used for performing feature extraction based on comparison learning; the neighborhood similarity reconstruction module is used for reconstructing intra-class and inter-class similarities of nodes to obtain similarity reconstruction loss Ls; and the learning target module is used for jointly completing a learning target through similarity reconstruction loss Ls and comparison loss Lc. According to the public service venue data and government affair data anomaly detection model based on unsupervised difficult sample mining, the anomaly detection performance in different scenes can be greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of data mining technology, and specifically relates to an anomaly detection model for public service venue data and government data based on unsupervised difficult sample mining. Background Art

[0002] As graph learning garners increasing attention in the field of data mining, anomaly detection on attribute graphs has gradually become one of the field's representative tasks. Its goal is to mine rare anomalous instances (nodes) due to attribute or structural anomalies from multiple normal instances (nodes). This task has been widely applied in a range of real-world security scenarios, including financial fraud detection, network intrusion monitoring, and digital government development. Currently, mainstream graph anomaly detection frameworks are primarily designed around feature extraction modules, such as contrastive learning modeling based on local subgraphs, multi-scale, and enhanced views. Despite their success, this unbiased approach lacks consideration of the anomalous samples that truly impact model performance.

[0003] As digital transformation deepens in government affairs and public service settings, massive amounts of data resources continue to emerge. Take government data, for example: government data is multi-source, heterogeneous, and dynamically interconnected, encompassing complex scenarios such as social security fraud detection, identifying anomalous behavior in public services, and analyzing cross-departmental collaborative networks. Government data governance faces challenges such as scarce annotations, strict privacy compliance requirements, and hidden anomalous patterns. Existing government anomaly detection methods rely on labeled data to build supervised models. However, in practice, high annotation costs, imbalanced sample distribution, and difficulty covering dynamically changing anomalous patterns limit model performance. Traditional rule-based or shallow statistical methods lack sensitivity to structural and contextual anomalies implicit in attribute graphs (such as anomalous associated nodes in cross-departmental collaborative networks and nodes tampering with social security data) and are susceptible to noise. Furthermore, the complex topological relationships in government data can easily lead to problems in information transmission: over-compression (abnormal node features are overwhelmed by normal node information), underreach (cross-level node dependency transmission fails), and oversmoothing (the representations of abnormal and normal nodes converge), limiting detection accuracy.

[0004] In view of this, the present invention proposes a new attribute graph anomaly detection model and method, which is an unsupervised difficult sample mining model HSGAD from the perspective of information transmission. It is also an unsupervised difficult sample mining framework for public service venues and government affairs. It eliminates redundant noise edges (such as abnormal associations in redundant approval processes) through neighborhood structure optimization, and combines contrastive learning with similarity reconstruction loss to enhance the distinguishability of abnormal node representation, breaking through the scarcity of labeled data and providing efficient detection tools for scenarios such as government service supervision and risk warning. Summary of the Invention

[0005] The purpose of the present invention is to develop an anomaly detection model for public service venue data and government data based on unsupervised difficult sample mining, and to solve the constraints of three types of difficult samples, namely over-compression, under-reachability and over-smoothing, on representation learning in abnormal samples from the perspective of information transmission. It can effectively alleviate the generation of abnormal samples caused by over-compression, under-reachability and over-smoothing, and greatly improve the anomaly detection performance in different scenarios.

[0006] In order to achieve the above objectives, the technical solutions adopted are:

[0007] Anomaly detection models for public service venue data and government data based on unsupervised difficult sample mining include:

[0008] Neighborhood structure optimization module: The input original graph is optimized through selective edge deletion strategy and structural enhancement to jointly optimize the neighborhood structure;

[0009] Feature extraction module, including: contrast example pair sampling module, GNN module, feature readout module, performs feature extraction based on contrastive learning;

[0010] Neighborhood similarity reconstruction module: reconstructs the similarity within and between node classes, and obtains the similarity reconstruction loss L s ;

[0011] Learning target module: Reconstruct the loss L through the similarity s and contrast loss L c , jointly complete the learning goal to enhance the similarity between nodes within a class and weaken the similarity between nodes between classes.

[0012] Furthermore, the neighborhood structure optimization module determines high-probability nodes by degree calculation and selects target edges for deletion.

[0013] Furthermore, the process of determining high probability nodes is as follows: calculating the average node degree of all nodes And After sorting in descending order, select the top-k1 nodes to form a node set The formula is as follows:

[0014]

[0015] in, Represents the boundary value corresponding to k1-th after all average node degrees are sorted in descending order;

[0016] The process of selecting the target edge to delete is as follows: Each node v i , calculate its neighboring nodes Node degree After sorting them in ascending order, select the top-k2 node set The formula is as follows:

[0017]

[0018] in, Represents node v i Neighborhood node v j The boundary value corresponding to k2-th after the node degrees are sorted in ascending order.

[0019] Furthermore, the structure is enhanced as follows: the side relationship e ij ∈[0,1] is enhanced, the formula is as follows:

[0020]

[0021] Furthermore, the sampling process of the comparative example is as follows:

[0022] First, in each round, all nodes in the graph are randomly traversed without duplication to determine the target node;

[0023] Then use random walk to determine the positive and negative instance pairs;

[0024] The information of the initial nodes in the positive and negative instance pairs is masked, and finally the target nodes are combined with the positive and negative instance pairs and sent into the GNN module.

[0025] Furthermore, in the GNN module, the positive and negative subgraphs are simultaneously processed by sharing the GCN. and With the target node v i The low-dimensional representation learning formula is as follows:

[0026]

[0027] in, Represents the subgraph adjacency matrix A i Normalized, self-connected subgraph adjacency matrix represents the degree matrix of the local subgraph, represents the weight matrix of the (l-1)-th layer, and d represents the dimension of embedding. and denote the hidden layer representations of the l-th layer and the (l-1)-th layer, respectively, and φ(·) denotes the activation function such as ReLU;

[0028] The weight matrix of GCN maps the target node and the local subgraph into the same embedding space, as follows:

[0029]

[0030] in, and Represent the target node v i The hidden layers learned by the l-th and (l-1)-th layers represent row vectors, and

[0031] Furthermore, the feature reading module uses average pooling to complete the information reading of the positive and negative local subgraphs, so that the node representation of the local subgraph matches the target node. The formula is as follows:

[0032]

[0033] Among them, (H i ) l Indicates H i The l-th row, n i express Number of midpoints

[0034] Furthermore, the similarity reconstruction loss L s The formula is:

[0035]

[0036] Where, α is a non-negative value that constrains the distance not to deviate; d(·) is the target node embedding h i and the embedding of positive and negative subgraphs e i + With e i - Similarity measure of ;

[0037] The contrast loss L c The formula is as follows:

[0038]

[0039] For a positive instance pair, y i =1, for negative instance pairs, y i =0.

[0040] Furthermore, the final objective function formula in the learning objective module is:

[0041] L=γL c +(1-γ)L s ;

[0042] Where γ is the equilibrium L c and L s Importance control parameters.

[0043] Another object of the present invention is to provide an anomaly detection method for public service venue data and government data based on unsupervised difficult sample mining, using the above-mentioned anomaly detection model to perform anomaly detection.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The anomaly detection model for public service venue data and government data based on unsupervised difficult sample mining described in the present invention discovered and established an unsupervised difficult sample mining model HSGAD from the perspective of information transmission. First, in order to alleviate the constraints of compressed difficult samples and reachable difficult samples, the present invention designed a two-stage neighborhood subgraph structure optimization module. Specifically, the target edge is first selected for deletion based on the calculation of the node degree, and then the structure is optimized in combination with the edge enhancement operation. Furthermore, in order to alleviate the limitations of smooth difficult samples, the present invention designed a neighborhood similarity reconstruction module, rethought the model training based on the similarity metric, and designed a novel similarity loss, which is jointly optimized with the contrast loss to enhance the similarity between nodes within the class and weaken the similarity between nodes between classes. HSGAD reduces the limitations of multi-class difficult samples generated during information transmission through neighborhood structure optimization and neighborhood similarity reconstruction, and efficiently completes attribute graph anomaly detection. The main innovations of the present invention are as follows:

[0046] 1. The technical solution of the present invention explores the problem of difficult sample mining in graph anomaly detection from the perspective of GNNs information transmission for the first time.

[0047] 2. The technical solution of the present invention systematically explores the problems of compressed difficult samples, accessible difficult samples and smoothed difficult samples caused by three types of phenomena in GAD's GNNs information transmission: over-compression, under-reachability and over-smoothing, and alleviates the generation of difficult samples in GAD by optimizing the neighborhood structure based on degree calculation and reconstructing the neighborhood similarity based on similarity measurement.

[0048] 3. In extensive experiments on six attribute graph anomaly detection datasets in synthetic and real scenarios, the technical solution of the present invention shows that HSGAD achieves the most outstanding performance on synthetic and real datasets, outperforming the suboptimal model by 4.59% and 22.6% respectively, demonstrating detection effectiveness and interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This paper provides a detailed description of the three problems of over-compression, under-reachability, and over-smoothing in graph anomaly detection. Figure (a) describes these three problems. Figure (b) describes the relationship between the three types of difficult samples caused by these three problems and the structural and contextual anomalies in GAD.

[0050] Figure 2 It is the overall framework of HSGAD;

[0051] Figure 3 ROC comparison of HSGAD and 7 baseline models on 6 datasets;

[0052] Figure 4 is the average node degree change of the data set before and after HSGAD processing;

[0053] Figure 5 The changes in similarity within the group during HSGAD training; (a) BC original (b) BC final (c) Flickr original (d) Flickr final (e) Weibo original (f) Weibo final

[0054] Figure 6 Visualize the embedding and topological relationships of complex data sets before and after HSGAD processing;

[0055] Figure 7 is the Loss balance parameter γw.rtAUC importance;

[0056] Figure 8 K1 and K2 values ​​wrtAUC in neighborhood structure optimization. DETAILED DESCRIPTION

[0057] In order to further illustrate the anomaly detection model of public service venue data and government data based on unsupervised difficult sample mining of the present invention and achieve the intended purpose of the invention, the following is a detailed description of the specific implementation method, structure, characteristics and efficacy of the anomaly detection model of public service venue data and government data based on unsupervised difficult sample mining proposed by the present invention in combination with the preferred embodiments. In the following description, different "one embodiment" or "embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0058] The following will further introduce the anomaly detection model of public service venue data and government data based on unsupervised difficult sample mining in detail with reference to specific embodiments:

[0059] Attribute graph anomaly detection is a key task in graph learning. Its goal is to capture anomalous nodes in a given graph network that differ from the majority node pattern. Currently, mainstream unsupervised graph anomaly detection models use carefully designed feature extraction modules to amplify the differences in representation between normal and anomalous nodes, thereby identifying anomalies. However, this unbiased approach lacks consideration of the difficult samples that truly impact model performance. In this example, we systematically analyze the difficult sample problems in graph anomaly detection caused by over-compression, underreachability, and oversmoothing during graph information transmission. Therefore, we propose an anomaly detection model for public service venue data and government data based on unsupervised hard sample mining, a novel High-Speed ​​Graph Detection (HSGAD) model. Specifically, we design a neighborhood structure optimization module to filter noisy edges and combine it with structural enhancement to optimize the graph structure, mitigating the generation of difficult compressed and difficult reachable samples. Furthermore, we design a similarity reconstruction loss based on similarity calculation, which mitigates the generation of difficult smoothed samples by guiding the model to increase inter-class differences and reduce intra-class differences. Extensive experimental results on six synthetic and real datasets demonstrate that HSGAD significantly improves the constraints of these three types of difficult samples, significantly enhancing anomaly detection performance in various scenarios.

[0060] In this embodiment, abnormal samples are also called difficult samples.

[0061] Example 1.

[0062] The specific steps are as follows:

[0063] ARelated Work

[0064] 1. Graph Representation Learning

[0065] The goal of graph representation learning is to learn representations of nodes or the entire graph to better perform downstream tasks (such as node classification and anomaly detection). Recent work has focused on optimizing graph representation learning from the perspectives of information transfer and graph structure learning. In graph structure learning, Pro-GNN achieves graph structure learning by eliminating carefully designed adversarial structures while preserving the intrinsic graph structure. SUBLIME optimizes graph structure in self-supervised contrastive learning by maximizing the consistency between the anchor graph and the learning graph. Most GNNs propagate node information through the information transfer paradigm. Research has shown that optimizing information transfer within the graph is a key approach to optimizing graph representation learning, and the ultimate goal of graph structure learning is to optimize graph information transfer. Two hot issues discussed in information transfer are over-compression and underreachability. To alleviate over-compression, BORF uses Ricci curvature to mitigate over-compression. SJLR addresses over-compression in GNNs by adding or removing edges during model training. PASTEL addresses over-compression in GNNs based on semi-supervised learning by enhancing connectivity between similar nodes. To alleviate the underreach problem, NodeMixup leverages graph structure to alleviate the underreach problem of GNNs in semi-supervised node classification by fusing neighbor connections of node pairs within a class. DiffWire designs CT-Layer and GAP-Layer based on the Lovász boundary to address this problem. EEGNN proposes an edge-enhanced GNN with a Bayesian non-parametric graph model to address this problem. To alleviate the oversmoothing problem, GraphCON alleviates this problem based on the discretization of second-order systems of ordinary differential equations (ODEs). DGN significantly alleviates this problem by normalizing the independence of nodes in the same group and separating the distribution of nodes in different groups. Existing studies have explored the oversmoothing problem in BERT from a graph perspective. Although the above work has explored the problems of over-compression, underreach, and oversmoothing from different perspectives, no work has systematically considered the GAD task. We hope to prevent the constraints of difficult samples in graph anomaly detection from the perspective of information transmission, thereby effectively improving the detection performance of the model.

[0066] 2. Anomaly Detection on Attribute Graphs

[0067] Anomaly detection aims to uncover rare data that deviates from the majority of normal data and is widely used in the real world. Compared to traditional anomaly detection techniques, attributed graphs have become a hot topic in anomaly detection due to their comprehensive consideration of both attribute and topological information. How to more fully exploit the data information in attributed graphs to better detect graph anomalies is a challenging problem. Some traditional shallow mechanisms attempt to detect graph anomalies based on residual analysis (such as Radar) and CUR decomposition (such as ANOMALOUS). After DGI first proposed contrastive learning in GNNs in an unsupervised manner, CoLA first proposed an unsupervised contrastive learning solution for graph anomaly detection. Contrastive learning has gradually become a mainstream approach for addressing anomalies in attributed graphs. ANEMONE proposed a multi-scale contrastive learning approach to learn consistency between patch and context-level instances. SL-GAD employed generative attribute regression and multi-view contrastive learning for graph anomaly detection. Sub-CR identified graph anomalies by jointly optimizing a multi-view contrastive learning module and an attribute reconstruction-based module. GRADATE proposed multi-scale contrastive learning with enhanced views for graph anomaly detection. In contrast, we analyzed and found that the causes of difficult samples that really affect the model detection performance are closely related to three types of problems: over-compression, underreach, and over-smoothing. We systematically explored these three types of problems and, based on neighborhood contrastive learning, designed neighborhood structure optimization and neighborhood similarity reconstruction modules to alleviate these three types of problems in GNNs information transmission, thereby completing attribute graph anomaly detection more efficiently.

[0068] BObservation and Rethinking

[0069] Graph neural networks (GNNs) aggregate and propagate information from neighboring nodes in an iterative manner, thereby updating the embedded representation of the target node. This embodiment takes into account that attribute graph anomaly detection is a category-biased node classification task, and the information transmission process also faces challenges such as over-compression, under-reachability, and over-smoothing of graph learning tasks, which makes it difficult to capture minority class samples (abnormal samples in GAD). Therefore, this embodiment specifically analyzes the generation process of over-compression, under-reachability, and over-smoothing phenomena in graph anomaly detection tasks. (In this embodiment, over-compression, under-reachability, and over-smoothing are not completely consistent with the current broad concepts. The definition, thinking, and solution of these three types of problems are mainly carried out for GAD tasks.) Combined with Figure 1 (a), specifically:

[0070] (1) Over-compression: When the information of abnormal node v1 and the information of multiple normal nodes at the same layer are transmitted to abnormal node v3 through only a single path, the information of multi-hop nodes is compressed into a fixed-size representation. That is, the information of node v1 is easily interfered by normal nodes, which causes the abnormal information aggregated by node v2 to be distorted. This makes nodes such as node v1 become difficult-to-compress samples in GAD.

[0071] (2) Underreach: When the information between abnormal nodes v2 and v4 needs to be transmitted through multiple hops to be obtained, the aggregation and propagation operations of GNNs are very likely to have insufficient information transmission within the receptive domain due to this long-range dependency, which makes nodes such as v2 or v4 become difficult-to-reach samples in GAD.

[0072] (3) Oversmoothing: Under the current definition of oversmoothing in GNN, this embodiment focuses on GAD. The difference is that GAD can be trained based on shallow GNN. Therefore, this embodiment mainly focuses on the problem of increasing abnormal nodes in the normal node representation area due to the low similarity of similar nodes and the high similarity of heterogeneous nodes, which causes difficult smoothing samples in GAD.

[0073] Figure 1 (b) describes the relationship between these three types of difficult examples and attribute and structural anomalies in the attribute graph. Structural difficult examples primarily include compression difficult examples and reachable difficult examples, while attribute difficult examples primarily include smoothing difficult examples. The intersection of attribute anomalies and structural anomalies encompasses all common anomaly examples. Therefore, combined with current research on difficult example mining in GNNs, addressing the constraints imposed by these three types of difficult examples on representation learning in GAD from the perspective of information transfer may provide a promising approach for attribute graph anomaly detection research.

[0074] C Method

[0075] 1. Problem Definition

[0076] Definition 1 (Attribute Graph Anomaly Detection): Define an attribute network where X∈R N×M Represents the attribute matrix, A∈R N×N represents the adjacency matrix. The node set of the network is x i ∈R M Represents node v i The i-th attribute of the network. The edge set of the network is E(|E|=E), e ij ={v i ,v j}∈E represents node v i With v j The goal is to calculate the anomaly score s through the model i =f(v i ) to measure Each node v i degree of abnormality.

[0077] Definition 2 (Hardness Score for sample): Given A node v in i , the embedding of positive and negative subgraphs e i + With e i - The similarity between -paradigm) and node degree Together they reflect the difficulty of the node. Calculate, d(·) is -norm distance( - paradigm), the difficulty score of the node is calculated as:

[0078] Definition 3 (Difficult Samples in Attribute Graph Anomaly Detection): Given a graph The hard samples that can be determined are based on the following assumptions: i A sample with a high difficulty score (i.e., its difficulty score is relatively large among all nodes) can be considered a difficult sample. It is worth noting that we use the sample difficulty score to quantify the probability of it being a difficult sample, and the difficulty of the sample is relative to other nodes in the graph.

[0079] This embodiment targets the attribute graph anomaly detection problem in Definition 1, uses Definition 2 as a measurement method for difficult samples to design a model, and comprehensively considers the difficult sample mining problem in Definition 3.

[0080] 2. Methods

[0081] Combine Figure 2 ,HSGAD’s neighborhood structure optimization module is implemented through selective edge deletion strategy and structure enhancement.,The feature extraction module is completed based on contrastive learning, and finally, the similarity reconstruction loss L,is used. s (reduce the intra-class differences and increase the inter-class differences) and contrast loss L c (Complete classification tasks) Jointly complete learning objectives.

[0082] The government attribute graph anomaly detection model based on unsupervised difficult sample mining of the present invention includes:

[0083] Neighborhood structure optimization module: The input original graph is optimized through selective edge deletion strategy and structural enhancement to jointly optimize the neighborhood structure.

[0084] The feature extraction module includes: a comparative example pair sampling module, a GNN module, and a feature readout module, which performs feature extraction based on contrastive learning.

[0085] Neighborhood similarity reconstruction module: reconstructs the similarity within and between node classes, and obtains the similarity reconstruction loss L s .

[0086] Learning target module: reconstruction loss L through similarity s and contrast loss L c , jointly complete the learning goal to enhance the similarity between nodes within a class and weaken the similarity between nodes between classes.

[0087] The data flow in this model is:

[0088] (1) Neighborhood structure optimization module (N-SO)

[0089] Data flow: The original adjacency matrix A is generated by edge deletion and enhancement to generate the optimized A enhanced , passed to the feature extraction module.

[0090] (2) Feature extraction module

[0091] Data flow: Optimized A enhanced The attribute matrix X is input into GCN to generate node embedding H, which is passed to the similarity module.

[0092] (3) Neighborhood similarity reconstruction module

[0093] Data flow: Node embedding H is input to the similarity module to calculate the loss, and the joint contrast loss is back-propagated to update the model parameters.

[0094] The specific details of each module are as follows:

[0095] (1) Neighbor-Structure Optimization (N-SO)

[0096] The impact of edge deletion on information transmission: The message passing mechanism of the k-th layer of GNN is defined as follows:

[0097]

[0098] in, and φ(·) represents an arbitrarily differentiable network. represents the feature representation of node i in the kth layer, Represents the node v i The message passing process is done by connecting the edge e ij Aggregate the feature information of adjacent nodes. The function Agg(·) defines the aggregation mode and the different aggregation functions corresponding to different GNN layers. However, the quality of the input graph will seriously affect the representation learning of the graph neural network. The messages gathered along the "noise" edge will reduce the quality of node embedding to a certain extent. This degree increases with the number of "noise edges" and the increase in aggregation effect. In the GAD task, from the perspective of information transmission, it is faced with Figure 1The risks of over-compression and under-reachability in a will increase sharply when multiple GNN layers are stacked.

[0099] Definition 4 (Graph noise filtering based on edge deletion): Given a graph Where X represents the attribute matrix and A represents the adjacency matrix. When edge deletion is used to optimize information aggregation, based on Selectively filter edge relationships, where is the optimized adjacency matrix, ⊙ represents the element-by-element product. Matrix C is an edge deletion matrix that determines whether the corresponding edge should be deleted. C is defined by Bernoulli distribution, where each element C ij ~Bernoulli(ρ). This means that C ij =1 is ρ, indicating that node v i and v j The edges between should be preserved, and C ij The probability that =0 is (1-ρ).

[0100] Based on the above principles, this module effectively solves the compression-hard and reach-hard-hard sample problems in GAD tasks through neighborhood structure optimization. The node degree in a graph network reflects the substructure of the node, and moderate adjustments to the node degree can effectively optimize graph learning. Therefore, this embodiment uses degree calculation to determine the edge deletion matrix C in Definition 4 for neighborhood structure optimization. After determining high-probability nodes, it further selects target edges for deletion to filter out noisy edge relationships, and combines this with structural enhancement to jointly optimize the neighborhood structure.

[0101] Specifically:

[0102] ①Identify high-probability nodes

[0103] Identifying high-probability nodes This part is used to determine the high-probability nodes that cause over-compression and under-reachability in GNNs information transmission. Figure 1 It is not difficult to find that the higher the average node degree, the higher the probability of being associated with difficult samples, because complex edge relationships are more likely to cause information distortion or remote dependency due to redundant structural information, which will directly lead to underreach and overcompression problems in the GAD process. Based on this idea, this embodiment calculates the average node degree of all nodes And After sorting in descending order, select the top-k1 nodes to form a node set The formula is as follows:

[0104]

[0105] in, Indicates the boundary value corresponding to k1-th after all average node degrees are sorted in descending order.

[0106] ②Select and delete the target edge

[0107] Based on the high probability nodes obtained in the previous step, this step further determines the target edges that are more likely to cause over-compression and under-reachability. Figure 1 It is not difficult to find that the difficult samples that the present invention wants to mine due to over-compression and under-reachability are more likely to be caused by the noise edge relationship between nodes with different structures. Therefore, this embodiment obtains v1 for each node v i , calculate its neighboring (first-order) nodes Node degree And sort them in ascending order (combined with the descending order in the previous stage to obtain the noise edges between nodes with dissimilar structures) and then select the top-k2 node set

[0108]

[0109] Similarly, Represents node v i Neighborhood node v j The k2-th node's boundary value after the node degree is sorted in ascending order. By removing edges to filter out graph structural noise and optimize the structure, HSGAD identifies the edges filtered by P as target edges that cause underreach and overcompression. By removing these target edges, HSGAD avoids the problem of insufficient model information capture caused by compression-hard and reach-hard samples in GAD information transmission.

[0110] ③Structural enhancement

[0111] Considering the problem that some edge information in the topological relationship is easily lost during the information transmission process due to the small connection weight, the following formula is used to calculate the edge relationship e ij ∈[0,1] is enhanced:

[0112]

[0113] (2) Feature extraction

[0114] The module consists of three parts: contrast instance sampling, GNN module, and feature readout, which realizes feature extraction based on contrastive learning.

[0115] ① Comparative example sampling

[0116] First, in each round, HSGAD performs a non-repeated random traversal of all nodes in the graph to determine the target node. Second, a random walk is used to identify positive and negative instance pairs (subgraphs), where the initial node of the positive instance pair is the target node, and the initial node of the negative instance pair is randomly selected from nodes other than the target node. Next, information masking is performed on the initial nodes of the positive and negative instance pairs, and finally, the target node is combined with the positive and negative instance pairs and fed into the GNN module.

[0117] ②GNN module

[0118] HSGAD uses a shared GCN approach to simultaneously align positive and negative subgraphs. and With the target node v i The low-dimensional representation learning formula is as follows:

[0119]

[0120] in, Represents the subgraph adjacency matrix A i Normalization of the subgraph adjacency matrix with self-connection represents the degree matrix of the local subgraph, represents the weight matrix of the (l-1)-th layer, and d represents the dimension of embedding. and denote the hidden layer representations of the l-th layer and the (l-1)-th layer, respectively, and φ(·) denotes the activation function such as ReLU.

[0121] Considering that the target node information is obscured, this embodiment uses the GCN weight matrix to map the target node and the local subgraph into the same embedding space. The formula is as follows:

[0122]

[0123] in, and Represent the target node v i The hidden layers learned by the l-th and (l-1)-th layers represent row vectors, and

[0124] ③Information readout (i.e. feature readout)

[0125] In order to make the node representation of the local subgraph match the target node, HSGAD uses average pooling to complete the information reading of positive and negative local subgraphs. The formula is as follows:

[0126]

[0127] Among them, (Hi ) l Indicates H i The l-th row, n i express The number of nodes in the middle. At this time, the anomaly score of the positive instance pair is:

[0128]

[0129] Among them, W s is a learnable weight matrix, and σ(·) is the logistic sigmoid function. Similarly, the anomaly score of the negative instance pair is

[0130] (3) Neighbor-Similarity Reconstruction

[0131] In order to effectively alleviate the problem of smoothing difficult samples, the present invention analyzes and finds that the key factor leading to the over-smoothing problem in GNNs is the excessive mixing of effective information and noise, which leads to insufficient distinction between nodes. Therefore, the similarity within and between nodes is reconstructed based on similarity.

[0132] ①Neighborhood similarity

[0133] This example focuses on how to effectively increase inter-class differences while reducing intra-class differences. Based on the joint loss, the following design is implemented:

[0134] α+d(h i ,e i +) <d(h i ,e i - )

[0135] Where d(·) is the target node embedding h i and the embedding of positive and negative subgraphs e i + With e i - Similarity measures (such as -norm), α is a non-negative value that constrains the distance not to deviate.

[0136] ②Similarity loss

[0137] This example carefully designs the above formula as a similarity reconstruction loss to guide model training:

[0138]

[0139] Ideally, as the model training converges, the similarity reconstruction L sThe intra-class similarity will be enhanced by expanding the similarity between positive and negative instance pairs, while reducing the inter-class similarity.

[0140] (4) Learning objective

[0141] This embodiment uses similarity reconstruction loss L s Effectively mine difficult samples caused by over-smoothing due to high similarity in graph anomaly detection tasks. At the same time, based on the contrast loss L c Complete node classification prediction for class imbalance in graph anomaly detection, L c The formula is as follows:

[0142]

[0143] For a positive instance pair, y i =1, for negative instance pairs, y i = 0. In order to achieve the above learning objectives, it is necessary to jointly optimize the two losses. Finally, the objective function formula of this embodiment is:

[0144] L=γL c +(1-γ)L s

[0145] Where γ is the equilibrium L c and L s Importance control parameters.

[0146] In order to prevent the accidental perception of abnormal information in local sampling, this embodiment completes the calculation of abnormal scores of nodes on the graph based on multiple rounds of sampling. i The prediction score of the positive instance pair is calculated according to Calculate, where R is the sampling round, The target node v is the r-th round pair i The prediction score of the positive instance pair. Similarly, we can get v i The prediction score of the negative instance pair Final node v i The anomaly score is calculated according to Calculate, where s(·) is the final node anomaly score evaluation function.

[0147] D experiment

[0148] This embodiment conducts experiments to answer the following questions.

[0149] Q1: Does HSGAD perform well on datasets of different scenarios and structural complexity?

[0150] Q2: What is the impact of HSGAD’s neighborhood structure optimization module and similarity reconstruction loss on the model?

[0151] Q3: What is the difference between the selective edge deletion strategy in the neighborhood structure optimization module and the random deletion of the same number of edges?

[0152] Q4: Does HSGAD effectively alleviate over-compression, underreachability, and over-smoothing in GAD?

[0153] Q5: How do key hyperparameters affect HSGAD performance?

[0154] 1. Experimental Setup

[0155] (1) Dataset: This embodiment selects public datasets with injected anomalies and real anomalies to cross-validate the effectiveness of HSGAD. Specifically, for the datasets with injected anomalies, we select two citation network datasets, Cora and Citeseer, and two social network datasets, Flickr and BlogCatalog, Following CoLA. This embodiment injects the same number of attribute anomalies and structural anomalies into the original dataset. For the real-world datasets, this embodiment selects the Books and Weibo datasets. These datasets contain different proportions of structural anomalies and attribute anomalies. Please see Table 1 for detailed statistical information. The last two columns are and S (see Definition 4) are the node average degree and difficulty score of the dataset respectively. The reason why the difficulty scores of the first three datasets are very low is that the node average degree is low, that is, their structure is simple, and the possibility of compression difficult samples and reachable difficult samples is extremely low. Therefore, they are no longer processed by the neighborhood structure optimization module. In this embodiment, the three complex structures ( The effectiveness of HSGAD in anomaly detection was verified in detail on datasets such as BlogCatalog, Flickr, and Weibo (which are often challenging to detect). To further explore its effectiveness on simpler datasets, we conducted supplementary experiments on Cora, Citeseer, and Books, which have different types and anomaly scenarios.

[0156] Definition 5 (Hardness Score for Dataset): Given a The difficulty score of all nodes is averaged as the difficulty score of the dataset. The average similarity of nodes is And normalize it to [0,1], the average degree of the node is The difficulty score of the dataset is

[0157] (2) Baseline model: This example selects seven well-known attribute graph anomaly detection models from 2018 to 2023 for comparative experiments. For detailed information, please see the first column of Table 2. The first model is a non-deep algorithm, and the others are all based on graph neural network methods.

[0158] Metrics: This is the evaluation method for the current mainstream graph anomaly detection model. This example also uses AUC as the evaluation metric.

[0159] Table 1. Statistics of the dataset.

[0160]

[0161]

[0162] 2. Experimental results and analysis

[0163] In this example, the performance of HSGAD is compared with that of 7 baseline models on 6 datasets of different types and abnormalities. The specific results are shown in Table 2. The corresponding ROC curves are shown in Table 2. Figure 3 shown.

[0164] Table 2 shows the comparison of AUC values ​​on the six datasets. The best performance on each dataset is highlighted in black, and the suboptimal performance is underlined.

[0165] Table 2

[0166]

[0167] Answer to Q1: Based on Table 1-2, we can observe that:

[0168] 1) HSGAD outperforms the baseline model on all datasets, indicating that it can effectively mine anomalies in both simulated and real scenarios.

[0169] 2) The performance of all models on the real anomaly datasets Books and Weibo is lower than that on the four synthetic anomaly datasets, indicating that the environment in real scenarios is more complex and anomalies are more difficult to capture.

[0170] 3) HSGAD’s advantage over the suboptimal model is even more pronounced on real anomaly datasets, with increases ranging from 0.28% to 4.59% on injected anomaly datasets and 7.16% to 22.6% on real anomaly datasets. This demonstrates that HSGAD is more sensitive to the complex contextual relationships of real anomalies.

[0171] 4) For the three complex datasets (BC, Flicker, and Weibo), HSGAD shows greater improvements regardless of whether they are injected with anomalies or real anomaly datasets. This is mainly because these datasets have serious problems generating three types of difficult samples during information transmission. The model's neighborhood structure optimization module and neighborhood similarity reconstruction module achieve significant systematic relief for these three types of problems in graph networks.

[0172] 3. Ablation Experiment

[0173] (1) Experiments on Q2: On 6 public datasets, this embodiment conducts comparative experiments on 3 variants of the HSGAD model: 1) w / o N-SO: remove the neighborhood structure optimization module. 2) w / o L s : Remove similarity reconstruction loss L s 3) w / o both: remove N-SO and L s , the results are shown in Table 3.

[0174] Answer to Q2: From Table 3, we can observe the following:

[0175] 1) After removing N-SO from the three complex datasets, the model performance decreased by 5.5%-38.4%, respectively, indicating that the proposed N-SO can effectively alleviate the over-compression and under-reachability phenomena.

[0176] 2) Remove the similarity reconstruction loss L s After that, the performance of all datasets decreased significantly, which further shows that the similarity reconstruction loss can effectively alleviate the over-smoothing problem, thereby improving the detection ability of HSGAD for difficult samples in different abnormal datasets.

[0177] 3) For the three data sets with complex structures, the N-SO module and L s The model performance further deteriorates, which indicates that the over-compression, under-reachability, and over-smoothing phenomena alleviated by HSGAD are complementary to hard sample mining.

[0178] (2) Experiments on Q3: This example further explores the difference between the selective edge deletion strategy (deleting target edges based on high-probability nodes) and random edge deletion in HSGAD using three complex datasets using the N-SO module. Specifically, this example is based on HSGAD and HSGAD w / o L s , replacing its edge deletion strategy with a random deletion of the same number of edges as a deformation, we get HSGAD* and HSGAD w / o L s *, the specific results are shown in Table 4.

[0179] Answer to Q3: From Table 4, we can observe the following:

[0180] 1) For HSGAD without L s * and HSGAD*, when their random edge deletion operations are replaced by the selective edge deletion strategy proposed in this paper, the model performance is improved by 0.8%-35.4% and 4.6%-10.7%, respectively. This shows that the random edge deletion operation may misjudge the noise edges, resulting in the loss of some effective information. The selective edge deletion strategy in N-SO can more accurately filter the noise edges based on the node degree calculation, thereby optimizing the graph structure.

[0181] 2) Without using similarity loss to alleviate the generation of smooth difficult samples, the random edge deletion and selective edge deletion strategies make the model perform similarly in the injected anomaly dataset, but the difference is too high in the real anomaly dataset, indicating that the selective edge deletion strategy increases the detection robustness of the model.

[0182] 3) When using similarity loss to alleviate the generation of smooth difficult samples, random edge deletion causes the model to show a significant performance degradation in various datasets compared to the selective edge deletion strategy, indicating that the selective edge deletion strategy effectively filters out noisy edges in various datasets, thereby improving the model's anomaly detection performance.

[0183] Table 3 Ablation experiment results of three variants of HSGAD.

[0184]

[0185] Table 4 shows the deformation ablation experimental results of HSGAD based on edge deletion. The ones marked with * are deformations in which the edge deletion strategy is replaced by random deletion of the same number of edges.

[0186] Table 4

[0187]

[0188] 4. Case Studies

[0189] Regarding the Q4 experiment: This example conducts a series of case studies to illustrate how the neighborhood structure optimization module and similarity reconstruction loss work. The neighborhood structure optimization module's performance is compared between the average node degrees of each dataset before and after processing using the neighborhood structure optimization module in HSGAD.

[0190] The results are as follows Figure 4 As shown. To alleviate the underreach phenomenon, this embodiment uses d(·) to quantify the similarity within the positive and negative groups (the similarity range is normalized to [0,1]), as shown Figure 5 shown.

[0191] In addition, this example further analyzes the relief of HSGAD on over-compression, under-reachability and over-smoothing from the perspective of representation space. The results are as follows: Figure 6-7 shown.

[0192] Answer to Q4: Observation Figure 6 The following findings were made: 1) The neighborhood structure optimization module can effectively reduce the average node degree of each dataset, indicating that HSGAD can effectively alleviate the over-compression and under-reachability problems in the dataset. 2) With the increase in training rounds, the (positive-positive, negative-negative) intra-group similarity of nodes gradually increases, while the inter-group similarity gradually decreases, indicating that HSGAD effectively alleviates the over-smoothing problem.

[0193] observe Figure 6-7 , we have the following findings: 1) In both synthetic and real datasets, the neighborhood structure optimization module effectively alleviates the generation of difficult samples due to over-compression and under-reachability through structural optimization. 2) In the latent space distribution of both synthetic and real datasets, L s The separation degree of nodes between classes is improved, and the separation degree of nodes within a class is reduced, which can effectively prevent the generation of difficult samples due to over-smoothing.

[0194] 5. Parameter analysis

[0195] Experiments on Q5: This example analyzes the key parameters of HSGAD on all datasets: 1) Contrast loss L in loss function L c and similarity reconstruction loss L s The analysis of the importance balance parameter γ is as follows: Figure 7 2) Analysis of k1 and k2 in edge deletion in the neighborhood structure optimization module, as shown in Figure 8 shown.

[0196] Answer to Q5: Combination Figure 7-8 .

[0197] exist Figure 7 The following observations are made: 1) On different datasets, the optimal performance of HSGAD corresponds to different balance parameters γ. This is because the performance of underreachability on different datasets is different. Therefore, HSGAD has different performance in processing L c and L s 2) In general, when γ∈[0.7,0.85], the performance of HSGAD is more outstanding because L s What is actually achieved is the L c Auxiliary effect of alleviating information transfer over-smoothing in GNNs in imbalanced classification tasks.

[0198] exist Figure 8The following observations are made: 1) For different data sets, the values ​​of K1 and K2 corresponding to the optimal performance of HSGAD are not the same. This is because the proportion of noise edges in different data sets is significantly different. 2) On each data set, the performance of the model basically shows a trend of first increasing and then decreasing with the increase of K1 and K2. This is because when the number of edge deletions gradually increases within the range of the number of noise edges, the filtering of the noise edges in the data set gradually tends to the optimal structure, effectively optimizing the information transmission process of GNNs. When the number of edge deletions exceeds the number of noise edges in the data set, the model randomly deletes edges carrying key information while filtering the noise edges, resulting in the loss of important information in the information transmission. In this embodiment, the K1 and K2 values ​​that optimize the graph structure to the best in each data set are set as the final values.

[0199] Example 2.

[0200] Based on Example 1, the key technologies and scenario adaptability are as follows:

[0201] 1. Social Security Fraud Detection (Unsupervised Difficult Sample Mining)

[0202] A random walk strategy is used to simulate abnormal trajectories in the social security medical record graph, such as generating a node sequence of "frequent cross-regional medical treatment + excessive drug purchases" as a negative sample, forming a contrasting example pair with normal medical treatment paths. The model of Example 1 automatically learns abnormal patterns (such as cross-regional coordinated reimbursement of specific drugs), identifying hidden gang fraud without relying on manual fraud labeling.

[0203] 2. Identification of abnormal behavior in government services (neighborhood structure optimization)

[0204] In the government approval process flow chart, an edge deletion strategy is used to filter out irregular collaboration links between departments (such as the unusually high frequency of data retrieval between the Housing and Urban-Rural Development Bureau and the Civil Affairs Bureau in real estate transfers). This strategy also enhances the edge weights of compliant approval paths, such as the "Market Supervision Bureau → Administrative Approval Bureau" process. This optimized graph convolutional network can more accurately capture illegal "extracorporeal" approval patterns.

[0205] 3. Monitoring of abnormal allocation of public resources (cross-domain collaborative detection)

[0206] In the data of the livelihood subsidy distribution graph, the edges of multi-source data such as household registration, assets, and consumption are integrated, and through structural enhancement, the contradictory correlation paths such as "low-income households → high-consumption places" are highlighted, and a cross-system "false poverty identification-illegal subsidy appropriation" collaborative model is discovered.

[0207] In combination with the above application scenarios, the technical solution of the present invention has the following advantages:

[0208] 1. Unsupervised difficult sample mining adapted to the scarcity of annotations: By comparing instance pair sampling with random walk strategies (such as simulating abnormal trajectories in social security data), positive and negative sample pairs are generated to capture hidden abnormal patterns without manual annotation.

[0209] 2. Neighborhood structure optimization improves governance robustness: Selective edge deletion strategy filters redundant associations (such as false cross-departmental collaboration relationships), structural enhancement operations strengthen key edge weights (such as high-frequency public service access paths), and optimizes the efficiency of information transmission in government graphs.

[0210] 3. Similarity reconstruction strengthens privacy compliance: A loss function based on intra-class and inter-class similarity constraints avoids reliance on sensitive label information and complies with the principle of minimizing the collection of government data.

[0211] 4. This technology can be applied to scenarios such as social security fraud detection and identification of abnormal behavior in government services, improving the ability to detect complex government risks such as "hidden violations" and "cross-domain collaborative anomalies."

[0212] In combination with the embodiments, it can be seen that the present invention has rethought the problem of difficult sample generation in attribute graph anomaly detection in information transmission, and proposed the HSGAD model. The present invention finds that the over-compression, under-reachability and over-smoothing problems in GNNs information transmission are key factors in the generation of difficult samples in GAD. Therefore, the technical solution proposed by the present invention, the HSGAD model, is intended to alleviate the generation of difficult samples caused by these three types of problems. The technical solution of the present invention, on the one hand, adopts a neighborhood structure optimization module to alleviate the generation of compressed difficult samples and reachable difficult samples by filtering noise edges combined with structural enhancement. On the other hand, a similarity reconstruction loss is designed based on the similarity metric to alleviate the generation of smooth difficult samples. A large number of experiments on data sets in 6 different scenarios show that compared with the current mainstream models, HSGAD effectively alleviates the problem of difficult samples and exhibits stronger detection performance and robustness.

[0213] The above is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the embodiments of the present invention. Any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the embodiments of the present invention are still within the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An anomaly detection model for public service venue data and government data based on unsupervised difficult sample mining, characterized by: include: Neighborhood structure optimization module: The input original graph is optimized through selective edge deletion strategy and structural enhancement to jointly optimize the neighborhood structure; Feature extraction module, including: contrast example pair sampling module, GNN module, feature readout module, performs feature extraction based on contrastive learning; Neighborhood similarity reconstruction module: reconstructs the similarity within and between node classes, and obtains the similarity reconstruction loss L s ; Learning target module: Reconstruct the loss L through the similarity s and contrast loss L c , jointly complete the learning goal to enhance the similarity between nodes within a class and weaken the similarity between nodes between classes.

2. The attribute graph anomaly detection model according to claim 1, characterized in that The neighborhood structure optimization module determines high-probability nodes by degree calculation and then selects target edges for deletion.

3. The attribute graph anomaly detection model according to claim 2, characterized in that The process of determining high probability nodes is as follows: Calculate the average node degree of all nodes And After sorting in descending order, select the top-k1 nodes to form a node set The formula is as follows: in, Represents the boundary value corresponding to k1-th after all average node degrees are sorted in descending order; the process of selecting the target edge to delete is: Each node v i , calculate its neighboring nodes Node degree After sorting them in ascending order, select the top-k2 node set The formula is as follows: in, Represents node v i Neighborhood node v j The boundary value corresponding to k2-th after the node degrees are sorted in ascending order.

4. The attribute graph anomaly detection model according to claim 1, characterized in that The structural enhancement is: the side relationship e ij ∈[0,1] is enhanced, the formula is as follows:

5. The attribute graph anomaly detection model according to claim 1, characterized in that The sampling process of the comparative example is as follows: First, in each round, all nodes in the graph are randomly traversed without duplication to determine the target node; Then use random walk to determine the positive and negative instance pairs; The information of the initial nodes in the positive and negative instance pairs is masked, and the target nodes are finally combined with the positive and negative instance pairs and sent into the GNN module.

6. The attribute graph anomaly detection model according to claim 1, characterized in that In the GNN module, the positive and negative subgraphs are simultaneously processed by sharing GCN. and With the target node v i The low-dimensional representation learning formula is as follows: in, Represents the subgraph adjacency matrix A i Normalized, self-connected subgraph adjacency matrix represents the degree matrix of the local subgraph, represents the weight matrix of the (l-1)-th layer, and d represents the dimension of embedding. and denote the hidden layer representations of the l-th layer and the (l-1)-th layer, respectively, and φ(·) denotes the activation function such as ReLU; The weight matrix of GCN maps the target node and the local subgraph into the same embedding space, as follows: in, and Represent the target node v i The hidden layers learned by the l-th and (l-1)-th layers represent row vectors, and 7. The attribute graph anomaly detection model according to claim 1, characterized in that The feature readout module uses average pooling to complete the information reading of positive and negative local sub-images. The formula is as follows: Among them, (H i ) l Indicates H i The l-th row, n i express The number of midpoints.

8. The attribute graph anomaly detection model according to claim 1, characterized in that The similarity reconstruction loss L s The formula is: In the formula, α+d(h i ,e i + ) <d(h i ,e i - ), α is a non-negative value that constrains the distance not to deviate; d(·) is the target node embedding h i and the embedding of positive and negative subgraphs e i + With e i - Similarity measure of ; The contrast loss L c The formula is as follows: For a positive instance pair, y i =1, for negative instance pairs, y i =0.

9. The attribute graph anomaly detection model according to claim 8, characterized in that The final objective function formula in the learning objective module is: L=γL c +(1-γ)L s ; Where γ is the equilibrium L c and L s Importance control parameters.

10. Anomaly detection method for public service venue data and government data based on unsupervised difficult sample mining, characterized by: Anomaly detection is performed using the anomaly detection model described in any one of claims 1 to 9.

Images