A graph neural network forgetting method based on semantic and structural counterfactual distillation

Abstract

This invention proposes a graph neural network forgetting method based on semantic and structural counterfactual distillation. This method generates semantic counterfactual representations through a maximum probability reversal strategy, achieving stable knowledge forgetting at the semantic level; and constructs structural counterfactual representations through a counterintuitive neighbor confusion strategy, effectively removing topological dependencies in the intermediate representation layer to prevent privacy leaks. Simultaneously, a representative node representation alignment mechanism and a flexible weight integration regularization term are introduced to maintain the model's predictive performance and generalization ability on the remaining data while ensuring the integrity of the forgetting process. This invention, through a two-layer counterfactual distillation framework of semantic and structural layers, systematically addresses the shortcomings of existing graph forgetting methods in terms of training stability, privacy security, model dependency, and utility preservation, significantly improving the integrity, universality, and computational efficiency of graph neural network forgetting, and providing an efficient and feasible technical solution for secure and controllable graph data management.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and machine learning technology, and is specifically applied to knowledge forgetting and privacy protection of graph neural network models. It relates to a graph neural network forgetting method based on semantic and structural counterfactual distillation. Background Technology

[0002] Graph Neural Network Unlearning (GNN Unlearning), as a technique for efficiently removing the influence of specified data from a model, has attracted widespread attention in the fields of artificial intelligence safety and privacy protection in recent years. With the widespread application of graph-structured data in scenarios such as social networks, bioinformatics, and recommender systems, how to quickly and thoroughly forget specific nodes, edges, or subgraphs while maintaining model performance and generalization ability has become an important problem in GNN research. While traditional model retraining methods can achieve forgetting, they are often costly and unstable when dealing with large-scale graph structures. Therefore, exploring efficient and stable GNN forgetting methods is of great significance.

[0003] Early research on GNN forgetting primarily focused on parameter correction and gradient cancellation strategies. For example, it estimated the contribution of target samples to model parameters by calculating the influence function, thereby achieving forgetting without complete retraining. However, these methods typically rely on complex gradient backpropagation, which is insufficient in capturing the topological dependencies of graph-structured data, easily leading to incomplete forgetting or a significant deterioration in model performance.

[0004] To address these issues, recent research has increasingly shifted towards forgetting frameworks based on knowledge distillation and counterfactual generation. These methods construct counterfactual samples at the semantic or structural level—generating alternative representations that are similar to the target node in features or topology but lack its sensitive information—thus achieving smooth knowledge deletion during the distillation process. For example, the CF-GNNForget model effectively weakens the influence of sensitive nodes in information propagation by utilizing a counterfactual node feature replacement mechanism; the GraphEraser framework achieves multi-level graph structure forgetting through local reparameterization and subgraph reconstruction strategies. These methods significantly enhance model interpretability and privacy protection while improving forgetting integrity and computational efficiency.

[0005] However, existing counterfactual distillation methods still suffer from semantic instability and structural remnants—the semantic level is prone to introducing ambiguous feature substitutions, leading to excessive forgetting; the structural level struggles to completely eliminate topological dependencies between the target node and its neighbors. To address these issues, recent research focuses on semantic-structural bilayer counterfactual modeling and flexible parameter regularization strategies to balance forgetting integrity with model utility preservation. Summary of the Invention

[0006] This invention proposes a graph neural network forgetting method based on semantic and structural counterfactual distillation. It generates counterfactual semantic representations through maximum probability reversal, constructs counterfactual structural representations through counterintuitive neighbor confusion, and combines node alignment and elastic weight integration regularization to achieve stable, comprehensive, and efficient graph data forgetting, solving the following technical problems:

[0007] (1) High computational cost and unstable training. Existing methods often rely on approximate retraining or inverse gradient erasure strategies to achieve forgetting, but these methods are prone to numerical oscillations or unbounded loss during the optimization process, resulting in unstable model training. In addition, although full retraining can achieve the ideal forgetting effect, it is difficult to apply to large-scale graph data and the computational cost is too high.

[0008] (2) Topological information embedding leads to privacy leakage risk. Graph neural networks encode the adjacency relationships of nodes into intermediate representations through multi-layer aggregation mechanisms, so that the structural information of sensitive nodes or subgraphs is deeply bound in the model's latent space. When a forgetting operation is performed, if only the output layer is erased, the intermediate representation layer may still retain the implicit privacy structural information, thus causing potential leakage under white-box attacks.

[0009] (3) Existing methods rely on specific model structures and lack generality. Many graph forgetting methods are based on specific GNN architecture assumptions (such as linear number of layers and fixed aggregation methods). When faced with deeper or heterogeneous graph models, they are difficult to transfer or reuse directly, which limits their adaptability in different tasks and models.

[0010] (4) It is difficult to balance forgetting and utility. While aggressive forgetting strategies can completely delete target information, they often destroy the model's effective representation of the remaining data; while conservative forgetting may leave behind sensitive information. Existing methods generally lack a dynamic balance mechanism between the integrity of forgetting and the utility of the model, and are prone to over-forgetting or under-forgetting problems.

[0011] (5) The additional computation and storage burden is relatively large. Some methods improve the erasure effect by adding a forgetting subnetwork or multiple iterative distillation, but at the same time, they introduce high storage and computation overhead, which is difficult to meet the application requirements in real-time or resource-constrained environments.

[0012] To address the aforementioned technical problems, this invention proposes a graph neural network forgetting method based on semantic and structural counterfactual distillation. This method generates semantic counterfactual representations through a maximum probability inversion strategy, achieving stable knowledge forgetting at the semantic level; and constructs structural counterfactual representations through a counterintuitive neighbor obfuscation strategy, effectively removing topological associations in the intermediate representation layer to prevent privacy leaks. Simultaneously, a representative node representation alignment mechanism and a flexible weight integration regularization term are introduced to maintain the model's predictive performance and generalization ability on the remaining data while ensuring the integrity of the forgetting process.

[0013] Overall, this invention systematically addresses the shortcomings of existing graph forgetting methods in terms of training stability, privacy security, model dependency, and utility preservation through a semantic and structural dual-layer counterfactual distillation framework. It significantly improves the integrity, versatility, and computational efficiency of graph neural network forgetting, providing an efficient and feasible technical solution for secure and controllable graph data management.

[0014] The specific technical solution of the present invention is as follows:

[0015] A graph neural network forgetting method based on semantic and structural counterfactual distillation includes the following steps:

[0016] Step 1: Obtain the text dataset and construct the graph structure, then use the result of deleting nodes in a simulated forgetting task as the training set;

[0017] Step 2: Semantic-aware counterfactual distillation; both the original model and the forgetting model are graph neural networks; obtain the output logic vector after each layer of linear mapping in the original model to generate the counterfactual output logic vector; calculate the KL divergence loss based on the counterfactual output logic vector and the original output logic vector to distill the counterfactual semantic knowledge into the forgetting model;

[0018] Step 3: Structure-aware counterfactual distillation; Based on the real neighbor nodes of the forgotten nodes, generate counterfactual neighbor nodes, and use KL divergence to constrain the distillation of real and counterfactual neighbor nodes;

[0019] Step 4: Semantic and structural pattern preservation; First, sample the one-hop neighbor nodes of the forgotten nodes and sample non-neighbor nodes based on the confidence of the original model output to obtain a representative node set; at the semantic level, use the original model as the teacher and its output logical vector as soft labels to distill the semantic information of the representative nodes; at the structural level, distill the structural knowledge of the representative nodes.

[0020] Step 5: Elastic weight integration mechanism; First, calculate the cross-entropy loss function of the original model to obtain Fisher information, and then apply regularization constraints to the parameters of the original model and the forgotten model based on the Fisher information.

[0021] Step 6: Perform joint optimization of the comprehensive loss function to obtain the trained forgetting model.

[0022] The counterfactual output logic vector is specifically obtained through a maximum probability inversion strategy or a variational autoencoder.

[0023] The maximum probability reversal strategy is as follows:

[0024] Given the output logic vector of the trained original model, a one-hot vector is first constructed according to the target category. The maximum probability of predicting the category is reversed by subtracting this vector, so that the model predicts towards the closest wrong category. A uniform noise term is injected during the reversal process to obtain the counterfactual output logic vector.

[0025] In step 3, counterfactual neighbor nodes are obtained through a counterintuitive neighbor confusion strategy or a subgraph rearrangement and edge probability sampling strategy.

[0026] The counterintuitive neighbor confusion strategy is as follows: given a degree of... Forgotten nodes and their real neighbors are injected into the original adjacency matrix. Create a false edge to generate counterfactual neighbors;

[0027] The subgraph rearrangement and edge probability sampling strategy is as follows: edges of forgotten nodes are randomly deleted or replaced, and global connectivity is preserved through probability control.

[0028] The counterintuitive neighbor obfuscation strategy creates counterfactual neighbors, and the local structure of the forgotten node is represented as follows: , its first Each element is obtained by the following formula:

[0029]

[0030] in, and These represent the nodes in the original model. and his neighbors The output logic vector; Represents the radial basis function kernel, used to calculate and The similarity score between them. Parameters Controlling the rate of similarity decay, when When, it indicates a node Structural similarity between it and its real neighbor nodes; when When, it indicates a node Structural similarity with its counterfactual neighbor nodes;

[0031] Node Local structure representation Divided by neighbor origin, including 2d elements, the first... The structural similarity of each element to its real neighbor nodes constitutes the real part. ;back The structural similarity of each element to its counterfactual neighbor nodes constitutes the counterfactual part. And the distillation of both is constrained by KL divergence:

[0032]

[0033] in, It is the number of layers in the GNN. Denotes KL divergence, It is a set of forgotten nodes.

[0034] Step 4 is described in detail below:

[0035] At the semantic level, the original model is used as the teacher, and its output logic is used as soft labels to distill representative nodes. Semantic information:

[0036]

[0037] in, Indicates semantic distillation loss. This represents the representative set of nodes, including non-neighbor nodes sampled based on the confidence scores of the original model output and one-hop neighbors of forgotten nodes. and These represent the output logistic vectors of the forgotten nodes in the teacher model and the distilled model, respectively. Softmax is used to normalize the logistic values ​​into a class probability distribution.

[0038] At the structural level, distillation represents the nodes. Structural knowledge:

[0039]

[0040] in, This represents the structural distillation loss, and L represents the number of layers in the model. and They represent the first time. In the layer, the node similarity matrix is ​​calculated by the forgetting model and the original model.

[0041] Step 5 is described in detail below:

[0042] The sensitivity of each parameter to the model output is evaluated using the Fisher information matrix:

[0043]

[0044] in, Indicates the parameter The second-order partial derivative operation is used to calculate the curvature of the loss function; The parameters are shown as Fisher information content; The cross-entropy loss function is calculated on the graph structure; This represents the set of parameters of the original, trained model.

[0045] EWC regularization is expressed as follows:

[0046]

[0047] in, and These represent the parameters of the original model and the forgetting model, respectively. This is the loss for EWC regularization.

[0048] The joint optimization of the comprehensive loss function is as follows:

[0049]

[0050] in, For the total loss function, and It is a hyperparameter that controls the intensity of forgetting and retention; The KL divergence loss in step 2 is calculated using the following formula:

[0051]

[0052] in, This represents the output logical vector after forgetting. It's a temperature parameter. Output a logic vector for counterfactual purposes.

[0053] The beneficial effects of this invention are as follows:

[0054] Compared to existing technologies, the main advantage of this invention lies in its ability to achieve efficient forgetting of sensitive information in graph neural networks through a two-layer counterfactual distillation of semantics and structure, enabling fine-grained control over sensitive nodes and their associated information. Existing graph forgetting methods typically rely on parameter correction, gradient cancellation, or complete retraining to remove sensitive information. However, in the case of large-scale graph data or complex topologies, these methods often suffer from incomplete forgetting, degraded model performance, or high computational costs. Furthermore, existing methods often neglect the residual topological information in intermediate representation layers, allowing sensitive information to still affect the feature distribution of the remaining data during node aggregation and information propagation, increasing the risk of privacy leakage.

[0055] This invention generates counterfactual node representations at the semantic level, smoothly replacing the feature information of sensitive nodes. Simultaneously, at the structural level, it constructs counterfactual structural representations through neighbor confusion and local topological perturbation, thereby weakening the influence of sensitive nodes in information propagation. This two-layer intervention strategy can effectively remove sensitive information while maintaining the overall semantics and structural patterns of the graph, avoiding semantic shifts or topological destruction caused by excessive forgetting. Furthermore, by combining node alignment and pattern preservation mechanisms with elastic weight integration and joint optimization, this invention can maintain model stability during forgetting operations, ensuring that predictive performance and generalization ability on the remaining data are unaffected.

[0056] Compared to traditional methods, this invention achieves a better balance between sensitive information removal and model utility preservation. When processing large-scale graphs or complex subgraphs, it completely forgets the target node and its neighborhood information while preserving the knowledge structure and semantic features of the remaining graph to the greatest extent possible. Furthermore, the model-independent design and efficient counterfactual distillation strategy of this invention make it applicable to different types of graph neural network architectures, with significantly lower computational costs than full retraining methods, while also reducing potential privacy risks. Therefore, this invention outperforms existing technologies in terms of forgetting integrity, model performance preservation, computational efficiency, and privacy protection capabilities, making it suitable for practical applications such as social network privacy management, knowledge graph updates, and user data deletion in recommendation systems. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of the inventive principle of the present invention. Detailed Implementation

[0058] This invention proposes a graph neural network forgetting method based on semantic and structural counterfactual distillation, aiming to effectively and stably remove specified graph data (nodes, edges, or subgraphs) from a trained graph neural network (GNN) model while maintaining the model's performance and generalization ability on the remaining data. This method achieves knowledge forgetting and utility retention simultaneously at both the semantic and structural layers through a two-layer counterfactual generation and distillation mechanism.

[0059] like Figure 1 As shown, the overall architecture of this invention mainly includes the following four core modules: a hybrid sampling strategy module, a structure-aware counterfactual distillation module, a semantic-aware counterfactual distillation module, and a flexible weight integration mechanism. The overall process is centered on a two-layer counterfactual distillation of semantics and structure, achieving multi-level knowledge forgetting through counterfactual semantics and topological perturbation, while introducing flexible regularization at the parameter level to maintain model stability and utility. The following provides a detailed description of each main module:

[0060] Step 1: Obtain the text dataset and construct the graph structure, then use the result of deleting nodes in a simulated forgetting task as the training set;

[0061] In this embodiment, when validating the proposed graph neural network forgetting method based on semantic and structural counterfactual distillation, four publicly available graph structure datasets were selected, including three classic citation networks—Cora, Citeseer, and PubMed—as well as the CS (Computer Science) Coauthor Network. Statistical information for the four datasets is shown in Table 1.

[0062] Table 1. Statistical information of the dataset

[0063] Dataset type node side Feature vector Node categories Cora Reference network 2708 5429 1433 7 Citeseer Reference network 3327 4732 3703 6 PubMed Reference network 19717 44338 500 3 CS Collaborator Network 18333 163788 6805 15

[0064] In all the datasets mentioned above, nodes represent document or author entities, and edges represent citations or collaborations. Each node contains a multi-dimensional feature vector, typically a bag-of-words or TF-IDF text feature; the node class label represents the paper topic or the author's research field, used for node classification tasks.

[0065] Graph structure construction. In this embodiment, each dataset is modeled as a directed or undirected graph. ,in: Represents a set of nodes. Represents the set of edges; The node feature matrix, For node category labels.

[0066] Data preprocessing and partitioning. To simulate a real-world forgetting task, this invention randomly selects a certain proportion of nodes (denoted as the forgetting rate) from each dataset. The node to be forgotten is set as the node whose directly associated edges and feature information are set to zero. In the experiments presented in this paper, the following settings are used: This involves deleting 10% of the nodes and their adjacent edges, leaving the remaining nodes to form a "preserved subgraph," which is used for maintaining the model's utility and for retraining validation.

[0067] The node features are input into the model after standardization, i.e.:

[0068]

[0069] in and , respectively, are the mean and standard deviation of all node features. For node features, These are the standardized node features.

[0070] For text-based data (such as Cora, Citeseer, PubMed), TF-IDF vectorization and low-dimensional projection are used, and for author collaboration networks (CS), average pooling features based on domain word embeddings are used.

[0071] The dataset was divided into three parts: Training set (60% of nodes), Validation set (20% of nodes), and Test set (20% of nodes). This division ensures a balanced distribution of classes to facilitate evaluation of node classification performance.

[0072] Step 2: Semantic-aware counterfactual distillation;

[0073] To address the unbounded loss and training instability issues caused by the inverse gradient operator in existing graph forgetting methods, this invention designs a semantic-aware counterfactual distillation module based on the Maximum Probability Reversal (MPR) strategy. This invention includes both an original model and a forgetting model, both structured as graph neural networks. In the graph neural network (GNN) structure used in this invention, the model first encodes the features of nodes and their neighborhoods into high-dimensional semantic representations through multi-layer message passing and neighbor aggregation mechanisms. After multi-layer propagation, each node obtains a semantic output vector after the last layer of linear mapping, which is used as input to the classifier. The semantic output vector, as the original output logical vector, generates semantic counterfactual samples by reversing the predicted probabilities at the node semantic output layer, achieving stable forgetting of semantic knowledge. The node semantic output layer refers to the linear output space of the graph neural network after aggregating neighbor information and before the classifier input; it can also be called the node semantic representation layer. In this layer, the output logical vector of each node corresponds to the distribution of semantic information aggregated in the graph structure, reflecting the node's category tendency and semantic association in the global graph.

[0074] As one implementation method, the method of generating semantic counterfactual samples by reversing prediction probabilities specifically involves, given the logistic vector output by the original trained graph neural network model, firstly, constructing a one-hot vector (representing the ideal probability distribution of the node across all category dimensions, where the target category position is 1 and the rest are 0) based on the target category (i.e., the true predicted category of the current node in the original model). Then, by subtracting the maximum probability of reversing the predicted category from this vector, the model's prediction is directed towards the closest incorrect category. Subsequently, a uniform noise term is injected into the reversed semantic logic to generate a smoother counterfactual semantic representation and expand the exploration space. The above process can be formalized as follows:

[0075]

[0076] in, , and These represent the inverted counterfactual output logic vector, the original output logic vector, and the uniformly distributed noise, respectively. This represents the one-hot encoding function. Represents a node The target category predicted in the output of the original graph neural network model; Control noise intensity.

[0077] As one implementation method, a Variational Autoencoder (VAE) can also be used to generate counterfactual output logic vectors. By encoding and reparameterizing the original output logic vector, the VAE can generate alternative vectors that are semantically similar to the original nodes but do not contain the original node information. This method does not require explicit computation of the maximum probability inversion, but achieves semantic substitution through latent space sampling, and can still effectively balance sensitive information removal with the preservation of the semantics of the remaining data, while also exhibiting good generative diversity and model stability.

[0078] After obtaining the counterfactual output logical vector, Kullback-Leibler (KL) divergence loss is used to distill this counterfactual semantic knowledge into the forgetting model. The forgetting model refers to updating the parameter state after performing knowledge forgetting of specific nodes or subgraphs. This enables the forgetting model to forget specific semantic knowledge at the logical level.

[0079]

[0080] in, This represents the output logical vector after forgetting. It is a temperature parameter that controls smoothness. It is a set of forgotten nodes. The output is a counterfactual output logical vector. The set serves as the semantic input for subsequent steps 3 (structure-aware counterfactual distillation) and 4 (semantic and structural pattern preservation). Kullback-Leibler divergence (also known as relative entropy) is used to measure the degree of difference between two probability distributions. In this invention, KL divergence is used to constrain the forgetting model to approximate its counterfactual semantic distribution in terms of logistic distribution, thereby guiding the model to smoothly forget the semantic information of specific nodes in the probability space and avoiding unstable training caused by drastic oscillations at the parameter level.

[0081] Step 3: Structure-aware counterfactual distillation.

[0082] In graph neural networks, the intermediate representations of nodes naturally incorporate topological adjacency information. If this information is forgotten without intervention, it may lead to privacy leaks or residual structural dependencies. To address this, this invention proposes a counter-intuitive neighbor confusion (CNC) strategy. This strategy injects spurious edges into the original adjacency matrix to generate counterfactual structures, thereby disrupting the true topological pattern of the target nodes.

[0083] Specifically, given a degree Forgotten nodes and its real neighbors By injecting into the original adjacency matrix A false edge effectively creates corresponding counterfactual neighbors. The local structure of a node is defined as follows: , its first Each element is obtained by the following formula:

[0084]

[0085] in, and These represent the nodes in the original model. and his neighbors The intermediate representation refers to the output logical vector of the node, which is used to characterize the fusion result of the node's local structure and semantic information. Represents the radial basis function (RBF) kernel, used to calculate and The similarity score between the two nodes. When the middle representation of two nodes... and The closer they are, the greater the Euclidean distance. The smaller the value, the closer the exponent term is to 1, indicating that the two nodes are highly similar in structural semantics; when the two nodes are far apart in the representation space, the term approaches 0, indicating that the correlation between the nodes is weak or unrelated; parameters This controls the rate of similarity decay; a larger value indicates a slower decay and a stronger emphasis on global structural relationships, while a smaller value emphasizes local structural differences. When, it indicates a node Its real neighbor nodes Structural similarity between them; when When, it indicates a node Its counterfactual neighbor nodes The structural similarity between them.

[0086] Node Local structure representation Divided by neighbor origin, including 2d elements, the first... The structural similarity of each element to its real neighbor nodes constitutes the real part. ;back The structural similarity of each element to its counterfactual neighbor nodes constitutes the counterfactual part. Obtaining a counterfactual structure Then, the local structure is represented. The data is divided into a true component and a counterfactual component, and the distillation of both is constrained by the KL divergence.

[0087]

[0088] Among them, among them, It is the number of layers in the GNN. In the forgetting model The module weakens topological relationships at the intermediate representation level, enabling directed forgetting of structural knowledge. The output is a set of counterfactual structural representations. This serves as the input for the semantic and structural pattern preservation stage in step 3.

[0089] As one implementation, the counterintuitive neighbor confusion strategy can be replaced by subgraph rearrangement or edge probability sampling strategies to achieve structural intervention. For example, forgotten nodes can be assigned based on the importance of their neighboring nodes. Edges are randomly deleted or replaced, and global connectivity is preserved through probabilistic control. This scheme avoids directly confusing neighbor order or perturbing the original topology; instead, it adjusts the propagation of structural information through statistical methods, effectively reducing the number of forgotten nodes. The impact on the model, while maintaining the overall topological stability of the graph.

[0090] Step 4: Semantic and structural pattern preservation;

[0091] To prevent excessive forgetting during the forgetting process and its detrimental effect on the performance of the remaining subgraph, this invention designs a hybrid sampling strategy module. This module combines neighbor sampling and confidence sampling to dynamically select representative nodes from the remaining subgraph for distillation, thereby maintaining the stability of global semantics and structural patterns.

[0092] Specifically, first, one-hop neighbors of forgotten nodes are sampled to capture locally affected regions; then, several non-neighbor nodes are sampled based on the classification confidence scores output by the original model. Nodes with lower confidence scores are more sensitive to the decision boundary and are sampled with higher probabilities, defined as follows:

[0093]

[0094] in, This represents the set of sampling nodes based on confidence level. This indicates the original model for nodes. The classification confidence score reflects the degree of certainty of the original model regarding the category to which the node belongs. This is the confidence threshold, used to control the sampling range. This represents a node in the remaining subgraph.

[0095] At the semantic level, the original model is used as the teacher, and its output logic is used as soft labels to distill representative nodes. Semantic information:

[0096]

[0097] in, Indicates semantic distillation loss. Represents a representative set of nodes, including and the one-hop neighbor of the forgotten node, This represents the Kullback–Leibler divergence, used to measure the difference in node semantic distribution between the forgetting model and the original model; and and represent the output logistic vectors of the forgotten nodes in the teacher model and the distilled model, respectively. Softmax is used to normalize the logistic values ​​to a class probability distribution. This loss term ensures that the counterfactual model maintains semantic consistency regarding the teacher model's knowledge.

[0098] At the structural level, distillation represents the nodes. Structural knowledge:

[0099]

[0100] in, This represents the structural distillation loss, and L represents the number of layers in the model. and They represent the first time. In the layer, the node similarity matrix (or neighborhood attention distribution) calculated by the forgetting model and the original model. It is also used to measure the difference in structural distribution between the two. This structural distillation loss enables the distilled model to maintain consistency with the teacher model at the topological level, thereby maintaining global structural stability without affecting the discriminative ability of the remaining subgraph.

[0101] Another viable alternative to preserving semantic and structural patterns is to use graph regularization or graph convolutional attention mechanisms to maintain pattern stability. By imposing regularization constraints on node embeddings or adjusting neighbor aggregation weights, the global semantic and structural distribution can be automatically preserved even when sensitive information is forgotten. This alternative does not require explicit alignment operations on each node, yet it can still maintain the patterns and model utility of the remaining data after sensitive information removal.

[0102] Step 5: Flexible weight integration mechanism;

[0103] To further mitigate performance degradation caused by excessive forgetting, this invention introduces the Elastic Weight Consolidation (EWC) mechanism to maintain the importance of key weights in the original model from a parameter space perspective. First, the Fisher Information Matrix (FIM) is used to evaluate the sensitivity of each parameter to the model output:

[0104]

[0105] in, Indicates the parameter The second-order partial derivative operation is used to calculate the curvature of the loss function; This represents the input graph data, where These are the node set, edge set, node feature matrix, and label set, respectively. The parameters are shown as Fisher information is used to measure the uncertainty of the model at that parameter point; For graph data The cross-entropy loss function calculated above; This represents the set of parameters of the original trained graph neural network model; when A larger value indicates that the parameter has a significant impact on the task results and should remain stable during the forgetting process; conversely, a smaller value allows for a wider range of adjustments to achieve the forgetting of the target information.

[0106] The EWC regularizer is described as follows:

[0107]

[0108] in, and These represent the parameters of the original model and the forgotten model, respectively. This mechanism can maintain stable model performance on non-forgotten data while forgetting specific node knowledge;

[0109] Step 6: Jointly optimize the objective

[0110] Ultimately, the overall optimization objective of this invention integrates four parts: semantic distillation, structural distillation, pattern preservation, and parameter regularization.

[0111]

[0112] in, and It is a hyperparameter that controls forgetting and retention strength.

[0113] Through the above-mentioned semantic-structural-parameter multi-layer collaborative optimization, this invention can achieve efficient and stable forgetting of specified graph data while maintaining model performance, and has good generalization and privacy protection capabilities.

[0114] In the joint optimization and elastic weight integration module, another feasible alternative is to use gradient pruning, parameter freezing, or multi-objective optimization strategies (such as weighted loss functions or adaptive learning rate adjustment) to achieve similar results. These methods allow for control of model parameter updates without altering the distillation objective, balancing the removal of forgotten information with the preservation of performance from remaining data, while reducing training instability and computational overhead, thus achieving the same forgetting and utility objectives as the original approach.

[0115] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A graph neural network forgetting method based on semantic and structural counterfactual distillation, characterized in that, Includes the following steps: Step 1: Obtain the text dataset and construct the graph structure, then use the result of deleting nodes in a simulated forgetting task as the training set; Step 2: Semantic-aware counterfactual distillation; both the original model and the forgetting model are graph neural networks; obtain the output logic vector after each layer of linear mapping in the original model to generate the counterfactual output logic vector; calculate the KL divergence loss based on the counterfactual output logic vector and the original output logic vector to distill the counterfactual semantic knowledge into the forgetting model; Step 3: Structure-aware counterfactual distillation; Based on the real neighbor nodes of the forgotten nodes, generate counterfactual neighbor nodes, and use KL divergence to constrain the distillation of real and counterfactual neighbor nodes; Step 4: Semantic and structural pattern preservation; First, sample the one-hop neighbor nodes of the forgotten nodes and sample the non-neighbor nodes based on the confidence of the original model output to obtain a representative node set; At the semantic level, the original model is used as the teacher, and its output logical vector is used as a soft label to distill the semantic information of representative nodes; at the structural level, the structural knowledge of representative nodes is distilled. Step 5: Elastic weight integration mechanism; First, calculate the cross-entropy loss function of the original model to obtain Fisher information, and then apply regularization constraints to the parameters of the original model and the forgotten model based on the Fisher information. Step 6: Perform joint optimization of the comprehensive loss function to obtain the trained forgetting model.

2. The graph neural network forgetting method based on semantic and structural counterfactual distillation according to claim 1, characterized in that, The counterfactual output logic vector is specifically obtained through a maximum probability inversion strategy or a variational autoencoder. The maximum probability reversal strategy is as follows: Given the output logic vector of the trained original model, a one-hot vector is first constructed according to the target category. The maximum probability of the predicted category is reversed by subtracting the one-hot vector, so that the model predicts towards the closest wrong category. A uniform noise term is injected during the reversal process to obtain the counterfactual output logic vector.

3. The graph neural network forgetting method based on semantic and structural counterfactual distillation according to claim 2, characterized in that, In step 3, counterfactual neighbor nodes are obtained through a counterintuitive neighbor confusion strategy or a subgraph rearrangement and edge probability sampling strategy. The counterintuitive neighbor confusion strategy is as follows: given a degree of... Forgotten nodes and their real neighbors are injected into the original adjacency matrix. Create a false edge to generate counterfactual neighbors; The subgraph rearrangement and edge probability sampling strategy is as follows: edges of forgotten nodes are randomly deleted or replaced, and global connectivity is preserved through probability control.

4. The graph neural network forgetting method based on semantic and structural counterfactual distillation according to claim 3, characterized in that, The counterintuitive neighbor obfuscation strategy creates counterfactual neighbors, and the local structure of the forgotten node is represented as follows: , its first Each element is obtained by the following formula: ； in, and These represent the nodes in the original model. and his neighbors The output logic vector; Represents the radial basis function kernel, used to calculate and Similarity score between; parameters Controlling the rate of similarity decay, when When, it indicates a node Structural similarity between it and its real neighbor nodes; when When, it indicates a node Structural similarity with its counterfactual neighbor nodes; Node Local structure representation Divided by neighbor origin, including 2d elements, the first... The structural similarity of each element to its real neighbor nodes constitutes the real part. ;back The structural similarity of each element to its counterfactual neighbor nodes constitutes the counterfactual part. And the distillation of both is constrained by KL divergence: ； in, It is the number of layers in the GNN. Denotes KL divergence, It is a set of forgotten nodes. In the forgetting model The true local structure.

5. The graph neural network forgetting method based on semantic and structural counterfactual distillation according to claim 4, characterized in that, Step 4 is as follows: At the semantic level, the original model is used as the teacher, and its output logic is used as soft labels to distill representative nodes. Semantic information: ； in, Indicates semantic distillation loss. This represents the representative set of nodes, including non-neighbor nodes sampled based on the confidence scores of the original model output and one-hop neighbors of forgotten nodes. and These represent the output logistic vectors of the forgotten nodes in the teacher model and the distilled model, respectively. Softmax is used to normalize the logistic values ​​to a class probability distribution. It is a temperature parameter; At the structural level, distillation represents the nodes. Structural knowledge: ； in, This represents the structural distillation loss, and L represents the number of layers in the model. and They represent the first time. In the layer, the node similarity matrix is ​​calculated by the forgetting model and the original model.

6. The graph neural network forgetting method based on semantic and structural counterfactual distillation according to claim 5, characterized in that, Step 5 is described in detail below: The sensitivity of each parameter to the model output is evaluated using the Fisher information matrix: ； in, Indicates the parameter The second-order partial derivative operation is used to calculate the curvature of the loss function; The parameters are shown as Fisher information content; The cross-entropy loss function is calculated on the graph structure; This represents the set of parameters of the original, trained model. EWC regularization is expressed as follows: ； in, and These represent the parameters of the original model and the forgetting model, respectively. This is the loss for EWC regularization.

7. The graph neural network forgetting method based on semantic and structural counterfactual distillation according to claim 6, characterized in that, The joint optimization of the comprehensive loss function is as follows: ； in, For the total loss function, and It is a hyperparameter that controls the intensity of forgetting and retention; The KL divergence loss in step 2 is calculated using the following formula: ； in, This represents the output logical vector after forgetting. Output a logic vector for counterfactual purposes.

Images