Federal causal forgetting learning method, device, equipment and medium

Abstract

The invention discloses a federal causal forgetting learning method and device, equipment and a medium, and belongs to the technical field of information security. The method comprises the following steps: constructing a causal relationship graph based on a to-be-forgotten data set, a residual data set, model features and a prediction result of a forgotten client; generating an anti-fact data set and a correlation link, and cutting off the correlation link of the to-be-forgotten data set; and adding the anti-fact data set to a local data set of the forgotten client, and updating parameters of the global model by performing federal forgetting training on the forgotten client and the remaining clients. According to the method, the influence of the to-be-forgotten data and the remaining data on model output is analyzed by constructing the causal relationship graph, the anti-fact data generation mechanism is introduced to cut off the causal path of the to-be-forgotten data, and the knowledge distillation technology is utilized to migrate the original model knowledge to the new model; and the model performance of the remaining data is kept while the influence of the to-be-forgotten data is eliminated.

Description

Technical Field

[0001] The present invention belongs to the technical field of information security and relates to a federated causal forgetting learning method, device, equipment and medium. Background Art

[0002] With the wide application of machine learning (ML) and federated learning (FL) technologies, data privacy protection has become an important issue in today's society. In many machine learning models, especially in the federated learning framework, the model stores sensitive information during the training process, and as the model iterates, this information may be extracted, leading to the risk of data leakage. To address this issue, researchers have proposed federated unlearning, aiming to completely remove the influence of certain data from the trained model to ensure that users' privacy is not leaked.

[0003] Currently, the implementation of federated forgetting faces many challenges. First, it is how to ensure the data forgetting effect while avoiding negative impacts on the performance of the remaining data. Machine learning models usually remember the feature information of the input data during the training process. This memory may not be limited to the target data only and may also affect other data. When certain data is deleted, the model often loses the effective learning of other data, resulting in performance degradation. Especially in the federated learning environment, the data of each client is usually non-independent and identically distributed, which means that the data distributions of different clients are different.

[0004] Secondly, storage overhead and computational complexity are also important challenges faced by federated forgetting. One of the characteristics of federated learning is distributed training. Usually, the device resources of the participating parties are limited, especially on edge devices, where the computing power and storage space are relatively limited. Traditional federated forgetting methods often require a large amount of historical data and gradient information for storage, which not only increases the storage overhead but may also lead to a decline in device performance, thus reducing the overall efficiency of the system.

[0005] In addition, how to accurately decouple the causal relationship between the forgotten data and the remaining data is also a challenge. In many practical applications, there are complex causal dependencies between data. Forgetting a certain part of the data may affect the learning effect of other data and even change the behavior of the model. Summary of the Invention

[0006] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a federated causal forgetting learning method, device, equipment and medium, which can forget the target data without affecting the learning effect of the remaining data.

[0007] To achieve the above object, the present invention is implemented by the following technical solutions:

[0008] In a first aspect, the present invention provides a federated causal forgetting learning method, including:

[0009] Construct a causal relationship graph based on the dataset to be forgotten, the remaining dataset, model features, and prediction results of the forgetting client;

[0010] Based on the causal relationship graph, generate a counterfactual dataset, and generate a relationship link between the counterfactual dataset and the remaining dataset, model features, and prediction results, and cut off the relationship link of the dataset to be forgotten in the causal relationship graph;

[0011] Add the counterfactual dataset to the local dataset of the forgetting client, where the local dataset of the forgetting client includes the dataset to be forgotten and the remaining dataset;

[0012] Update the parameters of the global model by performing federated forgetting training on the forgetting client and the remaining clients to forget the influence of the dataset to be forgotten on the global model.

[0013] Furthermore, the number of the forgetting clients is one or more; the number of the remaining clients is zero, one, or more.

[0014] Furthermore, constructing a causal relationship graph based on the dataset to be forgotten, the remaining dataset, model features, and prediction results of the forgetting client includes:

[0015] A feature relationship link is set between the dataset to be forgotten and the remaining dataset;

[0016] A causal relationship link is set between the dataset to be forgotten and the model features; a causal relationship link is set between the dataset to be forgotten and the prediction results;

[0017] A causal relationship link is set between the remaining dataset and the model features; a causal relationship link is set between the remaining dataset and the prediction results;

[0018] A causal relationship link is set between the model features and the prediction results.

[0019] Furthermore, generating a counterfactual dataset based on the causal relationship graph includes:

[0020] Select samples from the remaining dataset , where is a feature, is the label of;

[0021] Generate an adaptive mask matrix for sensitive regions based on the model features , where the mask intensity is positively correlated with the gradient magnitude of the global model with respect to ;

[0022] Apply a non - linear perturbation to to generate counterfactual features , where ⊙ denotes element - wise multiplication;

[0023] According to the prediction confidence distribution of the model with respect to , select the class with the lowest confidence as the counterfactual label , and construct a counterfactual sample ;

[0024] By iteratively optimizing the mask matrix, maximize the intra - class distance between in the feature space and the dataset to be forgotten.

[0025] Furthermore, through federated training of the forgetting client and the remaining clients, it includes:

[0026] In the first training round, the forgetting client uses the global model as the teacher model and initializes the student model;

[0027] It also includes:

[0028] Send the global model updated after the K (K≥1) - th training round to the forgetting client and the remaining clients, making it the student model for the (K + 1) - th round of training of the forgetting client and the local model for the (K + 1) - th round of training of the remaining clients;

[0029] The remaining clients use their local datasets to train the local model separately to obtain local model parameters;

[0030] The forgetting client uses its local dataset and, under the guidance of the teacher model, trains the student model, and adjusts the parameters of the student model through the knowledge distillation loss to obtain student model parameters;

[0031] Aggregate the local model parameters and the student model parameters to update the global model.

[0032] Furthermore, the calculation formula of the knowledge distillation loss is:

[0033] ,

[0034] where is the KL - divergence loss of the feature layer, is the cross - entropy loss of the prediction layer, is the balancing weight, is the temperature parameter;

[0035] Among them, the KL divergence loss of the feature layer is:

[0036] ,

[0037] where is the KL divergence loss; is the adjustment coefficient; is the feature distribution of the student model; is the feature distribution of the teacher model; is the feature representation extracted by the teacher model from the input data; is the remaining data set; is the feature representation extracted by the student model during the forgetting process; is the data set to be forgotten; is the counterfactual data set;

[0038] The cross-entropy loss of the prediction layer is:

[0039] ,

[0040] where is the cross-entropy loss; is the adjustment coefficient; is the remaining data label; is the counterfactual data label; is the label prediction distribution of the teacher model for the input data; is the adjusted label prediction distribution of the student model during the forgetting process.

[0041] Furthermore, the parameters of the global model are updated, including:

[0042] After the round of training is completed, weights are dynamically allocated based on the local data volume of the client, and the parameter update of the student model of the forgetting client and the parameter update of the local model of the remaining clients are calculated;

[0043] The two types of client updates are weighted and aggregated according to the data volume ratio, and the aggregation formula is:

[0044] ,

[0045] where is the number of local data sets of the forgetting client; is the number of local data sets of the remaining clients; is the number of forgetting clients; is the number of remaining clients.

[0046] In a second aspect, the present invention further provides a federated causal forgetting learning device, which includes:

[0047] A causal relationship graph construction module, configured to construct a causal relationship graph based on the dataset to be forgotten, the remaining dataset, model features, and prediction results of the forgetting client;

[0048] A counterfactual dataset generation module, configured to generate a counterfactual dataset based on the causal relationship graph, generate a relationship link between the counterfactual dataset and the remaining dataset, model features, and prediction results, and cut off the relationship link of the dataset to be forgotten in the causal relationship graph;

[0049] A counterfactual dataset addition module, configured to add the counterfactual dataset to the local dataset of the forgetting client, where the local dataset of the forgetting client includes the dataset to be forgotten and the remaining dataset;

[0050] A dataset to be forgotten forgetting module, configured to update the parameters of the global model by performing federated forgetting training on the forgetting client and the remaining clients, so as to forget the influence of the dataset to be forgotten on the global model.

[0051] In a third aspect, the present invention further provides a computer device, including:

[0052] A memory, configured to store a computer program;

[0053] A processor, configured to execute the computer program to implement the steps of the above-mentioned federated causal forgetting learning method.

[0054] In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the program is executed by a processor to implement the steps of the above-mentioned federated causal forgetting learning method

[0055] Compared with the prior art, the beneficial effects achieved by the present invention:

[0056] The federated causal forgetting learning method provided by the present invention analyzes the dependence relationship between the dataset to be forgotten and the remaining data of the forgetting client by constructing a causal relationship graph, introduces a counterfactual data generation mechanism, cuts off the causal path of the dataset to be forgotten, and transfers the influence of the dataset to be forgotten to the counterfactual data. And by using knowledge distillation technology combined with KL divergence loss and cross-entropy loss, the knowledge of the original model is transferred to the new model, realizing the elimination of the influence of the dataset to be forgotten while maintaining the model performance of the remaining data.

[0057] The federated causal forgetting learning method provided by the present invention not only solves the problems of storage overhead and computational complexity, but also avoids the influence of the data to be forgotten on the remaining data, thereby improving the overall efficiency and accuracy of the federated forgetting method. Through experimental verification, the method of the present invention has shown excellent effects on various data sets, especially achieving a good balance between efficiency and forgetting accuracy. Description of the Drawings

[0058] Figure 1 It is a schematic flowchart of a federated causal forgetting learning method provided by an embodiment of the present invention;

[0059] Figure 2 It is a causal relationship graph before adding the counterfactual data set in an embodiment of the present invention;

[0060] Figure 3 It is a causal relationship graph after adding the counterfactual data set in an embodiment of the present invention;

[0061] Figure 4 It is a schematic framework diagram of the federated forgetting learning method provided by an embodiment of the present invention;

[0062] Figure 5 It is a schematic diagram for comparing the backdoor accuracy of the data to be forgotten after applying different forgetting learning methods on the MNIST data set in an embodiment of the present invention;

[0063] Figure 6 It is a schematic diagram for comparing the backdoor accuracy of the data to be forgotten after applying different forgetting learning methods on the CIFAR-10 data set in an embodiment of the present invention;

[0064] Figure 7 It is a schematic structural diagram of a federated causal forgetting learning device provided by an embodiment of the present invention;

[0065] Figure 8 It is an internal structural diagram of a computer device provided by an embodiment of the present invention. Detailed Embodiments

[0066] The technical solution of the present invention will be described in detail below through the drawings and specific embodiments. The same reference numerals in the drawings denote the same or similar components or parts. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. The embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Without conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.

[0067] As used herein, the term "and / or" is merely a description of the relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the character " / " in this text generally indicates that the associated objects before and after are in an "or" relationship.

[0068] Example 1:

[0069] As Figures 1 to 6 shown, an embodiment of the present invention provides a federated causal forgetting learning method. Figure 1 For the flow chart of the federated causal forgetting learning method, this flow chart only shows the logical order of the method described in this embodiment. On the premise of non - conflict, in other possible embodiments of the present invention, the steps shown or described can be completed in a different Figure 1 order than that shown.

[0070] The federated causal forgetting learning method provided in this embodiment can be applied to a terminal and can be executed by a federated causal forgetting learning device. This device can be implemented in software and / or hardware and can be integrated into the terminal.

[0071] See Figure 1 , the method of the embodiment of the present invention specifically includes the following steps:

[0072] Step 1: Based on the dataset to be forgotten, the remaining dataset, model features, and prediction results of the forgetting client, construct a causal relationship graph.

[0073] Federated Unlearning is usually carried out during or after Federated Learning. In Federated Learning, there is a central server and multiple clients. Each client has its own independent local original dataset, and the data between clients is not interoperable.

[0074] In the embodiment of the present invention, the clients include forgetting clients and remaining clients. The number of forgetting clients can be one or more, and the number of remaining clients can be zero, one, or more. Due to privacy considerations, the forgetting client sends a forgetting request to the server, asking to erase the contribution of the dataset to be forgotten in its local dataset to the global model. In steps 1 to 4 of the embodiment of the present invention, except for the final update of the parameters of the global model, the operations are all for one of the forgetting clients. If there are multiple forgetting clients, the same or similar processing is performed.

[0075] As Figure 2 shown, the causal relationship graph contains four core variables: the dataset to be forgotten , the remaining dataset , Model Features and the prediction result , the prediction result is specifically the label prediction distribution.

[0076] Among them, the remaining dataset and the dataset to be forgotten are interconnected through two-way feature relationship links ① and ②. This connection stems from the fact that the dataset to be forgotten and the remaining dataset may belong to the same category or have similar features, making it difficult for the model to distinguish the influence of the two during learning. There is a causal relationship link ④ between the dataset to be forgotten and the model features , and there is a causal relationship link ③ between the remaining dataset and the model features , indicating that the dataset to be forgotten directly affects the model features through relationship ④, while the remaining dataset directly affects the model features through relationship ③, showing that both types of data shape the intermediate representation of the model through the feature extraction process.

[0077] At the label prediction level, there is a causal relationship link ⑥ between the dataset to be forgotten and the prediction result , and there is a causal relationship link ⑤ between the remaining dataset and the prediction result , indicating that the remaining dataset and the dataset to be forgotten directly affect the predicted label distribution through relationships ⑤ and ⑥ respectively, which means that the information in the original data directly participates in the final classification decision.

[0078] In addition, there is a causal relationship link ⑦ between the model features and the prediction result , indicating that the intermediate feature representation of the model is further mapped to the final classification result.

[0079] The causal relationship graph reveals the complex dependencies between the data to be forgotten and the model output, especially the combined impact of the data to be forgotten on the model behavior through multiple paths.

[0080] Step 2: Based on the causal relationship graph, generate a counterfactual dataset, and generate the relationship links between the counterfactual dataset and the remaining dataset, model features, and prediction result, and cut off the relationship links of the dataset to be forgotten in the causal relationship graph.

[0081] Such as Figure 3As shown, the dataset to be forgotten is replaced by the counterfactual dataset , the model features and the prediction results are respectively updated to and . The counterfactual dataset affects the features through relationship ②, and directly affects the label prediction through relationship ④, while the remaining dataset affects and respectively through the original relationships ① and ③. An indirect mapping of relationship ⑤ is retained between the model features and the label prediction . It should be noted that all causal paths (original relationship ④, relationship ⑥) of the dataset to be forgotten have been cut off. The counterfactual data simulates a hypothetical scenario that the model has never learned through perturbation generation and label replacement. This structural change effectively isolates the impact of the dataset to be forgotten on the global model, and at the same time guides the model to adjust features and predictions through the counterfactual dataset to ensure that the causal paths of the remaining data are retained. Finally, the forgotten model drives decisions only through and independent paths, achieving a balance between the forgetting goal and the model performance.

[0082] Specifically, based on the causal relationship graph, a counterfactual dataset is generated, including:

[0083] Select samples from the remaining dataset , where is the feature, is the label of;

[0084] Generate an adaptive mask matrix based on the sensitive area of the model features, and the mask intensity is positively correlated with the gradient magnitude of the global model for ;

[0085] Apply non - linear perturbation to to generate counterfactual features , where ⊙ is element - wise multiplication;

[0086] According to the prediction confidence distribution of the model for , select the category with the lowest confidence as the counterfactual label , and construct a counterfactual sample ;

[0087] By iteratively optimizing the mask matrix, make in the feature space and the data to be forgotten to maximize the intra-class distance.

[0088] Step 3: Add the counterfactual dataset to the local dataset of the forgetting client, where the local dataset of the forgetting client includes the dataset to be forgotten and the remaining dataset.

[0089] The number of forgetting clients in the present invention can be one or more, and the calculated counterfactual dataset is added to the local datasets of each forgetting client.

[0090] Step 4: Update the parameters of the global model by performing federated forgetting training on the forgetting clients and the remaining clients, so as to forget the influence of the dataset to be forgotten on the global model.

[0091] In the first training round, the forgetting client uses the global model as the teacher model and initializes the student model.

[0092] Send the global model updated after the Kth (K≥1) training round to the forgetting clients and the remaining clients, so that it becomes the student model for the (K + 1)th round of training of the forgetting clients, and becomes the local model for the (K + 1)th round of training of the remaining clients; the remaining clients use their local datasets to train the local model alone to obtain local model parameters; the forgetting clients use their local datasets, and under the guidance of the teacher model, train the student model, and adjust the parameters of the student model through the knowledge distillation loss to obtain student model parameters; aggregate the local model parameters and the student model parameters to update the global model.

[0093] Among them, the calculation formula of the knowledge distillation loss is:

[0094] ,

[0095] Among them, is the KL divergence loss of the feature layer, is the cross-entropy loss of the prediction layer, is the balancing weight, is the temperature parameter;

[0096] Among them, the KL divergence loss of the feature layer is:

[0097] ,

[0098] Among them, is the KL divergence loss; is the adjustment coefficient; is the feature distribution of the student model; is the feature distribution of the teacher model; is the feature representation extracted by the teacher model from the input data; is the remaining data set; is the feature representation extracted by the student model during the forgetting process; is the data set to be forgotten; is the counterfactual data set;

[0099] The cross-entropy loss of the prediction layer is:

[0100] ,

[0101] where is the cross-entropy loss; is the adjustment coefficient; is the remaining data label; is the counterfactual data label; is the label prediction distribution of the teacher model for the input data; is the adjusted label prediction distribution of the student model during the forgetting process.

[0102] Update the parameters of the global model, including:

[0103] At the After the completion of the round of training, dynamically allocate weights based on the local data volume of the client, and calculate the update of the student model parameters of the forgetting client and the update of the local model parameters of the remaining clients

[0104] Weighted aggregate the updates of the two types of clients according to the data volume ratio, and the aggregation formula is:

[0105] ,

[0106] where is the number of local data sets of the forgetting client; is the number of local data sets of the remaining clients; is the number of forgetting clients; is the number of remaining clients.

[0107] Figure 4 shows the framework schematic diagram of the federated forgetting learning method in the embodiment of the present invention.

[0108] In order to verify the effectiveness of the federated forgetting learning of the present invention in the context of sample forgetting, we trained the corresponding CNN models on the MNIST and CIFAR-10 datasets as the target models to be forgotten. The model trained on the MNIST dataset achieved a prediction accuracy of more than 98% on the test set, while the model trained on CIFAR-10 achieved an accuracy of more than 78%. Subsequently, experiments with different forgetting learning methods were conducted on these models. According to the number of clients involved in the sample forgetting scenario, three client number configurations of N = 6, N = 8, and N = 10 were set. Experiments were conducted for each configuration, where the proportion of samples to be forgotten in the forgetting client was set to δ = 5% and δ = 10%.

[0109] Figure 5 The backdoor accuracy of the data to be forgotten after applying various forgetting methods on the MNIST dataset is shown. Figure 6 The backdoor accuracy of the data to be forgotten after applying various forgetting methods on the CIFAR-10 dataset is shown. Among them, RT represents complete retraining, Ours represents the federated causal forgetting learning method of the present invention, FedAvg represents the data of the initial forgetting model, and PGD is a comparative method of forgetting by reverse gradient descent. The experiments show that among the three methods, RT achieves the best performance, while our method matches the performance of RT very well and consistently outperforms PGD in all experiments, proving its effectiveness in removing forgotten data.

[0110] Table 1 shows the performance of each forgetting method on the remaining data after forgetting. In general, the method of the present invention achieves a non-learning effect comparable to RT and outperforms other methods, especially on the CIFAR-10 dataset, where it outperforms the PGD method by 2.71% on average. The reason for the advantage is that during the sample learning process, the forgotten data and the remaining data share similar classes, and there is a strong correlation between their samples and features. Completely eliminating the model's memory of the forgotten data will inevitably affect the performance of the remaining data. On the contrary, restoring the performance of the model on the remaining data may recall some of the forgotten data. By utilizing causal analysis, the method of the present invention focuses on the correlation between the data to be forgotten and the remaining data, and by introducing counterfactual data in the forgetting process, clarifies the relationship between the two, thereby ensuring good forgetting performance while maintaining the accuracy of the remaining data.

[0111] Table 1 Accuracy of the remaining data under different forgetting learning methods

[0112] In addition, the embodiments of the present invention selected some of the above-mentioned experiments and measured the learning efficiency of each method according to the time required to achieve the maximum learning effect. As shown in Table 2, the method of the present invention is superior to the comparative method and significantly exceeds RT. The reason for the advantage is the use of knowledge distillation in the causal forgetting process, which allows for the rapid acquisition of local forgetting models on each client. In addition, our method does not require a separate recovery training phase; the process of causal forgetting basically combines forgetting and recovery, and is more efficient compared to PGD.

[0113] Table 2 Learning Efficiency of Different Forgetting Learning Methods

[0114] To achieve the forgetting effect, the method of the present invention introduces causal analysis into federated forgetting learning, uses counterfactual data to separate the causal relationship between the forgotten and retained data, so as to achieve more precise forgetting. Knowledge distillation further strengthens this process by aligning the feature representations and label predictions of counterfactual data with those of the original model, ensuring the elimination of the influence of forgotten data while retaining the data. Experimental evaluations on multiple datasets show that the method of the present invention achieves effective learning and maintains the accuracy of the model, providing a practical and adaptable solution for federated forgetting learning.

[0115] Embodiment 2:

[0116] Based on the same inventive concept as Embodiment 1, the embodiments of the present invention also provide a federated forgetting learning device for implementing the above-mentioned federated causal forgetting learning method. The implementation solutions provided by this device to solve problems are similar to those described in the above method. Therefore, the specific limitations in the embodiments of the federated causal forgetting learning device provided below can refer to the limitations on the federated causal forgetting learning method in the above text and will not be elaborated here.

[0117] As Figure 7 shown, the embodiments of the present invention provide a federated causal forgetting learning device, including:

[0118] A causal relationship graph construction module, configured to construct a causal relationship graph based on the dataset to be forgotten, the remaining dataset, model features, and prediction results of the forgetting client;

[0119] A counterfactual dataset generation module, configured to generate a counterfactual dataset based on the causal relationship graph, generate a relationship link between the counterfactual dataset and the remaining dataset, model features, and prediction results, and cut off the relationship link of the dataset to be forgotten in the causal relationship graph;

[0120] A counterfactual dataset adding module, configured to add the counterfactual dataset to the local dataset of the forgetting client, where the local dataset of the forgetting client includes the dataset to be forgotten and the remaining dataset;

[0121] A dataset to be forgotten forgetting module, configured to update the parameters of the global model by performing federated forgetting training on the forgetting client and the remaining clients, so as to forget the influence of the dataset to be forgotten on the global model.

[0122] Embodiment 3:

[0123] The embodiment of the present invention further provides a computer device, which may be a server, and its internal structure diagram may be as Figure 8 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used for the processor to exchange information with external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements the federated causal forgetting learning method in the foregoing embodiments.

[0124] Those skilled in the art can understand that Figure 8 the structure shown in

[0125] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0126] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps of the following method:

[0127] Construct a causal relationship graph based on the dataset to be forgotten, the remaining dataset, the model features, and the prediction results of the forgetting client;

[0128] Based on the causal relationship graph, generate a counterfactual dataset, and generate a relationship link between the counterfactual dataset, the remaining dataset, model features, and prediction results, and cut off the relationship link of the dataset to be forgotten in the causal relationship graph;

[0129] Add the counterfactual dataset to the local dataset of the forgetting client, where the local dataset of the forgetting client includes the dataset to be forgotten and the remaining dataset;

[0130] Update the parameters of the global model by performing federated forgetting training on the forgetting client and the remaining clients to forget the influence of the dataset to be forgotten on the global model.

[0131] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0133] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device realizes the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby the instructions executed on the computer or other programmable apparatus provide steps for realizing the functions specified in one process or a plurality of processes and / or blocks Figure 1 one process or a plurality of processes and / or blocks Figure 1 steps of the functions specified in one block or a plurality of blocks.

[0135] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope protected by the present invention and the claims. These all fall within the protection scope of the present invention.

Claims

1. A federated causal forgetting learning method, characterized in that, Including: Construct a causal relationship graph based on the to-be-forgotten dataset, remaining dataset, model features, and prediction results of the forgetting client; Based on the causal relationship graph, generate a counterfactual dataset, and generate the relationship link between the counterfactual dataset, the remaining dataset, model features, and prediction results, and cut off the relationship link of the to-be-forgotten dataset in the causal relationship graph; Add the counterfactual dataset to the local dataset of the forgetting client, and the local dataset of the forgetting client includes the to-be-forgotten dataset and the remaining dataset; Update the parameters of the global model by performing federated forgetting training on the forgetting client and the remaining clients, so as to forget the influence of the to-be-forgotten dataset on the global model.

2. The federated causal forgetting learning method according to claim 1, characterized in that The number of the forgetting clients is one or more; the number of the remaining clients is zero, one, or more.

3. The federated causal forgetting learning method according to claim 1, characterized in that, Construct a causal relationship graph based on the to-be-forgotten dataset, remaining dataset, model features, and prediction results of the forgetting client, including: A feature relationship link is set between the to-be-forgotten dataset and the remaining dataset; A causal relationship link is set between the to-be-forgotten dataset and the model features; a causal relationship link is set between the to-be-forgotten dataset and the prediction results; A causal relationship link is set between the remaining dataset and the model features; a causal relationship link is set between the remaining dataset and the prediction results; A causal relationship link is set between the model features and the prediction results.

4. The federated causal forgetting learning method according to claim 3, wherein Generate a counterfactual dataset based on the causal relationship graph, including: Select samples from the remaining dataset , where is a feature and is the label of Generate an adaptive mask matrix for sensitive regions based on the model features , where the mask strength is positively correlated with the gradient magnitude of the global model for ; Pair Apply a non - linear perturbation to generate counterfactual features , where ⊙ represents element - wise multiplication; According to the prediction confidence distribution of the model select the category with the lowest confidence as the counterfactual label and construct a counterfactual sample ; By iteratively optimizing the mask matrix, maximize the intra-class distance between the feature space and the dataset to be forgotten.

5. The federated causal forgetting learning method according to claim 1, wherein By performing federated training on the forgetting client and the remaining clients, including: In the first training round, the forgetting client uses the global model as the teacher model and initializes the student model; It also includes: Send the global model updated after the K (K≥1) -th training round to the forgetting client and the remaining clients, so that it becomes the student model for the (K + 1) -th round of training of the forgetting client, and becomes the local model for the (K + 1) -th round of training of the remaining clients; The remaining clients use their local datasets to train the local model separately to obtain local model parameters; The forgetting client uses its local dataset, and under the guidance of the teacher model, trains the student model, and adjusts the parameters of the student model through the knowledge distillation loss to obtain student model parameters; Aggregate the local model parameters and the student model parameters to update the global model.

6. The federated causal forgetting learning method according to claim 5, wherein, The calculation formula of the knowledge distillation loss is: ， Among them, is the KL divergence loss of the feature layer, is the cross-entropy loss of the prediction layer, is the balancing weight, is the temperature parameter; Among them, the KL divergence loss of the feature layer is as follows: ， Among them, is the KL divergence loss; is the adjustment coefficient; is the feature distribution of the student model; is the feature distribution of the teacher model; is the feature representation extracted by the teacher model from the input data; is the remaining dataset; is the feature representation extracted by the student model during the forgetting process; is the dataset to be forgotten; is the counterfactual dataset; The cross - entropy loss of the prediction layer is: ， Among them, is the cross-entropy loss; is the adjustment coefficient; is the remaining data label; is the counterfactual data label; is the label prediction distribution of the teacher model for the input data; is the adjusted label prediction distribution of the student model during the forgetting process.

7. The federated causal forgetting learning method according to claim 6, characterized in that, Update the parameters of the global model, including: After the completion of the nth round of training, the weights are dynamically allocated based on the local data volume of the clients, and the parameter updates of the student models of the forgotten clients are calculated and the parameter updates of the local models of the remaining clients ; Weighted aggregate the updates of the two types of clients according to the data volume ratio, and the aggregation formula is: ， Among them, is the number of local data sets of the forgotten clients; is the number of local data sets of the remaining clients; is the number of forgotten clients; is the number of remaining clients.

8. A federated causal forgetting learning device, characterized in that, Including: A causal relationship graph construction module, which is used to construct a causal relationship graph based on the to-be-forgotten dataset, remaining dataset, model features, and prediction results of the forgetting client; A counterfactual dataset generation module, which is used to generate a counterfactual dataset based on the causal relationship graph, and generate the relationship link between the counterfactual dataset, the remaining dataset, model features, and prediction results, and cut off the relationship link of the to-be-forgotten dataset in the causal relationship graph; A counterfactual dataset adding module, configured to add the counterfactual dataset to the local dataset of the forgetting client, where the local dataset of the forgetting client includes the dataset to be forgotten and the remaining dataset; A dataset to be forgotten forgetting module, configured to update the parameters of the global model by performing federated forgetting training on the forgetting client and the remaining clients, so as to forget the influence of the dataset to be forgotten on the global model.

9. A computer device, characterized in that, Comprising: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the steps of the federated causal forgetting learning method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the steps of the federated causal forgetting learning method according to any one of claims 1 to 7 are implemented.

Images