Sparse perception gradient rising federal forgetting learning method and device
By sparse the initial local model of the target client and using projection gradient rise to perform forgetting learning, the problems of high computational cost and catastrophic forgetting in the existing technology are solved, and efficient data privacy protection and model updates are achieved.
Patent Information
- Application Number
- CN202510436537.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-07-25
AI Technical Summary
The existing federal forgetting learning methods have shortcomings in computing costs, storage burdens, and catastrophic forgetting, making it difficult to efficiently implement data privacy protection and model updates.
By sparse the initial local model of the target client and use the projection gradient rise to perform forget learning training, combined with the sparse reference model as a constraint, it reduces the computational cost and prevents catastrophic forgetting.
It effectively reduces computing costs, improves forgetting effect, prevents catastrophic forgetting, and achieves efficient data privacy protection and model updates.
Smart Images
Figure CN120373364A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of information security and relates to a sparse-aware gradient ascent federated forgetting learning method and device. Background Art
[0002] With the development of large models, a large amount of data has been collected and used for training models, which makes enterprises and individuals increasingly concerned about data privacy. Relevant laws stipulate that if a company or individual's data is used for training a model, they have the right to request the withdrawal of the data's contribution to the model to protect their right to be forgotten. Retraining from scratch is an accurate federated forgetting learning method, and all clients must retrain their local models from scratch and aggregate them into a new global model. However, retraining requires a very large computational overhead, and it is impractical to use this method in resource-constrained situations.
[0003] Current federated forgetting learning methods, some divide clients into different clusters. When forgetting, only the clients in the target client's cluster need to be retrained from scratch, without waiting for other clusters to train the model. However, this method requires continuously updating the storage history, bringing a huge storage burden; some send forgetting requests through the server to forget the low-quality data in the client and achieve forgetting without having to store historical updates. However, this method requires the server to have additional data as a reference, so it is difficult to meet in practical applications; there is also by introducing term frequency-inverse document frequency to quantify the category discrimination of channels and pruning the most relevant channels in the target category to forget its contribution to the model. However, this method requires each client to calculate an approximate inverse Hessian matrix, resulting in high computational costs. Summary of the Invention
[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a sparse-aware gradient ascent federated forgetting learning method and device that can reduce computational costs and prevent catastrophic forgetting.
[0005] To achieve the above object, the present invention is implemented by the following technical solutions:
[0006] In a first aspect, the present invention provides a sparse-aware gradient ascent federated forgetting learning method, including:
[0007] Obtain a global model for federated learning training and use the global model as the initial local model of the target client; wherein, the participants in the federated learning include the target client and the remaining clients, and the local data of the target client includes data to be forgotten;
[0008] Sparsify the initial local model to obtain the local model of the target client;
[0009] Perform forgetting learning training on the local model by means of projected gradient ascent to forget the influence of the data to be forgotten on the local model;
[0010] Use the local model that has completed forgetting to update the global model.
[0011] Further, sparsify the initial local model, including:
[0012] ,
[0013] wherein, represents the initial local model of the target client ; represents a sparsified mask matrix with values of 0 or 1; represents element-wise multiplication; represents the initial local model after sparsification.
[0014] Further, obtain the local model of the target client, including:
[0015] Based on the sparsified initial local model, calculate the reference model of the target client :
[0016] ,
[0017] wherein, represents the total number of the target client and the remaining clients; represents the global model;
[0018] Use the reference model as the local model for performing forgetting learning training of the target client .
[0019] Further, train the local model by means of projected gradient ascent, including:
[0020] ,
[0021] wherein, represents the projection operator, represents the local model of the target client ; represents the forgetting learning rate; represents the local model loss function of the target client ; represents at the gradient on;
[0022] Add additional constraints during the forgetting process , where the value of is the average of the L2 distances between
[0023] and ten random models. Furthermore, let the target client have a local dataset , where is a feature, and is the corresponding label; the local dataset is divided into a dataset to be forgotten and a remaining dataset , ;
[0024] The optimization objective of the forgetting learning training is:
[0025] ,
[0026] where represents the loss of the sample on the local target model of the target client ; represents the number of the dataset to be forgotten in the target client .
[0027] Furthermore, the method further includes: when the target client performs forgetting learning training on the local model, one or more of the remaining clients continue to perform federated learning.
[0028] In a second aspect, the present invention further provides a sparse-aware gradient ascent federated forgetting learning device, and the device includes:
[0029] An initial local model acquisition module, configured to acquire a global model for federated learning training and use the global model as the initial local model of the target client; where the participants in the federated learning include the target client and the remaining clients, and the local data of the target client includes data to be forgotten;
[0030] A local model acquisition module, configured to sparsify the initial local model to obtain the local model of the target client;
[0031] A local model training module, configured to train the local model by means of projected gradient ascent to forget the influence of the data to be forgotten on the local model;
[0032] A global model update module, configured to update the global model by using the local model that has completed forgetting.
[0033] In a third aspect, the present invention further provides a computer device, including:
[0034] a memory for storing a computer program;
[0035] a processor for executing the computer program to implement the steps of the above-mentioned sparse-aware gradient ascent federated forgetting learning method.
[0036] In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the program is executed by a processor to implement the steps of the above-mentioned sparse-aware gradient ascent federated forgetting learning method
[0037] Compared with the prior art, the beneficial effects achieved by the present invention are:
[0038] The sparse-aware gradient ascent federated forgetting learning method provided by the present invention sparsifies the initial local model of the target client, and mathematically proves that the sparsity operation can narrow the forgetting effectiveness gap between exact forgetting and approximate forgetting; based on the sparsified initial local model, a reference model is designed for the target client, and the reference model is used as a constraint to execute the gradient ascent process and obtain a forgetting model. Experiments prove that the sparse-aware gradient ascent federated forgetting learning method of the present invention can reduce the probability of random forgetting, improve the forgetting effect, and prevent catastrophic forgetting. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 is a schematic flowchart of a sparse-aware gradient ascent federated forgetting learning method provided by an embodiment of the present invention;
[0040] Figure 2 is a schematic framework diagram of the sparse-aware gradient ascent federated forgetting learning method in an embodiment of the present invention;
[0041] Figure 3 is a schematic diagram of the forgetting effect of the model in an embodiment of the present invention under different sparsities;
[0042] Figure 4 is the forgetting effect of using random gradient ascent for 2 rounds of forgetting learning training in an embodiment of the present invention;
[0043] Figure 5 is the forgetting effect of using random gradient ascent for 3 rounds of forgetting learning training in an embodiment of the present invention;
[0044] Figure 6 is a schematic diagram of the time cost of different forgetting methods in an embodiment of the present invention;
[0045] Figure 7Schematic structural diagram of a sparse-aware gradient ascent federated forgetting learning device provided by an embodiment of the present invention;
[0046] Figure 8 Internal structure diagram of a computer device provided by an embodiment of the present invention. Detailed implementation manners
[0047] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The same reference numerals in the drawings denote the same or similar components or parts. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. The embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Without conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.
[0048] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the preceding and following associated objects.
[0049] Embodiment 1:
[0050] As Figures 1 to 6 shown, an embodiment of the present invention provides a sparse-aware gradient ascent federated forgetting learning method. Figure 1 is a flowchart of the sparse-aware gradient ascent federated forgetting learning method. This flowchart only shows the logical order of the method described in this embodiment. On the premise of non-conflict, in other possible embodiments of the present invention, the steps shown or described can be completed in a different order from Figure 1 shown.
[0051] The sparse-aware gradient ascent federated forgetting learning method provided in this embodiment can be applied to a terminal and can be executed by a federated forgetting learning device. This device can be implemented in a software and / or hardware manner and can be integrated in the terminal.
[0052] Refer to Figure 1 , the method of the embodiment of the present invention specifically includes the following steps:
[0053] Step 1: Obtain a global model for federated learning training and use the global model as the initial local model of the target client; wherein, the participants in the federated learning include the target client and the remaining clients, and the local data of the target client includes data to be forgotten.
[0054] Federated Unlearning is usually carried out during or after Federated Learning. In Federated Learning, there is a central server and multiple clients. Each client has its own independent local dataset, and the data between clients is not interoperable. Due to privacy considerations, the target client needs to send a forgetting request to the server, asking to erase the contribution of the data to be forgotten in its local dataset to the global model. In the embodiments of the present invention, the number of target clients is one, and the number of remaining clients is at least one.
[0055] Step 2: Sparsify the initial local model to obtain the local model of the target client.
[0056] Among them, sparsifying the initial local model includes:
[0057] ,
[0058] Among them, represents the initial local model of the target client ; represents a sparsified mask matrix with values of 0 or 1; represents element-wise multiplication; represents the initial local model after sparsification.
[0059] The following proves the role of the above sparsification method in federated unlearning. The error characteristics between the gradient ascent forgetting model and the retrained model from scratch is the weight distance between them,
[0060] ,
[0061] Among them, is the big symbol (big notation, representing complexity), is the learning rate, represents the global model at the th iteration and the weight distance between the initial global model is the average of the maximum singular values.
[0062] Next, prove that the boundary value of the error can be written as In the form of the boundary, according to the paper by Thudi et al. (Thudi A, Deza G, Chandrasekaran V, et al. “Unrolling sgd: Understanding factors influencing machine unlearning,” in Proc. IEEE European Symposium on Security and Privacy (EuroS&P), 2022, pp. 303-319), stochastic gradient descent is used for training during federated learning, and the trained model is sparsified; the sparse model is as follows:
[0063] ,
[0064] where is the sparsified model, is the round of client sample data, , and the initialized model for all clients is ; while can be expressed as:
[0065] ,
[0066] , in order to forget , it is necessary to add , but calculating all the terms including takes a lot of time cost; taking as the method of sparse forgetting, the remaining part is used to calculate the boundary, which can be obtained by ignoring all terms in except
[0067] .
[0068] Because , and is the dominant term, thus, it can be obtained that:
[0069]
[0070]
[0071]
[0072] .
[0073] Inspired by the formula in the paper by Thudi et al., we can obtain:
[0074]
[0075] ,
[0076] where, , and is the largest singular value of;
[0077] From the above formula, we can draw the conclusion that the error of can be written in the form of the boundary of.
[0078] According to the form of the boundary of, it can be seen that the error of the model decreases with the increase of sparsity. Therefore, sparsifying the model is beneficial to narrowing the gap in forgetting effects between exact and approximate forgetting. However, when the sparsity level is too high, the accuracy of the remaining data may decrease. Therefore, it is very important to balance forgetting performance and effectiveness.
[0079] In addition, obtaining the local model of the target client also includes:
[0080] During the forgetting process, in order to prevent catastrophic forgetting of the remaining data, the average value of the remaining client models is used as a reference, which can be expressed as ; however, in the target client, the parameters of other clients are unknown. Therefore, based on the initial local model after sparsification, the present invention calculates the reference model of the target client :
[0081] ,
[0082] where, represents the total number of the target client and the remaining clients; represents the global model;
[0083] Taking the reference model as the local model for forgetting learning training of the target client .
[0084] Step 3: Perform forgetting learning training on the local model by means of projected gradient ascent to forget the influence of the data to be forgotten on the local model.
[0085] Among them, training the local model by means of projected gradient ascent includes:
[0086] ,
[0087] Among them, represents the projection operator, represents the local model of the target client ; represents the forgetting learning rate; represents the local model loss function of the target client ; represents on gradient.
[0088] To avoid catastrophic forgetting, the present invention adds additional constraints during the forgetting process , among which, the value of is the average of the L2 distances between
[0089] In an embodiment of the present invention, it is assumed that the target client has a local dataset , the sample , among which, is the feature, is the corresponding label; the local dataset is divided into a dataset to be forgotten and the remaining dataset , , ;
[0090] The optimization objective of the forgetting learning training is:
[0091] ,
[0092] Among them, represents the loss of the sample on the local target model of the target client ; represents the number of the dataset to be forgotten in the target client .
[0093] Step 4: Use the local model that has completed forgetting to update the global model.
[0094] When the target client performs forgetting learning training on the local model, the remaining clients of the present invention can wait for the target client to complete forgetting, or can continue with federated learning and continuously update the global model.
[0095] Using the local model that has completed forgetting to update the global model includes:
[0096] ,
[0097] Among them, represents the local model that has completed forgetting, represents the local models of the remaining clients .
[0098] Figure 2 Fig. shows a schematic framework diagram of the sparse-aware gradient ascent federated forgetting learning method in an embodiment of the present invention.
[0099] In order to verify the effectiveness of the sparse-aware gradient ascent federated forgetting learning method of the present invention in practical applications, a series of experiments were conducted.
[0100] As Figure 3 shown, first, the forgetting effects of the model under different sparsities were compared. Target refers to the accuracy of the forgetting model for the data to be forgotten, and Other refers to the accuracy of the forgetting model for the remaining data. The lower Target is and the higher Other is, the better the comprehensive effect of the forgetting model. It can be found that when the sparsity is 0.2, the forgetting effect is good and the accuracy of the remaining data is also high. Therefore, considering all factors, the effect is best when the sparsity is 0.2.
[0101] Since projected gradient ascent (PGA) is used in the present invention, PGA can control the movement of the forgetting learning model in the direction of the target data rather than in the direction of the remaining data. Once the sparsity is too high and the distance of the model exceeds the threshold, correction will be performed, resulting in that too high sparsity will instead make the forgetting effect of the model worse.
[0102] Figure 4 Fig. shows the forgetting effect after the model is sparsified and two rounds of forgetting learning training are performed using stochastic gradient ascent ( ); Figure 5 Fig. shows the forgetting effect after the model is sparsified and three rounds of forgetting learning training are performed using stochastic gradient ascent ( ). It can be found that when , catastrophic forgetting will occur regardless of the sparsity setting. When , as the sparsity increases, the forgetting effect is generally improved. However, when the sparsity reaches a certain level, the accuracy of the forgetting model for the remaining data will be greatly reduced and catastrophic forgetting will occur. Therefore, it can be concluded that high sparsity brings a better forgetting effect of the model.
[0103] The sparse-aware gradient ascent federated forgetting learning method (Ours), federated learning (FL), federated retraining (FR), and projected gradient descent (PGD) of the present invention are respectively used for comparative experiments on the CIFAR10 dataset, MNIST dataset, and SVHN dataset. The experimental results are shown in Table 1. It can be found that federated learning has the highest accuracy, but it does not perform the forgetting operation. In the forgetting experiments on these three datasets, federated retraining has the lowest accuracy for the data to be forgotten, forgets the most thoroughly, and has the highest accuracy for the remaining data. However, the time cost of implementing federated retraining is too high, and it can only be used as the optimal standard. The accuracy of the projected gradient descent method for the data to be forgotten is much higher than that of the method proposed in the present invention. That is to say, the method of the present invention forgets more thoroughly. Because the method of the present invention reduces the influence of the data to be forgotten and the remaining data on the model while sparsifying, and through projected gradient ascent optimization, the forgetting of the remaining data is very small. Generally speaking, the method proposed in the present invention has better results than the projected gradient descent method, and at the same time does not require retraining from scratch, saving time costs.
[0104] Table 1. Accuracy of different forgetting methods on multiple datasets
[0105]
[0106] The embodiment of the present invention also tests the time costs of three methods: the method of the present invention (Ours), federated retraining (FR), and projected gradient descent (PGD). The experimental dataset used is the MNIST dataset. As Figure 6 shown, when the number of clients participating in federated forgetting learning is 5, the time cost of the method of the present invention is 0.8 seconds, the time cost of the projected gradient descent method is 0.11 seconds, and the time cost of federated retraining is 104.11 seconds; when the number of clients participating in federated forgetting learning is 10, the time cost of the method of the present invention is 0.8 seconds, the time cost of the projected gradient descent method is 0.11 seconds, and the time cost of federated retraining is 104.11 seconds. The method proposed in the present invention has a slightly increased time cost compared to the projected gradient descent method due to the sparse operation, but it is still much lower than federated retraining.
[0107] Embodiment 2:
[0108] Based on the same inventive concept as Embodiment 1, the embodiment of the present invention also provides a sparse-aware gradient ascent federated forgetting learning device for implementing the above-mentioned sparse-aware gradient ascent federated forgetting learning method. The implementation solution provided by this device for solving problems is similar to the implementation solution described in the above method. Therefore, the specific limitations in the embodiment of the sparse-aware gradient ascent federated forgetting learning device provided below can refer to the limitations on the federated forgetting learning method in the above text, and will not be repeated here.
[0109] As Figure 7 shown, an embodiment of the present invention provides a sparse - aware gradient - ascent federated forgetting learning device, including:
[0110] An initial local model acquisition module, configured to obtain a global model for federated learning training and use the global model as the initial local model of the target client; wherein, the participants in the federated learning include the target client and the remaining clients, and the local data of the target client includes data to be forgotten;
[0111] A local model acquisition module, configured to sparsify the initial local model to obtain the local model of the target client;
[0112] A local model training module, configured to train the local model by means of projected gradient ascent to forget the influence of the data to be forgotten on the local model;
[0113] A global model update module, configured to update the global model by using the local model that has completed forgetting.
[0114] Embodiment 3:
[0115] An embodiment of the present invention further provides a computer device, which may be a server, and its internal structure diagram may be as Figure 8 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non - volatile storage medium and an internal memory. The non - volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non - volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with external terminals through a network connection. When the computer program is executed by the processor, it implements the sparse - aware gradient - ascent federated forgetting learning method in the foregoing embodiments.
[0116] Those skilled in the art can understand that Figure 8 the structure shown in
[0117] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.
[0118] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the following method are implemented:
[0119] Obtain a global model for federated learning training, and use the global model as the initial local model of the target client; wherein, the participants in the federated learning include the target client and the remaining clients, and the local data of the target client includes data to be forgotten;
[0120] Sparsify the initial local model to obtain the local model of the target client;
[0121] Perform forgetting learning training on the local model by means of projected gradient ascent to forget the influence of the data to be forgotten on the local model;
[0122] Use the local model that has completed forgetting to update the global model.
[0123] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0124] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.
[0125] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.
[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby the instructions executed on the computer or other programmable apparatus provide steps for realizing the functions specified in one process or a plurality of processes and / or one block or a plurality of blocks. Figure 1 one process or a plurality of processes and / or blocks Figure 1 steps for realizing the functions specified in one block or a plurality of blocks.
[0127] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope protected by the present invention and the claims. All of these fall within the protection scope of the present invention.
Claims
1. A sparse-aware gradient ascent federated forgetting learning method, characterized in that, Including: Obtain the global model for federated learning training and use the global model as the initial local model of the target client; wherein, the participants in the federated learning include the target client and the remaining clients, and the local data of the target client includes the data to be forgotten. Sparsify the initial local model to obtain the local model of the target client. Perform forgetting learning training on the local model by means of projected gradient ascent to forget the influence of the data to be forgotten on the local model. Use the local model that has completed forgetting to update the global model.
2. The sparse-sensing gradient ascent federated forgetting learning method according to claim 1, wherein Sparsifying the initial local model includes: ， Among them, represents the initial local model of the target client ; represents a sparsified mask matrix with values of 0 or 1; represents element-wise multiplication; represents the initial local model after sparsification.
3. The sparse-aware gradient ascent federated forgetting learning method according to claim 2, wherein Obtaining the local model of the target client includes: Calculate the reference model of the target client based on the sparsified initial local model : ， Among them, represents the total number of the target client and the remaining clients; represents the global model; Take the reference model as the target client and perform local model forgetting learning training.
4. The sparse-sensing gradient ascent federated forgetting learning method according to claim 3, wherein Training the local model by means of projected gradient ascent includes: ， Among them, represents the projection operator, represents the local model of the target client ; represents the forgetting learning rate; represents the local model loss function of the target client ; represents the gradient on ; Adding additional constraints during the forgetting process , where the value of is the average of the L2 distances between 5. The sparse-aware gradient ascent federated forgetting learning method according to claim 4, wherein Set the target client owns a local dataset , samples , where is a feature and is the corresponding label; the local dataset is divided into a dataset to be forgotten and the remaining dataset , ; The optimization objective of the forgetting learning training is: ， Among them, represents the sample at the target client on the local target model loss; represents the target client in the dataset to be forgotten quantity.
6. The sparse-sensing gradient ascent federated forgetting learning method according to claim 1, characterized in that Also included: When the target client performs forgetting learning training on the local model, one or more of the remaining clients continue with federated learning.
7. A sparse-aware gradient ascent federated forgetting learning device, characterized in that, Including: An initial local model acquisition module, configured to obtain the global model for federated learning training and use the global model as the initial local model of the target client; wherein, the participants in the federated learning include the target client and the remaining clients, and the local data of the target client includes the data to be forgotten. A local model acquisition module, configured to sparsify the initial local model to obtain the local model of the target client. A local model training module, configured to train the local model by means of projected gradient ascent to forget the influence of the data to be forgotten on the local model. A global model update module, configured to use the local model that has completed forgetting to update the global model.
8. A computer device, characterized in that, Including: A memory, configured to store a computer program. A processor, configured to execute the computer program to implement the steps of the sparse-aware gradient ascent federated forgetting learning method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the sparse-aware gradient ascent federated forgetting learning method according to any one of claims 1 to 6.